The second goal of the UTFI Vision for 2025 is to “expand constituent engagement by developing meaningful relationships with University of Tennessee alumni, students, and friends.” In order to measure our progress toward that goal, the UTFI Business Intelligence team developed a custom Alumni Engagement Score, which established a FY2019 System-wide baseline of 203,617 engaged alumni. Our goal is to increase that count by 3.5% annually, which translates to 210,740 for FY2020.
How does the UTFI Alumni Engagement Score work?
The UTFI Alumni Engagement Score is descriptive. It provides a useful measure of engagement by calculating a relative score (0-5) for each of our alumni. In other words, it tells us who has been engaged, and to what extent they’re engaged, rather than predicting who will become engaged.
Every alum who has an Alumni Engagement score of 1-5 is counted as engaged. Those with a score of 0 are unengaged.
The Alumni Engagement Score was influenced, however, by the insights we’ve gained from our many prospect models, which are predictive. Our models look at hundreds of potentially important data points and identify those that most strongly correlate to the likelihood of a constituent making an annual gift, a major gift, a planned gift, etc.
The six factors in the Alumni Engagement Score were selected because they are descriptors of engagement and because they are also significant predictors in the Annual Giving Model – the logic being that one crucial outcome of our efforts to engage alumni should be increasing the likelihood of them becoming donors.
Alumni Engagement Factors
- Activities, Events, and Athletics: 43%
- Email Engagement Score: 28%
- Committees: 10%
- Awards & Honors: 8%
- Volunteer Activities: 7%
- Years of Giving to UT over Past 5 Years: 4%
What’s the difference between “Activities” and “Volunteer Activities”?
Let’s start here: Four of the six Alumni Engagement factors are grouped together on the ANDI Entity Overview.
The Alumni Engagement Score is simply looking at those tables for signs of engagement. So, the most accurate answer to this question is: “An activity is something that was entered in the ANDI activities table, and a volunteer activity is something that was entered in the ANDI volunteer activities table.”
Historically, we as an organization have defined these terms rather loosely. Why? Because we were not trying to measure alumni engagement! There was no external motivation for us to consistently enter this data in ANDI. Remember that these conversations about data/metrics began quite recently.
One byproduct of the creation of the Alumni Engagement Score is Policy 1.3.1: Engagement Factors, which is an important first step toward more clearly defining our terms. The following definitions draw from Policy 1.3.1 and from other sources of data that feed into the score.
Activities, Events, and Athletics (43%)
An Activity is an ongoing engagement factor that does not have a set date or time. Examples include weddings at UTC’s Patten Chapel, travel on a UTAA tour, and consulting with UTAA Career Services.
An Event is a one-time occurrence hosted by UT with alumni/friend attendees. Examples include fundraising galas, tailgates, chapter events, and receptions.
We also pull five types of UTK athletics attendance data: baseball, football, men’s basketball, softball, and women’s basketball.
- Scoring: 0-1. Anyone who is noted as having engaged with activities, events, and athletics is assigned a 1. Everyone else is a 0.
Email Engagement Score (28%)
The Email Engagement Score is a regularly updated measure of an alum’s level of email engagement.
- Scoring: 0-5. Currently, the basic calculation is (Emails Opened + Emails Clicked) / Emails Received, which is then weighted for recency (an email opened this year counts more than an email opened in 2017) and converted to 0-5 according to a scale.
Committees (10%)
A Committee is a group of individuals appointed by the University for a specific function. Examples include Board of Directors, Alumni Councils, and Dean’s Advisory Boards.
- Scoring: 0-1. Anyone who is noted in ANDI as having participated on a committee is assigned a 1. Everyone else is a 0.
Awards & Honors (8%)
Awards and honors are bestowed on an individual by the University or a related entity. Examples include Distinguished Alumni, College/Unit ‘Hall of Fame’ Awards, and Service Awards.
- Scoring: 0-1. Anyone who is noted in ANDI as having won an award or honor is assigned a 1. Everyone else is a 0.
Volunteer Activities (7%)
A Volunteer Activity occurs when an individual volunteers for service or other actions for the benefit of the University. Examples include Career Guides, Write-a-Vol, and Big Orange Give Ambassadors.
- Scoring: 0-1. Anyone who is noted in ANDI as having participated in a volunteer activity is assigned a 1. Everyone else is a 0.
Years of Giving to UT over the Past Five Years (4%)
This is a simple count of the number of years of giving over the previous five years.
- Scoring: 0-5
Questions about how to classify or enter your engagement data?
Contact User Experience at ask@utfi.org or (865) 974-4153. They offer one-on-one and group training.
How were the weightings determined?
Again, the Alumni Engagement Score is a descriptive measure built from the insights gained by the predictive Annual Giving Model. After we remove the factors in the Annual Giving Model that are not descriptors of alumni engagement (things like job title, wealth capacity, zip code, etc.), we rebalance the six remaining factors so that they add up to 100.
For example, in the Annual Giving Model, “Activities, Events, and Athletics” was ~54% more significant than “Email Engagement Score” for predicting someone’s likelihood to make an annual gift. That relative performance is reflected also in the Alumni Engagement Score: 43% weighting for Activities, Events, and Athletics versus 28% weighting for Email Engagement.
So why is John Doe rated a 4 for alumni engagement but Jane Doe is rated a 2?
Every alum in ANDI has been scored on each of the six Alumni Engagement factors. The weighting is then applied and the resulting total is converted to an Alumni Engagement Score based on the following scale:
- AE5: >=80
- AE4: 60-79
- AE3: 40-59
- AE2: 20-39
- AE1: 1-19
- AE0:0
John Doe is an AE4 because he has attended events, he opens and clicks on an above-average number of emails (score 3/5), he has served on a committee and volunteered, and he has given to UT in one of the past five years (1/5).
| Factor | Score | Weight | Weighted Score |
| Activities, Events, and Athletics | 1 | 43 | 43 |
| Email Engagement Score | 0.6 | 28 | 16.8 |
| Committees | 1 | 10 | 10 |
| Awards & Honors | 0 | 8 | 0 |
| Volunteer Activities | 1 | 7 | 7 |
| Years of Giving to UT over the Past Five Years | 0.2 | 4 | 0.8 |
| John Doe | 100 | 77.6 = AE4 |
Jane Doe is an AE2 because she lives far from campus in an area without a chapter, so she hasn’t attended any events or volunteered, but she stays engaged with UT by reading and clicking on emails (score 4/5).
| Factor | Score | Weight | Weighted Score |
| Activities, Events, and Athletics | 0 | 43 | 0 |
| Email Engagement Score | 0.8 | 28 | 22.4 |
| Committees | 0 | 10 | 0 |
| Awards & Honors | 0 | 8 | 0 |
| Volunteer Activities | 0 | 7 | 0 |
| Years of Giving to UT over the Past Five Years | 0 | 4 | 0 |
| Jane Doe | 100 | 22.4 = AE2 |
If the Alumni Engagement Score was inspired by the Annual Giving Prospect Model, then why is years of giving weighted at only 4%?
Generally speaking, fundraising metrics are relatively easy math: add up the dollars, add up the transactions. Measuring the return on our investment in alumni programming is a much bigger challenge and has, for years, been a significant topic of conversation among the alumni programming leadership team.
Years of giving is weighted relatively low because the Alumni Engagement Score is designed to provide one measure of the effectiveness of our alumni programming and communication efforts. While those efforts should ultimately increase the likelihood of alumni giving, that is not the primary goal of our colleagues in alumni programming and communications. Or, to put it more plainly, the job performance of a Director of Alumni Programming is not typically measured by the annual donor count.
If John Doe had made gifts in each of the past five years, instead of only one, he would be an AE5 instead of an AE4. Even at only 4% weighting, that factor can be a useful identifier of our most engaged alumni.
What is the practical value of this metric?
The Alumni Engagement Score was designed to generate the largest reasonable number. In Advancement Services we often talk about our percentage of “active” alumni, which is a count of alumni who have an active email address, home mailing address, and/or phone number (currently ~98% of UT alumni are active). It’s a measure of our data health.
Active alumni are people who we are capable of reaching. The Alumni Engagement Score calculates the number of alumni who have been reached.
What’s the difference between our Alumni Engagement Score and the new CASE Alumni Engagement Survey?
In 2019, CASE conducted its first annual Global Alumni Engagement Metrics Survey, which collects engagement data in four categories: Philanthropic, Volunteer, Experiential, and Communication. We submitted our numbers in December 2019—one survey each for UTC, UTHSC, UTK, and UTM.
There are two very significant differences between the two measures. The first is that our Alumni Engagement Score draws from historical data, whereas CASE is only measuring the previous fiscal year.
The second difference is the treatment of email. The Communication portion of the CASE survey counts alumni who contacted us in the previous year, whereas our score includes email engagement, which is a measurement of our success in contacting them over time.
As a first step toward a CASE Communication count, we requested a custom report from iModules that identified all alumni who had logged into one of our sites or completed any form submission other than an online gift or event registration, which were counted separately under Philanthropic and Experiential. In other words, we identified the alumni who contacted us through this single channel, iModules.
If Jane Doe has never logged into or submitted a form on one of our iModules websites, then she would not have been counted in the CASE survey. And there are a lot of alumni like Jane. As a result, the CASE survey count is significantly lower than our Alumni Engagement count: UT Knoxville’s baseline of Engaged Alumni is 127,816, whereas we reported 31,112 to CASE.
The Alumni Engagement Score and the CASE survey are both valuable data points because they describe our levels of alumni engagement in different ways and can, therefore, inform different types of strategic conversations.
How can my team help the Foundation hit our Alumni Engagement goals?
First, we can only count alumni engagement for events, committees, alumni awards, and volunteer activities if that information is entered properly in ANDI. One reason we’re looking at historical data for those factors, rather than focusing only on recent activity, is because tracking that data is a work-in-progress for all of us in the Foundation. Knowing that activities, events, awards, and committees have not been entered consistently into ANDI over the years, we chose to count everything we could find.
Second, as you can see from the examples of John and Jane Doe, the more alumni we touch through alumni programming, email communications, and annual giving (acquisition, retention, and reactivation), the more alumni we will count.
Advancement Services, led by the Online Engagement team, is developing a more proactive email win-back and data acquisition strategy to help meet both the Alumni Engagement goal and also the new Email Engagement goal.
Because our alumni currently do not get an Email Engagement Score until they’ve received at least 20 emails, the current Alumni Engagement Score likely underrepresents new grads. As part of our new email strategy proposal, we are also recommending a significant refinement of the Email Engagement Score.