Despite considerable investment in student success programs over the last decade, student success outcomes remain stagnant at many colleges and universities. It’s still unclear who needs help, services are underutilized, staff capacity is limited, information is siloed, and aligning approaches across departments is challenging.
Institutions successfully moving the needle on student outcomes are doing something different – and it starts with comprehensively evaluating their student success initiatives to improve them iteratively and intelligently. Regular evaluation of student success practices provides the specific insight needed to make informed decisions and take action to incrementally improve student outcomes and generate more revenue over time.
After working with hundreds of higher education student success and assessment leaders and helping evaluate more than 1,000 student success initiatives, we’ve learned there’s a better way to know which student success initiatives are working (and for whom) and how to use that insight to drive institutional improvement.
Step 1: Identify All Existing Student Success Initiatives
It’s important to begin with a thorough audit of what initiatives exist today. We’ve worked with many higher education leaders who aren’t quite sure what student success initiatives exist across their institutions. Many can list the most obvious or visible, like the tutoring center or those closely related to their own work. However, when they set out on a mission to understand the full scope of initiatives across all departments on campus, they’re often surprised by how many they uncover.
Initiatives can range from ongoing services and support programs to sporadic, one-off resources and events. Examples might include counseling or mental health services, guest speaker events, faculty professional development, or a student job fair.
When you begin your audit, you might start by interviewing department heads, deans, career center leaders, students, advisors, faculty members, etc. Throughout the process, document what you learn. Make sure you uncover the following:
- What is the initiative designed to do?
- Who is the initiative designed for?
- What department, person, or team oversees the initiative?
- Who is eligible to participate in the initiative?
- Does it potentially overlap with another initiative?
This foundational understanding will serve as an important input in Step 2.
Here is an example of how you might document the information you gather in step one. Want your own copy? Download the inventory template here.
Step 2: Understand the Efficacy of Each Initiative
Take a look at your initiative inventory and select a handful of initiatives that warrant further investigation. Some of our most successful customers choose to hone in on services that are either clearly duplicative, services that require measurable outcomes to meet funding requirements, or services that require significant investment each year.
During this step, answer questions like:
- Who is mostly taking advantage of this service?
- How much do different student groups benefit from this service? (consider quantitative and qualitative evidence)
- Are the benefits we see clearly linked to the outcome the service intends to achieve? If not, why not?
Here is an example of how you might document the information you gather in step two. Want your own copy? Download the inventory template here.
Step 3: Prepare for Action… Then Take It
Once you have decided where to focus and completed your preliminary discovery, it’s time to ensure you have enough information to take action.
During this step, consider more nuanced questions like:
- Where overlap exists, is it intentional or simply duplicative?
- Would there be a benefit to combining duplicative efforts after better understanding the relative impact of each?
- What is the “theory of change” of this program… why would we assume it achieves these outcomes?
- If Student Learning Outcomes (SLOs) are the focus, do they have any logical or evidence-based relationship to persistence, or are they geared toward some other outcome of value to the institution?
- How has this initiative changed over time, and has that been in response to data of some kind?
- Does the initiative have any special cultural, historical, or emotional significance to the campus community, and why?
- Is there a relationship between what the initiative means to people and what it achieves?
- Most importantly … what should we do now?
Some institutions have used the information they gather to change policy or seek additional funding for certain services. In contrast, others have sent highly targeted nudges to students most likely to benefit. For example, one partner university updated the required number of supplemental instruction visits, and the success team sent personalized messages to students shown as most likely to benefit from their Center for Academic Reading.
The key here is that with the right level and kind of intelligence, it is possible to focus your efforts and begin the process of iterative improvement.
Improve Upon What Works, Rethink What Doesn’t
Someone with the best of intentions created every student success initiative. As we seek to understand and improve the effectiveness of these interventions, we need to appreciate them in context. Why did they begin, how have they evolved and changed, what challenges do they face today, and upon what concepts are they based?
We find that a process like appreciative inquiry, aided by great measurement, helps ground our investigation of efficacy in strengths and creates a safe space for improvement. Almost every program is effective for someone… so how can we value the initiative’s intention, recognize and improve upon the pieces that work, and rethink what doesn’t?
