Evaluation and Learning and your Post Secondary Mental Healthy Campus Strategy
Evaluation – A friend or a painful requirement?
What do you think of when you hear the word ‘evaluation’? Recognizing your automatic assumptions may be useful – especially if your funder means something different.
Does it conjure up visions of a watchdog? Something that sets out to judge how well you’ve done with your programs and services? Something that sets perfection as the minimal acceptable standard? Or maybe it makes you think about something that generates lots of useless information and reports that sit on shelves. Or something you’d do just because you have to, as a condition of getting funds.
Can you imagine that evaluation might be your friend? This approach aims to position evaluation and learning as activities that help you know the impact you’re having (or not). This helps you to know whether you will have a greater impact if you adapt what you’re doing, and also helps you know when conditions have changed so that what you did before is no longer as effective. Evaluation can provide you useful information for the decisions you’ll need to make, as well as being accountable to your funder. It can provide you useful information for advocacy, and for celebrations.
What do we mean by ‘learning’?
One kind of learning is receiving knowledge from someone more knowledgeable on the subject – the kind we most often associate with school. The kind of learning we’re referring to here is that which comes from experience. Sometimes called action learning, or experiential learning. You’ve perhaps heard the saying ‘This person has 20 years of experience, the other person has 1 year of experience repeated 20 times’. Having 20 years of experience indicates that you’ve paid attention to the impact of your actions, reflected on whether there are changes you could make that would improve the impact you’re trying to create, and made adaptations (which starts a new cycle of Do, Reflect, Adapt). This is the lifelong process that helps you move from being a novice to being an expert and then beyond to being a master – able to adapt your actions to the particular circumstances.
If you would like more background on Evaluation and Learning, check out the Evaluation and Learning Background document and references.
Developing your Evaluation and Learning Plan
Like anything else, it will help you to have a plan for your evaluation and learning process. It’s not a good idea to wait to the end to think about having the information you need for your report and find that no one has been keeping track as your initiatives rolled out.
An evaluation and learning plan has sections. They are developed by considering a number of key questions:
Note you may not work through these questions in order – different people might jump around, start from the top and work through, or start from the end and work back. There are a few examples in the Background document of ‘working backward’ to get at your answer to a question.
- For what purposes are you doing the evaluation and learning? In other words, what do you want to be able to learn or decide as a result of the information? (Meeting your funder’s requirements is enough, but you may want to add some for your own purposes).
- Who’ll be responsible for making sure the evaluation and learning processes are done? Are there other roles that need to be identified?
- Who are the audiences for the information from the evaluation and learning? The reporting you’ll need to do for the ACMHI funding is the minimum, but again, you may want to do more with the information. Other audiences might include your SA leadership team, perhaps the Counseling department or other campus partners, your SA Advocacy team?
- What kinds of information are needed to support your learning? What information do you need to make what decisions? What do you want your intended audiences to know or do differently? Sometimes it helps to identify the questions you’d like answers to, and then identify the information you’d need to answer those questions. In some cases another department might need information from you to improve their services and activities. If this is the case, include it here.
- One type of information is a description of what services and activities you’ve implemented as a part of your ACMHI strategy, and the context of your campus. You can get these from your Maturity Model worksheets. The Maturity Model also helps you explain the logic or rationale of your actions –why do you think they are likely to improve the state of your mentally healthy campus?
- See the Evaluation and Learning Background document for different kinds of information.
- Which stakeholders will it help to have involved? At what stage?
- From what sources should the information be collected? This might include students; insurance company benefits used (e.g. proportion of medication claims that are mental illness related); faculty (e.g. do you see more or less impacts from poorly managed stress); counselors (e.g. Are the students they see having more or less trauma because they have the added benefit of the SA’s programs? or Are student leader led initiatives filling an important gap or are they just nice to have?).
- How can that information be collected in a reasonable fashion, e.g., questionnaires at each event; periodic surveys, interviews or focus groups, examining documentation, or by asking people to draw images or take photographs, create poems describing their experiences, etc.
- Include your planned process for making sure that enough of your students complete the ACMHI survey so you can get information about your campus.
- When is the information needed (so, by when must it be collected and your analysis and sense-making processes completed?)?
- What will you need in your report to your funder(s), what ways will you want to tell your story to others? It will include a high level description of your Big Picture Strategy (described in the Start Here document) and the range of services and actions you delivered, advocated or worked with others to provide to students. Will you add to your story with tables of data, quotes from interviews or focus groups, perhaps a case study or video if you want to communicate a complex process. ACMHI reports are due at the end of each term. For the format and required information, see the ACMHI requirements.
- What resources are available for all the parts of the evaluation?
- Develop an evaluation workplan with activities, timelines and resources and add that to your overall workplan. Who has what roles and responsibilities?
- Following the evaluation and learning process you may want to reflect on what evaluation capacity you’d like to build or improve in yourself or others. This may be something that becomes more relevant later in your work or career as more and more funders demand more and more carefully done evaluations.
These steps are available as a pdf ACHMI ELF Planning and Management Table if you want to keep them handy.
What you’re actually after – a culture of inquiry and adaptation
You’re not trying to get a grade in a class or a PhD or trying to be a professional evaluator. Evaluation and learning processes can seem very overwhelming and challenging and something to avoid if you can. If you can find ways to make them fun and obviously useful that will be a great achievement! If you can, think of how you design and implement your activities and services as a serious but exciting experiment.
In the end, what you actually want is to build a culture of inquiry and adaptation. This is what will be most helpful to the SA and your ability to do the best job at improving students’ well-being and reducing stigma. This is what will consistently get you the ability to learn and adapt your strategy, and to get quality information for making decisions or assuring your funder that they made a good investment in your project.
Take a look at what other student leaders are doing but don’t assume you all need to do the same things. There will always be differences in how much capacity different campuses have to take on evaluation and learning processes. Seeing creative ways that other members use to learn what works without much effort helps all the members learn from each other. You might find that you’re able to collaborate with other campuses – creating a single survey instrument that everyone uses for example. You may find that if you can do some things together over the course of the year – like holding a shared learning circle – it will be more fun and give some different insights.
Creating a culture of inquiry and adaptation for yourself – seeing evaluation and learning as your friend rather than your judge – will also be the approach that helps you most throughout your career. You can think of your evaluation and learning approach for your PSE mentally healthy campus strategy as part of your professional development.