There are strong feelings in the community sector whether social improvements can or even should be subject to measurement. My preoccupation with this tension started in grad school learning program evaluation and then the first five years of my career as I helped charities in Saskatoon evaluate programs. It was never straightforward or easy, but always possible.
Now I get to do this work every day! 18 charities we fund set over 100 targets to achieve with and for their clients. In 2017/18 91% of these targets were met or exceeded!!
These are big and tough outcomes to accomplish, because they matter. For the first time in New Brunswick, to my knowledge, we know that 77% of women who were supported by Second Stage here in Saint John, remained free of violence for 3 years AFTER they moved out on their own. Now we don’t just believe it works, we know these programs help families break the cycle of violence. For each mom, there are children who are safe and not witnessing violence in their safe space, their homes.
There is so much reason to be encouraged. These great organizations are getting clearer and better at identifying and measuring progress towards important milestones along the way to ultimate outcomes. For example, three years ago the TRC measured the percent of youth who were attending school regularly and the credit accumulation of youth what came to the TRC. These are good things to measure to see if you’re helping youth successfully transition to adulthood. This year, however, the TRC has become crystal clear. They knew their service is more effective the more time they have with youth. Now they track the success of kids who attend at least 3x per week (79%) and they have segmented outcomes by grade level. For example, it is not a reasonable expectation for a youth in grade 9 to have post secondary plans, but it is for grade 11 (92%). Each grade now, there are unique milestones the TRC is working towards with the youth.
I share these results to demonstrate it is possible and essential to measure the results of programs. It is important for funders, of course, but it is essential for decision making by organizations.
My Two Cents
Measurement should be most useful to the charity; it should be part of doing business, not an afterthought to satisfy funding requirements. Information about program effectiveness should direct time and resources. Results should teach and challenge. Data should be meaningful.
Funders, let’s pay attention to the most meaningful measures of success, as determined with and by our partners. Let’s not micro manage and require tonnes of reporting that produces information we are not going to use. Let’s focus on outcomes rather than outputs. The TRC could increase their participation numbers from 80 to over 100 to show they serve more kids by counting every youth who stops in occasionally; but it is less meaningful. The question we should ask is not how many came, but how many were better as a result.