By Shelley G. Trebesch: Two to four years into a church plant? How do you know the church is making a difference? Courageous enough to seriously ask?
How do we know our ministries, whether preaching, outreach, serving the poor, children’s programs, or life groups, make a difference? Measuring impact tends to be elusive and often downright avoided, especially in Christian circles. However, I offer a fun and insightful method for discerning impact called the Most Significant Change (MSC).[1]
MSC is a qualitative, monitoring and evaluation methodology centered around asking people the question “What is the most significant change that has taken place as a result of …” and then asking them to tell a story that illustrates the change. MSC allows respondents to share freely and results in many stories exemplifying personal, relational, and/or leadership changes in people’s journey with God, family, workplace, community, and so on. Indeed, the range of life-change themes for effective programs can be vast and unexpectedly surprising.
This process works best in an interview setting where the interviewer asks two questions then captures a story.
- What have been the three most important benefits of ________to you?
- What has been the most significant change in your life (or your colleagues’ lives or your family member’s lives) as a result of this _____ ?
- Please share a story that illustrates this change in your life (or your colleagues’ lives or your family member’s lives).
To conduct an evaluation, select a team of people who will interview and collect stories. For a small program, endeavor to collect stories from all participants. For a larger program or church, collect at least 10 percent of participants’ stories. Make sure the stories are told in personal ways, using pronouns such as “I” and “we.” Either take notes while listening, record the conversation, or have the person write the story.
Once the team collects the stories, have members read the stories individually and select the most important stories to them. Then, gather as a team and read the selected stories out loud. As a team, make a list of the emerging themes and how often they occur—repeated themes demonstrate significance. Select stories that represent the most themes i.e. the most significant themes. I find it helpful to use a spreadsheet to correlate how many stories have the same or differing themes. For example, if five out of ten persons reveal in their story that they practice reconciliation as a result of conflict management training, that theme occurs in 50 percent of the stories.
Different sets of eyes may notice new themes, potentially because of their relative importance to another team. Send the above, along with the list of themes, to the leadership team. The leadership team then reads the stories and captures the themes they see. A third group, say program directors, may capture other themes. Then create a spreadsheet that shows the three teams’ (columns) and their selected themes from highest (occurs in more individual stories) to lowest percentages and observe what can be learned from the three columns. What is important to whom?
In my experience with MSC, individual stories often reflect multiple themes that are important markers of the change process. This allows a church to understand their change theory—what activities, programs, practices, etc. actually instigate change?
Although MSC is a qualitative technique, given a sufficient number of stories, it is possible to conduct some quantitative analysis. For example, the percentages of the most common themes, as well as differences in themes that emerge in different contexts or as the result of different participant characteristics. One may notice the frequency and difference of themes amongst men and women or age categories or ethnicities or marital status, etc. This analysis enables leaders to plumb the depths of what activates change for certain groups, as they compare the themes to elements of the program.
Give it a try and discover the impact of your church!
[1] See mande.co.uk for a guide to MSC. Wikipedia also has a reasonable summary of the methodology at wikipedia.org.