Generating ideas, connections, and action

How to Keep Learning and Strengthen Your Program : Lessons for leadership programs from an evaluation perspective

I enjoy doing leadership development evaluation and getting into the nitty gritty of how program staff think about and implement their leadership development efforts. Some of the most important work of evaluations happens upfront and it’s work that programs would benefit from doing themselves. In the spirit of our former board chair, Eugene Kim (Faster than 20), I found myself thinking about how to share some of what we are learning more broadly to give programs a leg up in clarifying program goals, design and expected change. I am not suggesting that it’s not useful to have an outside perspective and new eyes on the program, but let’s face it, a lot of programs don’t have the resources for external evaluations. So for the DIY folks here are a few ideas:

 

  1. State the goal of your program: In the old days, we often used to hear that the goal was to develop stronger leaders. In the past, people were less inclined to be explicit about their societal goals since most evaluation practices were focused on whether people were better leaders rather than the contribution of their improved leadership skills, connections, or capacities to the field in which they worked. It is helpful in designing leadership supports to stretch further and talk about leadership to what end? Clarity about the desired social change benefits will influence who you select and the supports that would be likely to help them contribute to your social change goal. For example, is your goal to improve health outcomes for children in a specific region, to reduce violence against women, or to tackle education achievement gaps? If you don’t already have one (and if you do, revisit it from time to time) state your program’s goal as clearly as possible. Now think about how you are communicating the program’s purpose and goal and how it informs the design, content and delivery of your program.

 

  1. Keep asking ‘why and what’ questions to unearth your assumptions. I have met with funders who were thinking about starting a leadership program, e.g. a program to strengthen civic leadership. I asked them a lot of ‘why’s and ‘what’s: What do you mean by civic leadership, and what is the problem you see that you associate with the quality or performance of civic leaders? Why do you think a different kind of civic leadership is the solution to that problem; what does stronger civic leadership look like; what do you know about approaches that try to develop this kind of leadership and what have been the results? In the end, this funder decided on a different strategy to address huge divides in political discourse that were undercutting community life. See if you can come up with the 3-4 assumptions that are underlying your program’s purpose and design.

 

  1. Prioritize your key strategies: What strategies flow from your program goal and assumptions, and anchor the design of your leadership work? People talk about best practices and while there are certainly some we would lift up, much of leadership work is contextual. For example, a program that has an explicit goal of trying to diversify the non-profit leadership pipeline may have a strategy of creating a network of mentors, while another program focused on addressing disparities in education may have a strategy of providing participants with a strong equity framework and introducing racial impact analysis tools. Can you identify the 3-4 most important strategies that connect the design of your leadership effort to your leadership program’s goal?

 

  1. Identify expected changes: Now that you have clarified your goals, made your assumptions explicit, and designed a program with the strategies that you think will help you make progress on your goals, how will you test your thinking and know if your program is succeeding? What are the changes you are hoping to see in the next year; and if those things are happening, what do you think will happen further down the road as a result, maybe in 3 years? This is the hard work since many of us have pretty lofty goals and don’t expect in the life of a program or association with it to see a significant reduction in carbon emissions or quality education for all children. Still, it’s hard to know if we are on the right track without some road signs. See if you can come up with 3-4 changes you hope to see more immediately and 3-4 things that you believe would happen 3-5 years out as a result of the more immediate changes.

 

  1. Find out what happened: How will you get information about changes that are occurring? This can be some of the more costly and time-consuming work of leadership programs, especially if you are doing interviews to understand in more depth about the experience of participants. A lot of the time when we ask participants how they feel about their program they appreciated the opportunity to participate and often say, “I would like to give back.”  Learning is a great way to give back. Convening participants to ask them about their experience can be very helpful and it’s great to ask people closest to the experience to make meaning not only of their own experience but what they are hearing from others in the group. You can go back to the changes you anticipated to help you formulate the questions for the group. People are also experimenting with using google forms to create dashboards as a way of collecting data over multiple points in time that helps participants see how they are doing and can identify areas that might need more attention. Approaches that support continuous learning and building the learning capacity of groups are, in my mind, best.

 

You may have guessed by now that this is a walk through a Theory of Change process. I have learned a lot from looking at other people’s ToCs. Some are very detailed like the Health Foundation for Western and Central New York’s ToC for their leadership program (which they have generously agreed to share), and some are much higher level. Here is a blank template you could use to gather your answers to these questions.

 

It would be cool if people wanted to share their ToC work so we could learn from each other. If you send them to us, we will create a central repository. I think the term ToC is daunting and prefer to simply call it getting clear about the program’s purpose and assumptions to support ongoing learning and improvement. I am not trying to put myself out of a job completely or dismiss the value of an external evaluation (that could be another post). I mostly want to encourage everyone to have fun digging deeper into your program and what you can learn and are learning.