Leadership Learning Community
in partnership with Connective Associates and The Berkana Institute
Notes prepared by Aerin Dunford
Claire Reinelt and Bruce Hoppe shared their paper, Social Network Analysis and the Evaluation of Leadership Networks and we addressed the question: What are we learning about leadership networks and the usefulness of SNA as a network facilitation and evaluation tool?
One of the challenges that is raised at the end of the paper is how useful SNA is for supporting emergent community networks. Debbie Frieze and Aerin Dunford joined us from the Berkana Institute. We used Berkana as a context to discuss some of the challenges they have had with traditional forms of evaluation, and attempted to explore in more depth the following topics:
• how SNA might be productively used to illuminate community networks,
• how communities might benefit from visualizing/understanding themselves as networks and/or networks of networks, and
• what some of the challenges and unintended consequences might be of using SNA.
Notes from our conversation:
Introduction – Claire Reinelt
Check in with images: Think of a time when you felt a deep sense of community with others with whom you were learning or taking some action with, what image are you drawn to that evokes some aspect of those feelings. Each participant shared their name, organization and an explanation of the image they chose.
Introduction to the Art of Learning Centering – The connection to place, practice and learning – Debbie Frieze and Aerin Dunford
Introduction to social network analysis and the facilitation and evaluation of field/policy networks and collective leadership networks and learning communities – Bruce Hoppe
What Berkana thinks about the predominant use of this kind of mapping work:
Berkana feels that much of the current work on networks displays old paradigm bias. In social network analysis, physical representations of the network are created by mapping relationships. This is useful for convincing people that networks exist, and people are often fascinated to see the network made visible. Other network analysts name roles played by members of the network or make distinctions between different parts of the network, such as core and periphery. It may not be the intent of these researchers, but their work is often used by leaders to find ways to manipulate the network, to use it in a traditional and controlling way.
What’s missing in these analyses is an exploration of the dynamics of networks:
• Why do networks form? What are the conditions that support their creation?
• What keeps a network alive and growing? What keeps members connected?
• What type of leadership is required? Why do people become leaders?
• What type of leadership interferes with or destroys the network?
• What happens after a healthy network forms? What’s next?
If we understand these dynamics and the lifecycle of emergence, what can we do as leaders, activists and social entrepreneurs to intentionally foster emergence?
These are complementary but different perspectives from what Bruce and Connective Associates have noticed in their work with SNA – some of the questions or criticisms are similar, but are based in different reasoning.
It is important to be thinking about both the product and what you do with network maps as well as considering the process of creating the map. In participatory evaluation, we often forget to pay attention to the process.
Explanation of different aspects of Social Network Analysis – Bruce Hoppe
Bridging and bonding: the importance of illustrating the connections within a network is that if you don’t take the time to illuminate collectively who is learning from whom and who is learning with each other then you may miss out on seeing the big picture of the community.
Bonding is a measurement of connectivity amongst players (individuals or organizations) inside a network. It helps to establish a greater sense of community and identity within an established system. Measuring the degree of bonding is helpful for measuring community/efficiency.
Bridging is looking at the connectivity from one system others – it occurs when individuals or organizations start to seek knowledge outside of the system they are a part of. Bonding assists with bringing in innovation from outside and having a greater overall impact.
One of the main aspects of Bruce’s work, and part of the focus of the paper written by Claire and Bruce is around looking at how different kinds of metrics or ways of evaluating work within different kinds of networks: Peer Leadership Networks, Organizational Leadership Networks, Field/Policy Networks and Collective Leadership Networks.
Clustering is noticing the various groups within a network map that are connected to each other and naming these. This is also related to the idea of centrality: the hubs and their relationship to clusters: who has access to what people and information and what is the culture within these clusters?
Structural Equivalence (Amazon.com is a great example) By asking different individuals what their interests or affinity is to different topics or modes of action we are able to see a new level of relatedness by clustering them according to their answers.
By considering both the people and the outside issue or topic, we can see where there is overlap and a new layer of information and perspective on the network can be gained.
Betweenness measures the likelihood that some piece of information goes through a particular hub or node in a map. Betweenness can also be useful in measuring efficiency: if you wanted something to travel very quickly from one part of a system to another, betweeness would tell you which particular hub to go to in order to move info quickly.
Sited the Green Healthy Building network and the Schott network as case-studies to illustrate the different uses and information that can be gleaned from social network analysis.
For more information on Bruce’s presentation, please see the pdf of his Power Point presentation or check out Bruce’s blog: http://connectedness.blogspot.com/
For more information on the background, please read Claire and Bruce's paper - http://www.leadershiplearning.org/node/588.
Risk of SNA: One of the issues that comes up when participants in a SNA study look at the results, there may be a degree of judgment about one’s place in the map: if they are or are not as involved in the group as others or have lower degrees of betweenness. How do people react to this? Encourage that there be an opening in working together on a project like this that it is just one window into the network, but emphasize that there are many other ways to look at the system out there.
What about the facilitator/bridger/weaver/builder in a network? How do the individuals/organizations that are particularly connected in a network play a role as a change agent to push the system or move it in particular directions?
In reference to applying this particularly to the Berkana Exchange: If we’re talking about a collective leadership network, if only the facilitator/caller (The Berkana Institute) is interested in learning from this tool, it might be a pitfall of using SNA in this particular community. It seems like SNA map might not be a good choice of evaluation tools if not everyone finds it valuable.
What difference does it make who you ask in an organization to describe the relationships?
What difference has been made from doing these kinds of maps? What has been done with them? – This is particularly interesting to The Berkana Institute, because we would like to use SNA and other similar tools to demonstrate that learning is happening in the Exchange. It seems like it would be necessary to evaluate what the network looks like before an Art of Learning Centering gathering and then some period after the gathering.
Is network mapping useful in helping a very internally bonded organization become more aware of how they are part of something larger.
We have a hypothesis that by virtue of doing the analysis some change will happen in the system - how do we demonstrate what kind of change has happened?
Where did these labels come from in the map? Who creates the categories? Are they named before hand by those conducting the analysis or are they generated by the members of the network themselves? In the case of the Schott network analysis, these were descriptions that people both gave to themselves and they were also deduced by those conducting the analysis from surveys done with participants prior to the mapping exercise.
It might be a good idea for Berkana to begin by looking at the end results of other SNA experiments; examining some of the things that we think might have been demonstrated by these maps, getting some examples of SNA, where is been useful, for whom and for what purpose for the Exchange community to look at before we begin the project. If we look at some of these other examples, it might then be worthwhile to test some hypotheses about the usefulness of SNA in the Berkana field.
Structural Equivalence and the practice areas or communities of practice. Within the Exchange community there are people and learning centers are learning about and practicing in different areas. Perhaps using a structural equivalence lens to look at these different practices and the people/learning centers involved in them, would help us know what communities of practice to put our attention into.
What is the use of network mapping for the participants in the Berkana Exchange? In terms of knowing learning is happening, they don’t have evidence but they just know. What’s the use to participants that we be able to demonstrate that this learning is unfolding? Why should it matter to participants?
The problem is that over time doubt seeps in about the value of an individual’s or learning center’s participation in the Exchange community. How do we justify spending time at the AoLC and how do we show to other staff members that it has value?
If you did a before and after analysis – something you could think about before hand would be producing a result with this evaluation that could really add inspiration to the individuals who participate in the gathering to go back and be more engaged with their work, more effective and more able to share what they are learning as a result of being engaged in the Exchange.
You might also be able to think about what extent do people continue to interact with others in the network and what is the value of those connections?
It sounds like you’re taking network mapping and making a connection between learning and effectiveness. There was some curiosity around how we came to the connection between measuring learning and measuring efficiency, which are two very different things. To what extent is this being used by learning centers to go out and effect change in their community? Will the connection between Communities of Practice and structural equivalence help to strengthen the CoPs? Who’s creating the community of practice and to what purpose – and is there any thing that you are adding by doing SNA that would add value to the Community of Practice? Is there something that you (The Berkana Institute) add to this community or CoPs by doing SNA?
Coming back to clarity on the question we are seeking to inquire into with this project: How do you create the conditions to really demonstrate that learning has occurred at the AoLC?
The struggle is that with the amount and types of diversity within the community and the health of the Exchange network being largely a factor of health in the learning centers themselves – the struggle is in the question: what’s enough value? Is there enough of a shared concrete way of asking a question that’s still going to be valuable? Have we really come up with a set of compelling questions to be able to demonstrate learning in a powerful and effective way?
When I first heard the question, I thought that the frame of reference was very narrow. I think the network map that you do should be about two levels out – the first level (which I don’t think you should measure) is how l.c.’s participation in the network shows them how their work at home is changing. The second is measuring how their communities are changing.
If we look at the lifecycle of communities and networks, it is important to really look at what is called for in the lifecycle now. At first you realized that you needed more time to be together – 10 days. It is a big deal for some learning centers to take off from their work for so long. Maybe people need fewer days at the AoLC now – maybe it should go back to being shorter.
For me there is the real question not only of demonstration that learning is happening but also being able to see and measure the degree of integration of learning. How is what people are learning at the AoLC being reintegrated back into their work with the learning centers in a meaningful, measurable way? What is the effect of their participation on other parts of the organization (either people, way of work or programs)?
My thought on using this tool for this purpose is that there’s not enough information yet about whether or not the tool would be useful for your purposes of evaluation – if you can begin to identify the places where learning integration has already happened as a base and then in real time you can start to track this kind of learning integration. So far there are not any hypotheses that have arisen out of the abstract on this project.
Debbie: The learning of impact, integration – what are the things you would advise us right now about how to do this?
Maybe using an Appreciative Inquiry process to evaluate what has happened at the gathering. A little hard to do that at the level of integration and larger scale impact.
You may want to start to integrate the evaluation process into the planning for the gathering now – look at the dynamics of that process – who are you talking to? Who’s talking to you? Who’s active in the planning of the AoLC process? Then using network mapping to look at what’s happening in dynamics after the gathering.
You could use questionnaires and other kinds of tools to identify behaviors that people begin to demonstrate back in their own learning centers, over and above the natural rewards system.
Highlight who the connectors and people of influence – perhaps unidentified leaders?
I would have you look at the basic OD 101 issue, which cautions against sending a changed person back into an unchanged environment. What’s the impact of inviting participants to the AoLC, having them become changed throught he experience and then sending them back into an environment where others have not had a similar experience.
In the work of measuring impact at the local learning centers you might want to think about whether the premise that a change at the individual level (in this case individual learning center) results in change at the community level. This might not necessarily be true. Test that hypothesis – if you really want to measure community change, does it really work to measure the change at the learning center level or not?
How are the communities that the learning centers work with changing? Is there a strengthening of social capital – who’s talking to whom? How do we even measure that? The quantification of links between people I don’t think really helps to prove that this strengthening is real. What does strengthening of social capital look like?
Is this project perhaps putting the cart before the horse? By putting the hypothesis forward first? We haven’t yet defined what the space might look like? In one instance, someone did an exercise where many individuals drew a map of what conditions they felt would be needed for transformational learning to happen in their organization – the maps that several people drew were almost identical – this helped us to be able to see where the common ground was and the kind of value that could be generated for the organization. Until there’s a better sense of the amount of value, and a better understanding of the value of this kind of evaluation by the whole community, perhaps we can’t begin to create hypotheses and test them.
You can’t really test out the idea of measuring learning integration, beyond self-reporting from the participants – you could ask people: what learning from the last AoLC meeting was integrated into recent changes in the learning center? You could then initiate learning conversation amongst those people who felt that they were a part of this integration process – what was it that enabled them to transfer the knowledge and implement the ideas of transferring the knowledge? Then maybe mapping later and seeing any correlation between these people, their relationship to the whole of the Exchange.
Question for Bruce and Debbie and Aerin: Do you really have a sense of what you want to do together?
If you do use a tool like this, there is an important question of how to introduce it? I would definitely suggest bringing it into the planning process right now.
Whenever you measure something it changes that which you measure. However you want to use the measurement in a way that you firmly believe in – in a way that the results actually enhances your view of the world. One never knows exactly what will change as a result of your measurement, but you have to take the leap that if you are using the results in a way that you really believe in and that is in alignment with your values, that it will be a useful exercise. You have to take that leap.
Check-out: Mention one thing you’re taking away from today – one learning.