Chapter
6 • Building
Leadership Development, Social Justice and Social Change in Evaluation
through a Pipeline Program: A Case Study of the AEA/DU Graduate Education
Diversity Internship Program
Prisca M. Collins and Rodney K. Hopson
Chapter 7 • From
The Inside Out: Evaluating Personal Transformation Leadership Efforts
Sally Liederman
Chapter
8 • Evaluating
Organizational Change/Transformation Efforts
Nancy Vollmer LeMay and Alison Ellis
Chapter 9 • The
Importance of Local Context in Leadership Development and Evaluation
Larry Peters and John Baum
Chapter 10 • Evaluating
Community Leadership Programs
Teresa Behrens and Maenette Benham
Chapter 11 • Evaluating Leadership as a Strategy
to Transform Complex Systems
K. Jinnett and T. Kern
Chapter 12 • Evaluating
Leadership Development for Social Change
Kim Ammann Howard and Claire Reinelt
Chapter 13 • Evaluating
Youth Leadership Development through Civic Activism
Hanh Cao Yu, Heather Lewis-Charp and Michele Gambone
Chapter 14 • Evaluating Leadership Efforts
for Neighborhood Transformation
Nilofer Ahsan
Part One • Part Two • Part Three • Book Home • Order
Building Leadership Development, Social Justice and Social Change in Evaluation through a Pipeline Program: A Case Study of the AEA/DU Graduate Education Diversity Internship Program
Prisca M. Collins and Rodney K. Hopson
This chapter presents the evaluation of leadership development pipeline programs with a case presentation of the American Evaluation Association/Duquesne University (AEA/DU) Graduate Education Diversity Internship Program, a pipeline effort aimed at developing leaders of color within the evaluation profession. Examples of existing pipeline programs are given, along with the design, planning, and implementation of such a pipeline plan into an existing leadership development program.
Chapter 6 Resources
The Association of American Colleges and Universities
http://www.aacu.org/irvinediveval/index.cfm
This site
provides an overview of the
James Irvine Foundation Campus Diversity Initiative and an evaluation of
its impact. The site also has a set of evaluation resources,
such as an Evaluation Project Resource Kit to help campuses create evaluation
plans to measure outcomes related to their campus diversity initiatives.
The American Evaluation Association
http://www.eval.org
An international
professional association of evaluators devoted to the application
and exploration of program
evaluation, personnel evaluation, technology, and many other forms of evaluation.
They serve as an important resource for tools, guides, and other materials
related to evaluation.
Diversity Web
http://www.diversityweb.org/research_and_trends/research_evaluation_impact/index.cfm
This
is a website created by professionals devoted to promoting diversity
in higher education.
It has descriptions and evaluations of various
diversity
efforts
at multiple colleges and universities, including leadership development
efforts.
W. K. Kellogg Foundation
http://www.wkkf.org
The foundation site offers a variety of information
sets, in particular toolkits and publications related to evaluation.
Onsite: “Evaluating
Outcomes and Impacts: A scan of 55 Leadership Development Programs” and
another entitled EVALULEAD: A Guide for Shaping and Evaluating Leadership
Development Programs.
The University Council for Educational Administration
http://www.ucea.org
The
council is a consortium of major research universities with doctoral
programs in leadership
and policy. Their own consideration of leadership development
and preparation and extant resources are a valuable reference for designing
and implementing
leadership development evaluations.
The
Wallace Foundation
http://www.wallacefoundation.orgThis foundation
has a document entitled, Beyond
the Pipeline: Getting the Principals We Need, Where They
Are Needed. This
policy document addresses the need to design appropriate
pipelines and policies to attract principals at school districts around
the country.
• • • return to top
Chapter
7
From The Inside Out: Evaluating Personal Transformation Leadership Efforts
Sally Liederman
Three ìinside-outî leadership programs are discussed in order to better understand how to best evaluate personal transformation in leadership development efforts. adult learning theory, theories of liberation theology and pedagogy, and theories of the role of the individual in social change are discussed in the context of these three organizations, as well as what makes these organizations so particularly effective.
Chapter 7 Resources
American Indians for Opportunity Ambassadors
http://www.aio.org/ambassador.html
Center for Assessment and Policy Development
http://www.capd.org
The full research brief for Healing the Heart of Diversity, as well
as Flipping
the Script: White Privilege and Community Building, may be found at
the Center for Assessment and Policy Development’s website.
• • • return to top
Chapter
8
Evaluating Organizational Change/transformation Efforts
Nancy Vollmer LeMay and Alison Ellis
How does leadership development contribute to measurable changes in organizational performance? This question steers an investigation into organizational performance evaluation. Items of discussion: the five guiding principles of a good organization, Leadership practices and competencies, workgroup climate, and evaluation implements.
Chapter
8 Resources
Management Sciences for Health
“Creating a Work Climate That Motivates Staff and Improves
Performance” in The Manager produced by Management Sciences for Health.
http://erc.msh.org/TheManager/English/V11_N3/V11_N3_En_Issue.pdf
M & L Leadership
Development Programs
http://www.msh.org
For
information on the design and content of M&L leadership development
programs, we recommend the publication by MSH: Managers Who Lead: A
Handbook for Improving Health Services. The handbook is available
for order through MSH’s e-bookstore located on the MSH website.
Another new manual recently developed by MSH provides guidance on applying
the WCA as well as tabulating and interpreting the results: Work Climate
Assessment: Guide for Facilitators. Copies are available for download from
the MSH Electronic Resource Center website (http://erc.msh.org/).
For
information on measuring leadership competencies, see The Leadership
Assessment Instrument
(LAI), a self assessment tool developed by Linkages,
Inc. with information at http://erc.msh.org/mainpage.cfm?file=96.50.htm&module=toolkit&language=English.
Finally, the full evaluation reports for the leadership programs in
Egypt, Nicaragua, and the VLDP are available from MSH on request. Brief
evaluation notes summarizing the results of these evaluations are available
at http://www.msh.org/projects/mandl/3.4.1.html.
International Society
for Performance Improvement
http://www.ispi.org
For information on the performance improvement
process that provides the foundation for our leadership programs.
We also recommend
the seminal works by Litwin and Stringer, who pioneered the study of climate
in corporate settings: Motivation and Organizational
Climate (Litwin and Stringer, 1968), and Leadership and Organizational
Climate (Stringer, 2002).
• • • return to top
Chapter
9
The Importance of Local Context in Leadership Development and Evaluation
Larry Peters and John Baum
This chapter discusses the difficulty in finding tangible and quantifiable
data supporting leadership development efforts, and how local context (representing
things like opportunity, management systems, HR systems, culture, and work
constraints) matters in any effort to fully evaluate a leadership effort.
Context issues in leadership development are discussed from multiple angles
around the argument that local context is a necessary factor to include in
any evaluation.
Chapter
9 Resources
No Resources Given
• • • return to top
Chapter
10
Evaluating Community Leadership Programs
Teresa Behrens and Maenette Benham
This chapter focuses on the evaluation of the first session
of the Kellogg Leadership for Community Change (KLCC) series,
including developed theories of community change, a detailed
analysis of KLCC evaluation design, and the overall design
of the KLCC program. Also explored are concepts and implications
of community leadership, as well as challenges for the KLCC
program as a whole.
Chapter
10 Resources
Atkinson,
P. Understanding Ethnographic Texts. Qualitative Research Methods Series
25. Thousand Oaks, CA: Sage Publications, 1992. This short, manageable monograph
provides practitioners and evaluators with a set of thinking points to guide
them in the complicated task of interpreting diverse contexts.
Czarniawska, B. A Narrative Approach to Organization Studies. Qualitative
Research Methods Series 43. Thousand Oaks, CA: Sage
Publications, 1998. This
is a must read for anyone conducting an organizational ethnography.
Durland, M.M. and Fredericks, K.A. (eds). Social Network Analysis in
Program Evaluation. Special Issue of New Directions for Program Evaluation (107).
Fall 2005.
Simmons, A. The
Story Factor: Inspiration, Influence, and Persuasion Through the Art of Storytelling. New York: Basic Books, 2001. Storytelling is an
ancient art that carries the soul and spirit of a community of people. For
the practitioner or evaluator to employ this art to inform, to persuade,
to entertain, and so on requires skill. The goal of this art is to be inspiring
and illuminating; the book provides fundamental thoughts that all authors
of organizational stories should know.
Stewart, A. The Ethnographer's Method. Qualitative Research Methods Series
46. Thousand Oaks, CA: Sage Publications, 1998. A simply written monograph
that provides the practitioner or evaluator with fundamental structures to
understand and implement ethnographic studies.
• • • return to top
Chapter
11
Evaluating Leadership as a Strategy to Transform Complex Systems
K. Jinnett and T. Kern
Using two separate but overlapping cases that focus on educational
leadership within urban K-12 school systems, this chapter
describes the challenges of evaluating leadership development
efforts within the broader context of complex systems. This
chapter illuminates how to design an evaluation around a
complex system, how to focus evaluation efforts for optimal
accuracy and honesty, and how to choose a research design.
Also discussed: key lessons and implications of evaluating
complex systems on overall leadership evaluation theory.
Chapter
11 Resources
University
of Toronto Leadership Site
http://education.umn.edu/CAREI/Leadership/default.html
Dr. Michelle LaPointe, Research Director
Stanford University
http://seli.stanford.edu/research/sls.htm
Leadership Assessment System
http://www.vanderbilt.edu/lsi/las/index.php
The Wallace Foundation’s Knowledge Center on Education Leadership
http://www.wallacefoundation.org/WF/KnowledgeCenter/KnowledgeTopics/EducationLeadership/
The site includes
reports, stories, and other resources about strengthening the performance
of education leaders to improve student achievement.
New Leaders for New Schools
http://www.nlns.org/NLWeb/Results.jsp
Preliminary results from the NLNS initiative may be found at this site.
Methods-related Resources
System Dynamics Society
http://www.systemdynamics.org
This group promotes the use
of system dynamics by researchers and practitioners. They publish newsletters
and the
journal System
Dynamics Review as well as holding annual conferences.
Donella Meadows, “Places to Intervene in a System.”
http://www.rmi.org/sitepages/pid790.php
International
Network for Social Network Analysis
http://www.insna.org/
The International Network for Social Network Analysis
is the main association for professionals (geared toward researchers) interested
in social network analysis. The organization sponsors an annual conference
and publishes Connections, a scholarly journal. More information can be
found at their website. The group also maintains
a useful listserv,
socnet, for those interested in exchanging ideas about the use of social
network analysis. Access at http://www.insna.org/INSNA/socnet.html.
Multilevel Modeling
There are a variety of resources on multilevel modeling.
The organizations that have developed software packages tend have useful
links to other resources. One of the most useful links pages is maintained
by MLWIN
software developers: http://www.mlwin.com/links/index.html. A useful listserve
for multilevel modeling is multilevel-mailbase@mailbase.ac.uk., which can
be accessed at http://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=multilevel
• • • return to top
Chapter
12
Evaluating Leadership Development for Social Change
Kim Ammann Howard and Claire Reinelt
This chapter discusses how to assess the impact of leadership strategies that are designed to seed and/or catalyze social change around issues, practices, and attitudes regarding family planning and reproductive health (FP/RH) in developing countries. While the chapter draws on different evaluations, it focuses on one evaluation in particular: the evaluation of six population leadership programs funded by the David and Lucile Packard and the Bill and Melinda Gates Foundations.
Chapter
12 Resources
Malcolm Gladwell, The Tipping Point
http://www.gladwell.com/tippingpoint/index.html
Gladwell
describes how ideas and innovations spread through the presence of certain
types of people (connectors, mavens, and salesmen), the contagiousness of
the message and the context or environment which influences how quickly the
innovations or ideas spread.
David Bornstein, How
to Change the World: Social Entrepreneurs and the Power of New Ideas
http://www.howtochangetheworld.org/
Bornstein
describes how Ashoka selects social entrepreneurs and supports them to scale
up their ideas and achieve breakthrough
results.
• • • return to top
Chapter
13
Evaluating Youth Leadership Development through Civic Activism
Hanh Cao Yu, Heather Lewis-Charp and Michele Gambone
In this chapter, traditional and popular conceptions of youth leadership are
used as the foundation for evaluating youth leadership development. Also discussed:
how traditional concepts of youth leadership can be expanded to encompass youth
leadership development within a community and civic context, presenting a case
study of an evaluation of a national youth leadership initiative.
Chapter
13 Resources
Evaluating Civic Activism: A Curriculum for Community
and Youth-Serving Organizations
http://www.evaluationtools.org/tools_main.asp
This curriculum was created
by Soukamneuth, S., Lewis-Charp, H., Lacoe, J., Heiman, L., and Kebede,
R.for the Innovation
Center for Youth Development and is a comprehensive self-evaluation curriculum
for youth and community
serving agencies.
Commissioning Multicultural Evaluation: A Foundation Resource Guide
http://calendow.org/evaluation/reports.stm
This
guide was created for The California Endowment by T. Endo
Inouye, H. Cao Yu and J. Adefuin and provides guidelines
for foundations and other organizations
to effectively conduct a multicultural health evaluation. The publication
also addresses the issues of monitoring and assessing the evaluation.
• • • return to top
Chapter
14
Evaluating Leadership Efforts for Neighborhood Transformation
Nilofer Ahsan
Using two multi-year documentation processes of resident leadership development, this chapter attempts to describe the documentation process and the ways it is both distinct from and contributes to the evaluation process, illustrate the complexities of evaluating resident leadership development, and provide practical guidance and thoughts on developing efforts to evaluate resident leadership development within neighborhood transformation efforts.
Chapter
14 Resources
Abramson, P. R. A Case for Case Studies. Thousand Oaks, CA:
Sage Publications, 1992.
Gillham, B. Case study research methods (real world research). London: Continuum,
2000.
Hamel, J. Case Study Methods. Qualitative Research Methods, Vol. 32. Thousand
Oaks, CA:
Sage Publications, 1993.
Making Connections Initiative
http://www.aecf.org/initiatives/mc/llp/chapter8_1.htm
A brief chapter on the documentation process and some guidelines for documentation
used within the Making Connections initiative can be found at their website.
Social Network Mapping Tools
InFlow 3.1-Social Network Mapping Software creates interactive visual models
of social networks.
http://www.orgnet.com/inflow3.html
UCINet
http://www.analytictech.com/ucinet.htm
UCINet is a general package for analysis
of social network data.
Pajek
Pajek is a free Windows based program for mapping social networks.
http://vlado.fmf.uni-lj.si/pub/networks/pajek/
An Overview of Computer Programs for Social Network Analysis
http://www.insna.org/INSNA/soft_inf.html
• • • return to top
Project Sponsors
