and Quasi-experimental Evaluations
S. Bartholomew Craig and Kelly Hannum
Experimental and quasi-experimental approaches to evaluation are the
focus of this chapter, and they provide a structured means to think about
evaluations. The chapter addresses a noticeable lack of effort to apply
experimental procedures to leadership evaluation, and highlights some
of the difficulties
and benefits of doing so. The chapter presents experimental and quasi-experimental
procedures as an effective way of measuring changes in leadership and leadership
outcomes, and determining the relationship between the leadership development
initiative in question and the changes measured.
Chapter 1 Resources
of Mental Measurements
can be identified and evaluated using information provided by the Buros
Institute of Mental Measurements, which has a searchable web-based
Helpful online calculators for estimating sample size requirements:
• • • return
Leading with Theory: Using a Theory of Change Approach for Leadership
Manuel Gutiérrez and Tania Tasse
When faced with the task of designing evaluations of leadership development
programs, evaluators are likely to wonder what types of evidence they
should look for in order to measure program success. This chapter outlines
implantation and utilization of pathway mapping as a first step in
development. The article describes multiple uses of pathway mapping
and describes in detail its effect on evaluation.
Chapter 2 Resources
Roundtable on Community Change
This resource describes the work of the Roundtable and includes
links to resources concerning theory of change and community-building topics.
source describes the mission and goals of the Eisenhower Fellowships
as program activities.
Leadership for a Changing World (LCW)
This resource describes the LCW program and includes announcements
about new and
current award winners and their social justice work and accomplishments.
Pathways Mapping Initiative
This resource provides information about
strategies that have been effective,
appear to be
working, or are promising, in community initiatives taking
place in particular locations. It also discusses how pathways are
about investments, programs, and policies made by multiple
Next Generation Leadership
includes information about the background of the Next Generation
program, and the program’s
W. K. Kellogg Foundation Logic Model Development Guide
• • • return
EvaluLEAD: An Open Systems Perspective on Evaluating Leadership
John Grove, Barry Kibel and Taylor Haas
One of the major challenges of leadership evaluation lies
in assessing the value added by a leadership development
to those directly
chapter introduces EvaluLEAD as a method to evaluate links
between program actions and organizational or systems-level
history/evolution of evaluLEAD, implementation of evaluLEAD,
possible problems with the method.
Grove, J., Kibel, B., and Haas, T. EvaluLEAD:
A Guide for Shaping and Evaluating Leadership Development Programs.
Foundation and Public Health
Institute, 2005. http://www.wkkf.org/DesktopModules/WKF_DmaItem/ViewDoc.aspx?LanguageID=0&CID=281&ListID=28&ItemID=2813740&fld=PDFFile This
guide provides detailed information about and guidance
for the EvaluLEAD process.
EvaluLEAD at http://www.evalulead.net provides access to the guide
as well as contact information for the authors and other
to implement EvaluLEAD and related open systems evaluation
Jacobs, J. The Nature of Economies. New York: Random
Senge, P., Jaworski, J., Scharmer C. O., & Flowers B.
S. Presence: Human purpose and the field of the future.
Cambridge, MA: Society for
Organizational Learning. (2004).
Wheatley M. J., & Kellner-Rogers, M. A simpler way. San
Francisco: Berrett-Koehler. (1996).
• • • return
Make Evaluation Work for the Greater Good: Fueling Generative
Visions of Provocative Possibility and Responsive Praxis
in Leadership Development
This chapter focuses on evaluation as a development resource
for excellence through processes that intentionally
blur the lines
program developer, and participant. The author explores
the potential (and positive)
ramifications of a successful leadership program and
also the implementation of a more “natural” form
Evaluation Training Institute, Howard University
program offers professional development to expand the number
capacity of mathematics
project evaluators who can plan and implement evaluations
that are technically sound,
contextually relevant, and have increased utility.
Reina, D., & Reina, M. Building trust in the workplace.
Access at http://www.trustinworkplace.com
Johnson, A. Privilege, power and difference. Mountain View,
CA: Mayfield, 2001.
Style, E. Curriculum as window and mirror. Social Science
Record. Fall 1996, 35-42.
Thomas, V. "Building a contextually responsive evaluation
framework." V.G. Thomas & F.I. Stevens (eds).
Co-constructing a contextually responsive evaluation
framework: The Talent
school reform. New
Directions in Evaluation 101. San Francisco, CA: Jossey
• • • return
Measuring ROI in Leadership Development
Jack J. Phillips, Ph.D. and Patricia P. Phillips, Ph.D.
Increased emphasis on leadership development programs
in the business, non-profit, and governmental sectors
lead to many
questioning the value
of individual development programs on their organizations.
This chapter offers methods to better quantify the return
A framework for measuring the ROI of development programs
is explored, along with a discussion of the many variables
can impede accurate
of a program.
Return on Investment
Each of the following organizations has members interested
Society for Training and Development
The ROI Institute
Society for Human Resource
The ROI Network
organized to share information. Case studies are available.
Phillips, J. Investing in Your Company’s Human
Capital: Strategies to Avoid Spending Too Little or Too
York, NY: AMACOM,
Phillips, J. Proving the Value of HR: How and Why to
Measure ROI. Alexandria, VA: SHRM, 2005.
Phillips J. Return on Investment in Training and Performance
Improvement Programs. 2nd Edition. Woburn, MA: Butterworth-Heinemann,
The second edition of
this bestselling book guides you through a proven, results-based
approach to calculating the Return on Investment in training
Phillips, J. and Schmidt, L. The Leadership Scorecard.
Woburn, MA: Butterworth-Heinemann, 2004.
This book expands
methods, incorporates ROI measurement and evaluation
methodology, sets out a step-by-step process, presents
and provides proven
and evaluation techniques.
Phillips, J., Phillips, P., and Hodges, T. Make Training
Evaluation Work. Alexandria, VA: ASTD Press, 2004. This
the learning professional—newcomer
or veteran—practical and specific ways to show
value and communicate results, select the right model
resources, and get management
buy-in and overcome resistance.
Phillips, J., Stone, R., and Phillips, P. The Human Resources
Scorecard: Measuring the Return on Investment. Woburn,
MA: Butterworth-Heinemann, 2001. This book
provides a comprehensive, step-by-step guide for measuring
the impact of human resources programs and includes seven
detailed case studies.
book is essential
for human resource executives, professionals, CEOs, CFOs,
consultants, professors and other managers concerned
with their business’s
Phillips, P. The Bottomline on ROI. Atlanta, GA: Center
Effective Performance, 2002.
This book offers the business case for ROI. It provides
the basics, benefits, and barriers to measuring training
and performance improvement programs.
Phillips, J. and Phillips, P. ROI at Work: Best-Practice
Case Studies from the Real World. Alexandria, VA: ASTD
case studies from the real world of measuring return
on investment, including a five case studies about leadership
programs. Case studies
come from the government sector as well as a range of
and provide valuable
lessons for professionals working to contribute to the
strategic goals of their organizations.
Phillips, P., Ed. Phillips, J., Series Ed. In Action:
Measuring Return on Investment, Volume 3. Alexandria,
VA: ASTD Press,
has 11 cases from a variety
of industries including telecommunications, computer
and technology, retail stores, automotive, and the government
Phillips, P. and Phillips, J. ROI Basics. Alexandria,
VA: ASTD Press, 2005.
This book provides the fundamental
evaluation, offering the reader basic skills in ROI,
tools for selecting appropriate programs
for ROI evaluation, and the ability to develop a strategy
to integrate ROI as part of the ongoing learning process.
• • • return