Part One: Designing Leadership Development Evaluations

Chapter 1 Experimental and Quasi-experimental Evaluations
S. Bartholomew Craig and Kelly Hannum

Chapter 2 Leading with Theory: Using a Theory of Change Approach for Leadership Development Evaluations
Manuel Gutiérrez and Tania Tasse

Chapter 3 EvaluLEAD: An Open Systems Perspective on Evaluating Leadership Development
John Grove, Barry Kibel and Taylor Haas

Chapter 4 Make Evaluation Work for the Greater Good: Fueling Generative Visions of Provocative Possibility and Responsive Praxis in Leadership Development
Hazel Symonette

Chapter 5 Measuring ROI in Leadership Development
Jack J. Phillips, Ph.D. and Patricia P. Phillips, Ph.D.

 

Part Two Part Three Book Home Order

Chapter 1

Experimental and Quasi-experimental Evaluations
S. Bartholomew Craig and Kelly Hannum

Experimental and quasi-experimental approaches to evaluation are the focus of this chapter, and they provide a structured means to think about designing evaluations. The chapter addresses a noticeable lack of effort to apply experimental procedures to leadership evaluation, and highlights some of the difficulties and benefits of doing so. The chapter presents experimental and quasi-experimental procedures as an effective way of measuring changes in leadership and leadership outcomes, and determining the relationship between the leadership development initiative in question and the changes measured.

Chapter 1 Resources

Buros Institute of Mental Measurements
http://www.unl.edu/buros/bimm
Existing measures can be identified and evaluated using information provided by the Buros Institute of Mental Measurements, which has a searchable web-based Test Directory.

Helpful online calculators for estimating sample size requirements:
http://www.surveysystem.com/sscalc.htm
http://www.stat.uiowa.edu/~rlenth/Power
http://www.isixsigma.com/library/content/c000709.asp

• • • return to top

 

Chapter 2

Leading with Theory: Using a Theory of Change Approach for Leadership Development Evaluations

Manuel Gutiérrez and Tania Tasse

When faced with the task of designing evaluations of leadership development programs, evaluators are likely to wonder what types of evidence they should look for in order to measure program success. This chapter outlines the implantation and utilization of pathway mapping as a first step in measuring leadership development. The article describes multiple uses of pathway mapping and describes in detail its effect on evaluation.

Chapter 2 Resources

Aspen Institute Roundtable on Community Change
http://www.aspeninstitute.org/site/c.huLWJeMRKpH/b.612045/k.4BA8/Roundtable_on_Community_Change.htm
This resource describes the work of the Roundtable and includes links to resources concerning theory of change and community-building topics.

Eisenhower Fellowships
http://eisenhowerfellowships.org
This source describes the mission and goals of the Eisenhower Fellowships as well as program activities.


Leadership for a Changing World (LCW)
http://leadershipforchange.org
This resource describes the LCW program and includes announcements about new and current award winners and their social justice work and accomplishments.

Pathways Mapping Initiative
http://www.pathwaystooutcomes.org
This resource provides information about strategies that have been effective, appear to be working, or are promising, in community initiatives taking place in particular locations. It also discusses how pathways are designed to guide choices about investments, programs, and policies made by multiple stakeholders.

Next Generation Leadership
http://www.nglnet.org
This resource includes information about the background of the Next Generation Leadership program, and the program’s alumni network.

W. K. Kellogg Foundation Logic Model Development Guide
http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf

• • • return to top



Chapter 3

EvaluLEAD: An Open Systems Perspective on Evaluating Leadership Development
John Grove, Barry Kibel and Taylor Haas

One of the major challenges of leadership evaluation lies in assessing the value added by a leadership development program to those directly served. This chapter introduces EvaluLEAD as a method to evaluate links between program actions and organizational or systems-level results. Subjects included: history/evolution of evaluLEAD, implementation of evaluLEAD, possible problems with the method.

Chapter 3 Resources

EvaluLEAD
Grove, J., Kibel, B., and Haas, T. EvaluLEAD: A Guide for Shaping and Evaluating Leadership Development Programs. W.K. Kellogg Foundation and Public Health Institute, 2005. http://www.wkkf.org/DesktopModules/WKF_DmaItem/ViewDoc.aspx?LanguageID=0&CID=281&ListID=28&ItemID=2813740&fld=PDFFile This guide provides detailed information about and guidance for the EvaluLEAD process.

EvaluLEAD at http://www.evalulead.net provides access to the guide as well as contact information for the authors and other practitioners who can work with organizations to implement EvaluLEAD and related open systems evaluation solutions.

Jacobs, J. The Nature of Economies. New York: Random House. (2000).

Senge, P., Jaworski, J., Scharmer C. O., & Flowers B. S. Presence: Human purpose and the field of the future. Cambridge, MA: Society for Organizational Learning. (2004).

Wheatley M. J., & Kellner-Rogers, M. A simpler way. San Francisco: Berrett-Koehler. (1996).

• • • return to top



Chapter 4

Make Evaluation Work for the Greater Good: Fueling Generative Visions of Provocative Possibility and Responsive Praxis in Leadership Development
Hazel Symonette

This chapter focuses on evaluation as a development resource for excellence through processes that intentionally blur the lines between evaluator, program developer, and participant. The author explores the potential (and positive) ramifications of a successful leadership program and also the implementation of a more “natural” form of evaluation.

Chapter 4 Resources

Evaluation Training Institute, Howard University
http://www.howard.edu/schooleducation/eti
This program offers professional development to expand the number and capacity of mathematics and science project evaluators who can plan and implement evaluations that are technically sound, contextually relevant, and have increased utility.

Reina, D., & Reina, M. Building trust in the workplace. Access at http://www.trustinworkplace.com

Other Resources
Johnson, A. Privilege, power and difference. Mountain View, CA: Mayfield, 2001.

Style, E. Curriculum as window and mirror. Social Science Record. Fall 1996, 35-42.

Thomas, V. "Building a contextually responsive evaluation framework." V.G. Thomas & F.I. Stevens (eds). Co-constructing a contextually responsive evaluation framework: The Talent Development Model of school reform. New Directions in Evaluation 101. San Francisco, CA: Jossey Bass, 2004.

• • • return to top



Chapter 5

Measuring ROI in Leadership Development
Jack J. Phillips, Ph.D. and Patricia P. Phillips, Ph.D.

Increased emphasis on leadership development programs in the business, non-profit, and governmental sectors has lead to many executives questioning the value of individual development programs on their organizations. This chapter offers methods to better quantify the return on investment (ROI) of leadership programs. A framework for measuring the ROI of development programs is explored, along with a discussion of the many variables that can impede accurate evaluation of a program.

Chapter 5 Resources

Return on Investment
Each of the following organizations has members interested in ROI.

The American Society for Training and Development
http://www.astd.org
The ROI Institute
http://www.roiinstitute.net
The Society for Human Resource Managers
http://www.shrm.org

The ROI Network
http://www.astd.org
This network has been organized to share information. Case studies are available.

Other Resources

Phillips, J. Investing in Your Company’s Human Capital: Strategies to Avoid Spending Too Little or Too Much. New York, NY: AMACOM, 2005.

Phillips, J. Proving the Value of HR: How and Why to Measure ROI. Alexandria, VA: SHRM, 2005.

Phillips J. Return on Investment in Training and Performance Improvement Programs. 2nd Edition. Woburn, MA: Butterworth-Heinemann, 2003.
The second edition of this bestselling book guides you through a proven, results-based approach to calculating the Return on Investment in training and performance improvement programs.

Phillips, J. and Schmidt, L. The Leadership Scorecard. Woburn, MA: Butterworth-Heinemann, 2004.
This book expands and discusses best practice leadership development methods, incorporates ROI measurement and evaluation methodology, sets out a step-by-step process, presents case studies, and provides proven measurement and evaluation techniques.

Phillips, J., Phillips, P., and Hodges, T. Make Training Evaluation Work. Alexandria, VA: ASTD Press, 2004. This book provides the learning professional—newcomer or veteran—practical and specific ways to show value and communicate results, select the right model and find resources, and get management buy-in and overcome resistance.

Phillips, J., Stone, R., and Phillips, P. The Human Resources Scorecard: Measuring the Return on Investment. Woburn, MA: Butterworth-Heinemann, 2001. This book provides a comprehensive, step-by-step guide for measuring the impact of human resources programs and includes seven detailed case studies. This book is essential for human resource executives, professionals, CEOs, CFOs, consultants, professors and other managers concerned with their business’s bottom lines.

Phillips, P. The Bottomline on ROI. Atlanta, GA: Center for Effective Performance, 2002.
This book offers the business case for ROI. It provides the basics, benefits, and barriers to measuring training and performance improvement programs.

Phillips, J. and Phillips, P. ROI at Work: Best-Practice Case Studies from the Real World. Alexandria, VA: ASTD Press, 2005.
This book presents best-practices case studies from the real world of measuring return on investment, including a five case studies about leadership development programs. Case studies come from the government sector as well as a range of industries and provide valuable lessons for professionals working to contribute to the strategic goals of their organizations.

Phillips, P., Ed. Phillips, J., Series Ed. In Action: Measuring Return on Investment, Volume 3. Alexandria, VA: ASTD Press, 2001.
This book has 11 cases from a variety of industries including telecommunications, computer and technology, retail stores, automotive, and the government sector.

Phillips, P. and Phillips, J. ROI Basics. Alexandria, VA: ASTD Press, 2005.
This book provides the fundamental steps in developing a comprehensive evaluation, offering the reader basic skills in ROI, tools for selecting appropriate programs for ROI evaluation, and the ability to develop a strategy to integrate ROI as part of the ongoing learning process.

• • • return to top

 

Project Sponsors