Week 4:
Logic Models and Program Theory

Description

This lesson focuses on describing the program by developing a logic model and explaining a program by providing a theory of why the intervention should work. A logic model is a visual diagram used by evaluators to represent, in a systematic way, the relationships between the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve. A program theory explains the reasoning behind why their particular intervention should work.

Learning Objectives

By the end of this week, students should be able to:

• develop research questions and objectives,
• understand and develop logic models, and,
• articulate program theory.

Describing the Program

All too often, programs are adapted from existing programs in other organizations or communities, or they are developed based on scientific literature, and many program developers have not reflected critically about the resources needed to implement a program (and the associated inputs and activities). Therefore, programs are sometimes launched without an understanding of how activities are linked or supported by science or evidence and how they will lead to the expected outcome. As a result, it can be a difficult task to understand the ideal or intended program and to identify linkages between program elements.

The Logic Model

The logic model is a conceptual framework that can be used by evaluators. It is a graphical representation that describes the program's rationale and the plan for an initiative, a program, policy, or service. The logic model also provides a basis for planning, program management, and internal/external communications. It can change the way evaluators and organizations think of their work. It is important to note that a logic model is only useful as a management tool if it is collectively owned by key stakeholder groups.¹

The first step of developing a logic model involves understanding the program’s situation, context, and priorities. These may be found as part of vision and mission documents, strategic planning session documents, high level discussions, or environmental scans. Typically, a logic model follows the steps outlined in Figure 1 below.²

Logic models can be as broad or specific as needed. However, all logic models should address the basic "how", "who", "what" or "why" questions. The "where" can complement the "who" (i.e., who is the intended audience?) as part of the consideration of reach. The "when" represents the timeline of when we expect the results (outcomes) to occur. All of the four connotations (how, who, what, and why) should be answered by viewing the logic model.¹

The “How” 
The logic model, in its simplest form, is a chart describing how an initiative operates. It describes inputs, activities, and outputs, as well as who the initiative reaches with its outputs.

The “Who”
The reach of an initiative includes not only users and clients, but also co-delivery groups, intermediaries, and other stakeholders.³

The “What”
The evaluator needs to understand the “what” for each of the groups reached by the initiative. For example, are we seeking a level of engagement, a reaction, or a behaviour change that results from a change in knowledge, abilities, skills, and/or aspirations?

The “Why”
The sum of behaviour changes is intended to lead to a final outcome, that is, the "why", or the reason that the program, initiative, policy, or service has been implemented.

Output vs. Outcomes

Outputs and outcomes are commonly confused in logic models.  As is highlighted in the visual below, the key difference between output and outcome is control and influence¹.

 

For example, the delivery of a seminar by a given group (such as a group in this health evaluation class) may be considered an activity leading to an output of the seminar. The fact that the audience may learn something new at the seminar can be considered an outcome. More precisely, the sequence of expected outcomes for this seminar would include:

• that people chose to attend the seminar in the first place;
• the audience participated in the assigned work and discussions;
• the audience believed participation in the seminar would be of value;
• the audience learned new information, or gained knowledge or abilities, before, during, and after the seminar;
• the audience applied the new knowledge gained outside of the seminar; and,
• application of this new knowledge resulted in decision making or action.

 

The ultimate or end outcome would be that individuals benefited from applying this knowledge elsewhere (e.g., at work). Ultimately, the last outcome (or end outcome) requires a leap of faith (since there’s no guarantee that individuals will apply this knowledge outside of the seminar). The longer-term outcomes therefore, are only marginally, attributable to any given initiative.

Test Your Knowledge: Mini-quiz 

Identify which of the following items are an output or an outcome. (Tip: ask yourself is it within the control of the initiative? Yes= output, No=outcome).

 

Logic Model Practice Recommendations

1. Start thinking with the end needs in mind. If you can define the desired results in terms of needs, you can help ensure the logic model and programs are grounded in reality.
2. Distinguish between control and influence (i.e., outputs versus outcomes).
3. Emphasize the steps of behaviour change.
4. Consider a systems thinking approach when developing logic models. In other words, think about reach of the program, the stakeholder relationships, and what the results will be. Also keep in mind that most real-world processes do not follow a linear path.

Program Theory

In addition to the graphical representations provided by logic models, programs also benefit from developing program theory, or narratives to describe how the components of the logic model are connected and should work.  “If-then” statements are very useful in developing these narratives or program theories⁴.

These narratives are developed to provide a detailed description of a causal model of why the intervention is expected to work. Often, program theories are referred to as “Results Chains” or “Theory of Change”. For each program theory, the process is essentially the same: think through and describe in full how and why a project should work. In addition to developing an explanation of the causal processes that you think will lead to desired outcomes, program theories can be used to:

• Come up with indicators/metrics of intended outcomes;
• Identify sources of evidence to support causal links;
• Connect information about processes to information about results;
• Strengthen causal inference; and,
• Support generalizability by identifying what works best in particular contexts.

Creating a Theory of Change

In addition to generating “if-then” narratives, program developers must also identify evidence that supports the need for their program, and explain the reasoning behind why their particular intervention should work. Identifying this evidence will help program developers articulate the program’s rationale and the set of beliefs, assumptions, and expectations that underlie how the program works, and how the target group is supposed to achieve desired benefits⁵. Often, programs will present this evidence as “assumptions” or “risks” and may even include them in their program logic model. These assumptions or risks can provide important context for the expected results of the program, and can also form the basis of important risk management discussions. Additionally, programs often look for “mediating processes” or “mediating theories”, which are often borrowed from sociological or psychological theories. Mediating processes or theories explain the mechanisms or processes that underlie a relationship between two variables. Mediating theories justify the rationale for the intervention, and support explanations for how and why the expected change occurs.

Here is a simplified example of a theory of change table developed for Gay ZONE, a health program in Ottawa that was established in 2007 to address concerns about the continuing high rates of HIV infection among gay/bi/trans men and men who have sex with men (MSM). This program provides culturally appropriate screening and treatment of STIs, anonymous testing for HIV, and educational programming (INPUTS) to reduce the transmission of STIs among gay men (OUTCOMES).

Similar to logic models, program theories or theories of change should always be done in consultation with the stakeholders. These theories help you to communicate activities and plan evaluations. It is your role as an evaluator to help find the evidence for a program’s rationale and the different mediating theories that influence the program. However, it is the stakeholders who decide whether the evidence and theory narratives make sense for the program.   As you can see from the above example, assumptions and risks address factors that influence the program. It should be noted that their articulation does not have to be presented in pairs, as depicted in the above example. How factors are presented is based on the evaluator and stakeholders’ preferences. 

So, what is the difference between a theory of change and a logic model?

If you look carefully at the examples provided in this lecture, you will see that the theory of change provides the bigger picture of why the intervention should work, while a logic model is a graphical representation, or description, of the change processes at the program and implementation level.  This difference is highlighted in the Logic models are descriptive, while a theory of change is explanatory. Theories of change are useful to demonstrate a clear plan for complex initiatives, while logic models summarise a complex theory into basic categories of inputs, outputs, and outcomes.  

Required Readings

1. U.S. Department of Health and Human Services. Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to Program Evaluation for Public Health Programs: A self-study guide. Atlanta, GA: Centers for Disease Control and Prevention, 2005. (Step 2 Only). (Course Reserves).
2. Public Health Agency of Canada: Porteous, N., Sheldrick, B. and Stewart, P. (1997).  Program Evaluation Tool Kit. (Step 1 Only). (Course Reserves).
3. Ontario Agency for Health Protection and Promotion (Public Health Ontario), Abdi S, Mensah G. Focus On: Logic models-a planning and evaluation tool. Toronto, ON: Queen’s Printer for Ontario; 2016. (Course Reserves).

Optional Readings

1. Goldman, K.D., & Schmaltz, J.K. Logic Models: The Picture Worth Ten Thousand Words. Health Promotion Practice, 7(1), 8-12. Society for Public Health Education. (Course Reserves).
2. W. K. Kellogg Foundation. (2004. Jan). Logic Model Development Guide. Michigan. (Course Reserves).

Activities and Assignments

Submit the Group Project: Team Contract

For due dates, please see the Course Schedule.

References

1. Goldman, K. D., & Schmaltz, J. K. (2006, January 1). Logic Models: The Picture Worth Ten Thousand Words. Health Promotion Practice, 7(1), 8-12. Society for Public Health Education. (Course Reserves). [This is an excellent reference for understanding logic models.]

2. W. K. Kellogg Foundation. (2004, January). Logic Model Development Guide. Battle Creek, MI: W. K. Kellogg Foundation.

3. Taylor-Powell, E., Jones, L., Henert, E. (2003). Enhancing Program Performance with Logic Models. Madison, WI: University of Wisconsin – Extension. [Start with Section 1 and Section 7, then go back to Sections 2 through 6].

4. Wilder Research. (2009, August). Program theory and logic models. St. Paul, MI: Wilder Foundation. 

5. Rogers, P. (2014). Theory of Change, Methodological Briefs: Impact Evaluation 2. UNICEF Office of Research-Innocenti.

6. King, R. (1999). Sexual behavioural change for HIV: Where have theories taken us? Geneva, Switzerland: UNAIDS.