Stakeholders are people or organizations invested in the program, interested in the results of the evaluation, and/or with a stake in what will be done with the results of the evaluation. Representing their needs and interests throughout the process is fundamental to good program evaluation.
Typical Stakeholders in Public HealthKey stakeholders for evaluations of public health programs fall into three major groups:
Clearly, these categories are not mutually exclusive; in particular, the primary users of evaluation findings are often members of the other two groups, i.e., the program management or an advocacy organization or coalition. While you may think you know your stakeholders well, the following categories help you to think broadly and inclusively in identifying stakeholders:
Why Stakeholders Are Important to an Evaluation Potential Stakeholders in Public Health ProgramsStakeholders can help (or hinder) an evaluation before it is conducted, while it is being conducted, and after the results are collected and ready for use. Because so many public health efforts are complex and because public health agencies may be several layers removed from frontline implementation, stakeholders take on particular importance in ensuring that the right evaluation questions are identified and that evaluation results will be used to make a difference. Stakeholders are much more likely to support the evaluation and act on the results and recommendations if they are involved in the evaluation process. Conversely, without stakeholder support, your evaluation may be ignored, criticized, resisted, or even sabotaged.
Use the evaluation standards to help identify those stakeholders who matter most. Give priority to those stakeholders who
In addition, to be proper/ethical and accurate, you need to include those who participate in the program and are affected by the program or its evaluation.
The worksheets at the end of this chapter are intended to help you identify key stakeholders. For example, in using the worksheets with the Childhood Lead Poisoning Prevention (CLPP) program, we identified the stakeholders in the sample worksheet 1A (see Table 1.1). Note that some stakeholders appear in more than one column; these are not exclusive classes of stakeholders so much as four ways of thinking about stakeholders to ensure thinking as broadly as possible. Second, note that not all categories have the same number of stakeholders. Indeed, for a simple project, there may be very few stakeholders and some categories may have none at all. The sample worksheet 1B (see Table 1.2) helped identify the perspectives and needs of the key stakeholders and the implications for designing and implementing the evaluation. Note in the CLPP example that while all stakeholders may applaud efforts to reduce EBLL in children, several stakeholders put priority on outcomes that might or might not agree with our priorities. For example, private physicians are most interested in “yield” of their screening efforts, while Congress cares about cost-effectiveness. Note that advocacy groups, in addition to priorities for specific outcomes, also have preferences related to data collection—such as stipulating “no surveys.” All of these insights are helpful at the start of an evaluation to ensure that the evaluation goes smoothly and the results are used.
Maternal and child health groups
| Stakeholders | What component of intervention/outcome matters most to them | |
|---|---|---|
| 1 | Physician associations | Sufficient “yield” of EBLL children to make their screening efforts “worth their time.” |
Clear referral mechanisms that are easy and work.
Support for families with EBLL children.
No survey data collection with families.
“Cost-effectiveness” of the effort.
What to Ask StakeholdersThroughout the evaluation planning process, you will be asking some or all stakeholders the following questions:
Stakeholder perspectives may influence every step of the CDC Framework. Stakeholder input in “describing the program” ensures a clear and consensual understanding of the program’s activities and outcomes. This is an important backdrop for even more valuable stakeholder input in “focusing the evaluation design” to ensure that the key questions of most importance will be included. Stakeholders may also have insights or preferences on the most effective and appropriate ways to collect data from target respondents. In “justifying conclusions,” the perspectives and values that stakeholders bring to the project are explicitly acknowledged and honored in making judgments about evidence gathered. Finally, the considerable time and effort spent in engaging and building consensus among stakeholders pays off in the last step, “ensuring use,” because stakeholder engagement has created a market for the evaluation results.
Stakeholders can be involved in the evaluation at various levels. For example, you may want to include coalition members on an evaluation team and engage them in developing questions, data collection, and analysis. Or consider ways to assess your partners’ needs and interests in the evaluation, and develop means of keeping them informed of its progress and integrating their ideas into evaluation activities. Again, stakeholders are more likely to support the evaluation and act on results and recommendations if they are involved in the evaluation process.
Engage your program’s critics in the evaluation, as well. Critics may help identify issues around your program strategies and evaluation information that could be attacked or discredited, thus helping you strengthen the evaluation process. This information might also help you and others understand the opposition’s rationale and could help you engage potential agents of change within the opposition. However, use caution: It is important to understand the motives of the opposition before engaging them in any meaningful way.
This emphasis on engaging stakeholders mirrors the increasing prominence of participatory models or “action” research in the research community. A participatory approach combines systematic inquiry with the collaboration of diverse stakeholders to meet specific needs and to contend with broad issues of equity and justice. As noted earlier, The Study of Participatory Research in Health Promotion, commissioned by the Royal Society of Canada, has published a set of guidelines for use by evaluators and funding agencies in assessing projects that aspire to be participatory. [14] The guidelines emphasize that traditional ways of conducting health research in populations must adapt to meet the educational, capacity-building, and policy expectations of more participatory approaches if the results of the research are to make a difference.