Section 3: Feedback and Evaluation

In this section, you’ll learn about different ways to collect feedback, key terms used in evaluation, and be introduced to tools to think more deeply about your RCO’s impact

Feedback and evaluation are both essential tools for nonprofits, particularly for RCOs. Feedback primarily involves gathering insights, opinions, and experiences from people and organizations engaged with your RCO. It provides an understanding of how programs and services affect people, emphasizing personal perspectives and lived experiences. On the other hand, evaluation is a more structured way to see how well your programs are working. It usually involves gathering and examining data and using set tools to track improvements and achievements. Having feedback and evaluation processes, no matter how simple, help RCOs form a better understanding of their impact, improve services, and better support the community.

As an RCO leader, ask yourself these questions as you move through this Section: 

• How well are we engaging with the recovery community to gather feedback on our programs and services? Are we actively seeking their input through a variety of means?
• Do we have systems in place to collect and analyze data to measure our effectiveness? If not, where can we start?
• Do we have a clear process for incorporating feedback into our program development?
• How can I foster open dialogue and actively invite input from individuals and communities who hold different perspectives from ours?

Feedback and Data

How can we encourage more stakeholders to provide feedback? What is our process for responding to feedback and making changes? 

Feedback provides a valuable opportunity for RCOs to gather insights from their stakeholders, such as staff, individuals in recovery, their families, community partners, and the broader recovery community. By actively seeking feedback, RCOs can better understand the needs, preferences, and experiences of their community. In fact, the intentional feedback loops that many RCOs have built in as standard practice are examples of evaluation in action. Peer Advisory Councils, community listening sessions, focus groups, participant surveys, all of these are sources of data that inform your decisions and programming before you start developing things and provide feedback on how to improve after people participate. 

Ensuring Inclusivity in Voices and Perspectives

To ensure that the voices and perspectives of all stakeholder groups are considered and valued, RCOs can establish advisory committees that include representatives from diverse community segments. Regular community forums where feedback and ideas can be openly shared are also effective. These inputs should not be just heard but actively integrated into decision-making and program development. Training staff in active listening and empathy can further reinforce an environment where all voices are genuinely valued.

Resource: Here are a few resources that could be helpful in getting started with asking for feedback or constructive criticism as well as program debriefs to capture ideas for improvement and celebration! 

Team-Based Inquiry

Appreciative Inquiry

Community Partner Feedback Form template

Helpful Terms to Know

Are we currently collecting any data and how? 

Quantitative: This type of data includes numbers and statistical information that can be measured. An RCO might collect data such as the number of individuals attending support meetings, the number of referrals made to community-based programs, or the number of calls made in a Telephone Recovery Support program.

Qualitative: This type of data is less structured and includes descriptive data, such as personal stories and experiences, feedback, and other non-numerical information. This can provide context and deeper understanding of a program's impact. This could include collecting personal stories from individuals who have been supported by the RCO, gathering feedback on the helpfulness of the programs, or understanding the personal experiences of participants in navigating recovery.

Outputs: These refer to the tangible products or services delivered by a program or intervention. They are the immediate, direct results of program activities, such as the number of workshops held or the number of participants served. For an RCO, this might be the total number of individuals served annually, the number of individuals attending a yoga class at the RCO, or the number of partnerships established with local businesses to support the RCO’s mission.

Outcomes: These are the changes or benefits that result from a program or intervention. For an RCO program, outcomes might include improvement in family relationships, enhanced financial stability among program participants, or an increase in the number of participants who gain employment.

Metrics: These are quantifiable measures used to track and assess the status of a specific process. This is measurable data that can help an organization understand its performance over time. An RCO might use metrics like the number of active participants in its programs, the retention rate in those programs, or the rate of growth in participant recovery capital over a certain period.

KPIs (Key Performance Indicators): These are a type of performance measurement that helps organizations define and track the process towards organizational goals. An RCO might set KPIs such as increasing the number of individuals served by a certain percentage each year, or the increase in community understanding and support for recovery.

Below is a basic table of when common forms of evaluation may occur in the development and implementation of a program. 

Evaluation

Do we have evaluation processes in place already? How will we use evaluation results to make decisions and improve our programs? 

Evaluation doesn’t need to be intimidating; it asks you to reflect on your work to identify what worked, what didn’t, and how you can improve. It helps your RCO in answering questions such as “Are our programs achieving their intended outcomes?” or “Are we effectively using our resources?” Evaluation often involves the use of quantitative and qualitative methods, and the information gathered is used to make data-driven decisions for the organization. The goal of evaluation is to improve the RCO’s ability to fulfill its mission and better serve the community.

Not all evaluations require professional evaluators or external evaluators. Consider what skills and tools you and your team already have, think about the questions that need answering, and then reach out to professionals when you cannot do the evaluation yourselves or you want additional support. Develop your organization’s evaluation skills—it is doable, it is always useful and applicable, and it will help create a sustainable organization for the long run.

A Theory of Change can help your team think deeply about the implications of your RCO’s programs, services, and existence to the communities you serve - your theory of what happens within your communities because of your RCO’s existence. Many grants focused on funding programs ask for a logic model and then want you to report on the outputs and outcomes that have resulted from funding that program. The information you gather to understand a program’s outputs and outcomes can also help you begin to test your theory of change and see what is or is not happening through your efforts. 

Resource: BetterEvaluation is a great place to get started on understanding and digging into evaluation. 

Mechanisms for Ongoing Evaluation of Stakeholder Engagement

To ensure ongoing evaluation of stakeholder engagement and equitable decision-making, RCOs can implement regular review processes, such as surveys, feedback forms, and community meetings. Establishing metrics to assess the effectiveness of engagement strategies and the diversity of participation can guide improvements. Additionally, periodic training for staff and leaders on health equity and inclusive practices ensures that these values remain a central focus in all organizational activities.

Context and Worldview

What is my worldview and how does it influence the questions I ask? What are we not asking?

Understanding is always influenced by the context in which it occurs. When conducting evaluations, it is important to consider factors such as culture, language, and context right from the beginning. Every question we ask, whether of ourselves or others, carries assumptions that are shaped by our own perspectives of the world, our work, and our expectations for the feedback we receive. It is vital to be aware of the assumptions we may be making when gathering feedback and to seek input from others who can provide insights that align with different cultures, contexts, perspectives, and ways of experiencing and navigating the world. 

Context and Worldview in Data Collection and Program Evaluation

The impact of context and worldview is critical in the processes of data collection, gathering feedback, and evaluating programs. Understanding the cultural, socioeconomic, and environmental factors that influence stakeholders' experiences and responses is essential. This requires adopting a culturally sensitive approach to data collection and analysis, ensuring that the tools and methods used are appropriate and respectful of the diverse populations served. Involving stakeholders in the design of these processes can also provide valuable insights and foster a sense of ownership and trust.

Resource: Here is an article from YW Boston about disrupting and identifying bias in evaluations, Your evaluations are likely biased. Here’s what you can do about it, 

Evaluation by Nonprofit Lifecycle Stage

Are we growing at a sustainable rate? How can we adapt our evaluation methods to our changing context? What can we learn from our challenges?

No matter where an RCO is in its growth cycle, evaluation can provide valuable insights and guidance. Below we explore concrete examples of how RCOs can evaluate their programs at various stages.

For RCOs just starting up:

1. Capturing current activities: Start by documenting the activities you are currently providing. Keep track of the types of programs, number of participants, and any relevant outcomes or changes observed. Consider using simple data collection methods such as attendance sheets, sign-in forms, or basic feedback surveys to gather information.

2. Conducting participant interviews: Engage with your program participants and ask for their feedback and experiences. Conduct interviews or focus groups to gain insights into their perceptions of the program's effectiveness and impact on their well-being. This qualitative data will provide valuable insights into the ways your programs are making a difference.

3. Tracking program metrics: Identify key performance indicators (KPIs) that align with your goals and track them over time. These could include the number of participants served, the percentage of attendees reporting positive outcomes, or the level of satisfaction measured through anonymous surveys. Regularly measuring these metrics will help you monitor progress and make data-driven decisions.

For RCOs in a growth stage:

1. Analyzing existing data: Dig deeper into the data you have been collecting and analyze it for trends or patterns. Look for connections between program activities and positive outcomes. Identify areas where your programs are excelling and areas that might need improvement. Use this information to support your case for new opportunities, funding, or program adjustments.

2. Conducting outcome evaluations: Move beyond basic data collection and focus on outcome evaluation to measure the long-term impact of your programs. Consider implementing pre- and post-program assessments to track changes in participants' lives. This will allow you to demonstrate the effectiveness of your programs and position yourself for funding opportunities or partnerships. The Repository of Useful Scales from the Health through Flourishing Research Program is a good place to look for simple, validated assessments for measuring improvements in Quality of Life, Emotional Wellbeing, Evaluation of Services and more.

3. Engaging external evaluators: Consider partnering with external evaluators or researchers to conduct a more comprehensive analysis of your programs. Their expertise can provide an objective perspective and help identify areas for improvement, effective strategies, and potential opportunities for growth.

For RCOs that have plateaued, or are in decline:

1. Conducting a program review: Step back and evaluate your programs against your mission and goals. Ask critical questions such as whether certain programs are still aligned with your organization's purpose or if resources are being allocated effectively. Identify programs that may no longer be meeting the needs of your community and be prepared to sunset them if necessary.

2. Engaging stakeholders: Engage your board members, staff, participants, and other stakeholders in open discussions to gather different perspectives on your organization's direction. Seek feedback and input on program relevance and impact. This self-reflection process can help realign your programs with your mission and ensure that you are making the best use of your resources.

3. Exploring new opportunities: Use the knowledge and experience gained from your evaluation process to identify new opportunities for growth and sustainability. This could involve exploring partnerships, seeking alternative funding sources, or restructuring your programs to better align with your community's evolving needs.

Reporting Impacts

What outcomes do we currently report on? Are we missing any opportunities to demonstrate our less obvious successes?

Reporting is a key part of the feedback and performing evaluation process. RCOs compile their findings and share them with people involved, like funders, board members, program participants, and the broader community. The reports should be easy to understand and short, focusing on successes, challenges, and what was learned. It is a way to be open, responsible, and demonstrate accountability. It’s also a starting point for learning and improving.

Check out these annual reports from other RCOs:

Unity Recovery’s 2020 Annual Report 

Doc’s Recovery House 2022 Annual Report

Section 3: Pause & Reflect

At the end of each section, take time to reflect on what was presented, consider how it applies to your unique situations, structures, and goals of your RCO, and identify possible next steps.