Annual Outcome Reports

Making Sure Programs Make a Difference

In 1997 United Way of the Midlands (UWM) Board of Directors committed to measuring outcomes of UWM-supported programs. Today, teams of volunteers annually review and evaluate programs supported with Community Impact funds. The purpose of the review is to assure that programs are "doing the right things in the right way for the right people" and getting intended results. 
 
Teams review Annual Outcomes Reports to make their program evaluations. An Annual Outcomes Report answers the following questions: 
  1. What are the expected results (outcomes) for the program? How will people or conditions have changed as result of the program? 
  2. Outcomes can take time to achieve, so it is important to determine progress toward a longer-term outcome. What target objectives (indicators) will demonstrate the program is making a difference? How close are the target objectives (expected indicators) compared with the actual results (actual indicators)? 
  3. "Doing the right things in the right way" means that the program is based on models and practices (activities) that already have demonstrated success. To what degree were program activities carried out according to the model? If changes or adaptations were made, what were those changes? Did they affect results?
  4. Doing things in the right way also means that the right resources (inputs), like specially qualified or trained staff, special curriculum materials, equipment and supplies were used. What resources (inputs) were required to implement this program?
  5. Many model programs are intended for specific target populations. Who are the "right people" for this program? Were special recruitment activities used to engage them? Did the target population participate in the program? 
  6. How often does program staff review program information? How is that review and analysis used to improve program services? Are program staff continually learning from their own work and the work of others in similar programs? 
A three-person evaluation team reviews each UWM-supported program. Each evaluator is trained in UWM outcomes reporting. 
 
Each program is "scored" using a four-point scale applied to 13 (weighted) items. The four points on the scale represent:
 
1= Does Not Met Expectations
2= Needs Improvement
3= Meets Expectations
4= Exceeds Expectations
 
Earning 3 points, "Meets Expectations," on every item results in a total score of 75. A program score is calculated by averaging the total scores of the three-person team. A score of 75 then, is a very strong score that reflects that a program is doing the right things in the right way for the right people and is delivering the expected results.
 
UWM posts results within the following range:
Scores > 80= Exceeds Expectations/Exemplary
Scores 65- 79= Meets Expectations/Acceptable
Scores50- 64= Needs Improvement/ Acceptable with Recommendations
Scores < 50= Does Not Meet Expectations/Conditional
 
The purpose of the evaluation is to assure that programs are on the right track and are making a difference for the clients who use them. When a program earns a low score, there is an expectation that with refinements and assistance, the program will produce or can demonstrate results. Thus, conditional programs must develop improvement plans and resubmit reports. Programs that score below 65 points may be asked, at Community Council discretion, to develop improvement plans and resubmit reports as well. Should a program fail to make improvements and progress, it will likely be de-funded.