Greetings SASP Administrators!
Administrators, coalitions and local programs want survivors to have good services that really support them on their healing journeys. You also want to be able to show how effective and vital sexual assault intervention services are, and you may also feel pressure to prove that grant funds are well spent. These reasons and more may lead you to seriously contemplate program evaluation strategies.
With program evaluation for sexual assault services, there are no easy answers, and it can actually be very challenging to even know the right questions. A number of intrinsic difficulties exist such as the nature of the work, including issues like anonymous hotline calls, one-time advocacy visits, and the highly sensitive and traumatic nature of the survivor's experience as well as the lack of a robust evidence base. It is critically important that any program evaluation work we do be trauma-informed and survivor- and stakeholder-centered. The most successful evaluation projects are built on a strong foundation of collaboration and stakeholder involvement. It is critical that all the people involved in the evaluation understand and support the measures, process, and outcomes. One of the most serious pitfalls an administrator can fall into is advancing a program evaluation agenda without adequate stakeholder buy-in from the state coalition and local programs. Involving the coalition, local programs, and in some cases, survivors and community members in designing and co-creating the evaluation process and tools is the best way to get sustainable, valuable, and valued evaluation results.
A number of states have successfully undertaken evaluation projects with these key factors in mind, and we share their journeys and materials with you in this newsletter. Please also make sure to look through the sturdy section of evaluation resources.
Finally, as you contemplate evaluation, it is important to remember the sources of data we already have. Sometimes, we simply have to ask new questions of existing data to get the answers we need. SASP Administrators submit an annual report to the University of Southern Maine Muskie School of Public Service Violence Against Women Act Measuring Effectiveness Initiative each year. These reports are a source of program evaluation data for OVW, Congress and other stakeholders. It is very important that Muskie is receiving accurate numbers and compelling stories about the work of SASP in your states. We know that the stories of survivors and anecdotes remain the most powerful way to talk about the importance of the work we do.
In 2008, the Kentucky Coalition of Sexual Assault Programs (KASAP) contracted with Dr. Stephanie Townsend to work with them to create evaluation tools for both services and systems response in Kentucky, and together they developed, "The Healing Voices Project." The Healing Voices Project includes evaluation of services at sexual assault programs as well as the systems response to survivors. Read the full article in the attachment below.
PCAR's Victim Service Program Evaluation Project
The Pennsylvania Coalition Against Rape's Victim Service Program Evaluation Project (VSPE) was started in 1996 by six victim service agencies in Western Pennsylvania with the intention of assessing the effectiveness of services and developing a more effective way of demonstrating to funding sources and the public the purpose and impact of providing services to victims of crime. Read about their current evaluation efforts and view their guidebook with history and surveys as well as their implementation checklist in the attachments below.
Possibilities for Practice-Based Evidence: A Presentation from Stephanie Townsend, Ph.D. at the Sexual Assault Services Program National Forum, 2011
This workshop explored the current evidence from research and evaluation on what makes for effective sexual assault services. Discussion included critical consideration of the limitations and gaps in the existing evidence. The presentation materials attached below also include surveys and information from KASAP's Healing Voices Project.
This is the most recent Muskie report on the SASP Formula Grant Program and includes data about services provided and positions funded as well as anecdotes from survivors, service providers, and funders.
This is a new website associated with the CDC's Veto Violence initiative. It includes a lot of background material on evaluation from the public health perspective.
The American Evaluation Association's mission is to improve evaluation practices and methods, increase evaluation use, promote evaluation as a profession, and support the contribution of evaluation to the generation of theory and knowledge about effective human action. These are the guiding principles they suggest for those pursuing evaluation projects.
This is an article from Blue Avocado, an online resource from The Magazine of American Nonprofits, that looks critically at the idea and role of evidence-based practice for small community organizations addressing complex issues.
Kentucky Association of Sexual Assault Programs Healing Voices Project: Procedures for Evaluating Counseling and Advocacy Services This document, attached below, is referenced in the article above on the The Healing Voices Project. It includes guidelines for the project as well as the surveys themselves.
Outcome Evaluation Strategies for Sexual Assault Service Programs: A Practical Guide by Cris M. Sullivan and Suzanne Coats. Published by the Michigan Coalition Against Domestic and Sexual Violence, 2000 This is a must-read comprehensive guide that looks at many critical aspects of evaluation for sexual assault services.
Pennsylvania Coalition Against Rape Victim Services Program Evaluation Manual This manual, attached below, is PCAR's comprehensive services evaluation guide for its programs and includes the surveys and materials for the project.
PCAR's VSPE Implementation Checklist This checklist, attached below, goes with PCAR's evaluation manual and provides considerations for programs implementing the evaluation of services.
Outcome Measures for Sexual Assault Services in Texas - Final Report University of Texas at Austin This guidebook from 2003 looks at the process of developing outcome measures for core services in Texas including logic models for each of the services.
Texas Association Against Sexual Assault Blog on Evaluative Thinking This blog from TAASA looks at the importance of determining the right questions when one is contemplating evaluation.
Washington Coalition of Sexual Assault Program's Newsletter on Program Evaluation (Spring, 2013) This is WCSAP's recent newsletter covering a number of important sexual assault evaluation topics.
Listening to Our Communities: An Assessment Toolkit from the Sexual Assault Demonstration Initiative This toolkit from the National Sexual Assault Demonstration Initiative focuses on key tools and skills for conducting community assessments in order to strengthen services for sexual assault survivors. It is written specifically for multi-service programs, but will be useful for most victim service programs. The toolkit includes:
- Fact Sheet on Community Assessments
- Guide on Data Analysis
- A Guide for Focus Groups
- Sample Interview Guide
- Tips for Interviewing
- Tools for Measurement
I want to thank the RSP Evaluation Workgroup for resources and ideas used in this newsletter. I wish you the best of luck in your evaluation endeavors, and I'm here to help you all along the way! I'm at (515) 401-8984 or firstname.lastname@example.org.
Terri Poore, MSW
SASP Technical Assistance Specialist
The Resource Sharing Project and this newsletter are supported by Grant No. 2009-TA-AX-K037 awarded by the Office on Violence Against Women, U.S. Department of Justice. Points of view in this newsletter are those of the authors and do not necessarily represent the official position or policies of the U.S. Department of Justice.