ETAC Resources

* Click on links for more information

Program Evaluation

  • ETAC Evaluation Matrix Template
    ETAC Evaluation Matrix Template

    The evaluation matrix is designed to help grantees refine their goals and strategies.

  • Program Evaluation Overview
    Program Evaluation Overview

    The document provides a summary of program evaluation by first distinguishing among the different types of evaluations, next laying out a detailed outline on how to plan for an evaluation, and finally describing the process of completing an evaluation. Understanding the process and carefully planning and executing the evaluation will allow for a thorough evaluation that will yield rich information.

  • Evaluation Reporting
    Evaluation Reporting

    This document provides a snapshot of how to effectively write comprehensive evaluation reports. Evaluators should align their reports to the overall purpose of the evaluation and keep in mind the needs of their stakeholders. The reports need to fully address all the questions raised in the evaluation plan and detail both the outcome data and process data in order to fully capture a program's progress.

  • Annual Program Evaluation Report – Qualitative Data
    Annual Program Evaluation Report – Qualitative Data

    This sample evaluation report contains data collected during the evaluation of the Red Apple Connect program. The report aligns the implementation, process, and outcome data to draw comprehensive conclusions. The data collection methods selected for the outcomes framework includes both qualitative and quantitative tools and methods including surveys, interviews, activity logs, observations and document review. The mixed-method approach provides for increased validity of measurement through triangulation, as well as insight into project activities and outcomes.

  • Annual Program Evaluation Report – Quantitative Data
    Annual Program Evaluation Report – Quantitative Data

    This sample evaluation report assesses the fidelity of implementation of elements of the Red Apple Connect and it includes professional development modules. The relative effectiveness of the individual intervention programs is also assessed to inform both process improvements and sustainability plans. By aligning implementation, process, and outcome data the evaluator is able to draw comprehensive conclusions.

Program Sustainability

  • Checklist for Sustainability Planning
    Checklist for Sustainability Planning

    This Checklist, presented at the 2011 Kick-off Conference, includes action steps for developing and implementing effective sustainability plans and practices. The action steps summarize the essential activities that should be performed to identify, and maintain support for, successful program components.

  • Sustainability Self-Assessment
    Sustainability Self-Assessment

    This self-assessment will help projects determine progress in preparing for the transition from grant supported to self-sustaining programs. Each component required to build support for sustainability is presented as an activity for a specific stakeholder group. The survey elements allow the team to rate progress on each activity and determine priorities for future action.

  • Program Sustainability in Challenging Economic Times
    Program Sustainability in Challenging Economic Times

    The Office of Safe and Drug Free Schools (OSDFS) devoted an entire issue of The Challenge to sustaining school-based initiatives begun under five OSDFS grant programs. It contains examples of successful strategies from actual programs, and provides several references and useful resources to use in planning for sustainability.

External Resources

* Click on links for more information

Regional Education Laboratories (REL) Tools and Reports

Evaluation Resources

Teacher Quality

  • Approaches to Evaluating Teacher Effectiveness
    Approaches to Evaluating Teacher Effectiveness

    The authors of this study evaluated the current research on teacher effectiveness and the various methods and tools used to measure it. The resulting research synthesis provides a thorough discussion of findings as well as implications current evaluations have on public and education policy. The authors consider an expanded definition of teacher effectiveness to include their overall impact on their students, colleagues, and environments, in hopes of providing substantial guidance in this vital field.

  • What Can Mixed-Methods Designs Offer Professional Development Program Evaluators?
    What Can Mixed-Methods Designs Offer Professional Development Program Evaluators?

    This paper considers the role of mixed methods designs and their applicability to evaluations of professional development programs in education. The authors study the literature regarding mixed methods designs and argue that professional development, with its emphasis on accountability and decision-making driven by data. The advantages of mixed methods designs are compared to the additional demands they require, and the authors find their potential benefits to be worthwhile for pursuit in future research.

  • Evaluation Research to Sustain and Expand an Established PDS
    Evaluation Research to Sustain and Expand an Established PDS

    The research, conducted with pre-service teachers in the UI-PDS and graduates of the UI-PDS in their first year of teaching, indicates teachers prepared within a PDS model are well equipped to meet the challenges of urban settings. They plan and implement lessons relative to students' diverse backgrounds, interests, and skills while simultaneously engaging in advocacy and collaboration to advance their students' achievement. This research also indicates components of the UI-PDS program that best supported pre-service and novice participants' learning to teach that they specified through interviews and surveys. Components included taking responsibility Evaluation Research 3 for teaching students, daily on-site support, collaborative practice, and personal attributes of participants.

Fidelity of Implementation

  • A Model for Fidelity of Implementation in a Study of a Science Curriculum Unit: Evaluation Based On Program Theory
    A Model for Fidelity of Implementation in a Study of a Science Curriculum Unit: Evaluation Based On Program Theory

    This paper proposes a conceptual framework for fidelity of implementation and its measures in the context of a quasi-experimental effectiveness study of a middle school science curriculum unit, Motion and Forces (M&F). In this study of M&F, measures of fidelity were based upon the intervention's program theory, and included the intervention's processes and structure from the points of view of both teacher and students. By using multiple measures of fidelity of implementation for treatment and comparison classrooms and correlating fidelity measures with student outcomes, fidelity of implementation not only provides evidence for the internal validity of the study, but also provides a rich view of how and why the implementation "works", and supports the intervention's program theory.

  • Implementation: Measuring and Explaining the Fidelity of CSR Implementation
    Implementation: Measuring and Explaining the Fidelity of CSR Implementation

    This study examines data collected by the National Longitudinal Evaluation of Comprehensive School Reform to determine how implementation of different comprehensive school reform (CSR) models varies according to what factors. The authors find variances between different components within certain CSR models as well as between different CSR model-implementation keys. Implementation can be predicted by several factors, and the study notes that implementation levels are closely connected to the leadership style of the principal.

  • The Evolving Definition, Measurement, and Conceptualization of Fidelity of Implementation in Scale-up of Highly Rated Science Curriculum Units in Diverse Middle Schools
    The Evolving Definition, Measurement, and Conceptualization of Fidelity of Implementation in Scale-up of Highly Rated Science Curriculum Units in Diverse Middle Schools

    In this paper, we report on our progress in defining, conceptualizing, and measuring fidelity of implementation as we complete the fourth year of a six-year study on the scale-up of highly rated middle school science curriculum units in a large diverse public school system within the metropolitan area of Washington, DC (Lynch, Kuipers, Pyke, & Szesze, in press). To begin, we describe our initial attempts to examine fidelity of implementation within our scale-up study. Given that evaluation studies are prerequisite to scale-up studies, we then discuss the dilemma we faced when studying fidelity of implementation of curriculum materials that had not yet been "proven" effective through evidence-based research designs. In addition, the focus of our research is on improving outcomes for diverse student populations, so it is critical that the effects of the curriculum materials on subgroups of students are determined to guide scale-up.

The inclusion of external links on this site does not imply ETAC or DoDEA endorsement of any content or materials hosted on those sites.