You can read BPE's summary of an evaluation of SAM-New York by Dr. Joan Talbert, a nationally recognized researcher of school leadership initiatives, and Dr. Nell Scharff of Baruch College, a partner of SAM-New York.  You can also view the original report, as presented to the American Educational Research Association in March 2008.

BPE has contracted with the University of Massachusetts Donahue Institute’s Research and Evaluation Group to conduct a two-year evaluation of the SAM program.  Researchers from the institute will conduct interviews with BPE facilitators, school principals, and all team members, administer a school-wide survey to all teachers, observe a team meeting at each school, and analyze student performance data to assess the program’s effect on accelerating student learning and changing school structures to move more students into the sphere of success.

The following are the key evaluation questions laid out in the evaluation design:

These questions are cited verbatim from the UMass Donahue Institute’s “Preliminary Design for an Evaluation of the Scaffolded Apprenticeship Model in the Boston Public Schools” dated December 20, 2007.

1. To what extent is the SAM model being implemented as designed by BPE and the participating schools?  And, what factors have served to support or hinder program implementation or participation?

2. To what extent do the school Inquiry Team and its individual members display the professional growth and increased knowledge anticipated by the model?  What factors have served to support or hinder the attainment of this growth or knowledge?

3. To what extent are key practices, approaches, competencies, and structures developed and implemented by members of the school Inquiry Team in evidence within the broader school community?  What are the structures or approaches by which these practices have most effectively been disseminated?

4. What is the effect of new practices, approaches, competencies, and structures on school culture, particularly with regard to how leadership is both perceived and exhibited?  What aspects of the intervention appear to be most effective in leveraging cultural change?

5. To what extent is SAM leading to improvements in learning and achievement among targeted students and among the broader school community?  To what extent are these changes linked explicitly to SAM or attributable to other school initiatives?

6. What inter-relationship exists among: program implementation characteristics; changes in Inquiry Team members and other school staff capacities and practice; changes in school community culture, capacities, and practice; and student learning outcomes at the target group and whole school level?  What are the root causes of SAM success or failure at the target-group, whole school- and program-levels?

7. In what ways, if any, is the SAM-Boston model evolving over the course of its implementation? How are decisions to alter strategy made and what are the reasons behind any modifications that are implemented?

Check back for the following reports:

Mid-term summary of findings – coming Summer 2008

Preliminary project report – coming May 2009

Final project report – coming September 2009