Summer STEM Project
Evaluation Findings 2014
About the Summer Science Project
The Summer Science Project supported the availability and quality of summer and science, technology, engineering, and mathematics (STEM) learning programs in Oakland, Concord and San Jose, California. The project was led by the Partnership for Children and Youth (PCY) and Techbridge, in collaboration with four project communities, which included Oakland, Mt. Diablo, Franklin-McKinley, and San Jose Unified School Districts.
Summer Science combined the resources and experiences of PCY’s Summer Matters Campaign and Techbridge’s informal STEM education curriculum to build the capacity of expanded learning staff to lead hands-on summer and science programming for 3rd-5th grade youth. Through this initiative, project communities received hands on curriculum, professional development, and coaching around best practices to engage youth in summer and STEM.
Summer 2014 was the final year of the three-year project. As part of this final year, project leadership at both PCY and Techbridge focused some efforts on setting programs up for continued success. Project leadership supported programs to find, modify and create their own summer science curricula and prompted programs to take on more autonomy in setting their own program plans and monitoring their own programs.
Programmatic Elements
The Summer Science Project supported summer STEM programming through:
- Training: From fall 2013-spring 2014, PCY and Techbridge conducted a series of Summer Quality and STEM professional development opportunities. This series included four trainings (16 hours) of STEM specific training for program staff at Oakland, Mt. Diablo, Franklin-McKinley, and San Jose Unified School Districts. These trainings addressed teaching strategies that promote inquiry- based, hands-on STEM.
- Coaching: Each project community received up to 80 hours of district and/or site-based coaching in the months leading up to and following summer programming. Coaching focused on best practices to engage youth in high-quality summer and STEM programming and ranged from providing resources to co- facilitating summer training. In summer 2014, PCY and Techbridge continued to train onsite, certificated teachers to serve as STEM Coaches for participating program staff. Site-based coaching is an integral component in ensuring that line staff feel confident, prepared, and able to lead high-quality STEM programming in summer. STEM Coaches were onsite daily, in most cases, to provide instructional coaching to support both the preparation and delivery of STEM lessons. Coaching also included one or two formal observations of each staff member leading an informal STEM activity. All observations were accompanied by a debriefing session between the instructional coach and the program line staff to review written feedback based on the STEM coaching rubric.
- Quality Assessment: Quality coaching in spring 2014 was followed by onsite quality assessment site visits using the Comprehensive Assessment of Summer Programs (CASP) Site Observation Tool and the Summer Learning Program Quality Assessment (SLPQA).
Partner Agencies
The Summer Science Project was a collaborative project of:
- Partnership for Children and Youth (PCY): In 1997, a group of concerned government, philanthropy and business leaders decided to do something about the persistent poverty and barriers to success faced by children and youth in Bay Area communities. The Partnership for Children and Youth was created to connect schools and their community partners in these underserved communities with available public and private resources, and to improve the effectiveness of funding streams and services for low-income children. PCY works around three key initiatives: Expanded Learning, Community Schools, and Policy and Advocacy. PCY ignites systems of collaboration, leadership and continuous learning among school districts, government agencies and community-based organization serving low-income children and youth by supporting community school, after school and summer partnerships through training, assessment, planning, policy and advocacy.
- Techbridge: Founded by Chabot Space & Science Center with support from the National Science Foundation, Techbridge was launched in 2000 to expand the academic and career options of girls and to help increase the representation of women and underrepresented youth in STEM. Building on 12 years of success, Techbridge spun off as an independent nonprofit organization in 2011. Techbridge has reached over 4,000 girls in the Bay Area through after school and summer programs for girls that offer innovative hands-on projects, role models and worksite visits, and academic and career guidance. Through partnering with school districts and community-based organizations, Techbridge has helped engage thousands more girls and boys in STEM.
- Oakland Unified School District After School Programs Office: The OUSD After School Programs Office oversees 76 state and federally funded elementary, middle and high school programs, supporting the implementation of quality academic and enrichment expanded learning time programs in close partnership with 15 community organizations to over 16,000 children and youth over the course of the school year.
- Mt. Diablo CARES: Mt. Diablo CARES administers elementary, middle and high school programs at 16 school sites with support from 23 community partners. The program is the result of an ongoing collaboration between the Mt. Diablo Unified School District, City of Concord Parks & Recreation, and Bay Area Community Resources. CARES is supported by several funding sources including grants from state and city initiatives.
- Franklin-McKinley School District/ Washington United Center: Franklin-McKinley includes 19 schools that serve over 10,000 elementary and middle school students. In partnership with the CORAL (Communities Organizing Resources to Advance Learning) after school program of Catholic Charities of Santa Clara County, Franklin-McKinley offers literacy, resiliency, and enrichment activities with support from state and foundation initiatives and funding. CORAL also provides after school programming at Washington United Youth Center in San Jose, offering homework club, sports and cultural activities. Catholic Charities of Santa Clara County has been serving individuals and families for more than 50 years.
Participating Sites
In summer 2014, Summer Science programming was implemented at the following elementary schools:
Mt. Diablo Unified
- Cambridge
- Delta View
- El Monte
- Fair Oaks
- Ygnacio Valley
Oakland Unified
- Allendale
- East Oakland Pride
- Global Family
Franklin-McKinley/ San Jose Unified
- Robert F. Kennedy
- Washington United Youth Center
Summer Science Project Goals
The Summer Science Project has six established goals. The project was effective in changing staff and youth attitudes, beliefs and knowledge and developed a top-notch system of technical assistance. Moreover, project leadership evolved how the project fit into California’s developing summer and STEM initiatives and effectively showcased the high quality work done by participating programs.
Table 1: Progress toward project goals
In many ways, the project met or exceeded the first goal: it went from 2 to 3 communities in Year 2 and sustained these three communities for Year 3. The project also served over 1,100 youth in Year 3, exceeding the target number of youth served. Moreover, the participating projects provided science programming every day for 4 weeks during the summer. The average hours of programming per youth, however, fell about 5% below the 120 hour goal. This gap reflects the strain on program budgets as providers attempt to deliver as many hours as possible, provide adequate STEM materials and invest in professional development. On the positive side, average daily attendance per child reached 85%, which is higher than at most summer learning programs, reflecting high levels of engagement and dosage per child. While the average hours of programming per youth fell slightly below the 120 hour goal, programs were able to provide a consistent, frequent summer science experience for a great many youth in the summer.
Lessons Learned
As the Summer Science Project concluded its third and final year, the Project compiled key lessons learned beyond the project goals. First, it required a significant investment to sustain a project of this nature. Participating programs needed to bring a certain level of readiness and structure; Project leadership needed to right-size the curricula front line staff used offering enough guidance but not too much; and the Project needed to provide significant time and resources in technical assistance, planning, assessment and training. Second, it was important to partner the approach and content of science with a focus on quality. They are natural partners, and may even be necessary partners. Third, participating programs reaped many additional, and some unexpected, benefits through their participation in the Project. These additional effects of the initial investment include stronger staff, improved program quality beyond the science activities themselves, and improved connections to the school day. These benefits are notable in large part because theses are the very areas in which programs often struggle. See ‘Lessons Learned’ starting on page 26 for the full discussion of these findings. See Appendix A for information on the data sources for this report.
Acknowledgements
We would like to acknowledge the following people for their active contribution to the Summer Science Project and this report:
We are grateful to the staff of Partnership for Children and Youth (PCY) and of Techbridge for their support of this evaluation.
- Andrea Broxton, Director, Technical Assistance, PCY
- Katie Brackenridge, Senior Director, Expanded Learning Time Initiatives, PCY
- Laura Beebe, Administrative Coordinator, Expanded Learning Time Initiatives, PCY
- Roshni Kasad, Senior Program Manager, Techbridge
- Jen Joyce, Director of Professional Development, Techbridge
We also extend our thanks to the Agency Directors, Site Coordinators, Instructional Coaches, and Front Line staff of Oakland, Mount Diablo, Franklin-McKinley, and San Jose Unified School District’s summer programs that provided services to the children of their respective Districts over the past three summers. Their active participation in the evaluation is key to the success of this report.
Public Profit Evaluation Team
- Jessica Manta-Meyer, Senior Research Associate
- Stephanie Kong, Research Assistant
- Venessa Laurel, Project Assistant
- Corey Newhouse, Project Director, Founder and Principal