Evaluation & Evidence
- Goal Measurement
All youth outcome and organizational goals are tied to indicators and data collection methods
- Data Collection Method
Data collection includes both qualitative and quantitative approaches. Program uses more than two standardized methods to collect data, such as survey, administrative data, focus group and formal or informal youth assessment.
- Collection and Analysis of Data by Staff
Year-round and seasonal staff is involved in both the collection and analysis of data related to program goals and stakeholder perspectives
- Stakeholder Perspectives Data
Program collects data on stakeholder perspectives from at least three groups (ex. staff, youth, families and partners).
- Average Daily Attendance
On average, participants attend at least 85% of the summer session.
- Youth Retention
At least 80% of eligible youth attend the program for at least two years.
- Staff Retention
Program retains all staff through the term of the program. Program retains more than 75% of its eligible staff from year to year.
Program Sustainability
Program develops and implements a clear strategic plan and aligned fundraising plan. Program shares information about the program with key stakeholders to promote sustainability.
- Comprehensive Strategic Plan
Summer program is included in long-term planning for the parent agency or organization. Summer program strategic plan sets the direction for 3-5 years of programming, evaluation, budgeting and administration.
- Strategic Plan Updating
Program dedicates collaborative planning time for all staff at least twice per year to assess and update the strategic plan.
- Diverse Funding
Program operating budget shows revenue from several diverse funding sources.
- Stakeholder Communication
Program prepares an annual report or publication and disseminates to several external stakeholder groups (families, board members, community leaders, funders) to communicate program outcomes.
- Advocacy
There is evidence that the program leadership or staff has presented information formally and informally to the media, community stakeholders, funders and policymakers educating them about summer learning loss and the program. Director consistently empowers other stakeholders, including youth, families and staff, to publicly advocate for the summer program.
- Community Engagement
Director understands the value of connections to the broader community and actively leads community groups or committees with common goals of the program.
Planning
Program is designed to allocate enough time, staff and resources to promote positive academic and developmental youth outcomes. Program has a proactive summer program planning process that is inclusive of all key stakeholders and connected to the goals of the program. Program has a comprehensive structure in place for all programming throughout the summer, in advance of the session.
PROGRAM DESIGN
- Adult to Youth Ratio
The average program activity has an adult to youth ratio of 1:8 or lower.
- Physical Activity
Program dedicates at least 30 minutes per 3 hours of programming daily for physical activity for all youth.
- Continuum of Programming
Programming is offered on a continuum over multiple summers for all young people. Offerings span the transition between elementary and middle school, middle school and high school, or both.
- Food Service
Appropriate for schedule, all meals (breakfast, lunch, snack) are provided for young people every day the program is in session
- Total Hours of Programming
Program offers a minimum of 150 hours of programming to each youth.
CURRICULUM AND ACTIVITIES
- Proactive Planning
Program director begins planning for the next summer session at the close of the current summer session.
- Youth Input
Program solicits input from most youth to inform program design and planning each year.
- Staff Collaborative Planning
Site coordinators and line staff have input into selection and development of the curriculum and activities for the summer through collaborative planning sessions at least three months prior to the start of the session.
- Backward Planning
Program staff and partners use youth outcome goals as the foundation for activity planning and curriculum development and/or selection. All programming is connected to measurable youth outcome goals.
- Lesson Plan Framework
Program provides site coordinators and line staff a framework for unit and lesson planning that is connected to program goals and includes instructional strategies and support for benchmarking. Staff consistently uses framework to plan units and lessons.
- Thematic Learning
Program is designed to make clear connections for youth among all regular classroom experiences, field trips and special events through projects or thematic units. Learning is reinforced across activities and experiences every day.
- Use of Certified Teacher
There is a certified teacher on staff or available for consultation from the beginning and throughout the duration of the curriculum development and activity planning process to assist staff.
- Field Trips
Most field trips are to new places youth have never been and/or provide the opportunity to do something they have never done before. Field trips provide incentives for attendance and good behavior.
- Complete Summer Program Schedule
Daily program schedule for the entire summer session, including field trips, is complete before youth recruitment begins.
- Comprehensive Summer Program Schedule
Summer program schedule outlines hourly activities including locations and staff responsible for each activity. Schedule is posted throughout the program space and is referenced daily by all staff. Daily and weekly schedules create a predictable routine for young people and for staff, with fluidity allowed for special opportunities/events.
Staff
Program’s recruitment and staffing process intentionally yields culturally competent staff with relevant skills. Program staff is empowered to manage the program and has a voice in organizational decisions. Program provides extensive opportunities for staff development and advancement before, during and after the session.
STAFF RECRUITMENT
- Staff Interview and Selection
Program’s staff recruitment process is designed to reach potential staff who are knowledgeable of the community’s demographics and have skills connected to program goals.
- Staff Recruitment Timeline
Seasonal staff recruitment begins at least six months prior to the start of the session. Job offers are made at least three months prior to the start of the session.
STAFF TRAINING
- Alignment of Staff Needs and Training
Program formally assesses staff abilities upon hire against a set of articulated competencies for each position and sets training objectives based on identified needs in those competencies.
- Staff Training Timeline
Program requires summer program-specific staff training, connected to identified staff needs, at least two weeks before the program starts and includes additional paid time for individual or group preparation.
- Support for Noncertified teachers
Program offers additional specialized training and support for staff with little or no instructional experience. Training includes classroom and behavior management and instructional strategies.
STAFF AND PROFESSIONAL DEVELOPMENT
- Staff Management
Site coordinators have a role in hiring, supervising and providing feedback to site staff.
- Site Management
Site coordinators manage information on site budget, field trips and vendors that enables them to make day-to-day decisions about the program.
- Staff Meetings
Site coordinator and staff meet at least weekly during the session to celebrate success, problem-solve and reflect on events of the program. All meetings have an agenda and are facilitated by a designated staff member.
- Staff Observation and Feedback
Site coordinator observes each staff member multiple times during the summer session and provides feedback connected to identified staff competencies at planned intervals.
- Staff Development During Session
Program uses multiple methods to deliver staff development during the program such as staff meetings, online discussions, peer coaching, mentoring, journaling or study groups. Topics are relevant to the needs of current staff and staff development is ongoing throughout the session.
- Staff Participation in Professional Development
All year-round and seasonal staff participates in professional development during the course of the year and have opportunities to lead/facilitate PD sessions.
Partnerships
Program builds and maintains strong linkages with partners, including community organizations, the public school system and government agencies, that are supportive of its mission and have a vested interest in the program’s success. Program has a formal structure for communication and data sharing with all key external partners. Program builds and maintains strong linkages with families.
SCHOOLS AND COMMUNITY ORGANIZATIONS
- Shared Mission
Program builds collaborative partnerships with entities that complement its mission and expand its access to information and expertise. Partners share buy-in to a broader set of goals for youth. Partners jointly identify and recruit participants.
- Advance, Collaborative Planning with Partners
Program includes partners in year-round planning practices for summer programming. Partners have a voice in the development or review of programming. Regularly scheduled meetings, and consistent informal communication (email, phone, face-to-face contact) is a regular part of doing business.
- Partner Staff Collaboration
Program staff and partner staff collaborate to align program activities and coordinate services for youth. Strategies such as joint training, team-teaching, co-facilitation, shadowing, and mentoring are implemented.
- Partner Communication Structure
Partners have a clear means for exchanging information and sharing resources. Memoranda of Understanding are in place and clearly articulate the roles and responsibilities of each partner as well as the distribution of resources.
- Joint Funding
The program and its partner organizations regularly pursue joint funding opportunities.
- Data Sharing
Program and its partner contribute data to a shared data system that is regularly accessed by both parties. Data system tracks participant attendance and progress toward other identified outcomes. Program and its partner use data to improve the program and report student progress.
FAMILIES
- Family Involvement
Families are considered primary stakeholders in the program and there are both required and voluntary opportunities for families to participate in the program.
- Year-round Communication with Families
Program communicates with families in advance of the summer program, at regular intervals during the program, and throughout the balance of the year. Families are given access to the complete daily program schedule.
- Relationship-building with Families
All staff makes an effort to learn the names of all parents and caregivers and greet them personally. All staff makes a regular effort to share positive information and constructive feedback with parents and caregivers about their child through both written updates and conversations.
Individualized
Program assesses young people’s needs and develops individualized strategies for meeting program goals.
- Youth Assessment
Program uses standardized methods to assess baseline knowledge, skill or attitude of all youth aligned with program goals the first week of the program or before the program begins.
- Individualized, Tailored Instruction
The results of the pre-program assessment(s) inform lesson planning and instruction, and staff is equipped with skills to differentiate activities for youth of different abilities or attitudes.
- Multiple Grouping Strategies
Most activities show a blend of large group, small group and individualized instruction that is planned in advance. Transitions between groupings are smooth for both youth and staff.
Intentional
Activity planning and execution shows intentional focus on meeting learning goals and use of research-based instructional methods.
- Advance Planning
All activities show evidence of a detailed lesson plan and begin and end on time. Materials are prepared in advance and easily accessible to all youth.
- Daily Learning Objectives
In all structured activities, staff communicates daily learning objectives at the group or program level that are connected to youth outcome goals.
- Clear Expectations
Staff sets clear expectations for activities by consistently framing and focusing the activity using these four strategies: Directions for the activity; discussion of how success will be achieved and assessed; discussion of the level of attention or interaction required; and clarifying statements to refocus participants as needed.
- Critical Thinking
In all structured activities, staff use open-ended questions and encourage youth to use critical thinking to extend their answers and draw conclusions through analysis of information.
- De-briefs/Checks for Learning
All staff use debriefing techniques, recall and checks for understanding throughout the duration and at the end of each activity. Staff may ask participants to recall factual information, make generalizations, inferences or real-world applications based on what they have learned. Recall connects previous learning to current activities.
- Staff Engagement
All staff have a clear role in the success of the activities they co-facilitate and are actively engaged throughout all activities.
- Skill Building
Most activities are intentionally linked to age-appropriate academic and developmental skills and work to build subject matter expertise and skill mastery through deep analysis of a subject or idea.
Integrated
Programming builds skills, knowledge and behaviors that promote academic success and healthy development. Activities show a blend of academic strategies and social/emotional development strategies throughout the entire day.
- Integration of Academic and Developmental Focus
Program schedule and activities show a full integration of strategies to promote academic growth and strategies to promote social or emotional growth. All staff leads integrated activities throughout the day.
- Staff/Youth Interaction
Program environment is characterized by mutual care and respect between all staff and all youth. (e.g., All staff demonstrate caring for all participants through warm tones and facial expressions. Staff recognize unique talents, qualities and experiences of all youth to create an emotionally safe and inclusive environment.)
- Positive Reinforcement
All staff use positive reinforcement to recognize movement toward specific group or individual goals in all activities. Positive reinforcement highlights specific behavior rather than praising or calling attention to a particular participant doing the behaving.
- Behavior Management
Program environment is characterized by an intentional, consistent behavior leadership style led by both staff and youth throughout the entire program day. Staff and youth hold each other accountable to a jointly developed behavioral contract that defines positive behavior expectations and consequences. Staff and youth consistently demonstrate how the contract is applied and what it looks like in practice.
- Inquiry-Based Learning
Most activities involve a hands-on, kinesthetic or project-based component that allows youth to engage in in-depth investigations with objects, materials, phenomena and ideas and draw meaning and understanding from those experiences.
- Forward-Thinking Activities
Most activities are designed to expose youth to something new- a place, idea, material, skill or talent. Activities reflect a forward-thinking approach and expand on the previous school-year’s content and activities. If remediation is necessary, program uses materials and lessons that are different from the school year.
- Shared Facilitation
Staff shares facilitation of most activities with youth and give youth opportunities to lead or self-direct some of the activity. Approach values long-term student empowerment.
- Youth Engagement
All youth appear to be intrinsically motivated to participate in and complete the activities. Little to no encouragement is needed by staff to keep youth on task. Youth regularly volunteer ideas and answers and ask questions.
- Collaborative Learning
Most activities promote collaborative learning and interdependence among youth. Successful completion requires youth to work together to produce a product or display of their shared learning.
- Creative Thinking
Most activities foster creative development and allow youth to choose active or artistic expressions to convey ideas and build skills. Creative activities are tied to age appropriate skills and learning comprehension.
- Youth-Produced Work
Most youth-produced work has a purpose and a value in the program. Presentation and sharing of youth work is a regular part of activities. Program projects and activities lead to a tangible end product that reflects the work of the youth over the course of the summer.
- Youth Voice
Over the course of the program, all youth have the opportunity to make major decisions, based on their interests, which impact what and how they learn in the program.
Unique Program Culture
Program creates a “summer culture” that is different from the school year and promotes a sense of community
- Program Spirit
Program creates a spirit of community and pride among all young people through: daily shared traditions, cheers, competitions, team designations and awards for positive actions or attitudes.
- Program Principles
Program has principles that set culture through continuous communication of key ideals, strengths or talents that describe participants and staff. Principles are integrated into most program activities and rituals.
- Culminating Event
A culminating event rewards youth for their success and gives all youth an opportunity to showcase their work or new skills to invited guests.
- Physical Environment
Physical environment promotes creative thinking and exploration. Activity space set-up is different from a traditional classroom, with furniture arranged to create an intimate, interactive environment. Activity spaces are named in relation to the theme or goals of the program. Decorations are youth-driven and thematic.
- Flexible Workspace
Staff provide flexible workspace in most activities to accommodate different learning styles. Youth are able to work sitting at a desk, on the floor, or standing, for example.
- Transitions and Meals
Arrival, departure, transitions and meal-times are positive, unique experiences. Program incorporates themes, entertainment or aspects of program culture into most of the unstructured time.