A Data-Driven Approach to Enhancing Summer Programs
Data-driven schools ensure they are measuring up and improving in areas that need improvement. The same goes for summer learning programs. A new report, “Measuring and Improving Summer Programs,” from EdResearch For Recovery and the Tennessee Education Research Alliance, identifies best practices for data collection. Broken down into four focus areas—demographics, patterns, experiences, and outcomes—the report highlights what data to collect as well as research-based recommendations for each that enable schools to get the most out of their data collection.
1. Demographics of Summer Enrollment
This area of data collection focuses on student demographic data and summer enrollment rosters. The report poses two key questions you can answer with this data:
- Are the students who are enrolling in the program the ones who would benefit the most from the additional academic support and school engagement opportunities?
- Which students should we target for additional recruiting efforts?
What Data to Collect
- Enrollment Data: Maintain a list for each program or site of all students who sign up, even if some of them never actually attend any days of the program.
- Student Information: Link enrollment data to the district’s Student Information System (SIS) so that enrollment is connected to student-level characteristics.
This will help you better understand whether students enrolling in the summer program are those who might benefit most from the additional academic support and school engagement opportunities. Use this information as a starting point to engage families and assess barriers to participation.
Analyzing enrollment, apart from attendance, can help identify barriers in the participation pipeline. Capturing this information year-over-year, and tracking no-show rates, provides a clearer picture of the percentage of students who enroll in the program and the share of those students who attend. This allows for more efficient program planning and staffing structures.
2. Summer Attendance Patterns
This area of data collection focuses on daily student attendance in the summer program. The report poses three key questions you can answer with this data:
- What are the broad attendance trends over the course of the program?
- Which groups of students are attending more regularly and attending more days?
- Which students are not attending regularly and might benefit from support plans?
What Data to Collect
- Attendance Data: Document which program or site students attended, the program dates, and the specific dates students attended.
- Student Information: Link enrollment data to the district’s SIS so that enrollment is connected to student-level characteristics.
By tracking daily attendance and connecting student characteristics with summer attendance, you can examine students’ summer attendance by differences in school year achievement or school year attendance. This will allow you to develop targeted attendance interventions to support the students who need it most.
Attendance is strongest when programs communicate the benefits of high attendance during recruiting, establish an enrollment deadline, follow-up with reminders about the program, provide transportation, and create an engaging site climate with positive adult-student relationships.
3. Teacher, Student, and Family Experiences
This area of data collection focuses on teacher, student, and family surveys. The report poses four key questions you can answer with this data:
- Did students have positive experiences in the program?
- Do students report higher levels of non-academic outcomes (self-efficacy, connection, etc.) at the end of the program?
- How do teachers and parents view the quality of the program?
- What do parents and teachers see as strengths and areas for growth for the program?
What Data to Collect
- Student Surveys: Administer pre- and post-program surveys to see changes over time, or administer post-program surveys to see retrospective insight on the program.
- Teacher and Family Surveys: Administer after the program to get feedback on teacher and family experiences; ensure that the survey is accessible for families.
Robust surveying allows districts to measure and assess program impacts and experiences beyond test scores. Accurately capturing these experiences allows for better planning and ultimately creates stronger programs that align with the needs of students, families, and teachers.
Consider sending a survey to non-participant families to gain a better understanding of what types of activities and opportunities your families are looking for when it comes to summer programming. This can also allow you to compare responses for families who attended summer programming and those who did not.
4. Academic Outcomes
This area of data collection focuses on student test scores from the spring and fall and pre- and post-tests. The report poses three key questions you can answer with this data:
- Did students attending the program improve on targeted academic skills?
- Did students who attended the program longer see more academic improvement?
- Did summer program participants perform better on benchmark exams compared with non-participants?
What Data to Collect
- Achievement Data: Leverage available data sources, including current benchmark exams, pre- and post-testing, and prior achievement data.
- Student Information: Link achievement data to attendance data and the student-level characteristics to analyze achievement in more detail.
While many analyses of academic outcomes should not be considered causal estimates of differences in the efficacy of summer school participation across subgroups of students, they can provide preliminary, and helpful, evidence for the purposes of continuous improvement.
To see academic benefits for the most students, programs should be structured so that the majority of participants are able to experience at least 20 days of instruction, with 25 total hours of math and 34 total hours of English Language Arts in a summer.
The report also highlights what trends you might see in your data-collection as well as resources like a research brief, a summer learning toolkit, and example surveys.