What Makes a Data-Driven School?

While the need for data-driven schools has become more of a focus in recent years, it is by no means a new concept. Tiered problem-solving and data-driven decision-making within schools have a long history in education dating back at least to the 1970s. Neither will the need for data-driven schools disappear as legislation, policy, or society changes, especially if data-driven schools are presented not as a new initiative, but as an integrated “way of doing business.”

In that spirit, we review the key tenets of a data-driven school. Following these key tenets is critical in building and sustaining a school culture that embraces and utilizes—rather than pushes out or discredits—data as part of the school’s efforts to improve student outcomes.

Strong Leadership With Buy-In From Key Stakeholders

School and district leaders are the key decision-makers in school districts; they hold the purse strings. Their support is essential to the success of initiatives within a data-driven school. That being said, they do not need to be the ones who initiate the transition to a data-driven school.

That impetus can come from other data leaders within the school—including general or special education teachers or related services personnel (e.g., school psychologists). If the movement toward creating a data-driven school is to gather momentum and ultimately garner success, however, administrators must be brought on board. Not only must the initiator of the data-driven movement—if not an administrator—have the support of administration, but this initiator also must be able to cultivate the buy-in of other key stakeholders.

Not all followers are created equal in a potential data-driven school. Malcolm Gladwell, in the popular book The Tipping Point, highlights this in a memorable way by describing the essential roles of “connectors,” “mavens,” and “salespeople” in facilitating systems change. Connectors know a lot of people; if they get inspired by a systems change, that inspiration is likely to spread quickly, given their numerous connections to others within the system.

Mavens are not only knowledgeable, but they are eager to talk to others about what they know. If they begin to learn about a change effort, they will share what they learn with others, helping to educate the system on the key components of the initiative. Finally, Gladwell highlights the need for salespeople—people who have the skills to persuade others unconvinced of what they are hearing from the connectors and mavens.

We have seen firsthand the importance of paying attention to these special roles of potential stakeholders when an elementary school at which one of the authors, Daniel M. Hyson, worked was adopting the professional learning communities (PLC) school reform model. The school’s principal at the time recognized that any proposed school reform effort was in danger of being rejected by the school’s veteran staff, since they had already been through many other unsuccessful reform efforts over the years.

As a result, Hyson intentionally invited a well-known and respected fifth grade teacher with more than 20 years of experience to be part of the select team of staff that would be involved in intensive initial training in the model. If this key connector bought into the potential benefit of PLCs, the movement would be much more likely to gain momentum among others in the school.

A Comprehensive Assessment System

Another key tenet of a data-driven school is a comprehensive assessment system. This system must include assessment data for multiple unique purposes:

1. Screening in which all students in a school are assessed using a benchmark formative assessment tool linked to the outcome(s) that the school is working toward—general outcome measures (GOMs), computer adaptive tests, or common classroom tests of learning standards identified by grade level or content area—to determine whether general education instruction is meeting the needs of all students and identify which students might need supplemental support to be successful.
2. Diagnostics in which data is used to identify which specific skill or what behavior deficits might be getting in the way for students not meeting standards.
3. Progress monitoring in which the growth of students who didn’t meet standards is tracked using GOMs or skill-specific mastery monitoring tools to see whether they are closing the gaps.
4. Outcomes in which data is used to assess the degree to which students have learned what they have been taught during a predetermined period (e.g., annual state accountability tests, classroom unit tests).

In a series of interviews conducted with school psychologists across Minnesota, Hyson confirmed that a comprehensive assessment system was seen by most interviewees as being a critical building block of a data-driven school. There is a difference, however, between seeing a comprehensive assessment system as a critical building block and implementing it with reliability and validity; assessments must be administered with fidelity if results from those assessments are to be used to make instructional decisions within a data-driven school.

Efforts to safeguard the reliability of the system must start with providing sufficient training to those administering or proctoring assessments within the system, as well as to those completing any necessary data entry tasks associated with the assessments. This training should include periodic initial trainings for new staff or staff changing roles, as well as refresher trainings and fidelity checks to guard against drift following an initial training.

Administrator support is critical to the success of these efforts. Not only must principals and superintendents provide sufficient time for trainings to occur, but if the assessments involve technology, they must also commit to providing the infrastructure and support within the school or district necessary to limit challenges associated with the technology requirements and ensure an efficient and effective response to challenges.

Within a comprehensive assessment system, assessments should be used only for the purposes for which they are reliable. For example, when differences among individual progress monitoring data points or outcome assessment test strands are analyzed, data leaders within data-driven schools should remain aware of the error associated with these scores and be careful not to over­interpret small differences.

The assessments used within a school’s comprehensive assessment system must demonstrate adequate validity, as well. One of the most critical forms of validity to evaluate within a comprehensive assessment system is criterion validity, or the degree to which scores from one assessment are associated with scores on another assessment. Data-driven school personnel should ask themselves: “Are the screening assessments we’re using related to the outcomes that our school views as important?”

Guaranteeing the “face validity” or acceptability of assessments within a comprehensive assessment system is equally essential. Staff and students must believe that assessments within a school’s comprehensive assessment system are valid and can and will be used to help teachers teach and students learn. If students believe assessments are face-valid, they will more consistently put forth their best efforts, making it more likely that the results will reflect their true ability or achievement.

Easy Access to Appropriate Data for All Staff

A third key tenet of a data-driven school is that all appropriate school staff must have easy access to the data gathered. Technology tools used to access the data need to be user-friendly and require just a few clicks for the everyday consumer to get to the most important and frequently used reports.

The school can’t be reliant on one or even a few data experts to create and interpret these reports. Data leaders must, in essence, be continuously working themselves out of a job by building the capacity of all staff to create and interpret reports themselves. For this to happen, however, the data included in reports has to be relevant to the questions that are important to teachers and administrators.

Technology tools should ideally provide users with an integrated “one-stop shopping” experience, including data addressing all four purposes of assessment within a comprehensive assessment system in one place. Teachers and administrators are busy people; the more websites they need to go to and the more passwords they need to remember, the less likely they will be to come back.

All of these issues must be balanced against the critical importance of maintaining data privacy and confidentiality. The Family Educational Rights and Privacy Act (FERPA) requires that only individuals with a “legitimate educational interest” in a student should have access to the student’s performance data.

The Time and Resources for All Staff to Examine the Data

School staff must be provided with the time and resources necessary to examine the data they access, and PLC proponents argue that a non-negotiable step in providing this time and these resources to staff is that the time be provided during the school day. Inviting teachers to meet with their colleagues to access, interpret, and use data to drive instruction solely before or after school communicates the message that this activity is an add-on. Scheduling meetings during the workday says that focused collaboration is an integral part of teachers’ jobs and just as important as student contact.

Time during these meetings needs to be spent on systems-​level, data-driven decision-making, including such tasks as:

1. Evaluating the effectiveness of instruction and identifying and assessing the impact of interventions designed to provide supplemental support;
2. Identifying students in need of supplemental support; and
3. Monitoring the progress of these students in response to the support.

Data leaders might facilitate data-driven discussions, provide face-to-face and virtual training, and help individual staff, teams, and schools or districts set and monitor progress toward appropriate data-driven goals.

Clear Connections Between Data and Potential Interventions at the District, School, and Classroom Levels

The fifth key tenet of a data-driven school is that administrators and data leaders clearly articulate the connections between data and potential interventions at the district, school, and classroom levels. Avoiding the creation of a data-rich, information-poor culture involves providing time and resources not only for staff to access and interpret data, but also for connecting that data with instruction and intervention.

In addition to providing staff with easy access, it is also critical to provide them with easy access to information about research-based interventions to address student needs identified through the data. In examining the data and attempting to use it to identify interventions, data leaders and staff must remain aware of the potential limitations of the data. Assessments must be both reliable and valid for the purpose for which they are designed for data leaders to be confident in using them to identify data-driven interventions. To do so, data leaders must have, or be able to develop, assessment and intervention literacy skills.

Daniel M. Hyson is assistant professor in the School Psychology Graduate Program at the University of Wisconsin–La Crosse.

Joseph F. Kovaleski is professor emeritus of Educational and School Psychology at Indiana University of Pennsylvania.

Benjamin Silberglitt is executive director of Research, Outcomes, and Implementation at Intermediate District 287 in Minnesota.

Jason A. Pedersen is a school psychologist in the Derry Township School District in Hershey, Pennsylvania.

Excerpted from The Data-Driven School: Collaborating to Improve Student Outcomes by Daniel M. Hyson, Joseph F. Kovaleski, Benjamin Silberglitt, and Jason A. Pedersen.
© 2020, The Guilford Press.