EVAAS

Predictive Methodology

Assessments analyzed with the Predictive methodology can be used with any assessment that has sufficient prior testing. This prior testing, or predictors, can include assessments in the same or different subjects.

This model generates a predicted score for each student. Entering achievement reflects students' achievement before the current school year or when they entered a grade and subject or Keystone content area.

A predicted score is the score the student would make on the selected assessment if the student makes average or typical growth. To generate each student's predicted score we build a robust statistical model of all students who took the selected assessment in the most recent year. The model includes the scores of all students in the reference group, along with their testing histories across years, grades, and subjects.

By considering how all other students performed on the assessment in relation to their testing histories, the model calculates a predicted score for each student based on their individual testing history.

To ensure precision in the predicted scores, for most subjects, a student must have at least three prior assessment scores. This does not mean three years of scores or three scores in the same subject, but simply three prior scores on state assessments across grades and subjects. There is one exception. To generate a predicted score for fourth-grade science, only two prior scores are required: third-grade math and ELA.

Let's consider an example. Zachary is a high-achieving student who has scored well on state assessments for the past few years, especially in math. To predict Zachary's score on the Keystone assessment, we:

1. Determine the relationships between the testing histories of all students and their exiting achievement on this assessment in the same year.
2. Use these relationships to determine what the expected score would be for Zachary, given his own personal testing history.

Based on Zachary's testing history, a score at the 83rd percentile would be a reasonable expectation for him.

In contrast, Adam is a low-achieving student who has struggled in math. Their prior scores on state assessments are low. Just as with Zachary, we use the relationships between the testing histories of all students and their exiting achievement on the assessment statewide to determine a predicted score for Adam. Based upon Adam's own personal testing history, a score at the 26th percentile would be a reasonable expectation for him.

Once a predicted score has been generated for each student in the group, the predicted scores are averaged. Because this average predicted score is based on the students' prior test scores, it represents the entering achievement in this subject for the group of students.

Next, we compare the students' exiting achievement on the assessment to their entering achievement. If a group of students scores what they were predicted to score, on average, we can say that the group made average, or typical, growth. In other words, their growth was similar to the growth of students at the same achievement level across the reference group. This is the definition of meeting the growth standard in the predictive methodology.

If a group of students scores significantly higher than predicted, we can conclude that the group made more growth than their peers across the reference group. If a group scores significantly lower than predicted, the group did not grow as much as their peers.

The growth measure is a function of the difference between the students' entering achievement and their exiting achievement. This value is expressed in scale score points and indicates how much higher or lower the group scored, on average, compared to what they were expected to score given their individual testing histories. For example, a growth measure of 9.3 indicates that, on average, this group of students scored 9.3 scale score points higher than expected. When generating growth measures for Teacher Value-Added reports, students are weighted for each teacher based on the proportion of instructional responsibility claimed during roster verification.