Since New York state released scores on English/language arts and math tests in grades 3-8 last week, there’s been public discussion about whether the state made it easier to achieve proficiency on the tests by altering how the test was scored.
In an Aug. 17 story, the New York Post reported that the tests had been “fixed” in this way and that scores subsequently surged—although it’s worth noting that statewide proficiency rates on the E/LA were flat, and increased by a relatively modest amount in math.
So did Empire State K-12 leaders goose student scores on tests aligned to the Common Core State Standards?
The New York department says no. Ken Wagner, the state’s deputy commissioner for curriculum, assessment and educational technology, wrote in an email that the “cut scores” required for students to be classified as proficient were not changed for the 2014 test. What did change, according to Wagner, was the difficulty of some of the tests.
On the New York assessments in question, a student’s raw score on tests (the number of questions a student answers correctly) is converted into a score on a scale. That scale score, in turn, corresponds to “performance” levels, such as whether or not a student demonstrated proficiency. The chart below from the New York education department shows how those scale scores correspond to performance levels. On the New York assessments in question, a student needed to achieve a Level 3 in order to demonstrate proficiency.
“If the test is slightly easier, the number of raw score points needed to earn a performance level may increase slightly in order to maintain the performance standard,” Wagner wrote. “If the test is slightly harder, the number of raw score points needed to earn a performance level may decrease slightly in order to maintain the performance standard.”
On the 4th grade E/LA test, for example, a student taking the test in 2013 had to achieve a raw score of at least 38 to achieve a scale score of 320, the minimum score for demonstrating proficiency. But in 2013 a raw score of only 36 was needed for that 320 scale score. You can find more such information on this page on assessments from the New York department.
Statewide, students’ raw scores went down on six tests—three in E/LA and three in math—indicating that those tests were slightly harder than last year’s, according to Wagner. That led officials to reduce the number of raw score points required for achieving certain performance levels. Students’ raw scores went up on four tests—two in E/LA and two in math—indicating slightly easier tests. On two tests, one in each subject, students’ raw scores remained the same.
So New York officials are saying that while cut scores did not change, the raw score points that correspond to those scale scores did change, although Wagner says these changes in raw scores were small.
So should that put to rest all doubts about the proficiency levels with respect to New York’s 2014 tests?
The Post article says that from 2006 to 2009, the state department decreased the raw scores required to demonstrate proficiency, and that the explanation from the department’s then-commissioner, Richard Mills, that the tests became more difficult during that period met with objections from analysts who looked at the assessments.
A 2010 story by The New York Times about the problems from that time period stressed “the state’s decision to create short, predictable exams and to release them publicly soon after they were given, making coaching easy and depriving test creators of a key tool: the ability to insert in each test questions for future exams.”
The state released questions from the 2014 assessments earlier this month. But Fred Smith, a former testing analyst for New York City schools, argued in the Post story that the information released by the state still left an incomplete picture of how students are actually doing.
Information on Economically Disadvantaged Students
On a different topic: When the New York test scores were initially released Aug. 14, we didn’t have information about the performance of economically disadvantaged students. But now the state has released the information.
In 2013, 20 percent of those students (largely those eligible for free and reduced-price meals) demonstrated proficiency in math. In 2014, that figure rose to 26 percent.
In E/LA, the news was not as good and mirrored flat scores on the test statewide: the percentage of economically disadvantaged students demonstrating proficiency rose from 19 percent in 2013 percent in 2013 to 20 percent in 2014.
I’ve updated the charts I created Aug. 14 below, to reflect this new information:
N.Y. State Scores in EnglishLanguage Arts N.Y. State Scores in Math | Create Infographics