Question & Answer: RAND Researcher Assesses Effects of ‘High-Stakes Testing’

May 08, 1991 3 min read
  • Save to favorites
  • Print

Although many educators have warned that the pressure to raise test scores may result in higher test scores but not improved student achievement, few researchers have actually studied the effects of so-called “high stakes” testing.

In one of the first studies of its kind, Daniel M. Koretz, a senior social scientist at the rand Corporation in Washington, and his colleagues at the National Center for Research on Evaluation, Standards, and Student Testing investigated whether a school district’s high test scores present a misleading picture of achievement.

Over the past four years, the researchers examined test results from several districts and administered similar tests to determine if the students’ scores reflect actual performance in a subject area. As educators may have feared, they found wide gaps between the two sets of scores.

At last month’s annual meeting of the American Educational Research Association, Mr. Koretz presented preliminary results from his study, which represented findings for 3rd graders in a large urban district with high concentrations of minorities and students from low-income families. He discussed the study with Associate Editor Robert Rothman.

Q. How much of a discrepancy did you find between test scores and achievement?

A. In some cases, it was quite large. The discrepancy was larger, generally, in math than in reading, which is not terribly surprising. We had in two cases a difference of 15 to 16 percentile points on a national distribution scale. ... In terms of academic months, that’s a difference of eight academic months. For kids in the spring of 3rd grade, that’s a sizable difference.

Q. What do you think caused the inflated test scores?

A. There is only one possible explanation: Instruction was closely enough tailored to the tests used in the district. Kids learned better the material that was on the test than they did the larger area [of curriculum content].

This has to be a case of narrowing of instruction people have been warning us about.

Q. Are there ways of designing tests to provide a more accurate picture of student achievement?

A. That’s a contentious issue now. My answer is, it’s not so much the test, but how it’s used. [Some people say] if you replace the test with something else, you won’t have that [narrowing of the curriculum]. It may in fact be worse.

[The reason is,] in performance assessments, there is relatively little similarity between performance across tasks. Performance on one task won’t predict how well you’d do on another. Performance assessment often involves very few tasks, while paper-and-pencil tests have 100-some questions, even more. [In performance assessment], you don’t get the effect of averaging out discrepancies.

People would like the problem to go away if you change the test that is used. I don’t see any reason to be confident it will.

Q. In a national test, which President Bush and others are proposing, would the problem of narrowing instruction be worse?

A. I would worry that a national test would produce the same things. There will be an incentive to worry about things on the test, and not worry about things not on the test. People will want to look good on the national test. ...

You could monitor the effect of tests, [however]. You could have another test, like the National Assessment [of Educational Progress], that people don’t teach to. If scores on the test that matters to people go up sharply, and scores on the other test don’t, you have to worry about inflated test scores.

Q. In presenting his proposal for “American Achievement Tests,” Secretary of Education Lamar Alexander said they would be tests “worth teaching to.” Is that an appropriate strategy?

A. A test can be worth teaching to and still give an inflated picture of what students know. ...

There are two separate issues. One is, are we giving tests worth teaching to? The other is, if people are teaching to them, good or bad, what sense can we make of the results? I’m afraid they have not been separated as much as they should be.

A version of this article appeared in the May 08, 1991 edition of Education Week as Question & Answer: RAND Researcher Assesses Effects of ‘High-Stakes Testing’


English-Language Learners Webinar Helping English-Learners Through Improved Parent Outreach: Strategies That Work
Communicating with families is key to helping students thrive – and that’s become even more apparent during a pandemic that’s upended student well-being and forced constant logistical changes in schools. Educators should pay particular attention
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Addressing Unfinished Learning in Math: Providing Tutoring at Scale
Most states as well as the federal government have landed on tutoring as a key strategy to address unfinished learning from the pandemic. Take math, for example. Studies have found that students lost more ground
Content provided by Yup Math Tutoring
Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Nearly a Million Kids Vaccinated in Week 1, White House Says
Experts say there are signs that it will be difficult to sustain the initial momentum.
4 min read
Leo Hahn, 11, gets the first shot of the Pfizer COVID-19 vaccine, Tuesday, Nov. 9, 2021, at the University of Washington Medical Center in Seattle. Last week, U.S. health officials gave the final signoff to Pfizer's kid-size COVID-19 shot, a milestone that opened a major expansion of the nation's vaccination campaign to children as young as 5. (AP Photo/Ted S. Warren)
Education How Schools Are Getting Kids the COVID Shot, and Why Some Aren’t
Some district leaders say offering vaccine clinics, with the involvement of trusted school staff, is key to helping overcome hesitancy.
5 min read
A girl walks outside of a mobile vaccine unit after getting the first dose of her COVID-19 vaccine, outside P.S. 277, Friday, Nov. 5, 2021, in the Bronx borough of New York. (AP Photo/Eduardo Munoz Alvarez)
Education Biden Administration Urges Schools to Provide COVID-19 Shots, Information for Kids
The Biden administration is encouraging local school districts to host vaccine clinics for kids and information on benefits of the shots.
2 min read
President Joe Biden, and first lady Jill Biden walk to board Marine One on the South Lawn of the White House in Washington, Saturday, Nov. 6, 2021. Biden is spending the weekend at his home in Rehoboth Beach, Del. (AP Photo/Alex Brandon)
Education Civil Rights Groups Sue Tennessee Over Law Against Transgender Student Athletes
The state law bars transgender athletes from playing public high school or middle school sports aligned with their gender identity.
3 min read
Amy Allen, the mother of an 8th grade transgender son, speaks after a Human Rights Campaign round table discussion on anti-transgender laws in Nashville, Tenn. on May 21, 2021.
Amy Allen, the mother of an 8th grade transgender son, speaks after a Human Rights Campaign round table discussion on anti-transgender laws in Nashville, Tenn. on May 21, 2021.
Mark Humphrey/AP