Opinion
Accountability Opinion

An Immodest Proposal

By skoolboy — July 01, 2008 3 min read
  • Save to favorites
  • Print

This year’s statewide fourth-grade math exam administered in New York State -- the one with the remarkably high gains -- contained the following item:

“Janice bought a notebook for $3.75 and a pencil for $0.47. She gave the cashier $5.00. How much money did Janice receive in change?”

The item might have looked a little familiar to fourth-grade teachers. In 2007, a similar item appeared:

“Tony bought art supplies that cost $19.31. He gave $20.00 to the cashier. How much money did Tony receive in change?”

And in 2006, an item read:

“Mr. Marvin spent $54.10 on pants and shirts. He gave the cashier $60.00. How much money should Mr. Marvin receive in change?”

Other similarities abound. In 2008, an item read:

During the year, one thousand eight hundred four books were checked out of the school library. What is another way to write this number?

A. 184
B. 1,084
C. 1,804
D. 1,840

There was an uncanny resemblance to an item on the 2007 test:

The number of people who live in Goodwin Falls is three thousand nine hundred eight. What is another way to write the same number?

A. 398
B. 3,098
C. 3,908
D. 3,980

To be sure, the test-takers in 2008 still had to answer these questions correctly to get credit for them. But the similarity in item formats across the years gives some credence to concerns that scores are inflated.

Dan Koretz discusses the problem of score inflation in his excellent new book, Measuring Up: What Educational Testing Really Tells Us. One source of the problem, he explains, is that all tests sample the subject-matter domains that they are supposed to tap. If the same kind of item shows up repeatedly on the test from one year to the next, teachers and administrators can focus on this restricted set of test item types, and neglect other item types that are still part of the domain that the test is intended to represent.

The National Assessment of Educational Progress (NAEP) is sometimes referred to as the “gold standard” for standardized tests, and claims about test score inflation in a test, such as an NCLB-mandated state test, are often grounded in a discrepancy between NAEP and the other test either in the level of or trend in performance . The characterization of NAEP as the “gold standard” reflects the fact that it is designed to measure a much larger sample of student performance in a domain than is the typical state test. No individual child takes all of the items in the NAEP item pool; instead, students complete test booklets with blocks of items. In the 2000 12th-grade mathematics NAEP, for example, students completed one of 26 different test booklets, each containing three 15-minute blocks out of a total of 13 different blocks of mathematics items. Each student was asked to complete about 40 items across the domains of number sense, properties, and operations; measurement; geometry and spatial sense; data analysis, statistics and probability; and algebra and functions.

Overall, enough students respond to all of the items in the NAEP item pool to be able to measure how well the population of students in a state (or large urban district) is doing. But NAEP is not designed to yield scores for individual students, because no student responds to enough items to yield a reasonably precise measure of performance.

With tongue firmly in cheek, skoolboy offers the following solution to test score inflation: more testing. Imagine if students completed the entire pool of NAEP items (or some other broad pool of items assessing performance in a domain), instead of the relatively restricted sample of items used in most state-level testing programs. If students were assessed on a broad array of items tapping subject matter competence, teachers and administrators would not be able to concentrate their attentions on a subset of item types, and hence would not be able to artificially raise students’ scores relative to their true learning of the subject. Sure, the burden of testing would increase; we’d need to invest in better and more expensive tests; and increased testing wouldn’t solve the incentive problems that high stakes create.

More testing. An idea whose time has come?

Nah.

The opinions expressed in eduwonkette are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Ensuring Continuity of Learning: How to Prepare for the Next Disruption
Across the country, K-12 schools and districts are, again, considering how to ensure effective continuity of learning in the face of emerging COVID variants, politicized debates, and more. Learn from Alexandria City Public Schools superintendent
Content provided by Class
Teaching Profession Live Online Discussion What Have We Learned From Teachers During the Pandemic?
University of California, Santa Cruz, researcher Lora Bartlett and her colleagues spent months studying how the pandemic affected classroom teachers. We will discuss the takeaways from her research not only for teachers, but also for

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Accountability Did Washington D.C.'s Education Overhaul Help Black Children? This Study Says Yes
Researchers said the district's "market-based" reforms accelerated achievement versus other districts and states.
5 min read
Accountability Opinion What Next-Gen Accountability Can Learn From No Child Left Behind
As we ponder what's next for accountability and assessment, we’d benefit from checking the rearview mirror more attentively and more often.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Opinion Let’s Make Transparency the Pandemic’s Educational Legacy
Transparency can strengthen school communities, allow parents to see what’s happening, and provide students more of the support they need.
3 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability The Feds Offered Waivers on ESSA Accountability. Here's Where States Stand on Getting Them
While they get less attention than testing waivers, flexibility related to low-performing schools is an important federal and state issue.
5 min read
Image of a student taking a test with a mask on.
Rich Vintage/E+