Federal

‘Value Added’ Pioneer Says Stinging Critique of Method Is Off-Base

By Debra Viadero — May 05, 2008 3 min read
  • Save to favorites
  • Print

Includes updates and/or revisions.

The statistician who pioneered the use of “value added” research techniques for documenting students’ year-to-year academic progress is disputing a critique of his approach that was published recently in a prominent academic journal.

In the article, which appeared in the March issue of Educational Researcher, Audrey Amrein-Beardsley, an assistant professor at Arizona State University in Phoenix, takes aim at the Education Value-Added Assessment System, or EVAAS. The EVAAS is an offshoot of the Tennessee Value-Added Assessment System developed by William L. Sanders, which is widely credited with helping to launch nationwide interest in value-added approaches for measuring students’ academic progress.

Ms. Amrein-Beardsley contends that the EVAAS is opaque, is possibly flawed, and has not been subjected to enough external review to merit its widespread use in school systems.

“No one is looking inside the model to see if the assumptions being made within it are true,” Ms. Amrein-Beardsley said in an interview. “The bottom line is to open this up for debate and not have these things be adopted blindly.”

‘Factual Errors’ Alleged

See Also

A recently posted response to Audrey Amrein-Beardsley’s critique is now available online. The response is written by William L. Sanders and S. Paul Wright.

But Mr. Sanders, for his part, contends that Ms. Amrein-Beardsley’s report is “rife with factual errors,” and he points to three studies in which researchers have duplicated and tested his model. Among the most recent was a 2007 study, published in the peer-reviewed Electronic Journal of Statistics, in which a team of researchers from the Santa Monica, Calif.-based RAND Corp. compared the Sanders method with other value-added techniques.

Now working in private industry as a senior manager for value-added assessment and research at the Cary, N.C.-based SAS Institute, Mr. Sanders has never shared the computational algorithms for the EVAAS. But he and a colleague described the model in a 1997 paper, which the RAND researchers and others have used to create their own software mimicking Mr. Sanders’ model.

Ms. Amrein-Beardsley also raises technical concerns with the EVAAS, some of which echo cautions that other experts are voicing about value-added modeling in general. (“Scrutiny Heightens for ‘Value Added’ Research Methods,” May 7, 2008.)

She notes, for instance, that the EVAAS does not take into account demographic and socioeconomic differences between students.

“Family income, ethnicity, ability, and other background variables unquestionably affect levels of student achievement and the progress that students make from year to year,” she writes.

Mr. Sanders, though, has long maintained that such statistical adjustments are not needed because the EVAAS compares students’ year-to-year academic growth with their own progress from previous years. “A kid is coming from the same home this year as he was the year before,” he said.

To illustrate her case, Ms. Amrein-Beardsley reanalyzed data from the appendix of a controversial 2005 study of North Carolina teachers that Mr. Sanders was recruited to do for the National Board for Professional Teaching Standards, an Arlington, Va.-based group that awards national credentials to teachers who meet the requirements of a process based on documentation of their performance.

While Mr. Sanders concluded that board-certified teachers did no better on average than those who had failed to earn certification and those who had never tried, Ms. Amrein-Beardsley said the data showed a positive student-achievement effect. She said that, on average, students of board-certified teachers outperformed those of teachers with only regular certification by three-quarters of a month’s worth of growth in mathematics and one week in reading.

Mr. Sanders said his conclusions stemmed in part from another finding in his study: The variation in student-achievement levels for teachers within the NBPTS-certified group was greater than it was for teachers across different categories.

Joseph A. Aguerrebere Jr., the board’s chief executive officer, said most studies of board-certified teachers, such as the one by Ms. Amrein-Beardsley, also find that students of those teachers outperform their peers.

Coverage of education research is sponsored in part by a grant from the Spencer Foundation.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Personalized Learning Webinar
No Time to Waste: Individualized Instruction Will Drive Change
Targeted support and intervention can boost student achievement. Join us to explore tutoring’s role in accelerating the turnaround. 
Content provided by Varsity Tutors for Schools
Student Well-Being K-12 Essentials Forum Social-Emotional Learning: Making It Meaningful
Join us for this event with educators and experts on the damage the pandemic did to academic and social and emotional well-being.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
K-12 Cybersecurity in the Real World: Lessons Learned & How to Protect Your School
Gain an expert understanding of how school districts can improve their cyber resilience and get ahead of cybersecurity challenges and threats.
Content provided by Microsoft

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal What the Federal 'Don't Say Gay' Bill Actually Says
The bill would restrict federal funds for lessons on LGBTQ identities. The outcome of this week's election could revive its prospects.
4 min read
Demonstrators gather on the steps of the Florida Historic Capitol Museum in front of the Florida State Capitol on March 7, 2022, in Tallahassee, Fla. Florida House Republicans advanced a bill, dubbed by opponents as the "Don't Say Gay" bill, to forbid discussions of sexual orientation and gender identity in schools, rejecting criticism from Democrats who said the proposal demonizes LGBTQ people.
Demonstrators gather on the steps of the Florida Historic Capitol Museum in Tallahassee on March 7, 2022. Florida's "Don't Say Gay" law was a model for a federal bill introduced last month.
Wilfredo Lee/AP
Federal Fed's Education Research Board Is Back. Here's Why That Matters
Defunct for years, the National Board for Education Sciences has new members and new priorities.
2 min read
Image of a conference table.
vasabii/iStock/Getty
Federal Opinion NAEP Needs to Be Kept at Arm’s Length From Politics
It’s in all our interests to ensure NAEP releases are buffered from political considerations and walled off from political appointees.
4 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Federal Feds Emphasize Legal Protections for Pregnant or Recently Pregnant Students, Employees
The U.S. Department of Education has released a new resource summary related to pregnancy discrimination in schools.
2 min read
Young girl checking her pregnancy test, sitting on beige couch at home.
iStock/Getty Images Plus