Federal

‘Value Added’ Pioneer Says Stinging Critique of Method Is Off-Base

By Debra Viadero — May 05, 2008 3 min read
  • Save to favorites
  • Print

Includes updates and/or revisions.

The statistician who pioneered the use of “value added” research techniques for documenting students’ year-to-year academic progress is disputing a critique of his approach that was published recently in a prominent academic journal.

In the article, which appeared in the March issue of Educational Researcher, Audrey Amrein-Beardsley, an assistant professor at Arizona State University in Phoenix, takes aim at the Education Value-Added Assessment System, or EVAAS. The EVAAS is an offshoot of the Tennessee Value-Added Assessment System developed by William L. Sanders, which is widely credited with helping to launch nationwide interest in value-added approaches for measuring students’ academic progress.

Ms. Amrein-Beardsley contends that the EVAAS is opaque, is possibly flawed, and has not been subjected to enough external review to merit its widespread use in school systems.

“No one is looking inside the model to see if the assumptions being made within it are true,” Ms. Amrein-Beardsley said in an interview. “The bottom line is to open this up for debate and not have these things be adopted blindly.”

‘Factual Errors’ Alleged

See Also

A recently posted response to Audrey Amrein-Beardsley’s critique is now available online. The response is written by William L. Sanders and S. Paul Wright.

But Mr. Sanders, for his part, contends that Ms. Amrein-Beardsley’s report is “rife with factual errors,” and he points to three studies in which researchers have duplicated and tested his model. Among the most recent was a 2007 study, published in the peer-reviewed Electronic Journal of Statistics, in which a team of researchers from the Santa Monica, Calif.-based RAND Corp. compared the Sanders method with other value-added techniques.

Now working in private industry as a senior manager for value-added assessment and research at the Cary, N.C.-based SAS Institute, Mr. Sanders has never shared the computational algorithms for the EVAAS. But he and a colleague described the model in a 1997 paper, which the RAND researchers and others have used to create their own software mimicking Mr. Sanders’ model.

Ms. Amrein-Beardsley also raises technical concerns with the EVAAS, some of which echo cautions that other experts are voicing about value-added modeling in general. (“Scrutiny Heightens for ‘Value Added’ Research Methods,” May 7, 2008.)

She notes, for instance, that the EVAAS does not take into account demographic and socioeconomic differences between students.

“Family income, ethnicity, ability, and other background variables unquestionably affect levels of student achievement and the progress that students make from year to year,” she writes.

Mr. Sanders, though, has long maintained that such statistical adjustments are not needed because the EVAAS compares students’ year-to-year academic growth with their own progress from previous years. “A kid is coming from the same home this year as he was the year before,” he said.

To illustrate her case, Ms. Amrein-Beardsley reanalyzed data from the appendix of a controversial 2005 study of North Carolina teachers that Mr. Sanders was recruited to do for the National Board for Professional Teaching Standards, an Arlington, Va.-based group that awards national credentials to teachers who meet the requirements of a process based on documentation of their performance.

While Mr. Sanders concluded that board-certified teachers did no better on average than those who had failed to earn certification and those who had never tried, Ms. Amrein-Beardsley said the data showed a positive student-achievement effect. She said that, on average, students of board-certified teachers outperformed those of teachers with only regular certification by three-quarters of a month’s worth of growth in mathematics and one week in reading.

Mr. Sanders said his conclusions stemmed in part from another finding in his study: The variation in student-achievement levels for teachers within the NBPTS-certified group was greater than it was for teachers across different categories.

Joseph A. Aguerrebere Jr., the board’s chief executive officer, said most studies of board-certified teachers, such as the one by Ms. Amrein-Beardsley, also find that students of those teachers outperform their peers.

Coverage of education research is sponsored in part by a grant from the Spencer Foundation.

Events

Student Achievement Webinar What Effective Tutoring Should Look Like—and Achieve
Join this webinar to learn how to sustain effective tutoring programs that help improve students' performance in reading and math.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Mathematics Webinar
Engaging Every Learner: Strategies to Boost Math Motivation
Math Motivation Boost! Research & real tips to engage learners.
Content provided by Prodigy Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
The Ripple Effect: Mental Health & Student Outcomes
Learn how student mental health impacts outcomes—and how to use that data to support your school’s IEP funding strategy.
Content provided by Huddle Up

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Opinion Trump's Barrage of Executive Orders for Education: How Significant Are They?
A Washington insider discusses the immediate—and long-term—implications of the administration's education goals.
8 min read
The United States Capitol building as a bookcase filled with red, white, and blue policy books in a Washington DC landscape.
Luca D'Urbino for Education Week
Federal Judge Temporarily Blocks Trump Admin. From Dismantling Library Services Agency
The president referred to the agency as "unnecessary" in a March executive order, after which it started winding down many operations.
2 min read
President Donald Trump arrives at Tuscaloosa National Airport, Thursday, May 1, 2025, in Tuscaloosa, Ala.
President Donald Trump arrives at Tuscaloosa National Airport, Thursday, May 1, 2025, in Tuscaloosa, Ala. A federal judge blocked the president's attempt to dismantle the Institute of Museum and Library Services.
Manuel Balce Ceneta/AP
Federal Opinion We’re All to Blame for What Has Become of the U.S. Dept. of Education
The trouble started decades ago with a flawed plan to improve America’s schools, writes a former New York superintendent.
Michael V. McGill
5 min read
Illustration of pointing fingers.
DigitalVision Vectors<br/>
Federal Trump Ends $1 Billion in Mental Health Grants for Schools
The Education Department told grantees that their awards reflected the Biden administration's priorities.
5 min read
Guests listen as President Joe Biden speaks during an event to celebrate the passage of the "Bipartisan Safer Communities Act," a law meant to reduce gun violence, on the South Lawn of the White House, July 11, 2022, in Washington.
Guests listen as then-President Joe Biden speaks during an event to celebrate the passage of the "Bipartisan Safer Communities Act," a law meant to reduce gun violence, on the South Lawn of the White House on July 11, 2022, in Washington. The U.S. Department of Education on April 29 told grantees that had received money to train and hire more mental health professionals in schools that it wouldn't renew their grants.
Evan Vucci/AP