Education Letter to the Editor

Researcher Questions Report’s Methodology

January 29, 2013 1 min read
  • Save to favorites
  • Print

To the Editor:

As a senior evaluation specialist and research methodologist for a large Southeastern urban school district, I specialize in literacy, public policy, and school reform. I am writing to question the validity of the rankings in your annual Quality Counts report (Jan. 10, 2013).

The equal weightings assigned to the dimensions of your state rankings render them invalid. Soft and attitudinal indicators are rated the same as measurable indicators. Standards-setting is often no more than wish fulfillment, and teacher-assessment systems are error-prone. Criterion-referenced tests are not comparable from state to state.

Measurable indicators cannot be weighted the same as nonmeasurable indicators, and outcome-based indicators cannot be weighted the same as non-outcome-based indicators. “Opportunities to learn” are meaningless unless such opportunities translate into outcomes.

Lofty “standards” and “accountability measures” are irrelevant unless mechanisms are put in place to ensure that such standards are met. “Teaching indicators” are also questionable when some states rate teachers with value-added models that are little more accurate than a coin toss.

Education Week should not be an enabler of states that seek to dismantle their public education systems, using the rankings to camouflage their actions.

The methodology should be revised by regressing composite achievement against selected state demographic and resource indicators and using the residual scores to rank the states. Then, the other pieces can be factor analyzed to come up with reasonable weights.

When states with poor systems get high rankings, the politicians benefit while the students suffer.

Steven M. Urdegar

Director I/Evaluation

Assessment, Research, and Data Analysis

Miami-Dade County Public Schools

Miami, Fla.

The letter reflects the writer’s personal opinions and does not represent the Miami-Dade County schools.

A version of this article appeared in the January 30, 2013 edition of Education Week as Researcher Questions Report’s Methodology


Student Well-Being Webinar After-School Learning Top Priority: Academics or Fun?
Join our expert panel to discuss how after-school programs and schools can work together to help students recover from pandemic-related learning loss.
Budget & Finance Webinar Leverage New Funding Sources with Data-Informed Practices
Address the whole child using data-informed practices, gain valuable insights, and learn strategies that can benefit your district.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
ChatGPT & Education: 8 Ways AI Improves Student Outcomes
Revolutionize student success! Don't miss our expert-led webinar demonstrating practical ways AI tools will elevate learning experiences.
Content provided by Inzata

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: May 3, 2023
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: April 26, 2023
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: March 29, 2023
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Quiz Quiz: Test Your Knowledge of Trending Education News
How well do you know the trending news in education? Test your knowledge by taking our quiz.