Find your next job fast at the Jan. 28 Virtual Career Fair. Register now.
Opinion
Education Letter to the Editor

Researcher Questions Report’s Methodology

January 29, 2013 1 min read

To the Editor:

As a senior evaluation specialist and research methodologist for a large Southeastern urban school district, I specialize in literacy, public policy, and school reform. I am writing to question the validity of the rankings in your annual Quality Counts report (Jan. 10, 2013).

The equal weightings assigned to the dimensions of your state rankings render them invalid. Soft and attitudinal indicators are rated the same as measurable indicators. Standards-setting is often no more than wish fulfillment, and teacher-assessment systems are error-prone. Criterion-referenced tests are not comparable from state to state.

Measurable indicators cannot be weighted the same as nonmeasurable indicators, and outcome-based indicators cannot be weighted the same as non-outcome-based indicators. “Opportunities to learn” are meaningless unless such opportunities translate into outcomes.

Lofty “standards” and “accountability measures” are irrelevant unless mechanisms are put in place to ensure that such standards are met. “Teaching indicators” are also questionable when some states rate teachers with value-added models that are little more accurate than a coin toss.

Education Week should not be an enabler of states that seek to dismantle their public education systems, using the rankings to camouflage their actions.

The methodology should be revised by regressing composite achievement against selected state demographic and resource indicators and using the residual scores to rank the states. Then, the other pieces can be factor analyzed to come up with reasonable weights.

When states with poor systems get high rankings, the politicians benefit while the students suffer.

Steven M. Urdegar

Director I/Evaluation

Assessment, Research, and Data Analysis

Miami-Dade County Public Schools

Miami, Fla.

The letter reflects the writer’s personal opinions and does not represent the Miami-Dade County schools.

A version of this article appeared in the January 30, 2013 edition of Education Week as Researcher Questions Report’s Methodology

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Branding Matters. Learn From the Pros Why and How
Learn directly from the pros why K-12 branding and marketing matters, and how to do it effectively.
Content provided by EdWeek Top School Jobs
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
How to Make Learning More Interactive From Anywhere
Join experts from Samsung and Boxlight to learn how to make learning more interactive from anywhere.
Content provided by Samsung
Teaching Live Online Discussion A Seat at the Table With Education Week: How Educators Can Respond to a Post-Truth Era
How do educators break through the noise of disinformation to teach lessons grounded in objective truth? Join to find out.

EdWeek Top School Jobs

BASE Program Site Director
Thornton, CO, US
Adams 12 Five Star Schools
Director of Information Technology
Montpelier, Vermont
Washington Central UUSD
Great Oaks AmeriCorps Fellow August 2021 - June 2022
New York City, New York (US)
Great Oaks Charter Schools
Director of Athletics
Farmington, Connecticut
Farmington Public Schools

Read Next

Education Briefly Stated Briefly Stated: January 13, 2021
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Obituary In Memory of Michele Molnar, EdWeek Market Brief Writer and Editor
EdWeek Market Brief Associate Editor Michele Molnar, who was instrumental in launching the publication, succumbed to cancer.
5 min read
Education Briefly Stated Briefly Stated: December 9, 2020
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated Briefly Stated: Stories You May Have Missed
A collection of articles from the previous week that you may have missed.
8 min read