Researcher Questions Report's Methodology

Article Tools
  • PrintPrinter-Friendly
  • EmailEmail Article
  • ReprintReprints
  • CommentsComments

To the Editor:

As a senior evaluation specialist and research methodologist for a large Southeastern urban school district, I specialize in literacy, public policy, and school reform. I am writing to question the validity of the rankings in your annual Quality Counts report (Jan. 10, 2013).

The equal weightings assigned to the dimensions of your state rankings render them invalid. Soft and attitudinal indicators are rated the same as measurable indicators. Standards-setting is often no more than wish fulfillment, and teacher-assessment systems are error-prone. Criterion-referenced tests are not comparable from state to state.

Measurable indicators cannot be weighted the same as nonmeasurable indicators, and outcome-based indicators cannot be weighted the same as non-outcome-based indicators. "Opportunities to learn" are meaningless unless such opportunities translate into outcomes.

Lofty "standards" and "accountability measures" are irrelevant unless mechanisms are put in place to ensure that such standards are met. "Teaching indicators" are also questionable when some states rate teachers with value-added models that are little more accurate than a coin toss.

Education Week should not be an enabler of states that seek to dismantle their public education systems, using the rankings to camouflage their actions.

The methodology should be revised by regressing composite achievement against selected state demographic and resource indicators and using the residual scores to rank the states. Then, the other pieces can be factor analyzed to come up with reasonable weights.

When states with poor systems get high rankings, the politicians benefit while the students suffer.

Steven M. Urdegar
Director I/Evaluation
Assessment, Research, and Data Analysis
Miami-Dade County Public Schools
Miami, Fla.

The letter reflects the writer's personal opinions and does not represent the Miami-Dade County schools.

Vol. 32, Issue 19, Page 26

Published in Print: January 30, 2013, as Researcher Questions Report's Methodology
Related Stories
Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.
Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.

Back to Top Back to Top

Most Popular Stories