To the Editor:
As a senior evaluation specialist and research methodologist for a large Southeastern urban school district, I specialize in literacy, public policy, and school reform. I am writing to question the validity of the rankings in your annual Quality Counts report (Jan. 10, 2013).
The equal weightings assigned to the dimensions of your state rankings render them invalid. Soft and attitudinal indicators are rated the same as measurable indicators. Standards-setting is often no more than wish fulfillment, and teacher-assessment systems are error-prone. Criterion-referenced tests are not comparable from state to state.
Measurable indicators cannot be weighted the same as nonmeasurable indicators, and outcome-based indicators cannot be weighted the same as non-outcome-based indicators. “Opportunities to learn” are meaningless unless such opportunities translate into outcomes.
Lofty “standards” and “accountability measures” are irrelevant unless mechanisms are put in place to ensure that such standards are met. “Teaching indicators” are also questionable when some states rate teachers with value-added models that are little more accurate than a coin toss.
Education Week should not be an enabler of states that seek to dismantle their public education systems, using the rankings to camouflage their actions.
The methodology should be revised by regressing composite achievement against selected state demographic and resource indicators and using the residual scores to rank the states. Then, the other pieces can be factor analyzed to come up with reasonable weights.
When states with poor systems get high rankings, the politicians benefit while the students suffer.
Steven M. Urdegar
Director I/Evaluation
Assessment, Research, and Data Analysis
Miami-Dade County Public Schools
Miami, Fla.
The letter reflects the writer’s personal opinions and does not represent the Miami-Dade County schools.
