Researcher Questions Report's Methodology
To the Editor:
As a senior evaluation specialist and research methodologist for a large Southeastern urban school district, I specialize in literacy, public policy, and school reform. I am writing to question the validity of the rankings in your annual Quality Counts report (Jan. 10, 2013).
The equal weightings assigned to the dimensions of your state rankings render them invalid. Soft and attitudinal indicators are rated the same as measurable indicators. Standards-setting is often no more than wish fulfillment, and teacher-assessment systems are error-prone. Criterion-referenced tests are not comparable from state to state.
Measurable indicators cannot be weighted the same as nonmeasurable indicators, and outcome-based indicators cannot be weighted the same as non-outcome-based indicators. "Opportunities to learn" are meaningless unless such opportunities translate into outcomes.
Lofty "standards" and "accountability measures" are irrelevant unless mechanisms are put in place to ensure that such standards are met. "Teaching indicators" are also questionable when some states rate teachers with value-added models that are little more accurate than a coin toss.
Education Week should not be an enabler of states that seek to dismantle their public education systems, using the rankings to camouflage their actions.
The methodology should be revised by regressing composite achievement against selected state demographic and resource indicators and using the residual scores to rank the states. Then, the other pieces can be factor analyzed to come up with reasonable weights.
When states with poor systems get high rankings, the politicians benefit while the students suffer.
The letter reflects the writer's personal opinions and does not represent the Miami-Dade County schools.
Vol. 32, Issue 19, Page 26
Vol. 32, Issue 19, Page 26
Get more stories and free e-newsletters!
- K-12 STEM Teaching Jobs Available in Charlottesville, Virginia Join Our Free Online STEM Job Fair
- Albemarle County Public Schools, Charlottesville, VA
- K-12 STEM Teaching Jobs Available in Boston, Massachusetts Public Schools Join Our Free Online STEM
- Boston Public Schools, Boston, MA
- K-12 STEM Teaching Jobs Available in Seattle, Washington Join Our Free Online STEM Job Fair
- Mercer Island School District, Mercer Island, WA
- K-12 STEM Teaching Jobs Available in Milwaukee, Wisconsin SchoolsJoin Our Free Online STEM Job Fair
- Milwaukee Public Schools, Milwaukee, WI
- K-12 STEM Teaching Jobs Available in Birmingham, AL Public Schools Join Our Free Online STEM Job Fai
- Birmingham City Schools, Birmingham, AL