Special Report
States

Methodology

December 31, 2008 4 min read
  • Save to favorites
  • Print

About the State Policy Survey

To collect information on state education policies for Quality Counts 2009, the Editorial Projects in Education Research Center sent surveys to the chief state school officers in all 50 states and the District of Columbia. The surveys, distributed electronically on July 7, 2008, included sections regarding English-language learners, transition and alignment initiatives, and other education policy issues.

Respondents were asked to answer the questions and provide appropriate documentation to verify that the reported policies were in place at the time of the survey or for the 2008-09 school year. Such documentation might include state statutes, administrative rules, or Web links for information available online.

To ensure that answers were accurate and that consistent standards were applied uniformly across the states, EPE Research Center staff members carefully evaluated each state’s responses and documentary evidence over a 10-week period. That process often included discussions with the respondents. In the absence of documentation, the center did not award credit or assume the policy was in place.

On or around Sept. 16, the EPE Research Center sent each chief state school officer a completed survey indicating the state’s initial responses and the independent determinations by the center based on the available evidence. Officials in the state were asked to review the final answers and supply any corrections or changes that could be supported by additional documentation.

All 50 states and the District of Columbia participated in the survey. The EPE Research Center would like to thank the many dedicated individuals at state education agencies who generously contributed their time and effort in providing information for this year’s report. The Quality Counts editors and the center staff hope this examination of policies across the states will inform the efforts of researchers, legislators, policymakers, and practitioners.

How We Graded the States

For Quality Counts 2009, the Editorial Projects in Education Research Center graded the states based on results from 35 distinct indicators spanning three policy and performance categories. Those data were based on original analyses of state and federal data by the EPE Research Center as well as the center’s annual policy survey conducted during the summer of 2008. Survey responses were carefully verified with documentation, such as a state statute or other evidence provided by the state. States receive credit only for policy indicators that are determined to be in place based on the center’s review of the documentation provided.

Quality Counts 2009 assigns scores and letter grades to the states in three areas: the Chance-for-Success Index, transitions and alignment policies, and school finance. As is customary, indicators related to this year’s special focus—English-language learners—are not graded. The scoring rubric used to grade within a particular category depends on whether that category consists of numerical indicators or indicators for whether an education policy is currently in place. For the former, we employ a best-in-class approach for grading; for the latter, a policy-implementation tally.

The print edition of Quality Counts 2009 does not assign overall, summative letter grades across sections. This reflects our recent move to a modular policy survey in which we rotate survey sections on an every-other-year basis.

Best-in-Class Grading: Chance for Success, School Finance

Categories consisting of numerical indicators—Chance for Success and school finance—are graded using a best-in-class rubric. Under this approach, the leading state on a particular indicator receives 100 points, and other states earn points in proportion to the gaps between themselves and the leader.

This calculation is straightforward for indicators with a clearly bounded measurement scale. Examples of such indicators might be the zero-to-100-point scale for the percent of students proficient in reading, or states’ per-pupil expenditures expressed in positive dollar amounts.

But some of the indicators—such as those related to the equity of education spending—use more complex scales where the minimum or maximum values are not as clearly defined. For such indicators, we evaluate a particular state based on its performance relative to the minimum and maximum values on that indicator. Those indicators are scored on a 50-point base, meaning that all states start with 50 points rather than zero.

To compute a state’s score for a given category, we average points across the respective set of indicators. The Chance-for-Success and school-finance categories consist of 13 and eight indicators, respectively. On a best-in-class scale, a state’s overall score for a category can be gauged against an implicit standard where 100 points would correspond to a state that finished first in the nation on each and every measure.

Policy Grading: Transitions and Alignment

The indicators reported in the transitions and alignment section of Quality Counts 2009 consist of non-numerical measures showing whether a state has implemented a particular policy or program. This section is graded on a 50-point base, with a state’s final score reflecting the percent of tracked policies that it has implemented. A state that has enacted all policies in this category would receive the perfect score of 100 points. The 14 policies in the transitions and alignment area receive equal weight in our grading.

The Grading Scale

Using the scoring rules described above, each state receives a numerical score for each of the three indicator categories. After rounding scores to the closest whole-number values, we assign letter grades based on a conventional grading scale, as follows:

A = 93 to 100

A-minus = 90 to 92

B-plus = 87 to 89

B = 83 to 86

B-minus = 80 to 82

C-plus = 77 to 79

C = 73 to 76

C-minus = 70 to 72

D-plus = 67 to 69

D = 63 to 66

D-minus = 60 to 62

F = Below 60

Related Tags:

In March 2024, Education Week announced the end of the Quality Counts report after 25 years of serving as a comprehensive K-12 education scorecard. In response to new challenges and a shifting landscape, we are refocusing our efforts on research and analysis to better serve the K-12 community. For more information, please go here for the full context or learn more about the EdWeek Research Center.

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
The Reality of Change: How Embracing and Planning for Change Can Shape Your Edtech Strategy
Promethean edtech experts delve into the reality of tech change and explore how embracing and planning for it can be your most powerful strategy for maximizing ROI.
Content provided by Promethean
Reading & Literacy K-12 Essentials Forum Reading Instruction Across Content Disciplines
Join this free virtual event to hear from educators and experts implementing innovative strategies in reading across different subjects.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Harnessing AI to Address Chronic Absenteeism in Schools
Learn how AI can help your district improve student attendance and boost academic outcomes.
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

States More States Are Testing the Limits Around Religion in Public Schools
A wave of state policies mixing public education and religion are challenging the church-state divide in public schools.
4 min read
An empty classroom is shown at A.G. Hilliard Elementary School on Sept. 2, 2017, in Houston.
An empty classroom is shown at A.G. Hilliard Elementary School on Sept. 2, 2017, in Houston. Texas's state school board has approved a curriculum with Bible-infused lessons, the latest of a wave of state policies challenging the church-state divide in schools.
David J. Phillip/AP
States A State Changed Anti-Bias Guidelines for Teachers After a Lawsuit. Will Others?
The lawsuit filed by a conservative law firm took issue with state guidelines on examining biases and diversifying curriculum.
5 min read
Students arrive for classes at Taylor Allderdice High School in the Squirrel Hill neighborhood of Pittsburgh on Jan. 23, 2024.
Students arrive for classes at Taylor Allderdice High School in the Squirrel Hill neighborhood of Pittsburgh on Jan. 23, 2024. As part of a recent court settlement, Pennsylvania will no longer require school districts to follow its set of guidelines that sought to confront racial and cultural biases in education.
Gene J. Puskar/AP
States Oklahoma Superintendent Prays for Trump in Video He's Requiring for Students
Two of the state's largest districts say they won't show the video, in which Superintendent Ryan Walters prays for the president-elect.
2 min read
Ryan Walters, Republican state superintendent candidate, speaks, June 28, 2022, in Oklahoma City.
Ryan Walters, Republican state superintendent candidate, speaks, June 28, 2022, in Oklahoma City.
Sue Ogrocki/AP
States In Deep-Red Florida, Voters Reject Partisan School Board Races
Florida voters rejected a constitutional amendment to make school board races partisan.
2 min read
Image of a board room.
Collage by Laura Baker/Education Week (Images: DigitalVision Vectors; E+; iStock/Getty)