For Quality Counts 2008, the Editorial Projects in Education Research Center compiled data on more than 150 indicators across six distinct categories. This year’s report also reintroduces the state-by-state grading that had been a staple of Quality Counts from its start through the 10th edition of the report in 2006. The 2007 report did not assign letter grades, as we launched the new cradle-to-career framework and placed the teacher and school finance sections on hiatus for further study.
The set of indicators tracked by Quality Counts has always undergone periodic revision, a practice that makes direct comparisons of letter grades across years inappropriate. This caveat carries particular importance for the 2008 report, since we have made significant changes to both the indicator framework and the rubrics for grading the states.
Quality Counts 2008 reports state scores and letter grades in six areas: chance for success; K-12 achievement; standards, assessments, and accountability; transitions and alignment; the teaching profession; and school finance.We also provide an overall grade and score capturing a state’s showing across all six categories. The scoring rubric used to grade within a particular category depends on whether that category consists of numerical indicators (such as achievement scores) or indicators reporting whether an education policy is currently in place. For the former, we employ a best-in-class approach to grading; for the latter, a policy-implementation tally.
Best-in-Class Grading (Chance for Success, K-12 Achievement, School Finance)
Categories consisting of numerical indicators—chance for success, K-12 achievement, and school finance—are graded using a best-in-class rubric. Under this approach, the leading state on a particular indicator receives 100 points, and other states earn points in proportion to the gaps between themselves and the leader.
This calculation is straightforward for indicators with a clearly bounded measurement scale. Examples of such indicators might be the zero-to-100-point scale for the percent of students proficient in reading, or states’ perpupil expenditures expressed in positive dollar amounts.
But some of the indicators—namely, those capturing changes in achievement over time, poverty gaps, and the equity of education spending—use more complex scales, where the minimum or maximum values are not as clearly defined. For example, the percent of students who are proficient in reading may increase over time in some states but decline in others, so the observed range of changes spans both positive and negative values. For such indicators, we evaluate a particular state based on its performance relative to the minimum and maximum values on that indicator. Those indicators are also scored on a 50-point base, meaning that all states start with 50 points rather than zero.
To compute a state’s overall score for a given area, we average points across the respective set of indicators. The chance-for-success, K-12-achievement, and school finance categories consist of 13, 18, and eight indicators, respectively. On a best-in-class scale, a state’s overall score for a category can be gauged against an implicit standard, where 100 points would correspond to a state that finished first in the nation on each and every measure.
Policy Grading (Standards, Assessments, and Accountability; Transitions and Alignment; Teaching Profession)
The bulk of the state data reported in Quality Counts 2008 consists of non-numerical measures showing whether a state has implemented a particular policy or program. In these categories—standards, assessments, and accountability (SAA); transitions and alignment; and the teaching profession—states are graded on a 50-point base, with a state’s final score reflecting the percent of tracked policies that it has implemented. A state that has enacted all policies in a given category would receive the perfect score of 100 points.
The standards, assessments, and accountability policy area consists of those three subcategories: standards, assessments, and school accountability.We first calculate state scores for each of these subareas, and then average them together to arrive at the total SAA score. Similarly, the teaching-profession category is composed of three equally weighted components: accountability for quality, incentives and allocation, and building and supporting capacity. The 14 policies in the transitions and alignment area receive equal weight in our grading.
Overall Quality Counts Grade
To compute a state’s overall numerical score, we take the average of the scores across the six individual categories. Each category contributes equally to the final score.
The District of Columbia and Hawaii are single-district jurisdictions. For them, as a result, it is not possible to calculate values for the school finance section’s equity measures, which reflect the distribution of funding across school districts within a state. School finance grades thus are not reported for either of them. Their overall grades are based on results from the remaining five categories.
The Grading Scale
Using the scoring rules described above, each state receives an overall numerical score for Quality Counts and scores for each indicator category. After rounding scores to the closest whole-number values, we assign letter grades based on a conventional grading scale, as follows:
A = 93 to 100
A-minus = 90 to 92
B-plus = 87 to 89
B = 83 to 86
B-minus = 80 to 82
C-plus = 77 to 79
C = 73 to 76
C-minus = 70 to 72
D-plus = 67 to 69
D = 63 to 66
D-minus = 60 to 62
F = Below 60
