This post is by Susan Fairchild, chief knowledge officer of New Visions for Public Schools, and Chris Soderquist, founder and president of Pontifex Consulting.
Student achievement in New York State may have improved from 2012 to 2013, though this might be news to most. John Cronin and Nate Jensen of Northwest Evaluation Association recently released a terrific report, The Phantom Collapse of Student Achievement in New York. They offer a nuanced interpretation of the drop in proficiency rates on New York State’s new Common Core-aligned assessments (from 55 percent to 31 percent in reading and from 65 percent to 31 percent in math). While these statistics are certainly sobering, Cronin and Jensen explain how a decrease in proficiency is not the same thing as a decrease in performance.
New York State rolled out statewide assessments that measure the Common Core Learning Standards adopted by the State Board of Regents in 2010. Then-Commissioner of Education John King noted that the lower scores from 2012 to 2013 were “the result of the shift in the assessments to measure the Common Core Standards, which more accurately reflect students’ progress toward college and career readiness.”
Cronin and Jensen liken this to jumping ability. When you raise the bar from 3 feet to 6 feet, the 50 percent of students who cleared the 3 feet bar but not the 6 feet bar have not lost their ability to jump, they simply have a new threshold to reach; in the case of New York State, we are talking about the Common Core Standards.
Cronin and Jensen explore changes in student ability from 2012 to 2013 using the Measures of Academic Progress (MAP) in six New York school districts. MAP is a useful measure because it correlates strongly with the 2012 and the 2013 version of the New York state assessment. When they looked at changes in MAP scores from spring 2012 and spring 2013 for fourth grade mathematics, student achievement improved.
Cronin and Jensen’s report offers an important and valuable way to unravel the complicated relationships among student ability, performance standards, and proficiency levels. A second way of understanding the drop in New York proficiency rates is through the lens of systems thinking.
Many systems have similar structures that produce common and predictable patterns of behavior, what Senge calls “archetypes.” One key archetype in education is “erosion of goals.” In this scenario, gaps between current performance and desired performance sometimes generate pressure on educators to improve via corrective action. But corrective action is often difficult, time consuming, and/or expensive. One response may be to lower expectations and goals, since this can often be done quickly and cheaply. Erosion of goals relieves pressure and reduces the need for corrective action, but often results in a vicious cycle of worsening performance.
In New York State we have the opposite phenomenon: we are attempting to raise the standards through the implementation of the Common Core.
To understand the dynamism inherent to the raising and lowering of standards, we need to understand the interdependencies among: 1) student ability, 2) performance standards, 3) assessments, and 4) proficiency categories.
In systems thinking mapping, the rectangle, or “stock,” represents the accumulation of skills and content mastery (the box labeled “1"), or student ability. Think of this as a bathtub of mastery. The bathtub is filled by an inflow of learning that occurs via curriculum delivered through courses (Flow 1a). Every day a student attends school, presumably she will learn some skills and content that fill up her bathtub of content and skill mastery. But if the student does not have opportunities to apply the skill within a certain period of time after learning it, the likelihood that she will forget the content or skill is high. The outflow in the image below represents the process of content and skill mastery “atrophying” (or being lost) over time (Flow 1b).
Figure 1. The Dynamics of Content and Skill Mastery
Figure 1 helps visualize an important point: a student’s level of content and skill mastery is determined by the rate of learning and the rate of atrophying. In other words, “mastery” is a dynamic, rather than a static, state--filling and draining at the mercy of these two flows.
Educators evaluate student skill and content mastery (student ability) against performance standards (the box labeled “2")--the criteria for what students should know and be able to do. In New York State, the Common Core State Standards represent higher performance standards (Flow 2a).
Educators evaluate student skill and content mastery (ability) against performance standards through the administration of assessments (3). New York State has designed assessments that have been aligned to the Common Core standards.
Students must answer a certain percentage of questions on the assessment correctly to be categorized as “proficient” relative to the performance standards. Students who do not meet that threshold are considered “below proficient.” The dynamism associated with raising performance standards is visualized in Figure 5. When a performance standard is raised, this means that the criteria for meeting that standard is more rigorous. As a result, fewer students are categorized as meeting the criteria for “proficient” (4), and they are now considered “below proficient.”
The erosion of goals archetype represents the opposite dynamic. When high percentages of students are off track (5), pressure is often placed on districts, educators, and policymakers to lower performance standards. When this happens, it is the lowering of standards that is the mechanism that moves students into higher performance categories--not student ability.
The ease with results can be improved simply by lower performance standards should give us all pause. What we do not celebrate as often as we should is the effort of educators and policy makers to raise standards. And a systems-thinking framework helps us visualize the relationships among ability, standards, and proficiency levels--and helps us to be more attuned to the dynamism associated with these relationships.