Accountability, or Mastery?
The assessment trade-off that could change the landscape of reform.
There is general agreement in American public education that the conventional high school is badly broken and in need of a complete overhaul. Some even argue that it is obsolete and should be replaced with new models of secondary education.
Overwhelming evidence of institutional failure has triggered this almost universal alarm, and educators and policymakers have tried to solve the problem in a number of ways: by breaking bigger schools into smaller “academies” or creating “schools within schools”; by instituting a senior-year or culminating project; by raising standards, increasing rigor, and adding Advanced Placement courses; and by adopting high-stakes tests that students must pass to graduate. Positive results have been negligible.
But now, two states—New Hampshire and Rhode Island—are pioneering a promising “new” approach to redesigning high schools by instituting performance-based or competency-based assessments that link content to skills and use multiple measures (not just a statewide standardized test) to evaluate students’ proficiency.
The dominant school improvement strategy of standards-based accountability stresses academic content and standardized tests. Performance assessment combines content with skills, and requires students to carry out tasks to demonstrate mastery of both. The tasks generally fall into three categories—performance, portfolios, and projects—and are designed to encourage students to think and to solve problems through hands-on activities. To demonstrate mastery, students may, for example, perform a musical recital, make a significant oral presentation, write a major essay, or submit a portfolio of cumulative work from different disciplines. The point is that students must produce evidence that shows they understand the content and can apply it in real-world situations.
Teachers are trained to judge the quality of student work using rubrics or criteria, which they often help develop. And what they learn from observing and rating student performance is directly related to teaching: how well the students have or have not learned the material, and where further work is most needed. Students become active participants in their learning, gathering information and analyzing, evaluating, relating, and applying it.
Because performance assessments focus on tasks linked to curricula and state standards, they drive testing in a positive way; the tasks are worth teaching to, because their mastery indicates that students have learned both the material and how to use it. In short, the assessment is in itself a form of teaching and learning.
The New Hampshire and Rhode Island systems have much in common, but differ in detail. Together, they embody the essential elements of a comprehensive performance- or competency-based system.
Both states use several metrics to assess students, including portfolios (often digital), exhibitions, certificates of initial mastery, and end-of-course examinations. Students still must take and pass courses in core academic subjects aligned with the state standards, but the courses have been redesigned, from content-heavy, traditional, top-down instruction, to competency-based.
Students take state exams, but graduation is not dependent on the scores. The scores count toward graduation, but never enough to prevent a student from graduating. In Rhode Island, they cannot count for more than 10 percent of the overall determination of proficiency for graduation.
Both states encourage students to pursue their interests and passions both inside and outside of school. The Rhode Island program says that most student passions “can reasonably become a subject for a demonstration of proficiency in content and applied skills.”
Under its program, New Hampshire will become the first state to regulate the elimination of the Carnegie unit. Student achievement will instead be assessed through a demonstration of mastery of the material in the course and the ability to apply it. Rhode Island retains the term “Carnegie units,” but it now applies to courses that are based on competence rather than seat time. The two states also emphasize real-world learning and allow students to pursue alternative approaches outside the classroom to acquire knowledge and skills.
Like any substantial change in education, performance assessments face stiff resistance from powerful interest groups with a stake in the current system. These include the testing industry, school boards, teachers’ unions, and colleges and universities committed to traditional admissions requirements.
Psychometricians are generally critical of performance assessments because they don’t meet the conventional demands of validity, reliability, and objectivity, and therefore cannot be used for individual and school accountability purposes or to compare one student or school with another. Because public schools operate with public funds, accountability is a paramount concern, and the basis for the ever- increasing use of standardized tests.
The federal No Child Left Behind Act has ratcheted up the pressure of standards-based accountability: doubling the amount of testing, raising achievement goals, increasing the penalties for poor performance, and reducing the little flexibility schools have had to experiment and innovate. In such a policy environment, with very high stakes involved, state-level policymakers and district and school administrators are reluctant to venture far outside the box. And performance assessments are still pretty far outside the box.
These statewide programs may be a catalyst for change. As the oft-repeated dictum holds, you get what you test for. In the same way that conventional assessments have, performance assessments will drive curriculum and instruction, which strongly influence scheduling, pedagogy, and teaching assignments. This, in turn, affects the distribution of funds and becomes an issue in collective bargaining.
The chain of effects surely will include student motivation and engagement, and the relationships between and among students and teachers. Changes in these fundamental aspects of the high school could eliminate the traditional barriers between academic disciplines and replace top-down leadership with distributive leadership. Similarly, out-of-school activities emphasizing real-world learning could bridge the worlds of school and work.
Although the new assessments in New Hampshire and Rhode Island are shifting the conversation in schools from being teacher-centered to student-centered, and are slowly changing how schools operate and how teachers teach, fundamental aspects of high schools remain unchanged. The new assessment systems have not yet changed the structure of the high school, nor have they affected the power relationship between unions and district leaders. It remains to be seen whether they will ultimately create the change some envision, and, if so, to what degree.
For now, no state has used performance assessments long enough to accumulate a persuasive record of success. Some limited but encouraging indicators, however, can be drawn from the experience of four New Hampshire schools that piloted a competency-based assessment project in 2000, incorporating most of the provisions of the state’s new assessment system.
The report suggests that results in these schools were very positive, with high levels of performance for almost all indicators. It adds: “While there is some year-to-year variation, as is to be expected, the data demonstrate a clear trend that more students are staying in high school, more students are graduating from high school, more students are better prepared for success beyond high school, and more students are planning to go on to postsecondary education.”
The data also showed improvement in SAT combined verbal and math scores and in scores in English language arts and mathematics on the New Hampshire Educational Improvement and Assessment Program’s tests. All four schools showed improvement in language arts, while three showed improvement in mathematics. There was a general trend toward more students’ scoring “advanced” or “proficient,” the two highest categories, and an equally important reduction in the percentage scoring “novice,” the lowest of the categories. All four schools exceeded the state average for the percentage of students scoring advanced or proficient in language arts, and two were above the state average in mathematics. In addition, over the five-year period studied, the dropout rate was cut in half in two of the schools and improved in a third.
This pattern of improvement occurred despite high staff turnover in three of the four schools, which is notable, given findings that stable leadership committed to reform is key to the successful implementation of performance-based assessment systems. Research shows that administrator turnover has a substantial negative effect on reform and on student achievement and postsecondary outcomes.
Can nontraditional strategies like performance assessment coexist and survive in an education system dominated for 15 years by a reform strategy that gives top priority to accountability? If so, the public, led by the nation’s educators and policymakers, must be willing to rethink the purposes and practices of public schools.
Performance assessments will never be objective enough, or standardized enough, to be totally reliable in comparing schools and students. Neither the rubrics nor the teachers who use them will ever be 100 percent objective in evaluating students. The human factor is an inextricable component of performance-assessment systems. So for the larger society, the trade-off may be between two goals: holding schools and students accountable, or demonstrating student mastery and encouraging good habits of mind and behavior.
For the good of society and the rising generations of young people, leaders must find the imagination and flexibility to use different instruments to accomplish both of these important goals.
Vol. 26, Issue 34, Pages 36, 44Published in Print: April 25, 2007, as Accountability, or Mastery?