What’s next? With all the frustration surrounding NCLB and big end of year tests, what’s the new policy framework that will replace grade level testing?
For a state ready to embrace personalized and competency-based learning, what are the next steps?
This post suggests the use of assessment pilots and innovation zones where groups of schools apply and become authorized to operate alternative assessment
systems. But first, some background.
Jobs to be done. To get at the heart of value creation, Clayton Christensen taught us to think about the job to be done. Assessment plays four important roles in school systems:
-
Inform learning: continuous data feed that informs students, teachers, and parents about the learning process.
-
Manage matriculation: certify that students have learned enough to move on and ultimately graduate.
-
Evaluate educators: data to inform the practice and development of educators.
-
Check quality: dashboard of information about school quality particularly what students know and can do and how fast they are progressing.
Initiated in the dark ages of data poverty, state tests were asked to do all these jobs. As political stakes grew, psychometricians and lawyers pushed for
validity and reliability and the tests got longer in an attempt to fulfill all four roles.
With so much protest, it may go without saying but the problem with week long summative tests is that they take too much time to administer; they don’t
provide rapid and useful feedback for learning and progress management (jobs 1&2); and test preparation rather than preparation for college, careers,
and citizenship has become the mission of school. And, with no student benefit many young people don’t try very hard and increasingly opt out.
But it is no longer necessary or wise to ask one test to do so many jobs when better, faster, cheaper data is available from other sources.
What’s new? There have been six important assessment developments since NCLB was enacted in 2002:
-
Student internet access has improved sufficiently to support an expectation of frequent online learning and assessment.
-
Performance assessment tools make it easier to construct, manage, and assess projects and standards-aligned prompts (see features onLDC CoreTools, and Buck Institute).
-
Embedded assessments are incorporated into many forms of digital content.
-
Formative assessment systems have improved dramatically. Platforms like MasteryConnect,Acuity, Edmodo, OpenEd, and Schoology make it easy to build, administer, and share
standards-aligned assessments. -
Adaptive assessment, such as MAPS from NWEA, is widely used. Adaptive learning, which combines adaptive
assessment and targeted tutoring, is gaining widespread use in blended learning models. Providers includeDreamBox (K-8 math) i-Ready from Curriculum Associates (k-8 math and
reading), ALEKS from McGraw Hill (mostly secondary use). -
Broader aims of student success, including self management and relational skills, are widely recognized as important and are being incorporated
into state and district goals. The hard to measure skills and dispositions require broader feedback systems than traditional standardized testing.
Progress has been faster on these individual vectors than on combining formative assessment from various sources into useful information for students,
teachers and parents. Simply tracking a checklist of skills won’t cut it, solutions will need to include micro-standard tagging grouped into skill
clusters, a two or three layer hierarchy supporting the ability to combine fine grain and broader performance assessments. Examples of progress on
alignment through tagging include:
-
Houston ISD has had some success requiring content and assessment partners to use the IMS Global Thin Common Cartridge, which enables content
interoperability using Learning Tools Interoperability (LTI). -
Some learning platforms combine multiple forms of assessment around units of instruction (see feature on Buzz from Agilix).
-
Scootle
is an Australian library of tagged learning resources suggesting that a state or network could encourage strong alignment.
And, as noted last month, Without going through the
trouble of tagging data or putting up with the limitations of a closed system, it is increasingly possible to subject to correlate data sequences from
different digital learning applications after the fact by analyzing thousands of data points. IMS’s Caliper Analytics standards support both post hoc and real-time data feeds for millions of
students daily.
Next steps. The first step for a state, according to the
The iNACOL State Policy Frameworks
is for the state board and/or legislature to, “Redefine Carnegie Units or credits as competencies aligned to state academic standards.” The Great Schools Partnership has supported movement towards “proficiency-based diplomas” in
Connecticut, New Hampshire, Maine, Vermont, (see a summary of state policies).
For states ready to embrace personalized and competency-based learning, CompetencyWorks, an online community and
resources supported by iNACOL, outlines five components of competency-based education (CBE):
-
Students advance upon mastery.
-
Competencies include explicit, measurable, transferable objectives that empower students.
-
Assessment is meaningful and positive learning experience for students.
-
Students receive timely, differentiated support based on their individual needs.
-
Outcomes include application of knowledge and development of important skills and dispositions.
The definition sets a high bar by requiring well stated learning targets, powerful learning experiences, better reporting systems, and new rules for
matriculation management. It focuses primarily on the first two jobs: student learning and progress management.
High school options. There are two test-based options to replace an end of the year standardized test in high school:
-
End of course exams. A system of end of course exams has advantages during secondary education as course taking broadens and rates of progress
diverge.They can be offered on a frequently scheduled basis so students can take them when ready. Like the New York Regents and Advanced Placement
exams, end of course test have been most common in high school but could be used in middle grades. They facilitate incorporation of online, AP, and
dual enrollment courses and promote reliability but lock in the structure of courses and credits and lack the validity and authenticity of student
work. -
College entrance exams. Since many college bound students take the SAT or ACT, they are used as part of the state testing system. Eighteen states
already administer the ACT and three more make it available to districts.
The Great Schools Partnership suggests two curriculum-aligned
approaches:
-
Body of evidence: using common criteria, educators review a portfolio of student work. It encourages students to take greater ownership over the
learning process but is time-consuming and can be inconsistently executed. -
Compilation of assessments: using a common formula to aggregate assessment results on performance indicators over time. Results are relatively
straightforward and easy to calculate but it may encourage students to narrowly focus on grades and numerical indicators of success.
The test based options tend to be more reliable while the student work product approaches are more valid and authentic. A jurying process for portfolios
can boost reliability but adds cost and complexity. A state could combine both approaches by requiring a series of several ACT tests (Plan, Explore,
Compass) and incorporating them into a body-of evidence or complication of assessments approach. A state could also combine short end of course exams with
a body of evidence approach to gain affordable validity and reliability.
New K-8 systems. When federal education legislation is reauthorized it is likely to retain the annual testing requirement in grades 3-8. That’s a good
thing because we don’t have alternative ways to comparably measure student learning levels and growth rates to verify school quality (job #4).
But the beginning of the end of the big test is in sight and two replacement options are being contemplated. The first is assessment flexibility pilots.
New Hampshire’s Performance Assessment of Competency Education (PACE) is the first
federally approved accountability system that offers a reduced level of standardized testing together with locally developed common performance assessments
designed to support deeper learning through competency education.
This year, four PACE implementing districts gave the Smarter Balanced assessment once in elementary school, once in middle school and once in high school,
in three grades instead of seven. The state is also conducting a readiness survey to identify the next eight districts to include in the pilot. Next year
PACE districts will administer common and local performance assessments developed by the districts. For comparability, Smarter Balanced Achievement Level Descriptors are the basis for
establishing cut scores. Regional calibration scoring sessions will be conducted to build inter-rater reliability and consistency in scoring across
districts.
Maria Worthen, iNACOL (where I’m a director) explains, “Far from fewer assessments, New Hampshire must continue to
administer the SBAC in three grade spans to evaluate and validate the new system of assessments. No other state is close to gaining approval for a similar
assessment for accountability purposes.” Federal reauthorization discussions have contemplated inclusion of an innovative assessment pilot in the bill.
The other alternative path forward is the use of innovation zones where systems earn the right to opt out of testing by demonstrating that their proposed
alternative assessment system can report comparable results. Districts, networks, and consortia could apply to become a ‘certified assessment system.’
These systems would need to certify that they could report learning levels and growth rates for students in ways comparable to (but not limited to) state
administered tests before being admitted into the innovation zone.
As noted last fall, the innovation zone strategy could be implemented by an a statewide
authorizer using performance contracting to engage districts and networks in experiments with new delivery strategies, talent development structures, and
measurement systems. With a combination of pressure and incentives over five to 10 years, an innovation zone could become the primary delivery and
accountability system for the state.
In summary, assessment pilots would run parallel systems to demonstrate comparability in action while innovation zones would require demonstrated
comparability before receiving a waiver. Federal reauthorization of elementary and secondary act should include provisions for both of these options. Susan
Patrick, iNACOL, said, “States desperately need the flexibility to be able to do this to innovate.” After a few dozen
districts in a handful of states demonstrate new lightweight testing systems the rest of the states are likely to follow suit.
For more see:
Mastery Connect, Edmodo are Learn Capital portfolio companies where Tom is a partner. Agilix, Curriculum Associates, and DreamBox Learning are Getting Smart Advocacy Partners. Tom is a director at iNACOL.