As America’s “next generation” assessments tied to common-core academic standards begin to take shape through two state consortia projects, researchers and test developers alike are beginning to worry that expectations for the tests may outpace states’ technology and budgets.
Michigan and Louisiana education officials and leaders of the two consortia tasked with developing the new assessments—the 31-state SMARTER Balanced Assessment Consortium, or SBAC, and the 25-state Partnership for the Assessment of Readiness for College and Careers, or PARCC—discussed challenges to the tests at a panel held here earlier this month at the annual meeting of the National Council on Measurement in Education. The panel was organized by the Council of Chief State School Officers, one of two Washington-based groups that spearheaded the Common Core State Standards Initiative to create new common standards for college and career readiness. Those standards are now adopted by 44 states and the District of Columbia.
The tests are expected to roll out in 2014, and “the amount of innovation we’ll be able to carry off in that amount of time is not going to be that much,” warned Joseph Willhoft, the executive director of the SMARTER Balanced consortium. “There’s an expectation that out of the gate this [assessment] is going to be so game-changing, and maybe after four or five years it will be game-changing, but not immediately.”
Both consortia received grants through the federal Race to the Top Assessment competition created in the federal economic-stimulus law to develop tests based on the common standards. Both groups must develop end-of-year summative tests that can be used by any state. These will be computer-based tests for each tested grade level in mathematics and English. Each consortium is also developing optional interim benchmarking tests to allow teachers to monitor how students progress and change instruction. The optional tests are intended to go beyond simply moving questions from a paper to a computer screen. Ideally, individual test items will be tagged with the accommodations allowed for students who require them based on a disability or limited English proficiency, according to Laura M. Slover, the senior vice president for the PARCC program for the Washington-based Achieve, Inc., the consortium’s project director. Moreover, both SBAC and PARCC are developing optional tests that will adapt the difficulty of the questions as students move through the test, according to Sandy Boyd, Achieve’s communications vice president.
Short Timelines
Yet all of that is still in the works.
“One of the biggest problems I’ve seen with state assessments and national assessments is they are typically not done on a budget and a timeline that allow people to go out and do the pilot testing and tryouts that you would like,” said Mark D. Reckase, a professor of measurement and quantitative methods at the Michigan State University. “I’ve looked at the timelines for this, and they are fast; there will be incredible pressure to just get it done.”
Moreover, making sure the tests will serve their intended accountability uses has become trickier in the wake of high-profile, test-based teacher evaluations, such as those done in Los Angeles last fall. “If we are trying to look in a crystal ball about educator evaluation … That is likely to be the most difficult use of any data we put out and therefore requires the most thought and care in designing the models,” said Joseph Martineau, the director of educational assessment and accountability for the Michigan Department of Education, part of the SBAC.
PARCC plans to train thousands of teachers both in how the assessments will work and how the resulting data can be used for accountability or classroom instruction, Ms. Slover said. “One of the purposes [of the consortia project] was to really change assessment, both the way it’s done and the way it’s experienced by the students and teachers in the classroom,” Ms. Slover said. “As we think about how to transform the test to make it more usable for teachers, teachers have to embrace it and think it’s something being done for them and with them—and not to them.”
Mr. Reckase warned that mistrust of the new tests during the transition could cause delays. “There’s a tendency to want redundant systems, computerized and paper-and-pencil, … but that causes a whole other set of problems because now you have to make sure the two tests are equivalent.”
‘New Way of Doing Things’
Ms. Slover said the consortia are “betting heavily” that emerging technology will help them create tests that can balance accountability on multiple levels—from annual student achievement reporting to evaluating programs and curricula—with formative test information to help teachers tweak instruction for different students throughout the year.
“One cannot be done at the expense of the other, so balancing those is critical, ” she said. “Innovation in technology happens at lightning speed, so we are betting heavily on the fact that in four years there will be a new way of doing things, that iPads will be easily accessible or that handheld devices will be very affordable and will change the way we do testing in our schools.
At this point, she added, “technology is not really fully embedded in the world of classrooms.”
Even among classrooms with computer and Internet access, state officials agreed there are few brick-and-mortar schools that fully integrate technology into instruction, which may make it harder for students to adapt to taking tests via computer.
Scott Norton, assistant state superintendent for student and school performance for Louisiana, and a member of PARCC’s governing board, said states must be careful to get the tests right in the first shot. While jointly developing tests was intended to save states money, the grants do not include money for administering the new assessments long-term, and it will be harder to make adjustments to the tests once they are completed, because so many states will need to sign off on changes.
“In today’s world if we have a [testing] cost problem, we own that,” Mr. Norton said. “I’m not sure that holds up when we don’t own it alone. If we get into a test we can’t afford, we’re really left holding the bag.”
