Corrected: An earlier version of this story included an outdated title for Marianne Perie. She is now the director of the Center for Educational Testing and Evaluation at the University of Kansas in Lawrence.
Includes updates and/or revisions.
When Georgia found that its bank of questions for annual student tests was a bit thin here and there last year, it turned to Kentucky and borrowed a few.
That kind of item-sharing agreement is becoming more commonplace now that most states share the same academic standards.
Georgia assessment leaders reached out to Kentucky during the 2013-14 school year because they knew that Kentucky had already begun using its new K-PREP test based on the common core. That meant the state had completed the lengthy and expensive process of designing, vetting, and field-testing hundreds of assessment items. Georgia was still developing its Georgia Milestones Assessment, which makes its debut this winter.
“When you are implementing a new assessment system, your [item] bank is not as robust as it is four or five years down the road when you’ve field-tested more items and have more freedom to put a [test] form together,” said Melissa Fincher, Georgia’s deputy superintendent for assessment and accountability.
Georgia has used only a few of Kentucky’s items so far, but it has benefited in particular from the Bluegrass State’s assortment of open-ended questions, since Georgia’s test has always been multiple-choice only, Ms. Fincher said. She is talking with a couple of other states about item-sharing agreements.
Item-sharing isn’t brand new; some states were doing it before they adopted the common core. Even with different academic standards, there was enough commonality of content that states could share some test questions. But testing officials say that sharing the Common Core State Standards—which 43 states and the District of Columbia now do—makes it easier.
“That was always the hurdle on sharing items,” said Ken L. Draut, Kentucky’s associate commissioner for assessment and accountability. “If you don’t have the same standards, it is much harder to link with another state.”
Cost and Time Savings
Kentucky has one item-sharing agreement with Georgia and another with Illinois. A key benefit, Mr. Draut said, is the time and money saved on item development. It typically takes nine to 18 months to have questions ready for inclusion on a test.
Even once a state has a fully stocked item bank, it must be continually replenished to ensure secure items are available, he said.
“You can’t use the same ones over and over. Say you get to the second or third year of a geometry test, and you’re running short, and all you need is a few more good items for 7th grade,” Mr. Draut said. “It’s good to have another place to turn to instead of undertaking a whole development process.”
Kentucky won’t be able to keep using items from Illinois’ newest, however. Illinois is moving to the Partnership for Assessment of Readiness for College and Careers, or PARCC, test this year, and that consortium’s policy prohibits nonmember states—such as Kentucky—from using individual test items.
Nonmember states may pay to use the test forms, but they must use the whole test, not just items from it, said PARCC spokesman David Connerty-Marin.
Matt Vanover, a spokesman for the Illinois education department, said Illinois still could share items from its old test with Kentucky, but won’t be able share PARCC items.
Kentucky and other states are considering yet another kind of item-sharing agreement, though. Through the Council of Chief State School Officers, some of the states that have adopted the Next Generation Science Standards have been discussing the possibility of building a shared bank of science-test items.
In some cases, item-sharing can be facilitated by having an assessment vendor in common. Before the common core, the Washington-based American Institutes for Research built tests for Delaware, Hawaii, and Oregon, and those states agreed to share items, said Jon Cohen, the president of the AIR’s assessment division. The three states retained ownership of the items, and the AIR acted as a “steward,” he said. Now, Arizona, Florida, and Tennessee—for which the AIR is building new tests or providing subcontracted services in test design—are licensing the item bank the AIR designed for Utah, another new test customer, Mr. Cohen said.
Customers can benefit from the cost savings that a vendor derives from a shared item bank, said Marianne Perie, the director of the Center for Educational Testing and Evaluation at the University of Kansas in Lawrence. Her organization is crafting new tests for Alaska and Kansas, and those states have agreed to share items.
There are tricky aspects to an item-sharing agreement, however. One is ensuring the right balance between questions that are generalizable to multiple states and those that are tailored to one, Ms. Perie said.
A reading passage about the Great Plains, for instance, might not pass muster in Alaska, Ms. Perie said. State assessment officials routinely conduct “bias and sensitivity” reviews to weed out items that refer to unfamiliar or inflammatory concepts.
On the other hand, one state can benefit from a question written with another in mind. The CETE sent a team to Alaska to write test questions last summer, and it included a woman who had raced in the famous Iditarod, Ms. Perie said. The reading passage she wrote about her experience proved fascinating and welcome in Kansas, too, Ms. Perie said.
An important element of item-sharing agreements is the coordination of the timing when test items are released to the public, said Scott Norton, a former assessment director in Louisiana who now oversees assessment and accountability at the CCSSO. If one state makes a test question public before its partners do, he said, it destroys the security of that question in states that are still using it.
