Editor’s Note: Click on the words highlighted in this story to pull up a definition and short research summary.
When is a reading test not a true reading test? Most of the time.
For years, cognitive scientists have lamented that the year-end standardized tests almost every state uses don’t truly measure reading comprehension. Because they rely on decontextualized reading passages, students who don’t have exposure to the topics at hand—whether mammals, archeology, travel, or literature—don’t tend to do as well as those that have been exposed to background knowledge on that topic.
As a result, a standardized reading test, they’ve pointed out, isn’t really so much a measure of how well a child can read as it is of how much they know, and the related vocabulary they’ve been exposed to.
In 2018, Louisiana became the first state to try to change the equation.
With federal approval, the state was the first to develop an English/language arts assessment that directly corresponds with its Guidebooks curriculum, a statewide, teacher-developed ELA curriculum available in grades K-8. The test measures students on content and texts that they’ve already been exposed to in class, not passages that are completely unfamiliar to them.
The idea was to create a test that aligns better to what research shows about reading comprehension and encourages teachers to focus on curriculum rather than generic test-taking skills.
“It’s not just that kids will do better in demonstrating their reading comprehension when they are tested on books that they’ve actually read before, knowledge that they’ve extensively studied,” said John White, the former state superintendent in Louisiana who oversaw the beginning stages of the pilot before leaving office in 2020. “It’s … actually assuring that kids have learned and have stored the knowledge that is conveyed by the texts that they’re reading.”
“It’s that shift in teaching that is really the heart of this project,” White said.
But while the project has big ambitions, its progress has been slow-going and its future is unclear. Few districts are participating, and the state has no immediate plans to expand the pilot. The Pelican State has also released few technical details about how the state’s students stack up on this new exam compared to those who take the more traditional reading test most students take.
The pilot could provide a wealth of information to other states and districts about what kinds of testing work, said Dan Willingham, a professor of K-12 psychology and science at the University of Virginia.
“One of the great benefits of this distributed local education system should be that we can experiment,” he said. “Different states will do things different ways and then states will learn from one another. There’s been remarkably little interest in doing that. It’s a great a idea to have this experimentation and I hope that districts and states actually follow through.”
The baseball experiment
The inspiration for the testing pilot came from an expanding research base showing that reading comprehension is heavily dependent on student knowledge. Among the best-known studies is one from 1988, known as “The Baseball Experiment.”
The researchers, Donna Recht and Lauren Leslie, divided a group of 64 junior high students into four groups based on their reading ability and knowledge of baseball. Each student read a passage about a half inning of a baseball game and were then asked to recount it.
The students who were at lower reading levels but had prior knowledge of baseball outperformed the students at higher reading levels with less knowledge of baseball, suggesting that background knowledge goes a long way in reading comprehension.
Subject-specific knowledge doesn’t automatically transfer to general knowledge, which is where the push for “knowledge building” curriculum comes in: These texts purport to build broader, conceptual knowledge by engaging students in deep, substantive ideas across content areas.
All that adds up to thinking about how a curriculum and aligned test could support the aims of literacy rather than detract from it, said Jessica Baghian, a former assistant state superintendent and chief academic policy officer at the Louisiana education department.
“If we could even the playing field so that kids, as much as possible, had an even understanding of the background knowledge that’s important for a particular study, we could really know more about how strong their reading was, rather than potentially—even though we try to avoid bias—assessing more about income and access than skill,” said Baghian, who helped oversee the development of the pilot before she left the department in 2020.
Two major developments helped lay the groundwork for the exam. The first was the creation of Guidebooks, the state’s homegrown curriculum, developed in the mid-2010s to respond to the Common Core State Standards. Teachers wrote the lessons included in the curriculum, which is uploaded on the state’s website available to anyone across the United States to use freely. Students’ reading scores were increasing under Guidebooks curriculum before the state started to align it with the pilot exam.
Second, federal policy shifted as Guidebooks came onstream, allowing states to submit plans to pilot less rigid testing models.
Louisiana was the first state to receive an Innovative Assessment waiver from the U.S. Department of Education, and in 2020, it received a $3 million grant to scale the new pilot program.
A change in teachers
The test-heavy federal requirements that have held sway for more than two decades have made teachers apprehensive about testing, and it was no different with the Louisiana pilot, said White, who studied the pilot after leaving the state education department. White is now the chief success officer at Great Minds, an organization that helps districts create and implement high-quality curriculum.
“Teachers’ view of their job is now almost universally in the core subjects anchored in standards and tests as defining instruments of success,” White said. “So the fear is that when those things change, people want to be opting into a system in which they feel secure.”
Ultimately, teachers warmed to the test because it largely supports their day-to-day work, White and Baghian said. They can continue teaching the curriculum content through the end of the course, rather than drop everything to turn to test prep, like they typically would.
The goal was to replace time spent on students practicing questions on random texts “with deep efforts to make meaning out of texts that were building knowledge that was essential for them to learn and ultimately to use in comprehending further texts,” White said. Teachers can spend more time having students discuss the novels, poems, and memoirs they read in class.
Louisiana educators aren’t the only ones who feel this way about tests. In a November EdWeek Research Center survey, 57 percent of teachers, principals, and district leaders said that state reading tests for students don’t provide an accurate assessment of reading comprehension.
Dana Talley, chief academic officer of the Lincoln Parish school district in Ruston, La., said it took some time to convince teachers in her district to focus on teaching the curriculum and not switch to test prep as the exams approached. But teachers ultimately came around because of the wealth of feedback they got under the pilot test.
Lincoln Parish is one of 20 districts using the pilot for 6th through 8th grades in the 2023-24 school year, according to the state’s 2023 performance report on the program sent to the U.S. Department of Education.
“The teachers are very aware of the data and they want that data to be good,” said Talley, who also worked at the education department with White and Baghian before joining Lincoln Parish in 2020. “What we were able to convince them of is that, one, we think the results are going to make more sense and be more tightly aligned to what you did in the classroom ... and then you’re going to get this data along the way that’s going to help you identify some gaps in some places that you really need to shore up on as you move through the year.”
But because the pilot is only operating in a small range of grades, many participating teachers still spend time practicing “cold reads” of unfamiliar books, poems, and articles. They feel a duty to prep kids for that format, which will show up on college-entrance exams.
“Teachers in particular are sensitive to the fact that ultimately when kids go on, they’re still going to encounter the ACT, which has no basis in particular bodies of knowledge or particular texts, and that still gives them some concern,” White said. “But there’s a lot of evidence that teachers are willing … to shift the way they use other tests and the way they prepare students for tests because of the nature of this test and because they are more confident that they understand what’s going to be on the test.”
An uncertain future
Although it’s been five years since the testing pilot began, it’s unclear where the test is headed or if it will be used statewide anytime soon. The 20 districts involved in the pilot in 2023-24 is an increase from the 16 districts that were involved last school year, but they account for just 28 percent of the state’s total districts. (Districts are given the opportunity to sign up at the beginning of each school year.)
It’s also available only in the 6th through 8th grades, said Thomas Lambert, the assistant superintendent of assessments, accountability, and analytics at the Louisiana education department. In total, that amounts to 10 percent of the state’s students in those grades, he said.
As of now, the state doesn’t have any immediate plans to expand the pilot beyond those grades.
“The fact that we only have 16 districts that signed on voluntarily [in 2022-23] means to me that folks are not ready for this to be assigned or adopted wide scale,” Lambert said.
The EdWeek Research Center asked principals, teachers, and other educators in a survey to explain why they felt their state reading tests did—or didn’t—provide a good gauge of students’ reading comprehension. Their responses dovetail with the concerns cognitive scientists have raised about reading tests.
- “They provide only one point of data, on a given day, and in that given context (a computerized test, surrounded by other testing students).”
- “The texts are modeled for the test, and do not take into account varying backgrounds or interests. Answering the questions requires skills related to test-taking that do not relate to comprehension of what they read.”
- “They are all forced to read the same passage, regardless of where their interest lies so if you’re not interested in the material, you won’t answer well or care.”
- “The texts are typically not interesting and based in topics for which our students have little or no background knowledge (not contextualized in previously taught science or social studies concepts).”
- “The texts and tasks can be very content-specific, so if they do not have any background information they are not able to do the task.”
- “The students are not equipped sometimes to read the amount of that time (stamina) and then respond with an answer without teacher guidance. This is not a real world situation—as no teacher should even just give an assignment such as this without some sort of scaffolding or instruction or discussion through the text. This is especially true for students with disabilities.”
The reasons why are complex to undo. Districts have expressed concerns about losing control of curriculum and uncertainty around the future of the exams, Lambert said. Choosing to use the exam effectively locks districts into using the Guidebooks materials.
“The background knowledge component is something that is important to make sure that students have an opportunity to show that they’ve learned,” Lambert said. “But [it’s also important] making sure that local districts are able to make curricular choices so that they can identify the programming that’s going to best serve their student population without an assessment defining what that curriculum needs to look like.”
Politics may further complicate things. In January, the Louisiana board of elementary and secondary education will have eight new members, and Jeff Landry, a Republican, will take over from Gov. John Bel Edwards, a Democrat.
The test also doesn’t yet have verified student data to back it up. Because it has only been used for a few years, the state isn’t going to release data showing how students performed on the pilot test compared with the traditional exams for some time, Lambert said.
Louisiana’s annual report on the program to the federal Education Department shows that students have scored similarly or slightly higher on the testing pilot than they do on the state’s typical reading tests. It’s still unclear if the test is helping close achievement gaps.
That doesn’t mean it’s dead in the water, though. With the right support, the test could have major implications for the future of standardized testing.
“We have to recognize that as important as testing is, and it is very important, the design of today’s reading test is holding teachers back from making the most out of the high-quality curriculum that so many of our school systems have chosen,” White said.
A version of this article appeared in the December 13, 2023 edition of Education Week as There’s a Design Flaw With Many Reading Tests