Responding to a potential cheating scandal uncovered by a recent newspaper investigation, Texas officials last week announced a sweeping review of test security and plans for a new monitoring scheme for the state accountability system, which has served as a model for other states as well as the federal No Child Left Behind Act.
“We take cheating very seriously in our state, and we will be taking whatever actions are necessary to maintain the integrity of our testing program,” Commissioner of Education Shirley Neeley said at a Jan. 10 press conference called to outline the state’s response. “This whole situation is embarrassing, … but we’re not putting our heads in the sand over this.”
The move came after an analysis of test scores by The Dallas Morning News found that results at as many as 400 schools out of 7,700 statewide—including one celebrated Houston elementary school—were suspect. The newspaper, which used a regression analysis of all school-level results on the Texas Assessment of Knowledge and Skills for 2003 and 2004, outlined unlikely leaps in TAKS scale scores from one year to the next or students’ inability to maintain high levels of achievement as they advanced in school. The analysis compares relationships between variables to gauge or predict consistencies, such as a school’s performance in reading over several grades.
Ms. Neeley said the newspaper’s use of scale scores—the average score a school achieves on a given test, as opposed to passing rates, may not provide the most accurate measure of how a school is performing. The commissioner acknowledged, however, that the state must devise a common formula for identifying questionable results.
Although reports of cheating on standardized tests are not uncommon across the country, the extent of the allegations in Texas appear to be unprecedented, observers say. The latest claims have renewed appeals from experts for states to institute better oversight of testing systems.
“This is an ethical failure on the part of the U.S. education system, not just on Texas,” said Daniel Koretz, a testing researcher at Harvard University. “There isn’t an expectation in this country that we will carefully evaluate the impact of holding people accountable for scores.”
Texas has long used test scores to determine whether schools are making enough progress in raising student achievement, and for issuing penalties to those that fail to do so adequately. Under the federal law, state test scores are a central factor in whether schools meet “adequately yearly progress.”
Monitoring Criticized
Texas does not regularly monitor or review school or district results on the statewide assessment, according to Debbie Graves Ratcliffe, a spokeswoman for the Texas Education Agency. Since it began testing more than two decades ago, the state agency has conducted limited reviews of school data, primarily to find instances in which large numbers of students were exempted from taking the test or absent on test day. It has also investigated specific allegations of cheating or cases of significant statistical anomalies.
Districts are primarily responsible for monitoring their results and investigating any irregularities or allegations of impropriety. They can then refer cases to the state agency for further review or action. The state probes only a handful each year.
Texas’ limited monitoring is standard practice throughout the nation, according to George Madaus, a senior fellow with the National Board on Educational Testing and Public Policy, a private body based at Boston College.
“The whole testing situation is almost totally unregulated,” said Mr. Madaus, who, along with other researchers, has pushed for better monitoring for years. “We’re dealing with a very useful, but a very fallible, technology.”
Officials in Pittsburgh and Indiana, however, recently announced the hiring of independent contractors to monitor their testing programs and make recommendations for detecting potential cheating.
Commissioner Neeley said Texas, too, would hire an outside expert to review its testing policies and procedures and would craft measures for analyzing test results for the nearly 3 million students in grades 3-11 who participate in TAKS each year.
Educators found to have cheated, or those who failed to report cheating, could face disciplinary measures, ranging from a formal reprimand to suspension of their professional certificates to jail time, she said.
National Model Tainted
The state’s plan followed similar announcements by superintendents in Houston and Dallas, Texas’ largest school districts. The Dallas Morning News analysis found irregularities in dozens of schools in those districts.
In Houston, Wesley Elementary, which gained national acclaim under then-Superintendent Rod Paige for getting nearly all students from the poor neighborhood it serves to grade level in reading, was among those accused, along with two other affiliated schools that form a charter school district known as Acres Homes. Mr. Paige is wrapping up four years as the U.S. secretary of education.
According to the newspaper’s analysis of 2003 reading scores, Wesley’s 5th graders were among the top performers in the state, scoring in the top 10 percent for the grade level. The following year, as 6th graders at Houston’s M.C. Williams Middle School, they fell to the bottom 10 percent in that subject and on the mathematics test, a trend that was not repeated elsewhere in the state.
“A Dallas Morning News investigation has found strong evidence that at least some of the success at Wesley and two affiliated schools comes from cheating,” the newspaper said in an article dated Dec. 31.
A former Wesley Elementary teacher, Donna Garner, had reported to the Houston school board 19 months earlier that teachers at the school were encouraged over several years to cheat, according to state and district documents.
In November, the district appointed an independent counsel to investigate Ms. Garner’s claims.
The school attracted considerable attention throughout the 1990s, particularly for its adherence to Direct Instruction, a scripted commercial reading program. Its principal at the time, Thaddeus Lott, was hailed in news stories for helping his students beat the odds.
As governor of Texas, George W. Bush drew on the district’s purported success to support his educational accountability program, which became a national model and the basis for the No Child Left Behind Act instituted after he became president.
Wesley Elementary’s success prompted accusations of cheating then as well, but the claims were dismissed after a state investigation. In a 1998 interview with Education Week, Mr. Lott chalked up those challenges to “ignorance,” saying “it is racist” to assume that poor minority children can’t learn.
Mr. Lott was not available for comment last week.
‘Highly Questionable’
After the latest charges, Abelardo Saavedra, who recently took over as the superintendent of the 212,000-student Houston district, said in a statement to The Dallas Morning News that the district agrees that the performance of Wesley and two other schools that form the school system’s Acres Home charter district was “highly questionable.”
Mr. Saavedra announced Jan. 6 that the district would establish an inspector general’s office to institute new controls over test procedures and to investigate any suspected wrongdoing. The district’s internal auditor, Robert Moore, a certified fraud examiner, will head the office.
The Houston district is also planning to hire outside monitors to visit schools on test days, some assigned to specific schools and others randomly throughout the district. Commissioner Neeley said the state was considering a similar strategy.
District superintendents in Dallas and Fort Worth, where the newspaper found a number of cases of questionable test-score trends, have also unveiled plans for stricter monitoring.
But some critics say the measures may be inadequate, given the state’s and districts’ interest in showing that student achievement is improving.
“To think the TEA is going to monitor the quality of data from districts is like asking the fox to guard the chicken coop,” charged Walter M. Haney, a professor of education at Boston College who has worked to debunk what he calls “The Texas Miracle” in raising student achievement. “Even if there’s not outright fraud, where people become so obsessed with raising test scores on one relatively narrow test,” cheating and other improprieties are likely to occur.
