2009 Report Identified Dozens of Pa. Schools for Possible Cheating

By Benjamin Herold — July 11, 2011 6 min read
  • Save to favorites
  • Print

Dozens of schools across the city and state were flagged in a study of 2009 state standardized test scores that sought to use statistical analysis to ferret out possible examples of cheating on the PSSA exam.

The analysis, prepared for the Pennsylvania Department of Education in July 2009, highlights roughly 60 schools with suspicious results due to multiple statistical irregularities, including 22 Philadelphia district schools and seven Philadelphia charters.

Among the Philadelphia district schools referenced in the report is Roosevelt Middle School, which has been at the center of a controversy this year involving alleged cheating on the PSSA. In 2009, the analysis reveals, results of both the reading and math PSSA exams taken by Roosevelt’s 7th and 8th graders showed a highly unlikely number of wrong answers that were erased and changed to the correct answer. The results also showed highly improbable increases over the previous year in the percentage of students who scored proficient or advanced.

For example, the odds that the wrong-to-right erasure patterns that showed up on Roosevelt’s 7th grade reading response sheets occurred purely by chance were slightly less than 1 in 100 trillion.

The “data forensics technical report,” prepared by the Data Recognition Corporation, was made available to the Notebook by the state. At the Notebook‘s request, Andrew Porter, dean of the Graduate School of Education at the University of Pennsylvania and an expert on testing, reviewed the report and some of its data.

Porter stressed that that statistical analysis alone, without witnesses or confessions, cannot definitively prove that there was cheating. But he added that the report “describes a reasonable approach to identifying schools where there may have been cheating.”

Nevertheless, it appears that the state never followed up with any further investigations. The forensic data analysis was discontinued in 2010, although state Department of Education spokesperson Timothy Eller said it would be reinstated this year.

“We’re going to reinstitute those reports this year and they are actually going to mean something,” said Eller. He criticized the 2009 report as “convoluted.”

Earlier this year, teachers from Roosevelt and FitzSimons High told the Inquirer that they had witnessed numerous instances of cheating at their schools in 2010 and 2011. The Roosevelt teachers also spoke to the Notebook. FitzSimons was not one of the schools flagged in the state report for suspicious results in 2009.

Eller acknowledged that the state Department of Education has received two complaints from Philadelphia about possible cheating and said the department had ordered the district to investigate. Earlier this week, district officials said they found no wrongdoing at Roosevelt or FitzSimons. On Friday, district spokesperson Shana Kemp told the Inquirer that the district is investigating 15 schools and would wait until all are complete before submitting the reports to the state.

Eller sought to minimize the relevance of the 2009 report, pointing out that it was done under the previous administration of Gov. Ed Rendell and saying that “it is difficult to glean anything from that information.”

Eller stopped short, however, of saying it was wrong, emphasizing rather that it had not been presented in comprehensible form.

“It’s not information the average person would understand,” he said.

The report and accompanying analysis cost the state $113,000.

A DRC spokespersonsaid that the company could not answer any questions about the significance or context of report’s findings without state Department of Education’s permission, which was not granted.

Prominent in the DRC report is an analysis of erasures on the state tests. The study says that schools flagged for irregular erasure patterns are “of particular interest” because “these results may strongly suggest that a testing irregularity occurred in the school.”

At some Philadelphia schools, erasure patterns were flagged across numerous tested grades and subjects. For example, Olney Elementary, a K-8 school, was flagged for its erasure patterns in both reading and math in every tested grade at the school.

But the study looked at other possible evidence of test cheating as well.

A Notebook review showed roughly 60 schools across Pennsylvania received three or more “flags” in a single grade for improbable jumps in students’ performance levels and unlikely increases in schools’ scale scores on reading and math tests, as well as unusual erasure patterns. Nearly half of those schools singled out for having particularly suspicious results were in Philadelphia: 22 district schools and six seven Philadelphia charter schools had “flags” on three or more measures in a single grade.

“Multiple flags mean that the school data were highly unusual on more than one indicator,” said Penn’s Porter. “There are many ways to cheat, so it’s not wise to look at the data in only one way.”

Six of the 22 district schools had more than three “flags” for more than one grade.

Wagner Middle School in West Oak Lane, for example, was flagged in the report six times for its 7th grade results and three times for its 8th grade results. Overall, the likelihood of the erasure patterns on the school’s 8th grade response sheets in reading and math occurred purely by chance was more than 1 in 100 trillion.

From 2007-08 to 2008-09, the percentage of Wagner 8th graders scoring proficient on the PSSA exam jumped almost 14 percentage points, from 56.5 percent to 69.3 percent.

Statewide, 22 high schools had three or more flags. Three of those were Philadelphia district high schools, including Northeast High, which was flagged on every analyzed dimension of its 11th grade math results. Three local charter high schools were also flagged, including Imhotep Institute Charter High School, which was flagged seven of a possible eight times, including for a wrong-to-right erasure rate on its 11th grade math results that had a one in 10,000,000,000,000,000,000,000,000,000,000,000,000,000 chance of occurring purely by chance.

Questions about test score cheating in Philadelphia come on the heels of the district’s recent announcement that Philadelphia public schools improved their test scores for the ninth straight year in 2010-11. Over the past several months there have been several cheating scandals across the country.

Earlier this year, USA Today published a major investigation analyzing several years of test scores in six states and the District of Columbia that found statistical evidence of potential cheating. The series prompted a probe in Washington, D.C., where unusually high erasure rates were found in more than half of the schools.

In Atlanta, the state ordered an investigation into cheating after years of reporting by the Atlanta Journal-Constitution on similar unusual statistical patterns and other testing irregularities. The results, released this week, confirmed cheating in 44 of the 56 schools state investigators examined. Based on more than 2000 interviews, it named 178 educators as engaging in unethical behavior, said 80 of them had confessed, and concluded that the administration of former superintendent Beverly Hall had ignored or covered up evidence. Hall issued a statement Friday apologizing for any “shortcomings.”

A focus in both of those scandals was on suspicious erasure patterns detected by testing companies on student response sheets.

Eller, of the state Department of Education, said that new Pennsylvania Secretary of Education Ron Tomalis, appointed by Republican Gov. Tom Corbett earlier this year, has directed that forensic analysis and security audits be reinstated for the current year.

Eller gave no timetable for the completion of these reports. The PSSAs were administered in March.

To some, the need for more scrutiny of test score results grows in tandem with the intense pressure for schools to do well on standardized tests.

“As a general rule, the higher the stakes for high test performance, the more incentive to cheat, [and] the more cheating will be done,” Porter said.

Republished with permission from The Philadelphia Public School Notebook. Copyright © 2011 The Philadelphia Public School Notebook.


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Achievement Webinar
How To Tackle The Biggest Hurdles To Effective Tutoring
Learn how districts overcome the three biggest challenges to implementing high-impact tutoring with fidelity: time, talent, and funding.
Content provided by Saga Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Reframing Behavior: Neuroscience-Based Practices for Positive Support
Reframing Behavior helps teachers see the “why” of behavior through a neuroscience lens and provides practices that fit into a school day.
Content provided by Crisis Prevention Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Math for All: Strategies for Inclusive Instruction and Student Success
Looking for ways to make math matter for all your students? Gain strategies that help them make the connection as well as the grade.
Content provided by NMSI

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Accountability Sponsor
Demystifying Accreditation and Accountability
Accreditation and accountability are two distinct processes with different goals, yet the distinction between them is sometimes lost among educators.
Content provided by Cognia
Various actions for strategic thinking and improvement planning process cycle
Photo provided by Cognia®
Accountability What the Research Says More than 1 in 4 Schools Targeted for Improvement, Survey Finds
The new federal findings show schools also continue to struggle with absenteeism.
2 min read
Vector illustration of diverse children, students climbing up on a top of a stack of staggered books.
Accountability Opinion What’s Wrong With Online Credit Recovery? This Teacher Will Tell You
The “whatever it takes” approach to increasing graduation rates ends up deflating the value of a diploma.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
Accountability Why a Judge Stopped Texas from Issuing A-F School Ratings
Districts argued the new metric would make it appear as if schools have worsened—even though outcomes have actually improved in many cases.
2 min read
Laura BakerEducation Week via Canva  (1)