School & District Management

Study Gives Revised View of Chicago School Improvements

By Jaclyn Zubrzycki — October 04, 2011 | Corrected: February 21, 2019 5 min read
  • Save to favorites
  • Print

Corrected: An earlier version of this story incorrectly identified the fund headed by Jason Cascarino. The correct name is the Chicago Public Education Fund.

A new study looking back at nearly 20 years of data on Chicago’s public schools suggests that changes in standards and in test-taking and data-reporting policies over time have led to misconceptions about the city’s progress in improving school and student performance.

Researchers from the Consortium on Chicago School Research, an independent research group based at the University of Chicago, said their analysis indicates that high school performance in the Chicago district improved more than elementary school performance did, contrary to what many local educators may believe, and that the achievement gap between African-American and other students has widened over the past 20 years.

The report details test scores and graduation rates from 1990 to 2009 across three eras: the launch of the city’s landmark school-decentralization effort and Paul G. Vallas’ and Arne Duncan’s separate tenures as chief executive officer of the Chicago schools. Though improvement efforts over those years have led to a number of policy changes, “in general they have built on each other,” said Elaine Allensworth, the chief researcher with the consortium.

Analyzing the data was a project that “took three Ph.D.s three years,” the researchers said in an interview, and the complexity of tracking down and analyzing the data led to one of the report’s key findings: “The publicly reported statistics used to hold schools and districts accountable are not accurate measures of progress.”

Scores can provide a useful snapshot, but the publicly reported data show something different with each change in policy, according to the report.

Stuart Luppescu, the chief psychometrist with the consortium, said the report includes all scores, not just those that are publicly reported, thus offering insights into performance trends across time.

Mr. Luppescu described some of the changes that affected the results: “In 2002-4, there was a break given in the middle of the reading section [of the Iowa Tests of Basic Skills].” A jump in scores at that time may be related to that practice, he added, and renorming over time also led to boosts in elementary school scores.

Former CEO Vallas said that he held off on renorming during his tenure to maintain rigor and had “five years of improved reading and math scores,” and that scores rose 5% after the test was renormed. The researchers agreed. “Students seem to be getting higher scores over time for the same level of skill,” Mr. Luppescu said.

In 2006, Chicago replaced the Iowa Tests of Basic Skills with the ISAT, or Illinois Standards Achievement Test, which Mr. Vallas, the former schools chief, described in an interview with Education Week as an “easier” test.

“Policies are made without a thought to how they will affect comparability of statistics,” Mr. Luppescu said.

The report says these changes in elementary-level tests made it appear “that high schools are less successful [than elementary schools] when, in fact, [high schools] are simply held to a much higher standard.” Tests taken by high school students are based on college-readiness standards, Mr. Luppescu noted.

Jason Cascarino, the chief operating officer of the Chicago Public Education Fund, a private foundation that invests in education programs in the city, said that finding was “the big reveal of this report. For a long time, we have thought that elementary schools were making incremental gains. ... But the definition of what is proficient is what matters.”

Raising the Bar

High schools have shown growth in meeting college-ready standards, increasing a full point between 2001 and 2009 on the ACT college-entrance exam. But the report says that most students in the 408,600-student school system come into high school far behind where they need to be. “If we want kids to be college-ready, we need to use college standards [sooner],” Mr. Luppescu said.

The report also finds that dropout rates have declined significantly, though changes in retention policy have caused the rate to fluctuate from year to year.

“In the early 1990s, students who entered Chicago high schools were equally likely to drop out as to graduate,” the report says. “Now they are more than twice as likely to graduate as to drop out.” While that statistic might hint at decreased standards, the rise in ACT scores suggests that academic performance in high schools has continued to rise with the graduation rate, the authors write. The study tracks dropout rates by age cohort rather than grade, which the authors said is a more accurate measure of dropout rates.

Which data are reported publicly has also changed over the course of 20 years, according to the report. “Prior to 2008, students’ test scores could be excluded from public reporting depending on their bilingual or special education status,” it says, and very often were. In the late ’90s, up to 25 percent of student scores were not reported to the public due to that practice.

That omission partially accounts for the consortium’s rosier depiction of 1990s reforms in its 2010 book, Organizing Schools for Improvement: Lessons From Chicago, which also begins its analysis in 1988 rather than 1990, as the current study does. And it makes it difficult to trace student-achievement trends over time.

Further muddying the numbers, the racial and ethnic composition of the district has also changed over time: The percentage of low-performing African-American students decreased, and the percentage of bilingual students whose scores weren’t always reported rose.

Numbers in Context

The report finds that focusing on specific student groups reveals some troubling trends. For one, the achievement gap between the lowest-performing African-American students and all other racial and ethnic groups is greater than the comparable nationwide gap and grew in all three school reform eras the report studied. Ms. Allensworth characterized the growth of that gap as “disheartening.”

Still, the study says, students of similar demographic backgrounds in other Illinois schools actually perform worse on standardized tests and are more likely to drop out. The report “does say Chicago is not doing worse by their kids than other districts as a whole,” Ms. Allensworth said.

The results lend support to the current Chicago school board’s focus on college readiness, said Mr. Cascarino.

Officials from the Chicago school district could not be reached for comment.

But the researchers conclude that there is still work to be done, as “the district has a long way to go before the average student graduates ready to succeed in college,” and that mandates won’t fix all of the school system’s programs.

A version of this article appeared in the October 05, 2011 edition of Education Week as Report Rewrites Academic-Progress Trends in Chicago


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
6 Key Trends in Teaching and Learning
As we enter the third school year affected by the pandemic—and a return to the classroom for many—we come better prepared, but questions remain. How will the last year impact teaching and learning this school
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Evaluating Equity to Drive District-Wide Action this School Year
Educational leaders are charged with ensuring all students receive equitable access to a high-quality education. Yet equity is more than an action. It is a lens through which we continuously review instructional practices and student
Content provided by BetterLesson
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Attendance Awareness Month: The Research Behind Effective Interventions
More than a year has passed since American schools were abruptly closed to halt the spread of COVID-19. Many children have been out of regular school for most, or even all, of that time. Some
Content provided by AllHere

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Letter to the Editor School Mask Mandates: Pandemic, ‘Panicdemic,’ or Personal?
"A pandemic is based on facts. A 'panicdemic' is based on fears. Today, we have both," writes a professor.
1 min read
School & District Management How 'Vaccine Discrimination' Laws Make It Harder for Schools to Limit COVID Spread
In Montana and Ohio, the unvaccinated are a protected class, making it tough to track and contain outbreaks, school leaders say.
4 min read
Principal and District Superintendent Bonnie Lower takes the temperature of a student at Willow Creek School as the school reopened, Thursday, May 7, 2020, in Willow Creek, Mont.
Bonnie Lower, a principal and district superintendent in Willow Creek, Mont., checks the temperature of a student as Willow Creek School reopened for in-person instruction in the spring.
Ryan Berry/Bozeman Daily Chronicle via AP
School & District Management Opinion 'Futures Thinking' Can Help Schools Plan for the Next Pandemic
Rethinking the use of time and place for teachers and students, taking risks, and having a sound family-engagement plan also would help.
17 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
School & District Management Opinion The Consequence of Public-Health Officials Racing to Shutter Schools
Public-health officials' lack of concern for the risks of closing schools may shed light on Americans' reticence to embrace their directives.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty