School & District Management

Study Gives Revised View of Chicago School Improvements

By Jaclyn Zubrzycki — October 04, 2011 | Corrected: February 21, 2019 5 min read
  • Save to favorites
  • Print

Corrected: An earlier version of this story incorrectly identified the fund headed by Jason Cascarino. The correct name is the Chicago Public Education Fund.

A new study looking back at nearly 20 years of data on Chicago’s public schools suggests that changes in standards and in test-taking and data-reporting policies over time have led to misconceptions about the city’s progress in improving school and student performance.

Researchers from the Consortium on Chicago School Research, an independent research group based at the University of Chicago, said their analysis indicates that high school performance in the Chicago district improved more than elementary school performance did, contrary to what many local educators may believe, and that the achievement gap between African-American and other students has widened over the past 20 years.

The report details test scores and graduation rates from 1990 to 2009 across three eras: the launch of the city’s landmark school-decentralization effort and Paul G. Vallas’ and Arne Duncan’s separate tenures as chief executive officer of the Chicago schools. Though improvement efforts over those years have led to a number of policy changes, “in general they have built on each other,” said Elaine Allensworth, the chief researcher with the consortium.

Analyzing the data was a project that “took three Ph.D.s three years,” the researchers said in an interview, and the complexity of tracking down and analyzing the data led to one of the report’s key findings: “The publicly reported statistics used to hold schools and districts accountable are not accurate measures of progress.”

Scores can provide a useful snapshot, but the publicly reported data show something different with each change in policy, according to the report.

Stuart Luppescu, the chief psychometrist with the consortium, said the report includes all scores, not just those that are publicly reported, thus offering insights into performance trends across time.

Mr. Luppescu described some of the changes that affected the results: “In 2002-4, there was a break given in the middle of the reading section [of the Iowa Tests of Basic Skills].” A jump in scores at that time may be related to that practice, he added, and renorming over time also led to boosts in elementary school scores.

Former CEO Vallas said that he held off on renorming during his tenure to maintain rigor and had “five years of improved reading and math scores,” and that scores rose 5% after the test was renormed. The researchers agreed. “Students seem to be getting higher scores over time for the same level of skill,” Mr. Luppescu said.

In 2006, Chicago replaced the Iowa Tests of Basic Skills with the ISAT, or Illinois Standards Achievement Test, which Mr. Vallas, the former schools chief, described in an interview with Education Week as an “easier” test.

“Policies are made without a thought to how they will affect comparability of statistics,” Mr. Luppescu said.

The report says these changes in elementary-level tests made it appear “that high schools are less successful [than elementary schools] when, in fact, [high schools] are simply held to a much higher standard.” Tests taken by high school students are based on college-readiness standards, Mr. Luppescu noted.

Jason Cascarino, the chief operating officer of the Chicago Public Education Fund, a private foundation that invests in education programs in the city, said that finding was “the big reveal of this report. For a long time, we have thought that elementary schools were making incremental gains. ... But the definition of what is proficient is what matters.”

Raising the Bar

High schools have shown growth in meeting college-ready standards, increasing a full point between 2001 and 2009 on the ACT college-entrance exam. But the report says that most students in the 408,600-student school system come into high school far behind where they need to be. “If we want kids to be college-ready, we need to use college standards [sooner],” Mr. Luppescu said.

The report also finds that dropout rates have declined significantly, though changes in retention policy have caused the rate to fluctuate from year to year.

“In the early 1990s, students who entered Chicago high schools were equally likely to drop out as to graduate,” the report says. “Now they are more than twice as likely to graduate as to drop out.” While that statistic might hint at decreased standards, the rise in ACT scores suggests that academic performance in high schools has continued to rise with the graduation rate, the authors write. The study tracks dropout rates by age cohort rather than grade, which the authors said is a more accurate measure of dropout rates.

Which data are reported publicly has also changed over the course of 20 years, according to the report. “Prior to 2008, students’ test scores could be excluded from public reporting depending on their bilingual or special education status,” it says, and very often were. In the late ’90s, up to 25 percent of student scores were not reported to the public due to that practice.

That omission partially accounts for the consortium’s rosier depiction of 1990s reforms in its 2010 book, Organizing Schools for Improvement: Lessons From Chicago, which also begins its analysis in 1988 rather than 1990, as the current study does. And it makes it difficult to trace student-achievement trends over time.

Further muddying the numbers, the racial and ethnic composition of the district has also changed over time: The percentage of low-performing African-American students decreased, and the percentage of bilingual students whose scores weren’t always reported rose.

Numbers in Context

The report finds that focusing on specific student groups reveals some troubling trends. For one, the achievement gap between the lowest-performing African-American students and all other racial and ethnic groups is greater than the comparable nationwide gap and grew in all three school reform eras the report studied. Ms. Allensworth characterized the growth of that gap as “disheartening.”

Still, the study says, students of similar demographic backgrounds in other Illinois schools actually perform worse on standardized tests and are more likely to drop out. The report “does say Chicago is not doing worse by their kids than other districts as a whole,” Ms. Allensworth said.

The results lend support to the current Chicago school board’s focus on college readiness, said Mr. Cascarino.

Officials from the Chicago school district could not be reached for comment.

But the researchers conclude that there is still work to be done, as “the district has a long way to go before the average student graduates ready to succeed in college,” and that mandates won’t fix all of the school system’s programs.

A version of this article appeared in the October 05, 2011 edition of Education Week as Report Rewrites Academic-Progress Trends in Chicago

Events

Classroom Technology Webinar Building Better Blended Learning in K-12 Schools
The pandemic and the increasing use of technology in K-12 education it prompted has added renewed energy to the blended learning movement as most students are now learning in school buildings (and will likely continue

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Polls About Lessons on Racism in Schools Can Be Eye-Opening, and Misleading
Opinion surveys may help district leaders host more-productive conversations, but how they're framed can lead to wildly different results.
11 min read
Hand holding smartphone with voting app. Online voting with mini people concept flat vector illustration with smartphone screen, voting box and voters making decisions.
iStock/Getty Images Plus
School & District Management Pandemic-Seasoned Principals Share Hard-Earned Leadership Lessons
The COVID crisis has tested principals’ resolve to an unprecedented degree, but many have gleaned valuable takeaways from the experience.
6 min read
Boat on the water with three people inside. Leader pointing  forward. In the water around them are coronavirus pathogens.
iStock/Getty Images Plus
School & District Management This Intensive Internship Helps Principals Get Ready For the Job
A two-year program in Columbus City Schools gives aspiring principals the chance to dive deep into the job before actually taking the reins.
10 min read
Sarah Foster, principal of North Linden Elementary School, talks with Katina Perry in Columbus, Ohio on November 30, 2021. Columbus City Schools has a program that lets principal “test out” the principal role, before actually fully taking it on. Through the program, they work in a school for two years under a mentor principal and fill in as principal at different schools during that time.
Katina Perry, right, principal of Fairmoor Elementary School in Columbus, Ohio, meets with Sarah Foster, principal of North Linden Elementary School and Perry's mentor in a school leader internship program.
Maddie McGarvey for Education Week
School & District Management Q&A School Libraries and Controversial Books: Tips From the Front Lines
A top school librarian explains how districts can prepare for possible challenges to student reading materials and build trust with parents.
6 min read
Image of library shelves of books.
mikdam/iStock/Getty