School & District Management

Study Gives Revised View of Chicago School Improvements

By Jaclyn Zubrzycki — October 04, 2011 | Corrected: February 21, 2019 5 min read
  • Save to favorites
  • Print

Corrected: An earlier version of this story incorrectly identified the fund headed by Jason Cascarino. The correct name is the Chicago Public Education Fund.

A new study looking back at nearly 20 years of data on Chicago’s public schools suggests that changes in standards and in test-taking and data-reporting policies over time have led to misconceptions about the city’s progress in improving school and student performance.

Researchers from the Consortium on Chicago School Research, an independent research group based at the University of Chicago, said their analysis indicates that high school performance in the Chicago district improved more than elementary school performance did, contrary to what many local educators may believe, and that the achievement gap between African-American and other students has widened over the past 20 years.

The report details test scores and graduation rates from 1990 to 2009 across three eras: the launch of the city’s landmark school-decentralization effort and Paul G. Vallas’ and Arne Duncan’s separate tenures as chief executive officer of the Chicago schools. Though improvement efforts over those years have led to a number of policy changes, “in general they have built on each other,” said Elaine Allensworth, the chief researcher with the consortium.

Analyzing the data was a project that “took three Ph.D.s three years,” the researchers said in an interview, and the complexity of tracking down and analyzing the data led to one of the report’s key findings: “The publicly reported statistics used to hold schools and districts accountable are not accurate measures of progress.”

Scores can provide a useful snapshot, but the publicly reported data show something different with each change in policy, according to the report.

Stuart Luppescu, the chief psychometrist with the consortium, said the report includes all scores, not just those that are publicly reported, thus offering insights into performance trends across time.

Mr. Luppescu described some of the changes that affected the results: “In 2002-4, there was a break given in the middle of the reading section [of the Iowa Tests of Basic Skills].” A jump in scores at that time may be related to that practice, he added, and renorming over time also led to boosts in elementary school scores.

Former CEO Vallas said that he held off on renorming during his tenure to maintain rigor and had “five years of improved reading and math scores,” and that scores rose 5% after the test was renormed. The researchers agreed. “Students seem to be getting higher scores over time for the same level of skill,” Mr. Luppescu said.

In 2006, Chicago replaced the Iowa Tests of Basic Skills with the ISAT, or Illinois Standards Achievement Test, which Mr. Vallas, the former schools chief, described in an interview with Education Week as an “easier” test.

“Policies are made without a thought to how they will affect comparability of statistics,” Mr. Luppescu said.

The report says these changes in elementary-level tests made it appear “that high schools are less successful [than elementary schools] when, in fact, [high schools] are simply held to a much higher standard.” Tests taken by high school students are based on college-readiness standards, Mr. Luppescu noted.

Jason Cascarino, the chief operating officer of the Chicago Public Education Fund, a private foundation that invests in education programs in the city, said that finding was “the big reveal of this report. For a long time, we have thought that elementary schools were making incremental gains. ... But the definition of what is proficient is what matters.”

Raising the Bar

High schools have shown growth in meeting college-ready standards, increasing a full point between 2001 and 2009 on the ACT college-entrance exam. But the report says that most students in the 408,600-student school system come into high school far behind where they need to be. “If we want kids to be college-ready, we need to use college standards [sooner],” Mr. Luppescu said.

The report also finds that dropout rates have declined significantly, though changes in retention policy have caused the rate to fluctuate from year to year.

“In the early 1990s, students who entered Chicago high schools were equally likely to drop out as to graduate,” the report says. “Now they are more than twice as likely to graduate as to drop out.” While that statistic might hint at decreased standards, the rise in ACT scores suggests that academic performance in high schools has continued to rise with the graduation rate, the authors write. The study tracks dropout rates by age cohort rather than grade, which the authors said is a more accurate measure of dropout rates.

Which data are reported publicly has also changed over the course of 20 years, according to the report. “Prior to 2008, students’ test scores could be excluded from public reporting depending on their bilingual or special education status,” it says, and very often were. In the late ’90s, up to 25 percent of student scores were not reported to the public due to that practice.

That omission partially accounts for the consortium’s rosier depiction of 1990s reforms in its 2010 book, Organizing Schools for Improvement: Lessons From Chicago, which also begins its analysis in 1988 rather than 1990, as the current study does. And it makes it difficult to trace student-achievement trends over time.

Further muddying the numbers, the racial and ethnic composition of the district has also changed over time: The percentage of low-performing African-American students decreased, and the percentage of bilingual students whose scores weren’t always reported rose.

Numbers in Context

The report finds that focusing on specific student groups reveals some troubling trends. For one, the achievement gap between the lowest-performing African-American students and all other racial and ethnic groups is greater than the comparable nationwide gap and grew in all three school reform eras the report studied. Ms. Allensworth characterized the growth of that gap as “disheartening.”

Still, the study says, students of similar demographic backgrounds in other Illinois schools actually perform worse on standardized tests and are more likely to drop out. The report “does say Chicago is not doing worse by their kids than other districts as a whole,” Ms. Allensworth said.

The results lend support to the current Chicago school board’s focus on college readiness, said Mr. Cascarino.

Officials from the Chicago school district could not be reached for comment.

But the researchers conclude that there is still work to be done, as “the district has a long way to go before the average student graduates ready to succeed in college,” and that mandates won’t fix all of the school system’s programs.

A version of this article appeared in the October 05, 2011 edition of Education Week as Report Rewrites Academic-Progress Trends in Chicago

Events

School Climate & Safety Webinar Engaging Every Student: How to Address Absenteeism and Build Belonging
Gain valuable insights and practical solutions to address absenteeism and build a more welcoming and supportive school environment.
Student Well-Being K-12 Essentials Forum Social-Emotional Learning 2025: Examining Priorities and Practices
Join this free virtual event to learn about SEL strategies, skills, and to hear from experts on the use and expansion of SEL programs.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Professional Development Webinar
Inside PLCs: Proven Strategies from K-12 Leaders
Join an expert panel to explore strategies for building collaborative PLCs, overcoming common challenges, and using data effectively.
Content provided by Otus

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Do Students Suffer When a Superintendent Leaves? A New Study Has an Answer
A new study is the first in a while to explore how students fare academically when there's turnover in the district's top office.
5 min read
A man places his hand on top of his head as he looks up at an upwardly pointing arrow turning downward as it turns a corner.
iStock/Getty Images
School & District Management What Latino Superintendents Say It Will Take to Grow Their Ranks
Three Latino superintendents talked about the direct and indirect paths to building a pipeline of future district leaders of color.
4 min read
Vector image of many professionals, diversity, highlighting hispanic.
Liz Yap/Education Week and iStock/Getty
School & District Management Opinion Your School Needs a Teacher-Mentorship Program
We all know how critical the first few years of teaching are. Here's how to set teachers up for success.
Pamela Slifer
4 min read
Mentorship development of young teachers. School leaders make the teaching profession more sustainable by developing a robust mentoring program in their school.
Vanessa Solis/Education Week via Canva
School & District Management School Leaders Rush to Manage Deportation Fears
School and district leaders describe a chaotic time amid changes to federal immigration policies.
9 min read
A line of school children with obscured faces board a school bus on their way to school.
E+/Getty