School & District Management

Study Gives Revised View of Chicago School Improvements

By Jaclyn Zubrzycki — October 04, 2011 | Corrected: February 21, 2019 5 min read
  • Save to favorites
  • Print

Corrected: An earlier version of this story incorrectly identified the fund headed by Jason Cascarino. The correct name is the Chicago Public Education Fund.

A new study looking back at nearly 20 years of data on Chicago’s public schools suggests that changes in standards and in test-taking and data-reporting policies over time have led to misconceptions about the city’s progress in improving school and student performance.

Researchers from the Consortium on Chicago School Research, an independent research group based at the University of Chicago, said their analysis indicates that high school performance in the Chicago district improved more than elementary school performance did, contrary to what many local educators may believe, and that the achievement gap between African-American and other students has widened over the past 20 years.

The report details test scores and graduation rates from 1990 to 2009 across three eras: the launch of the city’s landmark school-decentralization effort and Paul G. Vallas’ and Arne Duncan’s separate tenures as chief executive officer of the Chicago schools. Though improvement efforts over those years have led to a number of policy changes, “in general they have built on each other,” said Elaine Allensworth, the chief researcher with the consortium.

Analyzing the data was a project that “took three Ph.D.s three years,” the researchers said in an interview, and the complexity of tracking down and analyzing the data led to one of the report’s key findings: “The publicly reported statistics used to hold schools and districts accountable are not accurate measures of progress.”

Scores can provide a useful snapshot, but the publicly reported data show something different with each change in policy, according to the report.

Stuart Luppescu, the chief psychometrist with the consortium, said the report includes all scores, not just those that are publicly reported, thus offering insights into performance trends across time.

Mr. Luppescu described some of the changes that affected the results: “In 2002-4, there was a break given in the middle of the reading section [of the Iowa Tests of Basic Skills].” A jump in scores at that time may be related to that practice, he added, and renorming over time also led to boosts in elementary school scores.

Former CEO Vallas said that he held off on renorming during his tenure to maintain rigor and had “five years of improved reading and math scores,” and that scores rose 5% after the test was renormed. The researchers agreed. “Students seem to be getting higher scores over time for the same level of skill,” Mr. Luppescu said.

In 2006, Chicago replaced the Iowa Tests of Basic Skills with the ISAT, or Illinois Standards Achievement Test, which Mr. Vallas, the former schools chief, described in an interview with Education Week as an “easier” test.

“Policies are made without a thought to how they will affect comparability of statistics,” Mr. Luppescu said.

The report says these changes in elementary-level tests made it appear “that high schools are less successful [than elementary schools] when, in fact, [high schools] are simply held to a much higher standard.” Tests taken by high school students are based on college-readiness standards, Mr. Luppescu noted.

Jason Cascarino, the chief operating officer of the Chicago Public Education Fund, a private foundation that invests in education programs in the city, said that finding was “the big reveal of this report. For a long time, we have thought that elementary schools were making incremental gains. ... But the definition of what is proficient is what matters.”

Raising the Bar

High schools have shown growth in meeting college-ready standards, increasing a full point between 2001 and 2009 on the ACT college-entrance exam. But the report says that most students in the 408,600-student school system come into high school far behind where they need to be. “If we want kids to be college-ready, we need to use college standards [sooner],” Mr. Luppescu said.

The report also finds that dropout rates have declined significantly, though changes in retention policy have caused the rate to fluctuate from year to year.

“In the early 1990s, students who entered Chicago high schools were equally likely to drop out as to graduate,” the report says. “Now they are more than twice as likely to graduate as to drop out.” While that statistic might hint at decreased standards, the rise in ACT scores suggests that academic performance in high schools has continued to rise with the graduation rate, the authors write. The study tracks dropout rates by age cohort rather than grade, which the authors said is a more accurate measure of dropout rates.

Which data are reported publicly has also changed over the course of 20 years, according to the report. “Prior to 2008, students’ test scores could be excluded from public reporting depending on their bilingual or special education status,” it says, and very often were. In the late ’90s, up to 25 percent of student scores were not reported to the public due to that practice.

That omission partially accounts for the consortium’s rosier depiction of 1990s reforms in its 2010 book, Organizing Schools for Improvement: Lessons From Chicago, which also begins its analysis in 1988 rather than 1990, as the current study does. And it makes it difficult to trace student-achievement trends over time.

Further muddying the numbers, the racial and ethnic composition of the district has also changed over time: The percentage of low-performing African-American students decreased, and the percentage of bilingual students whose scores weren’t always reported rose.

Numbers in Context

The report finds that focusing on specific student groups reveals some troubling trends. For one, the achievement gap between the lowest-performing African-American students and all other racial and ethnic groups is greater than the comparable nationwide gap and grew in all three school reform eras the report studied. Ms. Allensworth characterized the growth of that gap as “disheartening.”

Still, the study says, students of similar demographic backgrounds in other Illinois schools actually perform worse on standardized tests and are more likely to drop out. The report “does say Chicago is not doing worse by their kids than other districts as a whole,” Ms. Allensworth said.

The results lend support to the current Chicago school board’s focus on college readiness, said Mr. Cascarino.

Officials from the Chicago school district could not be reached for comment.

But the researchers conclude that there is still work to be done, as “the district has a long way to go before the average student graduates ready to succeed in college,” and that mandates won’t fix all of the school system’s programs.

A version of this article appeared in the October 05, 2011 edition of Education Week as Report Rewrites Academic-Progress Trends in Chicago

Events

Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Privacy & Security Webinar
Navigating Cybersecurity: Securing District Documents and Data
Learn how K-12 districts are addressing the challenges of maintaining a secure tech environment, managing documents and data, automating critical processes, and doing it all with limited resources.
Content provided by Softdocs

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management The Eclipse Is Great for Learning. But It's Tough on School Logistics
A total solar eclipse will cross a large swath of the country on April 8, sparking tough management choices for leaders of the districts in its path.
5 min read
A woman and stands outside with her arm on the back of a boy as they look up at the sky while wearing special paper glasses made for viewing a solar eclipse.
Jackie Johnson and her son Bradley Johnson, 9, watch a partial solar eclipse at the Frost Science Museum on Oct. 14, 2023, in downtown Miami. In 2024, some districts are planning to delay or cancel school on the day of a total eclipse, out of safety concerns.
Matias J. Ocner/Miami Herald via AP
School & District Management Opinion A Good Principal Knows When It's Time to Leave
I didn’t leave my job because of burnout; I stepped away from being a school leader because it was in everybody’s best interest.
Matthew Ebert
4 min read
Conceptual illustration of someone handing off a baton to someone else over a completed puzzle.
Vanessa Solis/Education Week via Canva
School & District Management Principals Tell Politicians on Capitol Hill: We’re Burning Out
Students' mental health top principals' growing list of concerns.
6 min read
People walk outside the U.S Capitol building in Washington, June 9, 2022.
Visitors walk outside the U.S Capitol building in Washington on June 9, 2022.
Patrick Semansky/AP
School & District Management Women Superintendents Experience Bias on the Climb to Leadership
Interpersonal slights and inequities make it hard for women to land the job and stay in it.
3 min read
Woman stands in front of a staircase in different colors. She is about to walk up the stairs. Concept of standing in front of a challenge and finding the right solution and courage to move on.
mikkelwilliam/E+