School & District Management

Study Gives Revised View of Chicago School Improvements

By Jaclyn Zubrzycki — October 04, 2011 | Corrected: February 21, 2019 5 min read

Corrected: An earlier version of this story incorrectly identified the fund headed by Jason Cascarino. The correct name is the Chicago Public Education Fund.

A new study looking back at nearly 20 years of data on Chicago’s public schools suggests that changes in standards and in test-taking and data-reporting policies over time have led to misconceptions about the city’s progress in improving school and student performance.

Researchers from the Consortium on Chicago School Research, an independent research group based at the University of Chicago, said their analysis indicates that high school performance in the Chicago district improved more than elementary school performance did, contrary to what many local educators may believe, and that the achievement gap between African-American and other students has widened over the past 20 years.

The report details test scores and graduation rates from 1990 to 2009 across three eras: the launch of the city’s landmark school-decentralization effort and Paul G. Vallas’ and Arne Duncan’s separate tenures as chief executive officer of the Chicago schools. Though improvement efforts over those years have led to a number of policy changes, “in general they have built on each other,” said Elaine Allensworth, the chief researcher with the consortium.

Analyzing the data was a project that “took three Ph.D.s three years,” the researchers said in an interview, and the complexity of tracking down and analyzing the data led to one of the report’s key findings: “The publicly reported statistics used to hold schools and districts accountable are not accurate measures of progress.”

Scores can provide a useful snapshot, but the publicly reported data show something different with each change in policy, according to the report.

Stuart Luppescu, the chief psychometrist with the consortium, said the report includes all scores, not just those that are publicly reported, thus offering insights into performance trends across time.

Mr. Luppescu described some of the changes that affected the results: “In 2002-4, there was a break given in the middle of the reading section [of the Iowa Tests of Basic Skills].” A jump in scores at that time may be related to that practice, he added, and renorming over time also led to boosts in elementary school scores.

Former CEO Vallas said that he held off on renorming during his tenure to maintain rigor and had “five years of improved reading and math scores,” and that scores rose 5% after the test was renormed. The researchers agreed. “Students seem to be getting higher scores over time for the same level of skill,” Mr. Luppescu said.

In 2006, Chicago replaced the Iowa Tests of Basic Skills with the ISAT, or Illinois Standards Achievement Test, which Mr. Vallas, the former schools chief, described in an interview with Education Week as an “easier” test.

“Policies are made without a thought to how they will affect comparability of statistics,” Mr. Luppescu said.

The report says these changes in elementary-level tests made it appear “that high schools are less successful [than elementary schools] when, in fact, [high schools] are simply held to a much higher standard.” Tests taken by high school students are based on college-readiness standards, Mr. Luppescu noted.

Jason Cascarino, the chief operating officer of the Chicago Public Education Fund, a private foundation that invests in education programs in the city, said that finding was “the big reveal of this report. For a long time, we have thought that elementary schools were making incremental gains. ... But the definition of what is proficient is what matters.”

Raising the Bar

High schools have shown growth in meeting college-ready standards, increasing a full point between 2001 and 2009 on the ACT college-entrance exam. But the report says that most students in the 408,600-student school system come into high school far behind where they need to be. “If we want kids to be college-ready, we need to use college standards [sooner],” Mr. Luppescu said.

The report also finds that dropout rates have declined significantly, though changes in retention policy have caused the rate to fluctuate from year to year.

“In the early 1990s, students who entered Chicago high schools were equally likely to drop out as to graduate,” the report says. “Now they are more than twice as likely to graduate as to drop out.” While that statistic might hint at decreased standards, the rise in ACT scores suggests that academic performance in high schools has continued to rise with the graduation rate, the authors write. The study tracks dropout rates by age cohort rather than grade, which the authors said is a more accurate measure of dropout rates.

Which data are reported publicly has also changed over the course of 20 years, according to the report. “Prior to 2008, students’ test scores could be excluded from public reporting depending on their bilingual or special education status,” it says, and very often were. In the late ’90s, up to 25 percent of student scores were not reported to the public due to that practice.

That omission partially accounts for the consortium’s rosier depiction of 1990s reforms in its 2010 book, Organizing Schools for Improvement: Lessons From Chicago, which also begins its analysis in 1988 rather than 1990, as the current study does. And it makes it difficult to trace student-achievement trends over time.

Further muddying the numbers, the racial and ethnic composition of the district has also changed over time: The percentage of low-performing African-American students decreased, and the percentage of bilingual students whose scores weren’t always reported rose.

Numbers in Context

The report finds that focusing on specific student groups reveals some troubling trends. For one, the achievement gap between the lowest-performing African-American students and all other racial and ethnic groups is greater than the comparable nationwide gap and grew in all three school reform eras the report studied. Ms. Allensworth characterized the growth of that gap as “disheartening.”

Still, the study says, students of similar demographic backgrounds in other Illinois schools actually perform worse on standardized tests and are more likely to drop out. The report “does say Chicago is not doing worse by their kids than other districts as a whole,” Ms. Allensworth said.

The results lend support to the current Chicago school board’s focus on college readiness, said Mr. Cascarino.

Officials from the Chicago school district could not be reached for comment.

But the researchers conclude that there is still work to be done, as “the district has a long way to go before the average student graduates ready to succeed in college,” and that mandates won’t fix all of the school system’s programs.

A version of this article appeared in the October 05, 2011 edition of Education Week as Report Rewrites Academic-Progress Trends in Chicago

Events

Jobs The EdWeek Top School Jobs Virtual Career Fair
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Teaching Live Online Discussion How to Develop Powerful Project-Based Learning
How do you prepare students to be engaged, active, and empowered young adults? Creating a classroom atmosphere that encourages students to pursue critical inquiry and the many skills it requires demands artful planning on the
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Safe Return to Schools is Possible with Testing
We are edging closer to a nationwide return to in-person learning in the fall. However, vaccinations alone will not get us through this. Young children not being able to vaccinate, the spread of new and
Content provided by BD

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management More Than 1 Million Students Didn't Enroll During the Pandemic. Will They Come Back?
Education Week analyzed state data to gather a more comprehensive understanding of this year's enrollment loss.
6 min read
Students participate in class outside at the Woodland Pond School, a private school  located near Bangor, Maine. Maine experienced one of the nation's largest drops in student enrollment this school year, according to an EdWeek analysis.
Students participate in class outside at the Woodland Pond School, a private school located near Bangor, Maine. Maine experienced one of the nation's largest drops in student enrollment this school year, according to an EdWeek analysis.
Photo courtesy of Woodland Pond School
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Sponsor
Drive Improvement in Your School With Harvard’s Certificate in School Management and Leadership
Aubree Mills had two dilemmas she needed to address: One was recruiting and retaining good teachers at the Ira A. Murphy Elementary School
Content provided by Harvard Graduate School of Education
School & District Management Opinion Are Your Leadership Practices Good Enough for Racial Justice?
Scratch being a hero. Instead, build trust and reach beyond school walls, write Jennifer Cheatham and John B. Diamond.
Jennifer Cheatham & John B. Diamond
5 min read
Illustration of leadership.
Collage by Laura Baker/Education Week (Images: DigitalVision Vectors, iStock, Getty)
School & District Management We Pay Superintendents Big Bucks and Expect Them to Succeed. But We Hardly Know Them
National data is skimpy, making it hard to know what influences superintendents' decisions to move on, retire, or how long they stay. Why?
8 min read
Conceptual image of tracking with data.
marcoventuriniautieri/iStock/Getty