School & District Management

Windy City’s Schools Look to Consortium For Practical Research

By Jeff Archer — April 09, 2003 10 min read

In a quiet corner of the University of Chicago, Shazia R. Miller is putting the finishing touches on what she calls her “little people.” The small, human- shaped figures on a bar chart represent students as they go through high school. Those at risk of academic failure are orange, while those who graduate are green. Dropouts are red.

The simple but striking illustration, created by sifting through a mountain of data on student mobility and performance, shows what actually occurred to students from a real Chicago school. Here at the Consortium on Chicago School Research, producing high-quality studies to meet the needs of local educators and policymakers is the guiding creed.

“You could send something out to a journal and reach a small sphere of people,” said Ms. Miller, an associate director with the consortium. “But it’s not the same as having this kind of impact.”

Amid a new push in education for more scholarship focused on the immediate concerns of practitioners, the work of the consortium represents an unusual example of “usable research” focused on one place. In a steady stream of reports, its researchers dissect Chicago district policy and paint detailed pictures of the conditions inside schools.

From teacher professional development to magnet schools, little that’s been done in the name of school improvement in the city has escaped the consortium’s scrutiny. Researchers have pointed out the shallowness of the content taught in many Chicago classrooms. And they’ve shown the important link between students’ academic success in their freshman year and whether they graduate from high school.

“They are not ivory-tower researchers,” said Arne Duncan, the chief executive officer of the 437,000-student Chicago district. “These are people who roll up their sleeves and get out to the schools and conduct research that is applicable to real situations.”

Accomplishing that goal has meant casting aside the usual relationship between researchers and their subjects. For the consortium, the local school community is both the object of analysis and its closest partner. Its members regularly cull the advice of education advocates and school officials in deciding what to investigate. And they keep them in the loop throughout the study process.

At times, it’s an uneasy marriage. The education leaders who help shape the consortium’s research agenda don’t always agree with the results. Its studies have fueled heated debate on a host of questions, such as how the city’s dropout rates should be calculated, and the wisdom of Chicago’s policy of requiring students to repeat grades if they don’t meet certain standards.

During the administration of former CEO Paul G. Vallas, who is now the head of the Philadelphia public schools, tensions ran so high that some feared a breakup with the consortium.

Anthony S. Bryk, the University of Chicago education professor who led the push to form the consortium, called that friction “the price of relevance.” What matters to him, he said, is not whether everyone agrees about the consortium’s conclusions, but that its studies raise the debate on school improvement in the city by grounding it in hard evidence.

“We’re informing the reform conversation,” said Mr. Bryk, who is now on sabbatical at Stanford University. “We’re constantly thinking about the big ideas that people can take away from our work that would make them think better about what they do.”

Setting the Agenda

In large measure, the Consortium on Chicago School Research owes its existence to the 1988 state law that sought to greatly decentralize the governance of the city schools. The measure gave broad new authority to decide issues of personnel, budget, and instructional programs to a local council of parents and educators at each school.

To many observers, it seemed obvious that such dramatic change warranted a major research effort to gauge its effects. Such a project might help schools use their new powers wisely, the thinking went, by providing new data on what worked and what didn’t.

The challenge was to find a way to give various stakeholders in the city the chance to help shape and inform the research agenda, while still producing rigorous, credible studies that didn’t appear biased toward any group’s perspective.

The answer was the consortium’s steering committee, forged in a series of meetings that Mr. Bryk, at the request of district officials, convened in 1990 with local school leaders, education advocates, and professors from area universities.

With 24 members, the panel includes academics, district leaders, union officers, and representatives of many of the city’s school-related interest groups. The body reflects a wide array of views.

Donald R. Moore, the executive director of the group Designs for Change, for instance, has long criticized the city’s efforts to end “social promotion” by holding back students based on test scores.

Meanwhile, Barbara Sizemore, a retired dean of DePaul University’s education school, has been one of the strongest supporters of the approach.

“There have been times when passions have been very high,” said Alfred G. Hess Jr., an education professor at Northwestern University who also sits on the panel. “But the steering committee has weathered those.”

While members of the panel offer advice on which studies to carry out, and even on how, the researchers retain ultimate control over the reports they produce.

The committee also doesn’t hold the consortium’s purse strings. With a staff of about 20, the consortium gets most of its $1.8 million annual budget from local philanthropies, primarily the John D. and Catherine T. MacArthur, Joyce, and Spencer foundations.

Still, the steering committee plays a crucial role in making sure the research staff stays focused on what matters most to Chicago’s education community, said Penny Bender Sebring, the consortium’s most veteran research associate.

“It ensures that the research we do is going to be a lot more useful, and that it’s going to be some answer to questions that people locally have,” she said. “It just keeps our feet on the floor.”

Mining Tools

Teaming up with the district gives consortium researchers access to an unusually rich source of information. To date, the group has amassed records on some 1.4 million current and former Chicago public school students, including test scores, attendance rates, and grade transcripts. That trove of data is what allows the consortium to follow what happens to students as they go through the system.

Dorothy Shipps, a former consortium researcher who is now a professor at Teachers College, Columbia University, points to a consortium study on truancy to explain why keeping close track of students is important. Mining the extensive database showed that, in fact, many students considered truant at one school had actually moved to another. So part of the truancy problem was really a mobility one.

“We came up with very interesting findings you could never get if you didn’t have this longitudinal database,” Ms. Shipps said. “Kids were moving among schools within a two- or three-mile radius.”

Some of the consortium’s most powerful findings have emerged by linking Chicago students with the district’s own survey results. Since 1991, the research group has carried out periodic polls of Chicago principals, parents, teachers, and students. Items on the questionnaires are meant to probe the quality of a school’s climate, leadership, staff training, and instruction.

Those surveys are what allow the researchers to track how major education policies play out in the schools. For instance, three years after the 1988 decentralization law, a consortium report showed that some 40 percent of the city’s elementary schools had well-conceived plans to raise student performance. But many others had just tacked on an assortment of unrelated programs—what the study memorably called “Christmas tree” schools.

Mr. Hess of Northwestern University says the snapshot made two important points: Despite some earlier predictions to the contrary, large numbers of schools were making good use of the new authority granted to them. Others clearly needed more guidance.

“Here was something that said some schools are doing better, and if some are, why aren’t the rest?” Mr. Hess said. “It kind of created the possibility for improvement.”

To help individual schools see what they most need to work on, the consortium has also created a kind of report card based on its surveys. Each school that returns enough questionnaires to make for a statistically valid sample gets a document that includes tallies of its own answers.

From the responses, the consortium calculates a series of indices meant to show the school’s strengths and weaknesses in certain critical areas, such as parental involvement, student discipline, and professional development.

The reports aren’t used for accountability purposes. In fact, to ensure candid responses, they aren’t even given the district’s central office. Instead, they’re used by school leaders and staff members to pinpoint problems.

Barbara J. Martin, the principal at Holmes Elementary School, for instance, said the survey responses helped her see the need for improving communication among her staff. When she received her first consortium report, she said, “we were just shocked to find that we didn’t do a lot of talking to each other.”

‘In the Middle of It’

By holding a mirror up to the district and its schools, however, the consortium often finds itself at the center of controversy. Said John Q. Easton, who was named executive director after Mr. Bryk left for Stanford: “We’re sort of in the middle of it, all the time.”

Especially contentious has been the consortium’s work focusing on the accountability measures put in place under Mr. Vallas, the district’s former chief executive.

Sent to run the schools by Mayor Richard M. Daley after the state gave the mayor control of the school system in 1995, Mr. Vallas established cutoff scores on standardized tests to determine whether students could go on to the next grade. The policy required students in danger of being held back to attend summer school, after which they’d be retained if they still didn’t meet the standard.

In general, the consortium’s studies have shown that the strategy has had mixed results. Although the summer school program has helped many students meet the requirements for promotion, the boost in academic achievement appears short-lived. And while the dropout rates of students who were held back haven’t spiked—as some critics of the effort had said would happen—they haven’t plummeted.

The findings contributed to a public debate on whether Mr. Vallas’ highly publicized plan was working, straining relations between the district and the research group, according to many members of the consortium’s staff and steering committee. Some even worried that the district might cut the consortium off from the student data that is its lifeblood. But it didn’t.

“There were concerns about the relationship breaking off,” said Paul D. Goren, the acting president of the Spencer Foundation. “There were efforts internally and externally to patch things up.”

Mr. Vallas now downplays the tension. Often, he said, his dispute was more with what he saw as a particular spin in the consortium’s press releases and the commentary in its reports—not with its findings. In fact, he has plans to replicate the consortium in his new district, Philadelphia, by starting a similar independent, privately supported research group dedicated to studying that city’s schools.

“I always viewed the consortium as an asset more than a liability,” Mr. Vallas said recently, “even during our own version of the ‘days of rage.’ ”

There is no doubt that the consortium enjoys good relations with the Chicago schools’ current leadership. Many of the policy initiatives of the current district chief, Mr. Duncan, appear to address concerns raised by the research group.

Mr. Duncan, for example, has broadened the criteria used to hold schools accountable for their results, adding factors meant to gauge students’ improvement, not just their performance.

And when he unveiled a major report last summer outlining new policy directions for the school system, it stressed many consortium findings, such as the importance of instructional “program coherence” to ensure that students aren’t taught the same things grade after grade.

The Chicago consortium is a rare example of scholarship that has been able to shape policy while also studying it, said Michael W. Kirst, a professor at Stanford who has observed the research group from afar.

“They’ve got to be critical and influential,” he said. “And they’re able to do that because of the depth of the research they do.”

Coverage of research is underwritten in part by a grant from the Spencer Foundation.

Related Tags:


Student Well-Being Webinar Boosting Teacher and Student Motivation During the Pandemic: What It Takes
Join Alyson Klein and her expert guests for practical tips and discussion on how to keep students and teachers motivated as the pandemic drags on.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
A Holistic Approach to Social-Emotional Learning
Register to learn about the components and benefits of holistically implemented SEL.
Content provided by Committee for Children
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
How Principals Can Support Student Well-Being During COVID
Join this webinar for tips on how to support and prioritize student health and well-being during COVID.
Content provided by Unruly Studios

EdWeek Top School Jobs

Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Interdisciplinary STEAM Specialist
Smyrna, Georgia
St. Benedict's Episcopal School
Arizona School Data Analyst - (AZVA)
Arizona, United States
K12 Inc.
Software Engineer
Portland, OR, US
Northwest Evaluation Association

Read Next

School & District Management Opinion New Resource Tracks School System Reopening
The Return to Learn Tracker identifies the current instructional model of all regular public school districts with three or more schools.
5 min read
Image shows a multi-tailed arrow hitting the bullseye of a target.
DigitalVision Vectors/Getty
School & District Management San Francisco School Board Pauses Renaming 44 Schools, Promises to Consult Historians
The renaming of 44 schools in the San Francisco Unified School District is apparently being put on hold after intense blowback.
Greg Keraghosian
1 min read
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.
A pedestrian walks below a sign for Dianne Feinstein Elementary School in San Francisco, on Dec. 17, 2020. The San Francisco Unified School District put the renaming of 44 schools, including Dianne Feinstein Elementary School, on hold after local and national blowback.<br/><br/>
Jeff Chiu/AP
School & District Management Superintendent Who Led During COVID-19 School Shutdowns Gets Top Honors
Michelle Reid of Washington state's Northshore district, one of the very first to close schools last March, was named National Superintendent of the Year.
3 min read
Michelle Reid, superintendent of the Northshore district in Washington
Michelle Reid, the superintendent of the Northshore district in Washington, was named National Superintendent of the Year.
courtesy of AASA, the School Superintendents Association
School & District Management Is Lunchtime the 'Weak Link' in School Reopening Plans?
It's risky when students are inside and unmasked, experts say. Here are five ways to mitigate that risk and make in-school meals safer.
11 min read
Elementary students in Brownsville, Texas, eat a socially distanced lunch in the school cafeteria. Experts say there are ways to mitigate the risk of contracting COVID-19 even when kids take their masks off to eat.
Elementary students in Brownsville, Texas, eat a socially distanced lunch in the school cafeteria. Experts say there are ways to mitigate the risk of contracting COVID-19 when kids take their masks off to eat.
Denise Cathey/The Brownsville Herald via AP