School & District Management

Windy City’s Schools Look to Consortium For Practical Research

By Jeff Archer — April 09, 2003 10 min read
  • Save to favorites
  • Print

In a quiet corner of the University of Chicago, Shazia R. Miller is putting the finishing touches on what she calls her “little people.” The small, human- shaped figures on a bar chart represent students as they go through high school. Those at risk of academic failure are orange, while those who graduate are green. Dropouts are red.

The simple but striking illustration, created by sifting through a mountain of data on student mobility and performance, shows what actually occurred to students from a real Chicago school. Here at the Consortium on Chicago School Research, producing high-quality studies to meet the needs of local educators and policymakers is the guiding creed.

“You could send something out to a journal and reach a small sphere of people,” said Ms. Miller, an associate director with the consortium. “But it’s not the same as having this kind of impact.”

Amid a new push in education for more scholarship focused on the immediate concerns of practitioners, the work of the consortium represents an unusual example of “usable research” focused on one place. In a steady stream of reports, its researchers dissect Chicago district policy and paint detailed pictures of the conditions inside schools.

From teacher professional development to magnet schools, little that’s been done in the name of school improvement in the city has escaped the consortium’s scrutiny. Researchers have pointed out the shallowness of the content taught in many Chicago classrooms. And they’ve shown the important link between students’ academic success in their freshman year and whether they graduate from high school.

“They are not ivory-tower researchers,” said Arne Duncan, the chief executive officer of the 437,000-student Chicago district. “These are people who roll up their sleeves and get out to the schools and conduct research that is applicable to real situations.”

Accomplishing that goal has meant casting aside the usual relationship between researchers and their subjects. For the consortium, the local school community is both the object of analysis and its closest partner. Its members regularly cull the advice of education advocates and school officials in deciding what to investigate. And they keep them in the loop throughout the study process.

At times, it’s an uneasy marriage. The education leaders who help shape the consortium’s research agenda don’t always agree with the results. Its studies have fueled heated debate on a host of questions, such as how the city’s dropout rates should be calculated, and the wisdom of Chicago’s policy of requiring students to repeat grades if they don’t meet certain standards.

During the administration of former CEO Paul G. Vallas, who is now the head of the Philadelphia public schools, tensions ran so high that some feared a breakup with the consortium.

Anthony S. Bryk, the University of Chicago education professor who led the push to form the consortium, called that friction “the price of relevance.” What matters to him, he said, is not whether everyone agrees about the consortium’s conclusions, but that its studies raise the debate on school improvement in the city by grounding it in hard evidence.

“We’re informing the reform conversation,” said Mr. Bryk, who is now on sabbatical at Stanford University. “We’re constantly thinking about the big ideas that people can take away from our work that would make them think better about what they do.”

Setting the Agenda

In large measure, the Consortium on Chicago School Research owes its existence to the 1988 state law that sought to greatly decentralize the governance of the city schools. The measure gave broad new authority to decide issues of personnel, budget, and instructional programs to a local council of parents and educators at each school.

To many observers, it seemed obvious that such dramatic change warranted a major research effort to gauge its effects. Such a project might help schools use their new powers wisely, the thinking went, by providing new data on what worked and what didn’t.

The challenge was to find a way to give various stakeholders in the city the chance to help shape and inform the research agenda, while still producing rigorous, credible studies that didn’t appear biased toward any group’s perspective.

The answer was the consortium’s steering committee, forged in a series of meetings that Mr. Bryk, at the request of district officials, convened in 1990 with local school leaders, education advocates, and professors from area universities.

With 24 members, the panel includes academics, district leaders, union officers, and representatives of many of the city’s school-related interest groups. The body reflects a wide array of views.

Donald R. Moore, the executive director of the group Designs for Change, for instance, has long criticized the city’s efforts to end “social promotion” by holding back students based on test scores.

Meanwhile, Barbara Sizemore, a retired dean of DePaul University’s education school, has been one of the strongest supporters of the approach.

“There have been times when passions have been very high,” said Alfred G. Hess Jr., an education professor at Northwestern University who also sits on the panel. “But the steering committee has weathered those.”

While members of the panel offer advice on which studies to carry out, and even on how, the researchers retain ultimate control over the reports they produce.

The committee also doesn’t hold the consortium’s purse strings. With a staff of about 20, the consortium gets most of its $1.8 million annual budget from local philanthropies, primarily the John D. and Catherine T. MacArthur, Joyce, and Spencer foundations.

Still, the steering committee plays a crucial role in making sure the research staff stays focused on what matters most to Chicago’s education community, said Penny Bender Sebring, the consortium’s most veteran research associate.

“It ensures that the research we do is going to be a lot more useful, and that it’s going to be some answer to questions that people locally have,” she said. “It just keeps our feet on the floor.”

Mining Tools

Teaming up with the district gives consortium researchers access to an unusually rich source of information. To date, the group has amassed records on some 1.4 million current and former Chicago public school students, including test scores, attendance rates, and grade transcripts. That trove of data is what allows the consortium to follow what happens to students as they go through the system.

Dorothy Shipps, a former consortium researcher who is now a professor at Teachers College, Columbia University, points to a consortium study on truancy to explain why keeping close track of students is important. Mining the extensive database showed that, in fact, many students considered truant at one school had actually moved to another. So part of the truancy problem was really a mobility one.

“We came up with very interesting findings you could never get if you didn’t have this longitudinal database,” Ms. Shipps said. “Kids were moving among schools within a two- or three-mile radius.”

Some of the consortium’s most powerful findings have emerged by linking Chicago students with the district’s own survey results. Since 1991, the research group has carried out periodic polls of Chicago principals, parents, teachers, and students. Items on the questionnaires are meant to probe the quality of a school’s climate, leadership, staff training, and instruction.

Those surveys are what allow the researchers to track how major education policies play out in the schools. For instance, three years after the 1988 decentralization law, a consortium report showed that some 40 percent of the city’s elementary schools had well-conceived plans to raise student performance. But many others had just tacked on an assortment of unrelated programs—what the study memorably called “Christmas tree” schools.

Mr. Hess of Northwestern University says the snapshot made two important points: Despite some earlier predictions to the contrary, large numbers of schools were making good use of the new authority granted to them. Others clearly needed more guidance.

“Here was something that said some schools are doing better, and if some are, why aren’t the rest?” Mr. Hess said. “It kind of created the possibility for improvement.”

To help individual schools see what they most need to work on, the consortium has also created a kind of report card based on its surveys. Each school that returns enough questionnaires to make for a statistically valid sample gets a document that includes tallies of its own answers.

From the responses, the consortium calculates a series of indices meant to show the school’s strengths and weaknesses in certain critical areas, such as parental involvement, student discipline, and professional development.

The reports aren’t used for accountability purposes. In fact, to ensure candid responses, they aren’t even given the district’s central office. Instead, they’re used by school leaders and staff members to pinpoint problems.

Barbara J. Martin, the principal at Holmes Elementary School, for instance, said the survey responses helped her see the need for improving communication among her staff. When she received her first consortium report, she said, “we were just shocked to find that we didn’t do a lot of talking to each other.”

‘In the Middle of It’

By holding a mirror up to the district and its schools, however, the consortium often finds itself at the center of controversy. Said John Q. Easton, who was named executive director after Mr. Bryk left for Stanford: “We’re sort of in the middle of it, all the time.”

Especially contentious has been the consortium’s work focusing on the accountability measures put in place under Mr. Vallas, the district’s former chief executive.

Sent to run the schools by Mayor Richard M. Daley after the state gave the mayor control of the school system in 1995, Mr. Vallas established cutoff scores on standardized tests to determine whether students could go on to the next grade. The policy required students in danger of being held back to attend summer school, after which they’d be retained if they still didn’t meet the standard.

In general, the consortium’s studies have shown that the strategy has had mixed results. Although the summer school program has helped many students meet the requirements for promotion, the boost in academic achievement appears short-lived. And while the dropout rates of students who were held back haven’t spiked—as some critics of the effort had said would happen—they haven’t plummeted.

The findings contributed to a public debate on whether Mr. Vallas’ highly publicized plan was working, straining relations between the district and the research group, according to many members of the consortium’s staff and steering committee. Some even worried that the district might cut the consortium off from the student data that is its lifeblood. But it didn’t.

“There were concerns about the relationship breaking off,” said Paul D. Goren, the acting president of the Spencer Foundation. “There were efforts internally and externally to patch things up.”

Mr. Vallas now downplays the tension. Often, he said, his dispute was more with what he saw as a particular spin in the consortium’s press releases and the commentary in its reports—not with its findings. In fact, he has plans to replicate the consortium in his new district, Philadelphia, by starting a similar independent, privately supported research group dedicated to studying that city’s schools.

“I always viewed the consortium as an asset more than a liability,” Mr. Vallas said recently, “even during our own version of the ‘days of rage.’ ”

There is no doubt that the consortium enjoys good relations with the Chicago schools’ current leadership. Many of the policy initiatives of the current district chief, Mr. Duncan, appear to address concerns raised by the research group.

Mr. Duncan, for example, has broadened the criteria used to hold schools accountable for their results, adding factors meant to gauge students’ improvement, not just their performance.

And when he unveiled a major report last summer outlining new policy directions for the school system, it stressed many consortium findings, such as the importance of instructional “program coherence” to ensure that students aren’t taught the same things grade after grade.

The Chicago consortium is a rare example of scholarship that has been able to shape policy while also studying it, said Michael W. Kirst, a professor at Stanford who has observed the research group from afar.

“They’ve got to be critical and influential,” he said. “And they’re able to do that because of the depth of the research they do.”

Coverage of research is underwritten in part by a grant from the Spencer Foundation.

Related Tags:


Student Well-Being Webinar After-School Learning Top Priority: Academics or Fun?
Join our expert panel to discuss how after-school programs and schools can work together to help students recover from pandemic-related learning loss.
Budget & Finance Webinar Leverage New Funding Sources with Data-Informed Practices
Address the whole child using data-informed practices, gain valuable insights, and learn strategies that can benefit your district.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Classroom Technology Webinar
ChatGPT & Education: 8 Ways AI Improves Student Outcomes
Revolutionize student success! Don't miss our expert-led webinar demonstrating practical ways AI tools will elevate learning experiences.
Content provided by Inzata

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management School Counselors Face 'Role Ambiguity.' This State Tried to Clarify Matters
New York's new regulations didn't always change how principals viewed or interacted with school counselors, research finds.
5 min read
Man trapped in maze.
Man trapped in maze.
iStock/Getty Images Plus
School & District Management Women of Color Bring Special Strengths to the Superintendency, New Research Suggests
They have deep expertise in instructional leadership and a facility for working with stakeholders and managing thorny social issues.
4 min read
Image of diverse faces.
School & District Management 3 Principles to Help School and District Leaders Build Better Relationships With Teachers
Communication, capacity building, and a willingness to fail are key tenets of relationship-building, school and district leaders say.
2 min read
051223 Lead Sym Alyson 2 jb BS
Chris Ferenzi for Education Week
School & District Management This Principal Says It's Critical to Infuse Students' and Teachers' Days With Joy
Part of a school leader's role is to guard against outside distractions so teachers can focus on kids, says Salome Thomas-EL.
2 min read
051223 Lead Sym Caitlynn jb BS
Chris Ferenzi for Education Week