School & District Management

Study Seeks More Data to Judge Rhee’s D.C. Changes

By Sarah D. Sparks — March 10, 2011 5 min read

The District of Columbia school system’s rising test scores won’t be enough to confirm the effectiveness of the nationally watched education overhaul guided by former Schools Chancellor Michelle A. Rhee, according to the district’s evaluator, and so far the District does not collect the comprehensive data needed to say whether the often-controversial policy changes should stay permanently.

The National Research Council of the National Academies released the first of a series of contracted evaluations of the District’s Public Education Reform Amendment Act last week, just as Ms. Rhee’s predecessor, Kaya Henderson, officially took over the reins of the 73,000-student-school system.

The 2007 reform measure transferred control of the city’s schools from an elected school board to then-Mayor Adrien M. Fenty, created a state education agency and chancellor of schools and instituted several organizational changes in the district, such as the creation of an ombudsman to which parents and community members could bring complaints.

The NRC’s Committee on the Independent Evaluation of D.C. Public Schools calls for the district to collect data and report annually on a portfolio of indicators across all aspects of the district, including teacher and administrator issues, student progress, community engagement, and facilities upkeep.

“The problem was the [test score] changes that seem to be going in the right direction can’t be attributed to the specific changes in the system,” said evaluation co-chairman Robert M. Hauser, the director of NRC’s division of behavioral and social sciences and education. “All districts should be cautious about generalizing from the kind of aggregate overview data that have been used to suggest successes of changes made in the district to date,” Mr. Hauser said. “A kid is going to spend 13 years in the system; it takes a long time to go through the system and … evaluate it.”

Mr. Fenty, a Democrat, and Ms. Rhee, who were ousted, via election and resignation, last year in what was widely seen as a referendum on their school policies, have pointed to rising student scores in reading and mathematics on state achievement tests and the National Assessment of Educational Progress to validate their legacy. On the District of Columbia’s test, the Comprehensive Assessment of Skills, the percentage of students proficient in math and reading began increasing in 2006, accelerated through 2009, and then flattened and began to decline in 2010. The NAEP showed similar improvements, with average scores in 4th grade math and reading and in 8th grade math significantly higher in 2009 than they had been in 2007. That track record also helped propel Ms. Rhee in December, after leaving her position as chancellor, to launch a national organization, called StudentsFirst, aimed at creating local environments conducive to “more aggressive” education reforms.

District officials would not comment on the story by deadline Wednesday afternoon.

The NRC evaluation task force said the test scores alone cannot prove students are learning better under the new policies, in part because Washington’s 45,000 traditional and 28,000 charter school student populations have changed demographically since 2007. The district’s poverty rate rose from 63 percent in 2006 to 70 percent in 2010, and the numbers of students with disabilities or limited English proficiency fell during that time. The district also had fewer black students and more white and Hispanic students by 2010. Moreover, the evaluators confirmed that students’ NAEP scores started to improve even before the overhaul law passed, as noted in a report last month by Alan Ginsburg, the former director of policy and program studies for the U.S. Department of Education.

The law’s implementation has been mixed, the evaluators note. For example, the District’s new interagency commission has developed citywide strategies to improve children’s health and reduce truancy and school violence since the law passed. It also established an ombudsman in 2008 to resolve community concerns but eliminated the position’s funding in fiscal 2010.

Data Progress Needed

To pave the way for more comprehensive evaluations, the District will have to beef up its longitudinal student-data system in a hurry. The system is not yet up and running, and it contains only six of the 10 elements considered essential by the Washington-based Data Quality Campaign. The state agency terminated its contract for the data system last year after slow progress and is in the middle of a search for a new group to oversee its data warehouse.

“Their capacity is better than it used to be but still not very strong,” said Michael Casserly, the executive director of the Council of the Great City Schools, also based in Washington. “Most urban systems have data systems that are far more sophisticated than D.C.’s so they often can do evaluations of their systems and progress in ways that D.C. has not been able to do so far.”

The evaluators agreed. For example, they noted that the NAEP’s background data collection revealed that the number of schools with less than 2 percent of teachers absent daily increased significantly from 68 percent in 2007 to 85 percent in 2009, and the average NAEP scale score at these schools was 246, significantly higher than the 234 average scale score at schools with teacher absenteeism of 6 to 10 percent daily. However, evaluators said it is impossible to know now whether higher teacher absenteeism actually caused lower scores, or whether both indicators were influenced by something else, such as unsafe school conditions.

“We really weren’t able to dig deeply enough to answer these questions, nor did we have the time and resources needed to do any really focused studies, so all we could do was look at initial implementation of structures that were created in [the law],” Mr. Hauser said.

“What needs to be done is look very closely at specific schools and specific areas. … If there are places in the city not getting the attention and resources they need, that’s the sort of thing a good indicator system can point out,” he said.

A version of this article appeared in the March 16, 2011 edition of Education Week

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Culturally Relevant Pedagogy to Advance Educational Equity
Schools are welcoming students back into buildings for full-time in-person instruction in a few short weeks and now is the perfect time to take a hard look at both our practices and systems to build
Content provided by PowerMyLearning
Classroom Technology Webinar Making Big Technology Decisions: Advice for District Leaders, Principals, and Teachers
Educators at all levels make decisions that can have a huge impact on students. That’s especially true when it comes to the use of technology, which was activated like never before to help students learn
Professional Development Webinar Expand Digital Learning by Expanding Teacher Training
This discussion will examine how things have changed and offer guidance on smart, cost-effective ways to expand digital learning efforts and train teachers to maximize the use of new technologies for learning.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Opinion Leaders, Your Communication Plan Needs to Start With Your Staff
Staff members are the point of contact for thousands of interactions with the public each day. They can’t be the last to know of changes.
Gladys I. Cruz
2 min read
A staff meeting around a table.
Vanessa Solis/Education Week and Getty Images
School & District Management L.A. Unified to Require Testing of Students, Staff Regardless of Vaccination Status
The policy change in the nation's second-largest school district comes amid rising coronavirus cases, largely blamed on the Delta variant.
Howard Blume, Los Angeles Times
4 min read
L.A. schools interim Sup Megan K. Reilly visits Fairfax High School's "Field Day" event to launch the Ready Set volunteer recruitment campaign to highlight the nationwide need for mentors and tutors, to prepare the country's public education students for the upcoming school year. The event coincides with National Summer Learning Week, where U.S. Secretary of Education Miguel Cardona is highlighting the importance of re-engaging students and building excitement around returning to in-person learning this fall. high school, with interim LAUSD superintendent and others. Fairfax High School on Wednesday, July 14, 2021 in Los Angeles, CA.
In this July 14, 2021, photo, Los Angeles Unified School District interim Superintendent Megan K. Reilly speaks at an event at Fairfax High School in Los Angeles. Reilly announced a new district policy Thursday requiring all students and employees of the Los Angeles school district to take weekly coronavirus tests regardless of their vaccination status.
Al Seib/Los Angeles Times via TNS
School & District Management Why School Boards Are Now Hot Spots for Nasty Politics
Nationalized politics, shifts in local news coverage, and the rise of social media are turning school board meetings into slug fests.
11 min read
Collage of people yelling, praying, and masked in a board room.
Collage by Gina Tomko/Education Week and Getty Images
School & District Management Opinion The Six Leadership Lessons I Learned From the Pandemic
These guiding principles can help leaders prepare for another challenging year—and any future crises to come.
David Vroonland
3 min read
A hand about to touch a phone.
Vanessa Solis/Education Week and Getty Images