College & Workforce Readiness Opinion

Distorting data

By Diane Ravitch — July 02, 2007 3 min read
  • Save to favorites
  • Print

Dear Deb,

Now that our mayor, Michael Bloomberg, is readying himself as a potential candidate for the Presidency, it is clear that education will be one of his signature issues. Sadly, he knows no more about education today than he did when he became mayor in 2001, based on his latest plan to pay poor kids to get higher test scores. That strategy seems to me to be an abject admission of cluelessness: When you don’t know anything about teaching or curriculum, then just pay for results.

I understand your frustration about the historical amnesia that you encountered. It seems to be the policy of our New York City Department of Education to wipe out all historical memory and at that they have been quite successful. Apparently, this administration wants the world to believe that whatever it is doing is historic, unprecedented, and of course dazzlingly successful. Part of their strategy is to launch one initiative after another, to persuade the public that they are on the move, when in fact they are merely lurching from one confused plan to another.

Last Friday, the Department released dazzling statistics about the graduation rates of its new small schools. The Department has created about 200 of them, of which 47 have a graduating class this year. The rate for these 47 schools is 73 percent, compared to a citywide rate of.... This is where it gets complicated. The city says it has a graduation rate of 60 percent. The state says the city’s graduation rate is 50%. Education Week‘s latest Diplomas Count report says the city’s graduation rate is 45 percent.

So the small schools graduation rate is impressive, right? Wrong. Dee Alpert, who publishes SpecialEducationMuckraker.com is a relentless devourer of data from the city and state education departments, and she sent out a bulletin describing her inability to decipher the real graduation rates at the “new small schools.” She says:

1. NYCDOE gives no list of the 'new, small schools' included in its calculations for the spinrelease, so... 2. They give percentages, not raw numbers, for their graduation rates: you can't even try to work backwards to see what was included. 3. Most importantly, they gave no numbers for 'still enrolled,' nor for 'discharged.' ...NYCDOE is notorious for mis-reporting dropouts as having enrolled elsewhere, i.e., discharged from a NYCDOE school's rolls. 4. The numbers for graduates who earned local v. Regents diplomas is also critical, and missing. In prior years, local hs diplomas predominated. According to the NYS Court of Appeals' CFE [Campaign for Fiscal Equity] decision, a local diploma resulting from passing RCT [low-level competency examinations] put a kid-depending on the subject—at between the 6th and 9th grade level. This isn't exactly college prep.

Dee Alpert concludes: “It’s all obviously part of the Bloomberg presidential campaign spin.” And she strongly recommends “an editorial moratorium on reporting NYCDOE spin numbers unless the complete data set accompanies the press release.”

I hope not to befuddle our national readership with too much news and talk about New York City, but there is an important point here. Alpert’s analysis reveals how easily education data are distorted for political purposes. We have seen this done before, but seldom so blatantly or so cleverly. It is likely the case that every school superintendent wants to release test scores and graduation rates that show what a success he or she has been. But what we have seen over the past five years is a determined political campaign—not just Bloomberg for President, although that may yet happen—but rather a political campaign to “prove” that mayoral control without any checks or balances is the absolute best way to manage a school system. The Department is incapable of impartial research. Its press releases are filled with the kind of P.R. spin that we have come to associate with politicians running for office, not with research departments where someone has his or her professional reputation on the line.

Whenever there is a new release of test score data from the state, I invariably get calls from reporters, asking what I think of the latest numbers. I have learned over the past few years never to answer their questions until I have had a chance to see the complete data set with my own eyes. I know that the Department has massaged the data and sought out every glimmer of good news while creating a narrative that distracts the reporters’ attention from anything unfavorable.

Forgive me, Deborah, but all this media manipulation persuades me that we need national tests (with no stakes), so that states and cities and districts can’t play games with the numbers. Failing that, I think that every state should have an independent agency to administer tests and report their results, and that these agencies should be run by professional psychometricians who can neither take credit nor blame when the scores rise or fall. In New York State, when the math scores went down last year, the State Education Department said that the test was harder; when the math scores went up this year, the State Education Department was congratulating itself and the Regents (our state board) for its wise policies. How about an agency that dispenses the facts without fear or favor or self-praise?


Related Tags:

The opinions expressed in Bridging Differences are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.