Accountability Measures Vary Widely

Fourth in a five-part series

Last month, the SouthEast Academy of Scholastic Excellence received an important visit—one that will ultimately help decide the charter school’s fate.

The board that issued its charter sent in a team of five evaluators to check up on the 8-month-old Washington school.

While the reviewers spent two full days observing classes and interviewing students, staff members, and parents, the exercise felt more like a class discussion than a pop quiz. Rather than insisting that the school meet certain predetermined criteria, the evaluators were guided by a 65-page “self-study” that the school’s administrators had turned in weeks before.

“It was not threatening,” Elizabeth Smith, SouthEast’s executive director, said of the evaluation. “It was a positive kind of approach to help us succeed.”

That is how the District of Columbia Public Charter School Board believes accountability should look in the age of greater school choice. By nurturing its charter schools and helping them when needed, the panel hopes it can avoid the drastic step of ever having to shut one down.

But it’s just one model among many. The way charter schools are evaluated varies widely from Arizona to Massachusetts, from Chicago to Colorado—and even within the nation’s capital itself.

Along with the Public Charter School Board, which was created by Congress, the local board of education also approves charters. Had Southeast been overseen by the school board, which has already closed two charter schools, it would have been evaluated by a review team armed with a “compliance checklist” instead of a self-study, and Ms. Smith would have been given just a few days’ notice before the inspection.

In part, the variety of approaches reflects both the sheer newness of the charter school movement and its spirit of experimentation.

At the same time, however, it means that—nearly eight years after the first charter school was launched—educators and policymakers have yet to agree on how the publicly financed but largely independent schools should be held accountable for their results.

“The hard part about developing these kinds of accountability plans, and doing site visits and inspections, is that there is no history of doing any of those things in our public schooling,” said Edward Kirby, a former director of charter school accountability for the Massachusetts Department of Education. “There hasn’t been a way of asking schools to really spell out clear expectations for performance and then scrutinizing how they measure up against those.”

No less daunting are the similar challenges posed by publicly funded voucher programs, in which parents can use taxpayer money to pay for tuition at private schools.

Pressure to reach some consensus on the issue is mounting. In theory, charter schools are given more autonomy than other public schools because they’re supposed to be held particularly accountable for performance. But even some of the movement’s most avid supporters, including President Clinton, acknowledge that such accountability isn’t always the case.

“The one problem we have had is that not every state has had the right kind of accountability for the charter schools,” Mr. Clinton said in a May 4 speech at City Academy in St. Paul, Minn., the nation’s first such school. “Some states have laws that are so loose that no matter whether the charter schools are doing their jobs or not, they just get to stay open, and they become like another bureaucracy. Unfortunately, I think even worse, some states have laws that are so restrictive, it’s almost impossible to open a charter school in the first place.”

Burying the Dead

Parents and regulators share joint responsibility for making sure charter schools live up to their end of the bargain.

As consumers, parents can close a failing school by voting with their feet. Charter sponsors—which can include such entities as school districts, state boards, and universities, depending on the state—can pull the plug on a school by revoking its charter agreement. “At least the charter school movement buries its dead,” said Bruno V. Manno, a senior fellow at the Baltimore-based Annie E. Casey Foundation and a co-author of the recent book Charter Schools in Action. “It doesn’t keep them on life support far beyond when their lives are over, like in most public schools.”

Nationwide, closures have been relatively rare, however. According to a federal study released this winter, 59 charter schools had shut down by last fall, representing a failure rate of just under 4 percent. And most closures have resulted from severe management or financial crises, not because of concerns over academic achievement, many observers say.

“The rhetoric is far advanced of the action,” said Paul A. Herdman, a doctoral student at Harvard University’s graduate school of education, who is writing his dissertation on charter school accountability. “The rhetoric is that if you don’t produce good results, you’ll be closed down. The reality is that virtually no schools have been shut down for performance.”

In part, that’s because the movement still is comparatively young. Nearly 1,700 charter schools are up and running in 34 states and the District of Columbia, but most of those have opened within the past four years, and so are only now starting to come up for renewal.

Still, charters also can legally be pulled at any time for flagrant violations. Some critics argue that the low closure rate is primarily due to lax oversight on the part of those charged with overseeing the schools.

“Most of these schools have not been shut down as much as they have collapsed, and I don’t think it shows anything for the system other than the great potential here for harming children,” said Alex Molnar, an education professor at the University of Wisconsin-Milwaukee and an outspoken critic of charter schools and voucher programs.

Even some proponents of the school choice movement agree that those who grant charters often don’t do enough to mind the store or provide needed guidance.

“Whenever there’s a problem, the fingers get pointed at the school,” said Jon Schroeder, the director of the Charter Friends National Network, based in St. Paul. “I think we ought to be looking closer at the authorizers. It may be that a charter should not have been granted in the first place, or the expectations were not clearly spelled out, or the problem could have been identified earlier.”

Arizona officials, for instance, didn’t revoke the charter of a Phoenix school called Citizen 2000 until the school itself filed for bankruptcy in late 1996. An investigation at the time prompted charges that the 14-month-old school had received $250,000 for which it wasn’t entitled, and that its founder had used $126,000 in school funds to buy, among other things, jewelry for herself and a home for her mother. In Ohio, the education department this winter closed the Riser Military Academy, after learning that the 7-month-old Columbus charter school lacked textbooks and computers and had failed to carry out needed facilities repairs. State officials also were concerned about public remarks made by the school’s founder that teachers may have struck students on more than one occasion.

Here in Washington, the school board pulled the plug on the Young Technocrats Math and Science Public Charter Lab School last year after a monitoring-team report revealed that the school suffered from lax student discipline, high staff turnover, and “a paucity of textbooks.” A year earlier, the board shut down the Marcus Garvey Public Charter School after the principal was charged with assaulting a local newspaper reporter; board officials later said they had uncovered “a pattern of fiscal mismanagement” at the school.

A recent report in The Washington Post stated that as many as a third of the 27 charter schools still operating here—including some approved by each of the two panels—were out of compliance with certain rules, such as filing financial audits on time and following contracting rules when hiring vendors.

Kevin P. Chavous, who chairs the education committee of the District of Columbia Council, responded that he favored creation of a new local authority to monitor charter school compliance, while allowing the two existing panels to continue to consider and approve charter applications.

“I don’t think these issues are fatal in and of themselves,” Mr. Chavous said, “but we do want to make sure they follow through with the law, because it’s when the T’s aren’t crossed and I’s aren’t dotted that sometimes trouble creeps in.”

Gathering Information

Washington’s Public Charter School Board has on occasion intervened to try to head off a crisis. When the panel worried, for instance, that the administration of one of its schools was being micromanaged by its board of trustees, it sent in a team of lawyers to evaluate the situation. The charter board then brought in the school’s leaders to try to clarify their roles.

“Had we done nothing, the school would have been in real danger,” said Nelson Smith, the executive director of the Public Charter School Board. “Our approach has been to be very supportive of the schools and very hands-on in terms of technical assistance, and then in terms of the review, we give them some breathing room and try to do something that’s improvement-oriented.”

Some, though, believe that approach could make it difficult for an authorizer to take drastic action when it’s warranted. Charter school authorizers in Arizona, Chicago, and Massachusetts, for example, have all opted against providing much assistance to their schools.

“Theoretically and practically, it becomes difficult to hold a school accountable if you’ve rolled up you’re sleeves and gotten involved in trying to fix the school, because you’re then almost as responsible for its failure as the people who hold the charter,” said Scott Hamilton, a former associate commissioner for charter schools in the Massachusetts Department of Education, who now is the managing director of the Donald & Doris Fisher Family Foundation.

That isn’t to say that Massachusetts has adopted a laissez-faire approach toward evaluating its charter schools—which, by law, must be authorized by the state board of education. In fact, the Bay State has honed one of the most elaborate such procedures, which state education officials recently used to carry out the fifth-year reviews of its first cohort of 14 charter schools.

At an average cost of about $14,000 per school, Massachusetts hired a private firm called SchoolWorks to assemble five-person teams of education experts, many of which have included an inspector who normally works as a school evaluator for the British Office for Standards in Education.

A team was then dispatched to each site for four days with the aim of gauging the school’s instruction, use of student assessments, teacher professional development, parent involvement, and governance. Each of the resulting reports—which include detailed analyses of student-performance data—ran for some 30 pages.

In Colorado, meanwhile, where charters are authorized by individual districts, the Colorado League of Charter Schools worked with state and local education officials to bring some consistency to the process.

“By and large, the districts had no idea what to do,” said James Griffin, the league’s director. “They’d send a principal in for half a day and write a report and look at test scores, and that was everything that was looked at to make a renewal decision. And we quickly realized how dangerous this was and the potential it created for arbitrary and political decisions.”

The process his group helped design includes goal-setting in the school’s first year, a self-study in year two, and reviews by external teams in years three and five. The plan has been adopted, at least in part, by 11 Colorado districts, which together oversee about a third of the state’s charter schools.

Other charter authorizers around the country have adopted a far more minimalist approach, believing that informal communications and the submission of annual audits are all that is necessary to make sure that red flags are raised when needed.

“I am convinced that you could not find funny business by visiting a school,” said Mary Gifford, the vice president of the Arizona State Board for Charter Schools, one of two statewide panels that issue charters there.

Ms. Gifford points out that Citizen 2000—the Phoenix school whose founder allegedly dipped deep into school funds for personal benefit—had welcomed the attention of outsiders.

“It was a showcase,” she said. “It was the one legislators would take people to. But that school was in horrible shape, as shown by its external audits.”

Unclear Expectations

Authorizers’ views of the academic expectations to which their charter schools should be held vary almost as much as their approaches toward monitoring them.

Generally, charter regulations call on schools and authorizers to agree on accountability plans that spell out the kinds of progress in student performance that should be attained for the charter to be renewed. But a common complaint is that, in practice, such goals are rarely well-articulated.

A report released last month by Florida’s Office of Program Policy Analysis and Government Accountability—an arm of the state legislature—bears out that contention. The report’s authors found that only six of the 33 accountability agreements between schools and their authorizers—which, in Florida, are all school districts—contained measurable goals and objectives.

Kate Foate Trewick, the chief academic officer for the St. Paul schools in Minnesota, said her district used to have the same problem. She cited a 1996 agreement between the district and a local charter school that required merely that the school would assess students using “competency expectations, exhibitions, or presentations,” evaluated against “national normed tests and community expectations.”

By contrast, a charter school approved this year under the district’s recently revamped review process had to promise that student attendance and achievement would at least equal that of the local county average.

Accountability “isn’t the kind of thing where you can build the ship as you set sail,” Ms. Trewick said. “You need to know from the beginning what direction you’re going in.”

Indeed, adding on assessments long after a charter has been granted often brings complaints of reregulation. That’s what happened in Massachusetts, where the state’s first generation of charters was issued four years before the state began to implement a new statewide testing system.

Theodore R. Sizer, one of the founders of the Francis W. Parker School, a charter school in Devens, Mass., said the Massachusetts Comprehensive Assessment System tests—which cover a wide range of knowledge—are at odds with his school’s emphasis on understanding a few topics in depth.

“There’s a terrible contradiction,” said Mr. Sizer, who also founded the Coalition of Essential Schools, a national network of public and private schools. “If you want new kinds of school, you’ve got to give them room. You can’t say, ‘Do what you want, but you must follow our frameworks and give our tests.’”

Carol Ascher, a senior research scientist at New York University’s Institute for Education and Social Policy, noticed that phenomenon last year when she carried out a series of case studies of charter schools in California, Louisiana, Massachusetts, and Texas.

“Our experience was that they were very much willing to be held accountable,” she said. “But they felt that being accountable to state assessments—particularly for schools with really innovative curriculums—reshaped what the charter schools thought it was that they were offering.”

Even when the schools and authorizers can agree on the right yardstick to measure progress, setting performance targets for individual schools can be tricky. A charter school’s founders often have little idea who their students will be, and what skills they’ll have, when they make their initial application. As a result, some authorizers wait until a school is well into its first year before asking for specific goals. Commonly, they also allow for some ratcheting up or down in subsequent years.

But others caution against allowing such adjustments.

“Individual negotiation leads to different standards for different schools,” said Greg Richmond, who heads the Chicago school district’s charter school office. “How can we say it’s OK for kids on the West Side of Chicago to learn less than those on the North Side? That’s what happens when you start arranging different deals.”

The accountability system Mr. Richmond helped design seeks to hold all of Chicago’s charter schools to the same standard. His office annually rates the charter schools according to their students’ performance on the Iowa Tests of Basic Skills and the Test of Academic Proficiency. Using a uniform matrix, each school is given a certain number of “high,” “middle,” and “low” ratings depending on how well students in each grade perform. As a result, every school is compared using the same formula.

“If the charter school movement can’t step up to the plate and do well with standardized tests, then we’re no better than the public schools that have been failing for decades,” Mr. Richmond said.

Oversight or Overkill?

Beyond the focus on test scores, some charter schools complain about the amount and frequency of paperwork requests imposed on them by their authorizers—from maintaining student-health and -residency records to filing financial audits and annual reports.

In Washington, for example, schools approved by the local school board are asked to produce such documents for review teams as many as four times in their first year.

“I feel we’re overmonitored,” said Bernida Thompson, the principal of the city’s Roots Public Charter School. “I agree with everything they look at; I just don’t need to be faxed every two weeks with another request.”

Some worry that such administrative demands place an undue burden on charter school operators.

“There are only so many hours in the day for charter school administrators,” said Mr. Schroeder of the Charter Friends network. “And if you have to spend all of your time complying with reporting requirements, then you don’t have time for designing and innovating curriculums, or developing new teacher-compensation systems, or scouting out new uses of technology. And you also might just get discouraged and burned out.”

Some, in fact, see an ulterior motive behind attempts to step up charter school regulation. What is presented as an attempt to enact more sensible oversight, they argue, is really an effort by charter opponents to rein in the entire movement.

“It’s a wonderful way for the other side to talk out of both sides of their mouths,” said David Zanotti, the president of the Ohio Roundtable, a public-policy group that strongly supports charters and voucher programs. “They don’t have to declare that they’re opposed to charter schools, but on the other hand, they can work to regulate them out of existence.”

But some school choice supporters believe charter schools shouldn’t fear all forms of regulation. The key, they say, is finding the right balance that allows for both ample autonomy and adequate oversight at the same time.

“I don’t necessarily think there’s a danger in overseeing charter schools—that’s a necessary obligation of a public agency,” said Margaret Lin, a Washington-based education consultant. “It’s when it crosses over the line and tries to control what a school is legally entitled to design on its own that it becomes a problem.”

A greater consensus may emerge over where that balance lies as jurisdictions work more to compare their approaches. Last year, the Charter Friends began to support an informal network—which Ms. Lin coordinates—to share successful accountability practices among both authorizers and private groups that support charter schools. Meanwhile, a small group led by Mr. Richmond of Chicago is working to form a new national organization of authorizers.

The result of such efforts, some suggest, could offer important lessons for the rest of public education. After all, they say, this alternative form of public schooling was created not only to test new ideas about teaching and learning, but also to try out new arrangements in school governance.

“Personally, I think this accountability issue probably has the most promise of anything that can some out of the charter school movement,” said Mr. Herdman, the Harvard doctoral student. “I don’t see the individual schools coming up with very many different ideas on how schools can function. But the larger trade-off of accountability for autonomy presents a real opportunity to redesign how schools are governed.”