One page from the Los Angeles Unified School District dashboard shows the status of available indicators and thier change over time. The blue wheel indicates very high performance, or in this case a very low suspension rate. The orange wheels indicate middle of the road performance. The state does not yet have complete data for the indicators marked n/a.
California released its multiple-indicator accountability dashboard last week. Criticism started about 30 minutes later. Check that. The criticism started before the release and was just repeated afterward.
The new dashboard troubles some because it broadens the concept of accountability and school success from a single number—the state’s discarded Academic Proficiency Index—that was largely a mashup of standardized test scores on math and language arts. In its place, the State Board of Education debuted multiple indicators of academic achievement and added in school climate, student engagement and other priorities.
The criticism was immediate and largely a rehash of things that had been said before during the numerous hearings about the dashboard design.
See John Fensterwald’s reporting on the dashboard rollout and initial criticism and his story about the Parent Revolution alternative that is also endorsed by Teach Plus and the Center for American Progress.
Here’s my take.
Dumb numbers create more harm than good. But people love ‘em.
The day the dashboard came out, I listened as NPR’s Marketplace program announced that the Dow was up. They even played “Happy Days...” in case the listener missed the significance. Problem is that this particular number, the Dow-Jones Industrial Index, is a very bad indicator of the state of the economy, and it tells the individual investor nothing about how to improve his or her portfolio. Indeed, a rising or falling Dow can send false messages: cause panic when none is warranted and create confidence when it should be sending warnings. Smart investors drill down, follow trends, and look for changes in fundamentals.
But people love the Dow, and journalists who know better continue to report it.
You might make an argument that the State Board of Education could have saved itself a lot of grief by keeping the old API and inserting the multiple indicator system underneath it. But I am persuaded that the old system had to go. Period. It had done some very bad things to California education as school systems narrowed their curriculum to the subjects covered by the standardized tests and the kinds of questions asked on the tests.
Accountability Dashboards Are Learnable, Teachable.
I bought a car recently. It had a dashboard. I’ve seen car dashboards before so this one wasn’t entirely novel, but it was new and a bit confusing. There are little switches to change the readouts, and there is a lot more information on my newish car than there was on the 1995 model that it replaced. I change cars about as often as the state changes accountability systems.
Guess what. The people at the dealership anticipated my confusion problem. Before I drove away, I spent an hour in a tutorial with a guy who had the job title “genius.” He walked me through all the vehicle’s electronics, including ways I can tweak the displays and even change the car’s performance. The genius also has an 800 number. I need to call for a refresher course.
Any new indicator system confuses people. But they learn, and providers of TV sets, software, refrigerators, microwaves all learn from customer experience. Maybe that’s why the state calls this version of the dashboard “a field test.” But the state needs to get much better at what designer Tim Brown calls “rapid prototyping”: quickly calling together groups of practitioners and community members to work with the new data and possible alternative ways of arranging it.
I think there is learning to be had. There are aspects of the Parent Revolution alternative dashboard that seem clearer to me, and there will be no shortage of organizations that will take the state’s data and convert it into other configurations. Good on them.
Moveover, the state is actually helping display data rather than hiding it, as some critics allege. The California Department of Education has released an online resource that allows users to search by school or district. EdSource has taken those data and created a searchable and sortable database that allows comparison among schools and districts on all listed indicators. Kudos to them.
Anyone with 15 or 20 minutes to spare can drill down on the indicators they value and create a comparison group for the schools or district they want to examine. That looks like the democratization of accountability that the state had in mind when it passed the Local Control Funding Formula and the attached accountability legislation.
Old Data Are Useless and Misleading
I went to the doc on Saturday. Great foot pain. Gout? Right symptoms for it, but blood tests came back negative. Broken toe, infection? Try an X-ray, maybe antibiotics. “Let’s just treat the symptom,” says the physician. “Come back if things don’t get better; then we’ll be looking at something more serious.”
The doctor based his diagnosis and prescription on that day’s results, not the two-year old readings he already had in his system. And the medical system had a testing and diagnosis system that provided new tests and feedback within an hour.
I join the school administrators who are appalled at getting score cards based on data from two years ago. The state’s whole testing and reporting system will crash and burn if it can’t create rapid turnaround.
Accountability or Privatization?
Inevitably, a state accountability system serves two, not entirely compatible ends. The system should signal when a school is in sufficient trouble that it requires drastic intervention. The old NCLB-era system was built entirely out of negative incentives, except for the highest-ranking schools where realtors touted the test scores. Low scoring schools, or mediocre ones, that were improving were often tagged with the failing school label. The new system tries to remedy that, maybe too much. But the system’s critics need to understand and take responsibility for pushing the intervention button too quickly.
The new system provides a great deal more data about how and where to intervene. In other words, it’s a smart (or at least smarter) system rather than a dumb one. And turning accountability into another political shouting match and questioning others’ motives, benefits no children.
All the parties to this political dust up are missing the privatization challenge from Betsy DeVos and the Trump Administration. The worse public schools can be made to look the easier it is to make private alternative look attractive. For the civil rights organizations that want equity and a robust public school system, dipping an oar in the water in favor of continuous improvement is the best way forward.
The opinions expressed in On California are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.