Schools Struggle to Balance Digital Innovation, Academic Accountability

When North Carolina’s Mooresville Graded School District launched a 1-to-1 laptop initiative three years ago, Superintendent Mark Edwards prepared himself for an"innovation dip,” a small drop in student performance as educators and students adjusted to the new approach.

He says he anticipated it would take time for students and teachers to master the use of laptop computers, digital curricula, and more personalized ways of teaching and learning. Though he believed that in the long run the approach would benefit students and be borne out in test scores, Edwards says he and the school board were mentally and philosophically prepared for a drop in scores over the first couple of years as the 5,600-student district worked out the kinks.

But just the opposite happened.

In three years, the district went from ranking 30th in the state in school performance measurements to fourth, and Edwards says he is gunning for first place this year. District officials saw boosts in other areas, too. Suspensions dropped at the high school level by 65 percent and districtwide by 50 percent, Edwards says.

“Students like using relevant tools and materials,” he says. “The kids are more engaged and excited about school. They’re doing things in class and saying, ‘I will do this in my future.’ ”

Balancing digital innovation and academic accountability is a tricky task for schools—one that is fraught with worries about what will work and what won’t. Schools want to utilize new tools and embrace different ways of teaching, but not at the expense of their performance on state achievement tests. Experts say finding that balance through trial and error is one of the keys to improving schools.

“The ways you measure quality and hold folks accountable are going to limit your ability to solve new problems,” says Frederick M. Hess, the director of education policy studies for the American Enterprise Institute, in Washington, and the author of the book Customized Schooling: Beyond Whole-School Reform. If K-12 policymakers insist on holding fast to grade-level assessments in reading and math currently used to determine whether a school or district is succeeding, Hess says, “what we’re going to do is limit the kinds of solutions that are going to emerge.”

‘Maintaining a Bad Situation’

Christopher Dede, a professor of educational technology at the Harvard Graduate School of Education, likes to use the analogy of a hospital when he talks about balancing innovation and accountability in education. If a hospital with a high death rate refused to try new, modern practices because they’d be unsure of the outcome or there might be a learning curve, “people would be upset because they’re maintaining a bad situation under the guise of being accountable,” he says.

The translation to education is the same, Dede says. Schools are often reluctant to incorporate new technology and apply 21st-century methods because they’re worried about a drop in test scores or other risks in shaking up the way things are done. But schools need to think about all the ways they’re accountable—not just through scores on state tests, Dede says. Schools are also accountable to students to provide a high-quality education, and to parents and local business leaders to produce students prepared for college and careers. And yes, they’re also accountable to state and federal educational leaders in the form of test scores, Dede acknowledges.

As a consequence, the approach to balancing innovation and accountability in K-12 schools needs to be particularly thoughtful, says John Danner, the co-founder and chief executive officer of Rocketship Education, which runs charter schools for 1,000 elementary-age students at three campuses in San Jose, Calif. At Rocketship, students spend part of the day in a traditional face-to-face classroom and part of the day in a Learning Lab, where they use computer software to improve their literacy and math skills. The schools, whose students are primarily low-income English-language learners, have quickly risen to the top on California test scores, outpacing many more affluent schools.

While Danner’s model relies on technology for innovation, that technology is only used when it adds value to the student-achievement equation."We have to be really careful that these programs are productive for kids,” he says. “If they’re not, you’re really costing kids learning time.”

But how do you move forward with such an innovation, when there isn’t definitive proof that it will work? That is the key question educators and policymakers are grappling with.

Under the education priorities of President George W. Bush’s administration, the catchphrase “research base” was drilled into educators” heads when it came to new programs and initiatives. If it wasn’t research-based, it wasn’t worth adopting.

But technology innovations occur so rapidly that it’s often impossible to do scientifically based trials proving effectiveness before schools embrace new approaches. Think of social-networking tools, iPads, and e-readers. And what other new digital-learning tools might also emerge well before scientifically based research can justify their use in classrooms?

Kathy Onarheim, the director of the Institute at the Cooperative Educational Service Agency #1 for 45 school districts in southeastern Wisconsin, helps districts incorporate innovative technology. The region often relies on rapid prototyping to determine which initiatives have potential. The design, implementation, and analysis of a pilot project takes months instead of years, and then the approach is quickly sent out to other districts to replicate, Onarheim says.

She says she encourages her districts to use the “80 percent” rule. “When we’re 80 percent sure of something, we go forward, rather than waiting until it’s 150 percent perfect. Otherwise, we’d never do anything” Onarheim says. “We need to take that challenge and move forward and innovate.”

One district, for example wanted to target middle school algebra and decided to use special instructional videos. The district was able to monitor in real time whether the effort was helping students grasp the material, she says. At the Florida Virtual School, which receives state funding based on how many students successfully complete a course, officials try out projects on a small scale for a short time before offering new programs to all students, says Pam Birtolo, the chief learning officer for the 97,000-student online school, based in Orlando.

When the school piloted its now-popular Conspiracy Code video game for studying American history, administrators gave any student enrolled in the experimental course the option to move at any time to the more traditional American history course offered by the school.

Florida Virtual officials also collected massive amounts of data from the students enrolled. The information wasn’t limited to student test scores. It also focused on whether students liked the graphics and interface of the course, Birtolo says.

“It’s not just an assessment; there’s an attitudinal thing going on” she says. “We want to know how the student engages and learns, and how the student retains information.”

Faster Research Tactics

Today’s new assessment technology makes it much easier to measure in real time whether a program is having its desired effect, giving educators more immediate incentives to take risks, says Tom Vander Ark, the chief executive officer of OpenEd Solutions, a Federal Way, Wash.-based blended learning service provider. Even just a few years ago, collecting such data was more difficult, often creating frustratingly long lag times between when assessments were given and when teachers and students saw the results and could act on them.

Now, “Instead of randomized long-term control trials, we can use rapid short-cycle control trials,” Vander Ark says."You can get good results in three hours and not three years.”

Often, educational innovations don’t translate into higher numbers on state tests, as they did in Mooresville. But that doesn’t mean they’re not valuable, some educators say.

Seven years ago, the 2,000-student Pascack Valley Regional High School District, in Bergen County, N.J., adopted a 1-to-1 laptop program, using digital curricula. The district, which serves students in grades 9-12, had reasonably high test scores already, and the high-tech initiative didn’t add to or detract from them, says Erik Gundersen, the district’s director of curriculum, instruction, and assessment, who was recently named superintendent and was a supervisor of science and technology education at the time the initiative began.

The overarching goal of the initiative, however, was not directed at raising test scores, Gundersen says. Instead, the district focused on embracing the 21st-century skills students need in today’s world. When students use project-based learning, employ videoconferencing to work with Arizona students on an ongoing immigration project, or collaborate using Ning or Google Docs, district officials and the school board believe students are benefiting, even if there is no measurable impact on test scores.

Still, it’s worth pointing out that the district doesn’t ignore accountability measures, either. Teachers work together to link students’ projects to state standards ‘so we make sure there’s a valid connection there, and we’re always under pressure to make sure that test scores are where they need to be” Gundersen says.

But he says the imbalance between innovation and accountability occurs because paper-and-pencil standardized exams don’t always measure those 21st-century skills the district is emphasizing.

Dede, of the Harvard Graduate School of Education, believes districts and states need to develop a “multidimensional scale of accountability” that values responsible innovation and change in addition to test scores.

“The structural problem is there’s a single mode of accountability being applied,” he says. “The only dimension is test scores.”

Officials at the U.S. Department of Education, where federal accountability requirements have shaped the educational landscape since the 2001 passage of the No Child Left Behind Act, also known as the Elementary and Secondary Education Act, seem to be also moving in that direction as the law comes up for renewal.

“As our blueprint for ESEA reauthorization describes, we have to create space and flexibility for schools to have room to make mistakes and recover without overly penalizing them,” says James H. Shelton, the department’s assistant deputy secretary for innovation and improvement.

But those with decisionmaking power often have only existing test scores to rely on, says Jeff Mao, the learning technology policy director for Maine’s Learning Technology Initiative, which aims to equip middle and high school students in the state with laptops. He says he runs into that problem when he goes to state lawmakers for financial support.

“We do have some data” to show the Maine initiative is working, he says, “but it’s hard to get a double-blind, gold-standard research study to speak to that notion.”

Lawmakers sometimes ask whether the initiative could use a control group and compare it with the group using the high-tech programs, Mao says. “But we asked legislators if they wanted their son or daughter to get the placebo versus the students who are getting the latest and greatest,” he says, pointing out that everyone wants to be in the group using the innovative techniques. “Education is not something that is as easily measured as a new widget. ... There are a million variables.”

Mao says he believes educators are wary of innovation because of concerns about having a negative impact on children’s education. “They’re not averse to change; they’re afraid to do something that’s not right for kids,” he says, “which makes it a real challenge to do anything.”

Accountability 2.0?

In New York’s 4,100-student Chappaqua Central School District, within commuting distance of New York City, officials have tried to create their own version of Dede’s multidimensional accountability scale. In 2007, the district’s school board adopted a strategic question as an underlying philosophy of education. The question asks how the district can go beyond content knowledge and focus on ensuring that all students “learn to think deeply, support their thinking, apply problem-solving skills, and actively participate in their learning.”

Based on that philosophy, a more thoughtful process of adopting innovative technologies to improve teaching and learning has occurred, says Darleen M. Nicolosi, the director of instructional technology for the district. “We try to move beyond the test scores to more creative and critical thinking for our students,” she says.

In doing so, the district doesn’t separate the use of new technology from that goal. “We incorporate emerging technologies in a natural progression” along with the educational goals in the district, she says.

As part of that approach, the district has developed its own Web 2.0 think tank of teachers and educational leaders to study emerging technologies and their place in the classroom. The district also has an administrators’ network that meets regularly to observe new technology in the classroom.

“We’re not focused on technology bells and whistles,” Nicolosi says. “We’re focused on what improves teaching and learning.”