College & Workforce Readiness

Slicing the K-12 Data on Governors Running for White House

By Alyson Klein — September 08, 2015 7 min read
  • Save to favorites
  • Print

In his campaign for the GOP presidential nomination, Wisconsin Gov. Scott Walker credits his push to get rid of teacher protections with boosting his state’s 3rd grade reading scores.

Louisiana Gov. Bobby Jindal, another contender in the crowded Republican field, claims his championing of expanded school choice helped hike his state’s high school graduation rates.

And former Florida Gov. Jeb Bush says his work on school accountability has translated into big achievement gains, including for minority students, in the Sunshine State.

So how much weight should voters give such claims?

Maybe not much, educational experts say. It’s very tricky to draw a direct cause-and-effect relationship between a particular policy—for instance, a governor’s decision to increase education spending or add more charter schools—and an improvement in student achievement, researchers say.

“In most cases, it’s almost impossible” to truly credit a particular policy or policymaker with gains in student outcomes, said Laura Hamilton, the associate director of RAND Education, a research organization, in Santa Monica, Calif.

“It’s hard to disentangle [changes that resulted from policy shifts] from changes in the teacher labor market, changes in the economy, other changes in the state,” she said. “It’s almost never appropriate to make a causal conclusion about a particular policy initiative that’s implemented statewide.”


Research Challenge

The best way to study a policy’s impact, researchers say, is to do a randomized controlled trial using two similar groups of students: one that got access to better-funded schools, for instance, and another that didn’t.

“With randomized controlled trials you can, with some confidence, attribute [gains] to the intervention,” said Laura Desimone, a professor at the University of Pennsylvania’s Graduate School of Education in Philadelphia.

But that sort of rigorous research is nearly impossible when it comes to new state K-12 policies, experts caution. For one thing, it’s hard to imagine voters would want some children—but not others—to benefit from a policy change, new program, or funding increase.

What’s more, candidates in some cases may be jumping the gun by citing an impact from policy changes that have yet to really take root.

Take Walker’s claim that the 2011 changes to teacher policies in Wisconsin increased 3rd grade scores on state tests. Weakening protections could arguably lead to a new and better teaching force.

But even if researchers designed an intensive study on the effects of Walker’s teacher policies on student achievement, it could take years to deliver a final verdict on their efficacy. That’s partly because it would take a while to get a new crop of teachers in place and for them to begin having any impact on student achievement, said Morgan Polikoff, an assistant professor of education at the University of Southern California’s Rossier School of Education.

There simply hasn’t been enough time since the policy changes were enacted to link them to any hike in test scores, Polikoff said.

Plus, 3rd grade reading scores on state tests haven’t risen consistently throughout Walker’s time in office, though they did part of the time, making it complicated for the candidate to offer sweeping conclusions about the effects of his policies.

Walker’s campaign did not return calls to discuss the issue.

‘Lagging Indicators’

And then there’s Jindal’s contention that graduation rates have gone up in Louisiana, thanks in part to an expansion of school vouchers that he signed into law in 2012.

It’s true that the Pelican state saw a rise in graduation rates during Jindal’s tenure, from 71 percent in the 2010-11 school year to 74 percent in the 2012-13 school year, according to the Adjusted Cohort Graduation Rate, or ACGR, which schools have been required to use since 2008. (Note: Louisiana’s graduation rates also rose during Jindal’s tenure as calculated by the older, Averaged Freshman Graduate Rate metric, as did the nation’s.)

But graduation rates rose nationally too, from 79 percent to 81 percent during the same time frame on the ACGR. And some states saw much bigger jumps than Louisiana’s. Alabama’s graduation rate shot up 8 percentage points over the same period on that measure, for instance.

“If you’re seeing this nationally, it’s hard to argue that a particular state’s policy was responsible,” Hamilton said.


Plus, outcomes like graduation rates are often what’s known in the research world as a “lagging indicator"—it could take a long time for an intervention put in place with elementary school students to bear fruit farther down the line.

Stafford Palmieri, a senior policy advisor to Jindal, who came into office in 2008, stood by the governor’s claims that his education-redesign plans, including expanded school choice and what she described as greater educational accountability, have made a significant difference on graduation rates and other student outcomes. Saying otherwise “doesn’t pass the sniff test,” she said.

She acknowledged that there hasn’t been a robust research study of Jindal’s policies, but said that “common sense” dictates that his efforts are responsible for some of the gains the state has seen.

“We’ve made incredible changes in the system,” she said. “I don’t think graduation rates would have gone up if we’d done nothing.” And she added, “Each state has to decide for themselves what’s the right way to give every child an excellent education. In Louisiana, our reforms have given thousands of children the shot at a better education and a better life.”

Who Has Bragging Rights?

An apples-to-apples comparison of educational outcomes under the nine governors or former governors now running for president doesn’t show any standout stars, according to an analysis by the Education Week Research Center.

See Also

Data: The Governors Running for President and Their K-12 Records

Most of those candidates—whether Republican or Democratic—didn’t see huge spikes in selected math and reading scores on the National Assessment of Educational Progress, or NAEP, the nation’s report card, during their tenure in office. Neither did they see major improvements in graduation rates under the AFGR metric. (The analysis doesn’t include graduation rates before 2003.)

One potential exception may be Bush, who led Florida from 1999 to 2007. That period corresponded with gains in 4th grade reading on NAEP. Scores jumped from 206 on a 500-point scale in 1998, the year before Bush took office, to 226 in 2009, when he’d been out of office for two years. (Nationally, scores on the same exam rose from 213 to 220 during the same period.) And when it comes to closing the achievement gap, Florida made some substantial progress with respect to reading and math gaps in 4th grade, between 2003 and 2013. Nationally, the black-white gap in grade 4 reading closed by 4 points compared to 8 points in Florida. In grade 4 math, the nation narrowed the black-white gap by 1 point compared with 5 points in Florida.

But there’s no way to say for sure that Bush’s leadership—and the policies he put in place, such as ending social promotion for some students—were solely responsible for those gains, said Sterling Lloyd, the senior research analyst for the Education Week Research Center.

It’s possible that they made a difference, he said, but it’s important to take other factors—such as demographic changes—into consideration. And the jump for students overall wasn’t nearly as impressive for 8th grade reading.

On the Democratic side, Martin O’Malley, the former governor of Maryland, has cited the Old Line State’s number one overall grade for several years in Education Week‘s annual Quality Counts report as evidence of his strong stewardship of the state’s K-12 system. It’s true that Maryland was ranked number one for five years running, 2009 through 2013. But it’s hard to attribute that track record to any one policy, or policymaker, Lloyd said. And Education Week recently changed its methodology, pushing Maryland down to third place nationally in the 2015 report.

Typical Fare

A few headshaking education researchers aren’t likely to stop governors-turned-presidential-candidates from trumpeting student achievement gains that occurred under their watch, said Patrick McGuinn, an associate professor of political science at Drew University in Madison, N.J., who has studied the role education plays in elections.

For one thing, the strategy gives governors a chance to highlight their state executive experience—a quality that sets them apart from candidates who have served in, say, the U.S. Senate or the business community.

And a bump in test scores is also an easy-to-grasp sound bite that can help paint a candidate as a leader who gets results, McGuinn said.

“Voters like to hear hard facts. This is empirical, it’s data, it sounds great, it sounds positive, it sounds like a real result,” McGuinn said.

Governors have a myriad of student achievement results to choose from in making their case, everything from state test scores for a particular grade to Advanced Placement course-taking and college graduation rates. “They get to cherry pick whatever indicators and facts they get to use,” McGuinn said.

That’s partly why it makes sense to look beyond any single number—such as a rise in ACT scores or 4th grade math scores—when considering a candidate’s record, said Desimone of the University of Pennsylvania. She urged anyone wanting to understand these governors’ records to “look for the evidence across a host of different kinds of tests.” If scores have gone up across the board—and the gains have been sustained—that could bode well for a particular policy, she said.

Education Week Librarian Holly Peele contributed to the reporting of this story, and compiled the data and data visualization.
A version of this article appeared in the September 09, 2015 edition of Education Week as Slicing the K-12 Data on Governors Running for President


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Achievement Webinar
How To Tackle The Biggest Hurdles To Effective Tutoring
Learn how districts overcome the three biggest challenges to implementing high-impact tutoring with fidelity: time, talent, and funding.
Content provided by Saga Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Student Well-Being Webinar
Reframing Behavior: Neuroscience-Based Practices for Positive Support
Reframing Behavior helps teachers see the “why” of behavior through a neuroscience lens and provides practices that fit into a school day.
Content provided by Crisis Prevention Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Mathematics Webinar
Math for All: Strategies for Inclusive Instruction and Student Success
Looking for ways to make math matter for all your students? Gain strategies that help them make the connection as well as the grade.
Content provided by NMSI

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

College & Workforce Readiness What the Research Says New Data Paint Bleak Picture of Students' Post High School Outcomes
Students are taking much longer to complete credentials after high school than programs plan.
2 min read
Student hanging on a tearing graduate cap tassel
DigitalVision Vectors/Getty
College & Workforce Readiness This East Coast District Brought a Hollywood-Quality Experience to Its Students
A unique collaboration between a Virginia school district and two television actors allows students to gain real-life filmmaking experience.
6 min read
Bethel High School films a production of Fear the Fog at Fort Monroe on June 21, 2023.
Students from Bethel High School in Hampton, Va., film "Fear the Fog"<i> </i>at Virginia's Fort Monroe on June 21, 2023. Students wrote, directed, produced, and starred in the film through a partnership between their district, Hampton City Schools, and two television actors that's designed to give them applied, entertainment industry experience.
Courtesy of Hampton City Schools
College & Workforce Readiness A FAFSA Calculation Error Could Delay College Aid Applications—Again
It's the latest blunder to upend the "Better FAFSA," as it was branded by the Education Department.
2 min read
Jesus Noyola, a sophomore attending Rensselaer Polytechnic Institute, poses for a portrait in the Folsom Library on Feb. 13, 2024, in Troy, N.Y. A later-than-expected rollout of a revised Free Application for Federal Student Aid, or FASFA, that schools use to compute financial aid, is resulting in students and their parents putting off college decisions. Noyola said he hasn’t been able to submit his FAFSA because of an error in the parent portion of the application. “It’s disappointing and so stressful since all these issues are taking forever to be resolved,” said Noyola, who receives grants and work-study to fund his education.
Jesus Noyola, a sophomore at Rensselaer Polytechnic Institute, stands in the university's library on Feb. 13, 2024, in Troy, N.Y. He's one of thousands of existing and incoming college students affected by a problem-plagued rollout of the revised Free Application for Federal Student Aid, or FASFA, that schools use to compute financial aid. A series of delays and errors is resulting in students and their parents putting off college decisions.
Hans Pennink/AP
College & Workforce Readiness How Well Are Schools Preparing Students? Advanced Academics and World Languages, in 4 Charts
New federal data show big gaps in students' access to the challenging coursework and foreign languages they need for college.
2 min read
Conceptual illustration of people and voice bubbles.