Federal

‘i3' Winners on Track to Meet Key Goal in Research Arena

By Michele McNeil — June 04, 2013 6 min read
  • Save to favorites
  • Print

Nearly three years after winning millions of federal dollars to help grow their ideas for improving education, the early recipients of Investing in Innovation grants appear to be largely fulfilling an important mission of one of the Obama administration’s signature programs: conducting solid research that shows whether those ideas ultimately work.

Among the 72 winners from the 2010 and 2011 seasons of the i3 program, 59 are on track to meet the rigorous research standards of the What Works Clearinghouse, according to the U.S. Department of Education. That includes all of the “scale up” and “validation” winners, which brought the most experience and research to the table and won the largest grants.

The i3 contribution to the research base could be considerable, given that the What Works Clearinghouse, which vets evidence on educational effectiveness on behalf of the Education Department’s Institute of Education Sciences, has previously identified only 30 single-study reviews that meet their standards.

An increase of nearly 60 studies receiving the clearinghouse’s top billing from a single Education Department program would be “particularly meaningful,” said Nadya Chinoy Dabby, the acting assistant deputy secretary for the department’s office of innovation and improvement.

“We are starting in a universe where there is not a super-rigorous research base,” she said. “This is a chance to figure out what is working and what is not working.”

Rigorous Evaluations

The Investing in Innovation program, launched as part of the 2009 economic-stimulus package passed by Congress, was designed to find innovative ideas to improve education and scale them up. But there was another goal as well: require the winners to conduct independent, rigorous evaluations of their projects that could be shared with the public.

A Look Under the Hood

The original 49 winners of the Obama administration’s Investing in Innovation competition are about halfway through their grants and in the middle of evaluating how well their ideas are working to improve student achievement. Research-plan highlights for the biggest grant winners include:

BRIC ARCHIVE

SOURCES: Grant Applications; U.S Department of Education

“What’s game-changing here is you’re providing an incentive for them to evaluate what they are doing in a way that’s rigorous and meaningful,” said Michele McLaughlin, the president of the Washington-based Knowledge Alliance, which represents education research organizations.

Since the inception of the i3 grants, nearly $1 billion has been awarded to 92 districts, schools, and their nonprofit partners during three rounds of competition. A fourth round is ongoing, for another $135 million in awards.

In each round, the largest grants—which in 2010 were $50 million each—go to the projects with the most evidence of past success, with small “development” grants going to those with promising ideas that haven’t been tested.

The winners spend from less than 10 percent to more than 30 percent of their awards on evaluations, which can be expensive and labor-intensive, especially when randomized controlled trials are being conducted.

Randomized Trials

Among the 2010 winners, about a dozen randomized controlled trials—the gold standard in research—are being done by independent evaluation firms. Those “experiments” involve randomly assigning students, schools, or other subjects to the “treatment,” while another group serves as the “control.” Such trials are widely used in fields such as medicine, but less so in education.

The other 2010 winners are using other methods, such as quasi-experimental statistical-matching studies, to evaluate their work.

The questions being asked in all of the i3 research—whatever the method—are more than just “Does the program improve test scores?”

For instance, the Harvard Graduate School of Education also wants to know which version of its summer reading program is most cost-effective. TNTP, formerly called The New Teacher Project, wants to know the effect of its teacher preparation program on content knowledge. And the WestEd professional-development program is examining whether that effort changes high school students’ reading behavior.

The Success for All Foundation, which provides intensive turnaround efforts in high-needs elementary schools, is studying their effects on reading outcomes and technology use. The foundation is using its $49 million grant to add 65 schools to its portfolio every year for five years.

The Baltimore-based Success for All recruited 40 schools across the country for the randomized part of its program: 20 that receive the services, and 20 that serve in the control group. For the 25-year-old organization, the i3 grant was a chance to update a randomized controlled trial that ended in 2006, said Robert E. Slavin, the director of the Center for Research and Reform in Education at the Johns Hopkins University School of Education and the co-founder and chairman of the Success for All Foundation.

“The program itself has evolved,” he said. “And perhaps the rest of the world has evolved.”

For example, when Success for All began, it used systematic phonics to teach reading though most other educators did not, Mr. Slavin said. Now, he said, almost everyone is using phonics.

For the eMINTS National Center at the University of Missouri, landing an i3 grant meant it could conduct a randomized controlled trial of its professional-development program—something the center couldn’t afford before. The “enhancing Missouri’s Instructional Networked Teaching Strategies” program, or eMINTS, uses interactive group sessions and in-classroom coaching and mentoring to help teachers integrate technology into their teaching.

“For us, that’s the whole driving factor behind why we wanted to apply,” said Lorie Kaplan, the executive director of the center, which is using about 15 percent of its $12.3 million grant to pay for its research trial, conducted by the American Institutes for Research.

“We’ve seen results, but this allows us to say we have unbiased results. It’s very much defensible,” she said.

Now, 58 schools in rural Missouri are split into three groups: a control group, a treatment group that’s receiving two years of eMINTS professional development and new technology, and a second treatment group that’s getting all of that plus an extra year of training through the Intel Teach program.

As an incentive to participate the control group will get the same professional development and technology as the first treatment group, but not until data collection ends in 2014.

More Time Needed

Barely three years after the first i3 grants were awarded, it’s too early to see whether any of the winners are showing statistically significant effects on student achievement. It usually takes at least two years—more often, three to five years—to see real change, researchers and grantees say.

But early findings of the eMINTS program are promising, according to research from the AIR evaluators presented at the 2013 American Educational Research Association conference in San Francisco.

Even after five years—the typical length of an i3 grant—it may be too early to determine a program’s effect. The Niswonger Foundation is using dual-enrollment, online and Advanced Placement courses to try to improve college and career readiness in 29 rural Tennessee high schools.

“We would like more time,” said Laura Holian, a senior research scientist at CNA, a not-for-profit research and analysis organization in Alexandria, Va., and the project director for the Niswonger research. “After five years we would hope there are changes in college readiness. We will know if there are differences in college enrollment. But how are those students performing in college? It will be too soon.”

Coverage of entrepreneurship and innovation in education and school design is supported in part by a grant from the Carnegie Corporation of New York. Education Week retains sole editorial control over the content of this coverage.
A version of this article appeared in the June 05, 2013 edition of Education Week as ‘i3' Winners on Track Toward Meeting Goal in Research Arena

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Federal Biden Calls for Teacher Pay Raises, Expanded Pre-K in State of the Union
President Joe Biden highlighted a number of his education priorities in a high-stakes speech as he seeks a second term.
5 min read
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
President Joe Biden delivers the State of the Union address to a joint session of Congress at the Capitol on March 7, 2024, in Washington.
Shawn Thew/Pool via AP
Federal Low-Performing Schools Are Left to Languish by Districts and States, Watchdog Finds
Fewer than half of district plans for improving struggling schools meet bare minimum requirements.
11 min read
A group of silhouettes looks across a grid with a public school on the other side.
Vanessa Solis/Education Week via Canva
Federal Biden Admin. Says New K-12 Agenda Tackles Absenteeism, Tutoring, Extended Learning
The White House unveiled a set of K-12 priorities at the start of an election year.
4 min read
U.S. Education Secretary Miguel Cardona participates in a roundtable discussion with students from Dartmouth College on Jan. 10, 2024, on the school's campus, in Hanover, N.H.
U.S. Education Secretary Miguel Cardona participates in a roundtable discussion with students from Dartmouth College on Jan. 10, 2024, on the school's campus, in Hanover, N.H.
Steven Senne/AP
Federal Lawmakers Want to Reauthorize a Major Education Research Law. What Stands in the Way?
Lawmakers have tried and failed to reauthorize the Education Sciences Reform Act over the past nearly two decades.
7 min read
Sen. Bill Cassidy, R-La., left, joins Senate Health, Education, Labor and Pensions Committee Chairman Bernie Sanders, I-Vt., left, as Starbucks founder Howard Schultz answers questions about the company's actions during an ongoing employee unionizing campaign, at the Capitol in Washington, on March 29, 2023.
Sen. Bill Cassidy, R-La., left, joins Senate Health, Education, Labor and Pensions Committee Chairman Bernie Sanders, I-Vt., left, at the Capitol in Washington, on March 29, 2023. The two lawmakers sponsored a bill to reauthorize the Education Sciences Reform Act.
J. Scott Applewhite/AP