The question, in a call last spring from the White House, seemed innocent enough: Would the Navy be interested in funding an expansion of a private-sector program encouraging high school students to take more rigorous math and science courses into schools that serve large numbers of military dependents? Its support would help promote a new, high-profile White House effort to help the nation’s armed forces and their families. But in agreeing to participate, the Navy also put itself ahead of the curve on federal education policy by embracing the untested and controversial idea of paying students who do well on standardized tests.
Cash incentives to students and their teachers are a core element of the 4-year-old, Texas-based National Math and Science Initiative, or NMSI, which targets low-achieving and low-income high schools with large minority populations. NMSI leaders believe that those schools can improve the quality of instruction by offering more Advanced Placement courses, on the assumption that their students can handle more challenging material if given the chance. The incentives are part of a well-structured program that also features extra lab equipment, Saturday classes, and special training for teachers.
The Obama administration has no official position on the use of cash incentives, and Congress has never addressed the topic in legislation. At the same time, U.S. Secretary of Education Arne Duncan has voiced support for the concept as a tool for raising student achievement, and a small departmental program to foster participation in AP courses allows officials to give money directly to students. However, only two of the 55 current grantees are doing so.
STEM Investment
Despite the fact that it would be entering uncharted waters, the Navy decided to proceed. Reaching schools that serve a large number of military families is a good way to address the Navy’s concern about finding enough U.S. citizens to fill the science- and technology-based jobs required to wage modern warfare, said Michael Kassner, the director of research at the Office of Naval Research and head of the Navy’s STEM—science, technology, engineering, and mathematics—office. “We know that students whose parents are in the military are more likely to go into the military,” he noted.
Mr. Kassner also had the money. The Navy is in the midst of doubling, to more than $100 million, its investment over five years in STEM education at all levels. “The AP test appears to be successful in improving a student’s skills,” said Mr. Kassner, a former professor of aerospace engineering at the University of Southern California. “And we thought NMSI looked like a good way to go.”
Mr. Kassner agreed to the White House’s offer to sign onto a program, the Initiative for Military Families, which brings the NMSI model to schools serving a large number of military families. The White House effort is part of Joining Forces, a program launched in April that received a national shout-out from first lady Michelle Obama before the start of this year’s first World Series game in St. Louis.
NMSI officials compiled a list of 154 schools around the country near military installations and screened them for their willingness to follow NMSI’s formula. At the end of September, the Office of Naval Research announced a $1.1 million grant to NMSI to conduct a three-year pilot at three public schools in Virginia and Hawaii. Mr. Kassner says the Navy is open to funding many more schools if the initiative shows it’s capable of priming the STEM pump to meet the Navy’s needs.
Missing the Target?
At one level, NMSI’s approach appears to be working well: In the midst of a nationwide surge in AP test-taking, the number of minority and female students passing those tests at 228 NMSI-sponsored schools this year increased four to 10 times faster than for the country as a whole. Apart from the initiative’s own data, however, there is scant evidence in the research literature that the program can deliver the talent sought by the Navy and a host of other employers.
Kirabo Jackson, a labor economist at Northwestern University in Evanston, Ill., has looked at the Texas precursor to NMSI. His 2008 study is perhaps the only one to take a comprehensive look at the role of incentives in the targeted population, although it doesn’t measure the impact of incentives apart from the other program elements.
Mr. Jackson found that the strategy increased AP participation rates and boosted the number of students with high scores on national college-entrance tests. At the same time, he found the program didn’t increase high school graduation rates or the number of students taking college-entrance exams. That result, he said, suggests it’s more likely to help high-achievers already headed to college than to raise the aspirations of those who hadn’t planned to continue their education.
The jury is still out, Mr. Jackson said, on whether efforts to promote AP courses with cash incentives actually result in the kind of payoff that the Navy is hoping for, namely, more students with college degrees in STEM fields from high-quality institutions. Still, his overall assessment of the program is positive.
“It’s one of the few programs that does something good for these students,” Mr. Jackson said. “Most programs haven’t been evaluated rigorously. And I don’t know if it can be expanded to other settings, with other populations. But if a school district had $1 million to spend, I think a program like this is a good investment.”
A 2010 study by economist Roland G. Fryer Jr. of Harvard University delivers a much more sobering message about the value of cash incentives for younger students. Mr. Fryer conducted a randomized trial of experimental programs in four large urban districts involving 38,000 children in grades 2 to 9. Although the program elements varied greatly from one district to the next, he found that paying for outputs, such as test results, didn’t work and in some cases resulted in lower scores. On the other hand, he found that paying students for inputs—showing up for class, staying on task, reading a certain number of books—had a positive effect.
Mr. Fryer speculated that students who don’t understand, for example, how hard work translates into better test scores are less likely to be motivated by the promise of cash rewards at the end of the term than by immediate reinforcements. “Student incentives based on inputs produce similar gains in achievement at lower costs,” he concludes in a paper posted by the National Bureau of Economic Research.
An Education Department official who requested anonymity acknowledges that paying students is controversial. “Poor kids don’t always know how to do well on tests by taking the time to study and by learning how to take a test,” the official notes. The official also admits that the NMSI approach assumes that “passing an AP test is a good thing—which is a big assumption.”
NMSI officials recognize that the program sets a high bar that may be difficult for some schools to clear. A school in which five students passed AP biology last year, for example, would be expected to record 37 passing scores this year, said Gregg Fischer, the director of the initiative’s AP training and incentive program. If its success rate was 16 percent last year, he said, that would mean enrolling some 200 students this year.
‘Lukewarm’ to Cash
At Green Run High School in Virginia Beach, Va., one of the schools getting Navy funding to pursue the NMSI program, many students “come in without all of the tools they need to succeed in an AP course,” said Principal George Parker. As a result, the school’s success rate to date on the end-of-year standardized tests has been in the single digits for some AP math and science courses. About one-fourth of the school’s 1,640 students come from the military community, which has a high student-turnover rate. The Joining Forces initiative hopes to provide a seamless transition for those students when their families relocate. (“Schools May Be Asked to Report on Progress of Military Children,” May 11, 2011.)
Students receive $100 for each math, science, and English AP test on which they score at least a 3, on a 5-point scale, and teachers earn $100 for every successful student, with the possibility of bonuses up to $3,000 for increasing AP participation, but Mr. Parker said money isn’t the real objective.
“The rigor of taking an AP course better prepares students for college, even if the students don’t pass the final exam,” he said. Many students are so far “lukewarm” toward the program, he added, because of the extra work required to be eligible for the payments. And Mr. Parker doesn’t think that the bonuses provide any additional motivation for his teachers to do their jobs well.
In many states, teachers’ unions are adamantly opposed to cash incentives, In addition to clashing with most labor agreements, incentives are viewed as undermining the learning process. In New York City, for example, the United Federation of Teachers declined to participate in that component of a program serving 31 schools that is otherwise modeled after the NMSI efforts. And in October, officials for the local nonprofit that runs the program, REACH, for Rewarding Achievement, decided to drop the student incentives as well when faced with a budget squeeze.
“We found that it prompted more students to take AP courses, but we didn’t see the magnitude effect that we had hoped,” said Kathrine Mott, REACH’s executive director. However, Ms. Mott said that her organization will continue to offer professional development for teachers, Saturday classes, and classroom grants.
