Includes updates and/or revisions.
Congress is weighing plans to scuttle a $5 million evaluation of the national Upward Bound program for low-income high school students because the federal study calls for randomly assigning students to either the program or a control group.
Established in 1965 as part of the “war on poverty,” Upward Bound provided summer learning activities, mentoring, college counseling, work-study jobs, and other services to an estimated 61,000 students last year. Its aim is to increase college-going rates among students from disadvantaged families, particularly when neither parent has attended college.
“I speak often of a ladder of opportunity that takes energy, persistence, and responsibility to climb,” Sen. Tom Harkin, D-Iowa, a critic of the study, said in a statement. “Young people deserve to know that programs like Upward Bound will be there for them as they climb that ladder and that they will not lose that access for the purpose of an evaluation.”
The Department of Education is requiring 101 of the 700-plus universities, colleges, and other agencies receiving 2007 funds to run local Upward Bound programs to take part in the evaluation, which got under way this summer.
But legislative amendments approved in recent weeks by the full House of Representatives and the Senate education committee could effectively cripple the study. The Senate amendment, introduced by Mr. Harkin and Sen. Sherrod Brown, D-Ohio, and attached to the Higher Education Amendments Act, would bar the department from forcing Upward Bound programs to participate in evaluations that deny services to control-group students. As of last week, that measure awaited passage in the full Senate.
The House amendment, sponsored by Rep. Robert C. Scott, D-Va., was added to a student-loan reform package that the House approved on July 11. It also seeks to rescind the requirement to take part in the study, as well as several other controversial changes to the program the department made in September.
Implications for Research
The concerns in Congress pose a challenge to the Bush administration’s ongoing efforts to seed and promote randomized experiments in education.
The Education Department’s top research officials see such scientific experiments, which are widely seen as the “gold standard” for determining whether a program works, as a way to transform education into an “evidence-based practice” not unlike medicine.
Meanwhile, conflict between the department and Capitol Hill over the long-running Upward Bound program dates back to at least 2005, when President Bush, in his proposed budget, called for killing it. His recommendation came after the White House Office of Management and Budget gave Upward Bound, and many other federal education programs, an “ineffective” rating.
Faced with congressional opposition to eliminating Upward Bound, the department decided instead to retool and re-evaluate it. Federal education officials in September contracted with Abt Associates, a Cambridge, Mass.-based research group, the Washington-based Urban Institute, and the Oakland, Calif.-based Berkeley Policy Associates to carry out the study that is now in jeopardy.
Both the rule changes and the evaluation effort have run into heavy resistance from grantees and their Democratic allies in Congress.
“You can’t tell a kid, ‘You’re going to be in this life-changing program,’ and then say, ‘No you’re only going to be in the control group,’ ” said Susan Trebach, a spokeswoman for the Council for Educational Opportunity, a Washington-based group that represents administrators of Upward Bound and other federal college-access programs. “We already have some people telling us the kids they deal with are devastated.”
Removing Bias?
Advocates for such programs contend randomized experimentation is the wrong way to study their efforts because program operators typically require students to undergo an elaborate application process as a way to demonstrate commitment to the program and winnow out candidates not likely to succeed.
“We have to get parents to actively support the whole process or … kids drop out,” Ms. Trebach said. But Education Department officials said that’s precisely why a randomized study is needed: Participants may be more motivated than students who don’t apply to Upward Bound. The only way to remove that potential bias, the department maintains, is to compare participants with applicants who failed to win a spot in Upward Bound.
“If that control group is removed, it would prevent us from estimating the impact on student outcomes, determining which practices are most promising, or determining whether the impact is more evident under certain circumstances than others,” said Ricky P. Takai, the associate commissioner for evaluation in the department’s National Center for Educational Evaluation and Regional Assistance, which is overseeing the study.
If it’s not halted, Mr. Takai said, the study will also enable federal officials to weigh the impact of last fall’s rule changes, which restricted the traditional program to students in 10th grade or younger and required grantees to set aside 30 percent of their slots for students deemed at high risk for academic failure.
“Some critics said changing the program to include at-risk children could have a negative effect on traditional students in the program,” Mr. Takai said. “This looks at the impact on all students.”
