School & District Management

Science ‘Proficiency’ Wide Ranging Across States

By Erik W. Robelen — December 14, 2011 4 min read
  • Save to favorites
  • Print

A student in New Hampshire or Rhode Island is likely to have a much tougher time achieving a passing score on a state science exam than one in Virginia or Tennessee, a new analysis suggests. But don’t blame it on the schools.

The reason is that states set the bar for science “proficiency” at widely varying levels, concludes the report, issued last week by the business coalition Change the Equation in collaboration with the American Institutes for Research.

Billed as the first-ever national analysis of how states define proficiency on science assessments, the study found that states have “radically different targets” for what their 8th graders should know and be able to do in science. And in many instances, what a state has deemed a proficient score is equivalent to below “basic” on the National Assessment of Educational Progress in science.

“At a time when the demand for robust skills and knowledge in science has gone global, ‘proficiency’ may have more to do with where you live than what you have learned,” the report says. “This hodgepodge undercuts a major reason why we have tests in the first place: to provide reliable information on how well we’re preparing students for the challenges of the global economy.”

‘Does It Matter?’

Such analyses of state cutoff scores in reading and math have been going on for some time, and generally have reached the same conclusion, including a study issued last summer by the National Center for Education Statistics.

The new report comes as a major effort is under way to develop a set of common, “next generation” science standards, which could be an important first step toward creating more aligned, and more rigorous, expectations for students around the nation.

The study, looking at 37 states in which relevant data were available, compares the passing scores states set on their 2009 8th grade science tests by measuring them against the 2009 NAEP in science. The researchers took each state’s passing score and mapped it onto the 300-point NAEP scale, allowing them to equate states’ standards for “basic,” “proficient,” and “advanced” with NAEP scores.

In 15 of the 37 states examined—California, Connecticut, North Carolina, Texas, and Virginia, among them—the state bar for proficiency was actually lower than the NAEP threshold for basic. New Hampshire and Rhode Island were the only states that had a higher proficiency threshold than NAEP, while in Massachusetts, it was about the same.

But Tom Loveless, a senior fellow at the Brookings Institution, a Washington think tank, cautioned that looking at state cutoff scores may not say much about the quality of education students in a given state receive. In a quick analysis, he found no clear connection between where a state sets its proficiency target and how its students performed on the 2009 NAEP in science. He said, for instance, that NAEP achievement was all over the board for the 15 states the study says set their proficiency bars below the NAEP basic level. Six of those states scored above the national average for 8th grade science, five scored at about the same level as the national average, and four scored below it.

“To me the question is: Does it matter?” Mr. Loveless said of where states set the bar for proficiency. “And it turns out, it doesn’t. ... There is no statistically significant relationship between how high states set their cut points and how well they score on NAEP.”

Charles Pyle, a spokesman for the Virginia education department, made much the same point about his state. Although, according to the report, Virginia’s proficiency bar in science is set lower than all of the other states examined, its achievement levels tell a different story.

Virginia students scored above the national average on the 2009 science NAEP for both 4th and 8th grades. In fact, its 4th grade scores were among the highest for all states.

Mr. Pyle also noted that one important factor in Virginia is that science assessments are considered high-stakes tests for both students and schools.

In addition, some experts have long suggested that NAEP’s definition of proficiency in various subjects is too stringent.

Confusing Parents

Still, it seems clear from the new study that states do not agree on what level of science learning is needed. And the study warns that parents in many states may be getting a distorted view of student achievement.

“It gets confusing,” said Claus von Zastrow, the research director for Change the Equation, a coalition of more than 110 corporate chief executive officers working to improve stem education. “In states that set a particularly low bar, a parent might conclude that their child is doing very well, … but that child could be performing in the bottom quartile of all schools nationally.”

The report is part of an ongoing effort by the business organization to examine the condition of learning in the stem fields of science, technology, engineering, and mathematics. Earlier this year, it released a set of state-by-state stem “Vital Signs” reports. At that time, the group also sent letters to all the nation’s governors calling for higher proficiency standards in science and math.

The study also seeks to put state proficiency standards in context by comparing them with the findings of a 2009 study by ACT Inc. It notes that while two-thirds of the states examined reported that most of their 8th graders were proficient in science, the act analysis found that only 8 percent of U.S. 8th graders were on track to do well in introductory college science courses.

A version of this article appeared in the December 15, 2011 edition of Education Week as Science ‘Proficiency’ in One State Misses the Bar in Another

Events

School Climate & Safety K-12 Essentials Forum Strengthen Students’ Connections to School
Join this free event to learn how schools are creating the space for students to form strong bonds with each other and trusted adults.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Reading & Literacy Webinar
Creating Confident Readers: Why Differentiated Instruction is Equitable Instruction
Join us as we break down how differentiated instruction can advance your school’s literacy and equity goals.
Content provided by Lexia Learning
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
IT Infrastructure & Management Webinar
Future-Proofing Your School's Tech Ecosystem: Strategies for Asset Tracking, Sustainability, and Budget Optimization
Gain actionable insights into effective asset management, budget optimization, and sustainable IT practices.
Content provided by Follett Learning

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Q&A How K-12 Leaders Can Better Manage Divisive Curriculum and Culture War Debates
The leader of an effort to equip K-12 leaders with conflict resolution skills urges relationship-building—and knowing when to disengage.
7 min read
Katy Anthes, Commissioner of Education in Colorado from 2016- 2023, participates in a breakout session during the Education Week Leadership Symposium on May 3, 2024.
Katy Anthes, who served as commissioner of education in Colorado from 2016-2023, participates in a breakout session during the Education Week Leadership Symposium on May 3, 2024. Anthes specializes in helping school district leaders successfully manage politically charged conflicts.
Chris Ferenzi for Education Week
School & District Management Virginia School Board Restores Confederate Names to 2 Schools
The vote reverses a decision made in 2020 as dozens of schools nationwide dropped Confederate figures from their names.
2 min read
A statue of confederate general Stonewall Jackson is removed on July 1, 2020, in Richmond, Va. Shenandoah County, Virginia's school board voted 5-1 early Friday, May 10, 2024, to rename Mountain View High School as Stonewall Jackson High School and Honey Run Elementary as Ashby Lee Elementary four years after the names had been removed.
A statue of confederate general Stonewall Jackson is removed on July 1, 2020, in Richmond, Va. Shenandoah County, Virginia's school board voted 5-1 early Friday, May 10, 2024, to rename Mountain View High School as Stonewall Jackson High School and Honey Run Elementary as Ashby Lee Elementary four years after the names had been removed.
Steve Helber/AP
School & District Management Quiz Quiz Yourself: How Much Do You Know About the School District Technology Leader?
The tech director at school districts is a key player when it comes to purchasing. Test your knowledge of this key buyer persona and see how your results stack up with your peers.
School & District Management Deepfakes Expose Public School Employees to New Threats
The only protection for school leaders is a healthy dose of skepticism.
7 min read
Signage is shown outside on the grounds of Pikesville High School, May 2, 2012, in Baltimore County, Md. The most recent criminal case involving artificial intelligence emerged in late April 2024, from the Maryland high school, where police say a principal was framed as racist by a fake recording of his voice.
Police say a principal was framed making racist remarks through a fake recording of his voice at Pikesville High School, a troubling new use of AI that could affect more educators. A sign announces the entrance to the Baltimore County, Md., school on May 2, 2012.
Lloyd Fox/The Baltimore Sun via AP