Assessment

Researchers Find ‘Surprises’ In Mass. Scores

By John Gehring — March 13, 2002 4 min read

Compared with their wealthy counterparts, many low-income school districts in Massachusetts are doing more with less when it comes to performance on state exams, a new analysis contends.

The recently released research, which evaluates a wide range of factors that could influence test performance, also found that school spending patterns can affect test scores. And it concludes that the more school districts spend on sports, the worse they do on the exams.

Because it identifies better-than-expected scores among some low-income schools, the study also challenges the belief that demographics are destiny when it comes to performance on the Massachusetts Comprehensive Assessment System, which students in the class of 2003 must pass in English and mathematics to graduate.

“Still near the top [of the performance rankings] are some affluent, high-scoring districts whose performance has been touted in the media on the basis of raw scores,” the report released Feb. 20 states."But the bottom of the list contains some stunning surprises, including several of the most affluent districts in the state.”

The study, “School District Performance Under the MCAS,” is one of the most comprehensive statistical reviews ever of the controversial MCAS exams.

Thomas Ferguson, a professor of political science at the University of Massachusetts, Boston, along with Jie Chen, a statistician there, wrote the report. But the 104-page study has drawn criticism, in part, because its statistical review places some of the state’s most affluent schools at the bottom of its performance rankings.

State Education Commissioner David P. Driscoll, for one, has dismissed the report as using “elaborate formulations” that produced results at odds with what has actually taken place in districts around the state.

“The bottom line is it doesn’t seem to match up with reality,” Mr. Driscoll said. Since the paper looks at MCAS performance from 1998 through 2000, Mr. Driscoll added that the analysis sheds little light on the dramatic improvements many districts made last year.

The MCAS became the centerpiece of reform efforts in Massachusetts after lawmakers passed a sweeping education reform act nine years ago. (“Mass. School Policies Praised as Test Scores Rise,” Oct. 24, 2001.)

The exams have gained strong support from advocates who argue that the MCAS has helped bolster academic expectations for all students. But critics, including Massachusetts’ largest teachers’ union and some students who have boycotted the exams, view the MCAS as a flawed, one-size-fits-all measure of achievement.

‘Value Added’

Among other areas, the authors found that teachers’ and superintendents’ salaries, spending on athletic programs, political races, and state aid all affected district MCAS results.The researchers evaluated every school district in the state by calculating their “grand average” on the exams, which were given to 4th, 8th, and 10th graders in 1998, 1999, and 2000.

Their model also examined districts by using a “value-added” analysis that attempts to estimate the three-year average score that would be predicted for each district purely on the basis of its wealth and demographics. This score is then compared with the district’s actual average to show how much the district over- or under-performed. Using this model, the authors ranked districts and found that some school systems like the Dover- Sherborn district and the Weston public schools, two of the wealthiest in the state, performed well on raw scores, but fared far worse on the MCAS after their economic and demographic advantages were taken into consideration.

In contrast, some less wealthy districts like Lenox, Belmont, and Nauset did far better than expected.

“Here, at last, is a list of high-performing schools whose entries cannot possibly be dismissed as simple reflections of socioeconomic advantage,” the authors write. Mr. Ferguson said in an interview that poverty still consistently remains a “major educational handicap,” but that much of education research has too often ignored other influences that fall outside teacher control.

For example, one of the most intriguing findings of this paper, he said, is that the more a district spends on athletics, the worse their MCAS performance. The researchers found that for each percent a district spends of the amount targeted for athletics in the state’s foundation budget for that district, MCAS scores fall by about a hundredth of a point.

While this may not sound significant, he said, some districts spend more than 300 percent of their targeted athletic spending and could be dragging down their scores three or four points."In terms of effects, this is huge,” Mr. Ferguson said. “The athletic spending point is an overwhelmingly important one because it’s not something under teacher control. If you don’t have a measure for athletic spending you have a major impact you’re not taking into account.”

Range of Factors

Perry P. Davis, the superintendent of the suburban 1,900-student Dover-Sherborn district that finished second-to-last among 226 districts ranked by the value-added research, said he plans to meet with the researchers. Mr. Davis doesn’t understand the authors’ statistical technique, but said he looks forward to learning more about how they evaluated test data. “As a good teacher, if I don’t understand something I want to figure it out,” Mr. Davis said.

The authors defended the value-added formula as a common technique that provides a more sophisticated analysis of data. “This is not a new technique. We didn’t invent this scheme,” Mr. Ferguson said.

Other findings suggest that MCAS scores have been improved moderately by the state’s almost $7 billion investment in K-12 education since 1993. Raising maximum teacher salaries and superintendents’ salaries by $1,000, researchers found, led to slightly higher MCAS scores.

Related Tags:

A version of this article appeared in the March 13, 2002 edition of Education Week as Researchers Find ‘Surprises’ In Mass. Scores

Events

School & District Management Live Online Discussion A Seat at the Table With Education Week: What Did We Learn About Schooling Models This Year?
After a year of living with the pandemic, what schooling models might we turn to as we look ahead to improve the student learning experience? Could year-round schooling be one of them? What about online
School & District Management Webinar What's Ahead for Hybrid Learning: Putting Best Practices in Motion
It’s safe to say hybrid learning—a mix of in-person and remote instruction that evolved quickly during the pandemic—is probably here to stay in K-12 education to some extent. That is the case even though increasing
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Mathematics Webinar
Building Equitable Systems: Moving Math From Gatekeeper to Opportunity Gateway
The importance of disrupting traditional American math practices and adopting high-quality math curriculum continues to be essential for changing the trajectory of historically under-resourced schools. Building systems around high-quality math curriculum also is necessary to
Content provided by Partnership for L.A. Schools

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion AP Exams Can't Be Business as Usual This Year
The College Board seems unconcerned with the collateral damage of its pandemic approach, writes an assistant superintendent of curriculum and instruction.
Pete Bavis
5 min read
Illustration of large boat in turbulent waters with other smaller boats falling into the abyss.
iStock/Getty Images Plus
Assessment Federal Lawmakers Urge Miguel Cardona to Let States Cancel Tests, Highlighting Discord
A letter from Democratic members to the new education secretary calls for an end to the "flawed" system of annual standardized exams.
3 min read
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
John Minchillo/AP
Assessment How Two Years of Pandemic Disruption Could Shake Up the Debate Over Standardized Testing
Moves to opt out of state tests and change how they're given threaten to reignite fights over high-stakes assessments.
9 min read
Image of a student at a desk.
patat/iStock/Getty
Assessment A Plan for Standardized Test Scores During the Pandemic Has Gotten States' Attention
A testing expert says his idea would provide helpful data with key context, but said other measures about student well-being are crucial.
7 min read
HS class 1257213326
Getty