School & District Management

Researchers Call For Making Data Widely Available

By Debra Viadero — November 05, 2003 8 min read
  • Save to favorites
  • Print

Although sharing is one of the first lessons children learn in school, a few prominent education scholars are suggesting some of their colleagues ought to take a refresher course in the subject.

The issue: Investigators are often reluctant to share data from their studies so other researchers can reanalyze the information, replicate the studies, or expand on the findings.

“We need cooperation and collaboration and sharing in a way we haven’t in the past,” said Barbara Schneider, a sociologist at the University of Chicago. “We’re in the heart of education reform, and we have tremendous amounts of resources being funneled into a lot of experiments.”

Ms. Schneider, who is a senior social scientist at the university’s National Opinion Research Center, has led some of the calls for researchers to be more forthcoming with the data they collect—particularly when the federal government is paying for their work.

She has been joined by other scholars, foundation officials, and policymakers, all of whom contend that their fellow researchers need to “share and share alike” for the field to amass a credible store of knowledge on what works in education.

The calls for collaboration still do not represent a consensus, however. Some researchers have cautioned against pushing too hard and fast to make data-sharing a standard practice. That’s especially true, they add, when research subjects’ privacy is at stake.

“It would absolutely kill ethnography,” said Mary Haywood Metz, a University of Wisconsin-Madison researcher who specializes in qualitative studies looking at the internal social and emotional workings of school communities. While it’s admirable that some individual researchers have taken it upon themselves to make their data public after their papers are published, she said, “to set that up as a standard for everybody to follow is quite another matter.”

Privacy High Wire

A panel organized by the National Research Council, an arm of the National Academies of Sciences, is exploring ways to make education research more “scientific.” The group, known as the Committee on Research in Education, was interested enough in the issue of data-sharing to include it in a workshop last summer on accumulating a knowledge base in education.

“They spend many years collecting their data, and they want to make sure they have first crack at it,” Kay Dickerson, a Brown University professor of community health and member of the panel, said of researchers’ attitudes. “On the other hand, sometimes researchers take a long time getting to it, and that’s frustrating.”

Sometimes, too, information goes unused because researchers’ grants do not cover the cost of “cleaning up” the data for public use. That means stripping away any information that could be used to identify study subjects and adding in enough context to make the data understandable.

Protecting subjects’ privacy, though, remains the main reason most researchers avoid opening up their files to outside inspection. In qualitative studies and small-scale quasi-experiments, researchers say, it’s hard to disguise everyone’s identity. A school might have just one art teacher, for instance. Or a small city might have a single high school principal.

Also, if subjects know that their views and actions will be widely shared, the University of Wisconsin’s Ms. Metz said, they might be less forthcoming—or refuse to participate in the study altogether.

“People won’t tell you what’s going on unless they can trust you,” she noted.

On the other hand, she said, when researchers remove too much identifying information, they risk losing some of the context that outside scholars need to understand what’s being studied.

Such debates are not unique to education.

Just last year, the federal National Institutes of Health, after vetting the opinions of researchers in the health fields, issued new rules requiring grant recipients to make all their research materials available to the public once their studies are accepted for publication.

Likewise, the Washington-based American Sociological Association—a group that also includes education researchers— revised its ethics code in the mid-1990s to explicitly encourage its members to share their data with colleagues.

“I don’t necessarily see educational research as behind in this,” said Felice Levine, the executive director of the American Educational Research Association, which also is located in Washington. Ms. Levine was the executive director of the sociological association when its ethics code was updated, and she served on a National Science Foundation panel on data-sharing.

In her current organization, she said, such policies are implicit in the “normative fabric’’ of the ethics code. “Could a code be more explicit?” she said. “It might be.”

Publication Pressures

Regardless of the status of data-sharing in other fields, Ms. Schneider of the University of Chicago and others suggest that the practice could be more ingrained in education research than it is now.

They offered this case in point: Researchers who publish in the journals Nature, Science, the American Sociological Review, the American Economic Review, and Demography all must agree to share their work when their articles are published. Researchers said few, if any, education research journals have similar requirements.

Similarly, the U.S. Department of Justice’s National Institute of Justice requires the researchers it supports to make their data public.

The U.S. Department of Education has no such blanket requirements for its researchers. That’s not to say that no department-financed studies are made widely available. All the data collected by the department’s National Center for Education Statistics or one of its contractors have long been open to the public. And the department’s Institute of Education Sciences, newly required by federal law to do the same, is making plans to follow NCES’ example.

Some of those federal data sets, though, are still a year or two from being widely available online or on CD-ROMs, according to Grover J. “Russ” Whitehurst, the institute’s director.

Likewise, Ms. Schneider is spearheading a project to develop a common database from all the information gathered from a federally funded, cross-agency research initiative on education.

Mr. Whitehurst said he doubts, however, that the Education Department will ever be able to demand that the individual researchers it subsidizes through the field-initiated-studies grant program or other smaller-scale research efforts put up their data for all to use.

While data-sharing may be a good way to get the “biggest bang for the buck” out of federal research expenditures, he said, “there are, in most cases, severe confidentiality issues that constrain distribution of that data.” But when it comes to informing public policy, some researchers say timely sharing of data becomes a moral obligation.

“If researchers want to hoard data, that’s fine, but I think the rest of the policy community should ignore the results until someone else has had the opportunity to analyze the data,” said Alan B. Krueger, a Princeton University economist who has sought access to data from school voucher experiments in New York City, the District of Columbia, and Dayton, Ohio.

Mr. Krueger said his own attempts to reanalyze the data from the three-city school choice experiments have been “eye-opening.” Although results from those experiments were released last year, the Princeton economist could not persuade Harvard University researcher Paul E. Peterson and his colleagues to part with their data from Dayton and Washington.

But Mathematica, the Princeton, N.J.-based research group that had worked with Mr. Peterson on his New York study, complied. Mr. Krueger reanalyzed the data using a different method for categorizing students by race. And he came up with some distinctly different results. (“Study: No Academic Gains From Vouchers for Black Students,” April 9, 2003.)

In an e-mail message, Mr. Peterson explained that he had declined to share the data because members of his research team are still publishing more papers from the results.

In its earlier analysis, the Mathematica-Peterson research team found that the privately financed tuition vouchers had led to sizable learning gains for the African-American students who used them to transfer to private schools. They outperformed a similar group of disadvantaged students who had also sought the vouchers, but didn’t win them in the lottery. Mr. Krueger’s refiguring, however, found those gains statistically insignificant.

That reanalysis did not end the debate on the benefits of school choice. Mr. Peterson continues to dispute Mr. Krueger’s conclusions. Experts say the example does, however, point up the need to cast a second eye on controversial studies that might be used to inform public policy.

Stepping up cooperation among researchers is important, they argue, because it offers another possible route for improving the quality of educational research.

One such proponent is Marshall S. Smith, the director of educational programs for the Menlo Park, Calif.-based William and Flora Hewlett Foundation, who was a top education official in the Clinton administration. The foundation has been pushing to make research data and other information resources from higher education more widely available.

“It introduces transparency, which creates a level of accountability which hasn’t existed before,” Mr. Smith said.

When someone is looking over their shoulders, he added, “you make people much more rigorous about carrying out their analyses.”

Coverage of research is underwritten in part by a grant from the Spencer Foundation.

Related Tags:


This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Data Webinar
Working Smarter, Not Harder with Data
There is a new paradigm shift in K-12 education. Technology and data have leapt forward, advancing in ways that allow educators to better support students while also maximizing their most precious resource – time. The
Content provided by PowerSchool
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
School & District Management Webinar
Deepen the Reach and Impact of Your Leadership
This webinar offers new and veteran leaders a unique opportunity to listen and interact with four of the most influential educational thinkers in North America. With their expert insights, you will learn the key elements
Content provided by Solution Tree
Science K-12 Essentials Forum Teaching Science Today: Challenges and Solutions
Join this event which will tackle handling controversy in the classroom, and making science education relevant for all students.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

School & District Management Democrats Try to Stamp Out School Closures as Omicron Surges
Democrats are speaking out more forcefully against COVID-19 school closures, recognizing a rising anger among parents.
6 min read
President Joe Biden greets Chicago Mayor Lori Lightfoot at O'Hare International Airport in Chicago, on Oct. 7, 2021. Democrats are speaking out against school closures even as the omicron surge puts additional pressures on public schools. Scattered teachers unions have called for closures, and a handful of districts have switched to virtual learning because too many educators have gotten sick.
President Joe Biden greets Chicago Mayor Lori Lightfoot at O'Hare International Airport in Chicago, on Oct. 7, 2021. Democrats are speaking out against school closures even as the omicron surge puts additional pressures on public schools. Scattered teachers unions have called for closures, and a handful of districts have switched to virtual learning because too many educators have gotten sick.
Susan Walsh/AP
School & District Management Explainer Teachers' COVID Sick Leave, Explained
Who gets paid? Are unvaccinated teachers eligible? How long should teachers be allowed to take off? Districts' answers will surprise you.
7 min read
sick leave 529156651 b
School & District Management Staff Shortages Are Bringing Schools to the Breaking Point
High rates of sick staff members have forced districts to move some of their schools temporarily back to remote learning.
6 min read
Image of staffing diagram.
Bill Oxford/E+
School & District Management Letter to the Editor The More Opportunities Students Get, the Better
Giving students jobs in schools shouldn't be controversial, writes U.S. Representative Virginia Foxx in this letter to the editor.
1 min read
Illustration of an open laptop receiving an email.