Working Behind the Scenes on Reports, Md. Company Finds Its Strength in Numbers
ROCKVILLE, MD--Over the past few weeks, the Colorado Department of Education released a study on student performance in geography, an international research organization put out a 32-nation study of reading literacy, and the National Education Goals Panel issued its second annual progress report.
The reports all made headlines and provoked debate over the current state of education.
But what these reports have in common is something that did not appear in the news stories about them: They all were put together with the help of Westat Inc., a research firm based here in this suburb of Washington.
Specializing in data collection and analysis, Westat is one of a half-dozen companies that actually conduct much of the work that goes out under the name of government agencies like the National Center for Education Statistics, the U.S. Labor Department, and the Environmental Protection Agency.
Although Westat does not "dominate the market,'' it usually ranks high in the fierce competitions for federal contracts, according to Paul D. Planchon, the associate commissioner of the N.C.E.S. for elementary and secondary education.
"We have gone to them consistently over the years,'' Mr. Planchon says. "They are a real solid survey organization.''
Some observers have suggested, however, that the government agencies should conduct studies themselves, rather than rely on contractors to do the work for them. In-house expertise is an irreplaceable commodity, these observers contend.
But Westat officials respond that a contractor, which can draw from work in a variety of fields, can bring skills that an agency employee, focusing on a single topic, can seldom provide.
"People can cut their own hair, and can buy a pair of clippers,'' says Alexander Ratnofsky, a vice president. But "most people go to a barber.''
Off the 'Technology Corridor'
Located just off the Interstate 270 "technology corridor,'' Westat is a testament to the growing complexity of the science of statistics.
Founded in 1961 by Edward C. Bryant, the head of the statistics department at the University of Wyoming (hence the name, from west statistics), the firm--which since 1977 has been owned by its employees--now has 700 employees and annual revenues of $100 million.
In addition to its main headquarters here, it also has three separate telephone-research and data-processing centers in nearby Gaithersburg, Md., in Frederick, Md., and in Oceanside, Calif.
Although it has recently branched out into a range of consulting services--including helping advise school districts on "total quality management''--it has hewed to its main strength, statistics.
"Most of the work we do as a company involves the process of data collection,'' Mr. Ratnofsky says.
That process, he notes, includes a range of services, from designing questionnaires to analyzing the data and writing reports.
"Any one contract could have some or all of those,'' he says.
'No Substitute for Experience'
Perhaps its main strength is in selecting the sample of people to interview.
Because of the sheer size of the school population, it is impossible to interview every student or teacher in order to conduct a study. So researchers choose a representative portion of the population.
But this must be done with care. Choose the wrong sample, and a reader can draw incorrect inferences from the results of the study.
The quality of samples has been a major issue, for example, in interpretations of international studies of student achievement. Some researchers have criticized cross-national studies of mathematics and science performance for failing to compare like populations of students. In an effort to correct for that problem, the Educational Testing Service in 1991 contracted with Westat to draw samples of students in the 20 countries that took part in the second International Assessment of Educational Progress.
"Westat is internationally known for its sampling,'' Mr. Ratnofsky says.
After drawing the sample, the firm also conducts surveys, sometimes in person; more often, this is done over the telephone. Westat's three research centers each include about 60 interviewing stations that enable employees to conduct interviews over the telephone and enter responses directly into a computer--the state of the art in survey technology, according to Mr. Planchon of the N.C.E.S.
Mr. Ratnofsky says that one of the firm's assets is its experience in conducting surveys. Partly because of employee ownership, the company has a low turnover rate, he says, and "there is no substitute for that experience.''
"The steps may be standard, but how they are implemented is not,'' he says. "It is much more an art than a science.''
"You can write in a book, 'Be sensitive to respondents,' '' he continues. "But until you've gone through several surveys, you just don't know'' how to do that.
David L. Bayless, another vice president, also notes that the company stresses the importance of not imposing on schools.
"When people go to training,'' he says, "we make one point: When you go to a school or a school district, you are a guest of that school or school district.''
"There is a burden to data collection,'' Mr. Bayless adds. "But schools are out there for teaching and learning.''
Range of Projects
A list of the firm's projects includes many of the major education studies the federal government conducts.
It has for years, for example, served as a subcontractor to the E.T.S. in designing samples for the National Assessment of Educational Progress, a Congressionally mandated project that tests about 100,000 students every two years in reading, writing, mathematics, and other subjects.
The company has also contracted with the National Center for Education Statistics to conduct the National Household Education Survey, a poll of some 60,000 households on such issues as early-childhood education, school safety, and discipline.
Mr. Bayless notes that the household survey is a "natural'' for Westat, because of its experience in surveying families on health and the environment. The new survey, he notes, represents the first attempt by the N.C.E.S. to gather education data outside of schools.
"People in education might take time getting the concept that you can get information from households about education,'' he says. "That's not a new subject for us to deal with.''
In addition to the survey work, Westat is conducting analyses of high school and college transcripts to learn about student and teacher course-taking.
For example, one project is examining the college transcripts of teachers who took part in the Schools and Staffing Survey, a major study by the N.C.E.S. Although teachers in the survey told interviewers what courses they took, the transcript study will provide more reliable information, according to Mr. Bayless.
"They'll say, 'Did I have a course in math 20 years ago?' '' Mr. Bayless says.
Drawing on its expertise in data collection, the company has also worked with school districts to help them improve their ability to gather and analyze information.
Last year, for example, Mr. Bayless headed a project to develop standards for data collection and reporting. And, under a contract with the U.S. Centers for Disease Control, the firm is working with state education agencies and 15 school districts to assist them in conducting surveys on risk factors associated with the virus that causes AIDS.
"We are sharing with these agencies, so they can see that they in
fact can do quality work,'' Mr. Bayless says. "We don't have to do it