Assessment

Test Industry Split Over ‘Formative’ Assessment

By Scott J. Cech — September 16, 2008 7 min read
Testing expert Richard J. Stiggins says he has stopped using the term “formative assessment.”

There’s a war of sorts going on within the normally staid assessment industry, and it’s a war over the definition of a type of assessment that many educators understand in only the sketchiest fashion.

Formative assessments, also known as “classroom assessments,” are in some ways easier to define by what they are not. They’re not like the long, year-end, state-administered, standardized, No Child Left Behind Act-required exams that testing professionals call “summative.” Nor are they like the shorter, middle-of-the-year assessments referred to as “benchmark” or “interim” assessments.

Or they shouldn’t be, at least according to experts inside and outside the testing industry, who believe that truly “formative” assessments must blend seamlessly into classroom instruction itself.

“It makes me want to scream and run out of the room,” said Ray Wilson, the executive director of assessment and accountability for the 33,000-student Poway Unified School District in Poway, Calif., referring to off-the-shelf commercial products labeled “formative assessment” that major test-makers try to sell him. “I still contend that so long as a teacher doesn’t have primary control [over assessment content],” he added, “you will never have truly formative assessment.”

“I had not heard, frankly, that there was some resistance to companies selling formative assessment,” said Robert Block, the associate vice president for K-12 learning and development at the Princeton, N.J.-based Educational Testing Service, which sells test questions in bulk to schools as “formative assessment item banks.”

But he pointed out that school districts’ requests for proposals often ask specifically for formative assessments.

“It has become the standard,” he said of the testing industry’s practice of labeling some assessment products as “formative.”

“I’m not sure if it’s good or bad—it’s just what the market is looking for.”

‘AssessMints’

The schism was in full view at the Council of Chief State School Officers’ National Conference on Student Assessment in Orlando last June. In the main exhibit hall, lined with competing assessment-makers, Peter D. Hofman, the vice president of marketing and business development at the Dover, N.H.-based testing company Measured Progress, handed out boxes of mints labeled “AssessMints” to strolling psychometricians and state-assessment directors.

“These are the only legitimate formative-assessment products,” declared Mr. Hofman, whose company does not sell formative-assessment materials.

He left unsaid his implicit message: The room’s rival exam firms—like almost every other testing company in the country—sell something that’s not what it purports to be.

It’s not just semantic consistency that’s at stake: Formative assessments are a more than half-billion-dollar business in the United States, according to the latest analysis by Outsell Inc., a Burlingame, Calif.-based research and advisory firm for the information, education, and publishing industries.

That total, including tests, test delivery, scoring, scoring analysis, professional development, and other services, but not such bundled products as textbooks, accounted for about 30 percent of the $2.1 billion in overall assessment revenue generated in the United States in the 2006-07 academic year—the most recent year for which statistics are available.

What’s more, said Laurence Bloom, an affiliate analyst for Outsell, the latest estimates project that formative-assessment revenue will climb at a rate of 10 percent to 20 percent per year through 2010, making it the fastest-growing segment in the assessment market.

That’s a lot of money being spent on something that experts say can’t really be sold—only practiced.

More Than Word Choice

Measured Progress is one of the few large testing companies taking a pass on the lucrative market. That’s mostly because Stuart R. Kahl, the company’s president and chief executive officer, takes his cues, as many other testing experts do, from a 1998 research-literature review of Paul Black and Dylan Wiliam, then at King’s College, University of London. The review, which predated the formative-assessment market, concluded that the research to date showed achievement gains using formative-assessment strategies that were “among the largest ever reported for educational interventions.”

“We use the general term ‘assessment’ to refer to all those activities undertaken by teachers—and by their students in assessing themselves—that provide information to be used as feedback to modify teaching and learning activities,” they wrote. “Such assessment becomes formative assessment when the evidence is actually used to adapt the teaching to meet student needs.”

Product or Approach?

“What they’re talking about is not what people are selling; it’s what teachers are doing day in and day out,” said Mr. Kahl, whose company produces state standardized tests used by Kentucky, Massachusetts, Montana, New Hampshire, Rhode Island, and Vermont.

“People say, ‘You’re just being picky about a word choice.’ I think it’s more serious than that,” Mr. Kahl said. “The consumers out there ... are being told [in advertisements and pamphlets] that these [products] satisfy their formative needs. They believe that and purchase it and don’t think they have to do anything more for formative assessment.”

With so much money at stake for products sold under the label “formative,” said W. James Popham, a professor emeritus of education at the University of California, Los Angeles, and the author of the 2008 book Transformative Assessment, “They’d call it ‘Clytemnestra’ if they thought it would sell,” referring to the mythological Greek figure best known for helping kill her husband.

When test companies sell things and call them formative, he said, “these vendors are being disingenuous—we used to call it lying.”

Calls seeking comment from the Maple Grove, Minn.-based Data Recognition Corp. and from Pearson Education, a division of the global media giant with headquarters in London and New York City, were not returned.

In response to criticism of commercial “formative assessment” products, Hillary Michaels, a senior research scientist for Monterey, Calif.-based CTB/McGraw-Hill, said at a National Conference on Student Assessment session that “when it comes to formative assessment, we need to rethink some of the ways we’re accustomed to thinking of testing.”

Avoiding the Term

Other experts acknowledge that the lack of widespread understanding about the nature of formative assessment makes the term vulnerable to appropriation, but don’t necessarily agree that testing companies are deliberately obfuscating the distinction.

“I would say that the interests of any business in maintaining their revenues is such that when the world around them changes, they redescribe what they have and repurpose what they have,” said Neal M. Kingston, an associate professor of the Program in Research, Evaluation, Measurement, and Statistics at the University of Kansas in Lawrence who has held high-ranking positions at Measured Progress, CTB/McGraw-Hill, and the ETS. He now helps create the state’s annual standardized Kansas Assessment Program exam.

“It’s not that there’s complete agreement on what makes something ‘formative’—it’s all how you’re using it,” Mr. Kingston said.

Richard J. Stiggins, the executive director of the Portland, Ore.-based Assessment Training Institute, mostly stays away from the term. “There’s not universal agreement on what it means—I’ve actually stopped using the word,” said Mr. Stiggins. He said that he now usually refers to formative assessment as “assessment for learning” to cut down on ambiguity.

Mr. Stiggins said he doesn’t believe testing companies are shading the truth, but he added, “Merely selling people a test on the assumption that it will be used formatively won’t cut it.”

In-House Differences

No one whom Education Week asked would identify a specific product or company that he or she thought was misusing the term. Many experts were leery of pointing to a company, in part because there are often differences of opinion within a company itself.

“It’s really hard to do that because even within companies, there’s a wide range,” said Mr. Kingston. His own university, for example, uses the word formative when referring to an assessment its staff created.

Another example is Mr. Stiggins. Although he has made the assertion “formative assessment isn’t something you buy—it’s something you practice” something of a mantra in the many talks he gives to educators around the country, his company is owned by the ETS, which sells item banks under the label “formative.”

Within any testing company, said Mr. Kingston, “you have your curriculum experts, you have your psychometric experts, you have your marketing and business people, and they all approach the needs that are out there in different ways.”

Mr. Kahl, the CEO of Measured Progress, said he sympathizes—to a point. “I know of individuals within those companies that feel [ambivalent about some products],” he said. “On the the other hand, you look at the [product] catalogs and you see things in there labeled ‘formative assessment,’ and you’ve got to wonder who’s driving the truck in those big companies.”

Coverage of new schooling arrangements and classroom improvement efforts is supported by a grant from the Annenberg Foundation.
A version of this article appeared in the September 17, 2008 edition of Education Week as Test Industry Split Over ‘Formative’ Assessment

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Mathematics Webinar
Building Equitable Systems: Moving Math From Gatekeeper to Opportunity Gateway
The importance of disrupting traditional American math practices and adopting high-quality math curriculum continues to be essential for changing the trajectory of historically under-resourced schools. Building systems around high-quality math curriculum also is necessary to
Content provided by Partnership for L.A. Schools
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Measuring & Supporting Student Well-Being: A Researcher and District Leader Roundtable
Students’ social-emotional well-being matters. The positive and negative emotions students feel are essential characteristics of their psychology, indicators of their well-being, and mediators of their success in school and life. Supportive relationships with peers, school
Content provided by Panorama Education
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
School & District Management Webinar
Making Digital Literacy a Priority: An Administrator’s Perspective
Join us as we delve into the efforts of our panelists and their initiatives to make digital skills a “must have” for their district. We’ll discuss with district leadership how they have kept digital literacy
Content provided by Learning.com

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Assessment Opinion Alternatives to Standardized Tests During a Pandemic Year
Three educators suggest alternatives to federally mandated standardized testing during this year undercut by COVID-19.
7 min read
Images shows colorful speech bubbles that say "Q," "&," and "A."
iStock/Getty
Assessment Opinion AP Exams Can't Be Business as Usual This Year
The College Board seems unconcerned with the collateral damage of its pandemic approach, writes an assistant superintendent of curriculum and instruction.
Pete Bavis
5 min read
Illustration of large boat in turbulent waters with other smaller boats falling into the abyss.
iStock/Getty Images Plus
Assessment Federal Lawmakers Urge Miguel Cardona to Let States Cancel Tests, Highlighting Discord
A letter from Democratic members to the new education secretary calls for an end to the "flawed" system of annual standardized exams.
3 min read
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
Jamaal Bowman speaks to reporters after voting at a polling station inside Yonkers Middle/High School in Yonkers, N.Y. on June 23, 2020.
John Minchillo/AP
Assessment How Two Years of Pandemic Disruption Could Shake Up the Debate Over Standardized Testing
Moves to opt out of state tests and change how they're given threaten to reignite fights over high-stakes assessments.
9 min read
Image of a student at a desk.
patat/iStock/Getty