Opinion
Education Opinion

Portfolio Folly

By Peter N. Berger — January 14, 1998 4 min read
  • Save to favorites
  • Print

It’s no secret that public schools today lack credibility. We sail from one fad to the next on a tidal wave of jargon, trailing the wreckage of our latest “instructional innovation” behind us, and then we wonder why regular people find it tough to believe we know what we’re doing.

Take portfolios, for instance. Here in Vermont they tell us we’re “leading the nation.” If so, the news isn’t good for the rest of the country.

What exactly are portfolios? They’re folders where kids keep their work. If this sounds like less than an education breakthrough, that’s only because it is less than an education breakthrough.

Not that there’s anything wrong with keeping portfolios. Plato probably carried one to Socrates’ classroom. But boosters today promote portfolios as “accurate measures of educational performance,” offering “credible,” “standardized” data. Unfortunately, portfolio scoring simply doesn’t work.

Portfolios are scored according to rubrics, which detail theoretically “objective” standards and criteria. In Vermont, for example, writing is judged in five “dimensions"--Purpose, Organization, Details, Voice, and Grammar, Usage, and Mechanics--and awarded one of four scores in each category--Extensively, Frequently, Sometimes, or Rarely. These scores translate into a four-point scale. Innovators naturally insist that portfolio 4.0’s and 3.0’s have nothing to do with old fashioned 4.0’s and 3.0’s--also known as A’s and B’s.

In the effort to improve student assessments, have educators lost sight of the purpose of good tests? Share your views in our on-line Town Meeting.

Suppose you’re trying to rate a piece’s Details. If you think the details are “explicit,” you score it an Extensively. If they’re only “elaborated,” of course, then you just give it a Frequently. Rating Organization requires that you distinguish between problems that “affect unity or coherence"--a Sometimes--and those that “make writing difficult to follow"--a Rarely. Scoring Voice is even easier. All you have to do is detect the difference between a “distinctive” tone, an “effective” tone, an “attempt” at an “appropriate” tone, and an “appropriate” tone that’s “not evident.” Next you can decide if the piece contains errors that “may distract the reader,” or errors that “interfere with understanding.”

This is “objective”?

By the way, portfolio folks promote all this on the grounds that parents “like the idea of clearly defined standards.” Unfortunately, most teachers--and most regular people--have a hard time deciding, with statistical reliability, whether they’re being “distracted” or their understanding is being “interfered” with.

In order to “calibrate” their judgment with that of the “experts,” teachers score sets of writing samples called benchmarks. Inconveniently, Vermont teachers statewide traditionally disagree with experts’ benchmark scores more often than they agree with them. Even the “experts” commonly disagree with each other. Which scores are the “right” scores? You tell me.

In an effort to eliminate the benchmark warfare that’s plagued scoring from the start, officials introduced “line pieces.” These new samples were supposed to indisputably define the absolute best a kid’s composition could be before it deserved the next higher score.

Unfortunately, this remedy rested on the breathtaking assumption that raters previously unable to agree on the difference between an “interference” and a “distraction” would somehow be able to agree on the difference between the very best “interference” and the very worst “distraction.” Not surprisingly, line pieces haven’t helped clear things up.

As a result, portfolio scoring remains notoriously unreliable. This means raters judging the same papers typically can’t agree with each other. The RAND Corp.--the state’s former assessment consultant--found initial Vermont scoring reliability “so low that most planned uses of data had to be abandoned.” RAND described scattered improvements in succeeding years as “slight” and “trivial,” with some reliability figures actually declining. “Large margins of error” have consistently contaminated data. RAND’s final report cited only “limited evidence from other programs that reliable scoring of writing portfolios is practical.”

According to the U.S. Department of Education and an array of testing specialists, portfolios also commonly suffer from validity problems. “Even perfectly reliable scoring,” they report, wouldn’t guarantee that portfolios are measuring the skills they’re intended to assess.

Portfolio debacles aren’t limited to Vermont. A report commissioned by Kentucky’s legislature blasted that state’s portfolio-based system as “seriously flawed,” concluding that “the public is being misinformed” about statewide results and “misled about the accomplishments of individual students.” Portfolio programs are also proving far costlier than anticipated. In the words of one RAND analyst, the “early infatuation” with portfolios was “unrealistic.”

Enthusiasts frequently exaggerate the role portfolios themselves play in learning. It’s wrong to credit portfolios with the benefits delivered by the writing process and sound problem-solving strategies simply because kids employ these techniques to complete the work you find in their portfolios.

It’s certainly wrong to misrepresent teacher support for portfolio keeping as an endorsement of portfolio assessment. You can use portfolios in your classroom with your kids without ever engaging in the pseudo-objective scoring, the voluminous record keeping, and the ghastly expense in both money and classroom time.

Portfolios were devised to provide “meaningful, useful data.” They don’t. That’s why as assessment tools they should be abandoned.

The last thing our schools need is more sound and fury, signifying nothing. Addiction to fashion and blindness to folly won’t build anyone’s confidence in public education.

Related Tags:

A version of this article appeared in the January 14, 1998 edition of Education Week as Portfolio Folly

Events

Jobs Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.
Ed-Tech Policy Webinar Artificial Intelligence in Practice: Building a Roadmap for AI Use in Schools
AI in education: game-changer or classroom chaos? Join our webinar & learn how to navigate this evolving tech responsibly.
Education Webinar Developing and Executing Impactful Research Campaigns to Fuel Your Ed Marketing Strategy 
Develop impactful research campaigns to fuel your marketing. Join the EdWeek Research Center for a webinar with actionable take-aways for companies who sell to K-12 districts.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Briefly Stated: March 13, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read
Education Briefly Stated: February 21, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: February 7, 2024
Here's a look at some recent Education Week articles you may have missed.
8 min read
Education Briefly Stated: January 31, 2024
Here's a look at some recent Education Week articles you may have missed.
9 min read