Opinion
Education Opinion

That Sinking Feeling

By John Merrow — October 17, 2001 6 min read
  • Save to favorites
  • Print
Bad tests and overreliance on test results are enemies of good standards. Just look at what happened in Atlantis.

To grasp the impact that high-stakes tests are having on school reform in this country, imagine a 51st state. Let’s say it’s an island, the state of Atlantis. Because this imaginary island-state is slowly sinking into the sea, its schools teach only one subject: swimming. When the movement for higher standards reached Atlantis, it set off a frenzy of reform debate, a perfect mirror of what is happening in our 50 states.

First Atlantis set about creating content standards—what every student needs to know about swimming. Some adults on Atlantis wanted a simple “survival swimming” curriculum, which would teach children to tread water. Others wanted a rich curriculum in which all students would be taught the four competition strokes: butterfly, crawl, backstroke, and breaststroke. Rather than make hard decisions, it proved easier to include just about everything, and so Atlantis schools now teach everything from the dogpaddle on up.

Establishing performance standards came next. Performance standards set the levels of skill and knowledge a student must demonstrate to pass or to earn distinction. After prolonged debate, Atlantis adopted four performance levels: advanced, proficient, basic, and below basic. “Advanced” swimmers must be able to swim four laps—one of each competition stroke—in less than nine minutes, while “proficient” swimmers must complete the laps in under 15 minutes.

Atlantis decided that “basic” swimmers would earn diplomas simply by finishing, no matter how long it took. This last decision angered educational conservatives, who accused Atlantis education officials of watering down the curriculum.

Students who fail to complete the four laps are in the “below basic” category. They either drown or drop out of school.

Atlantis simply ignored opportunity to learn standards. The idea behind this third type of standard is that it is unfair to hold students to performance standards unless conditions exist to give them a fair chance of meeting them. So, for example, for all Atlantis students to have a realistic chance of mastering the four strokes, they would be taught in lap pools by trained swimming coaches. However, trained swimming coaches are hard to find on Atlantis. Most schools have access to water (it’s an island, after all), but swimming instruction is generally left to adults who swim recreationally.

Measuring learning was the next hurdle. Teachers wanted classroom grades and their professional judgment to count, but Atlantis decided that students had to prove they could swim. Then, however, they couldn’t agree on how to test them. Requiring each student to swim a lap of at least one of the four strokes would be what educators call a valid test, just as testing writing skills by requiring students to write an essay is valid. Student flaws could be identified precisely and corrected immediately, and students could jump back in the pool and try again. Unfortunately, direct demonstration of skill is costly and time-consuming, and so Atlantis opted for a machine-scored, multiple-choice test of swimming. It ruled that students had to pass this test in order to graduate.

Direct demonstration of skill is costly and time-consuming, so Atlantis opted for a machine-scored, multiple choice test of swimming.

This decision had unintended effects on the curriculum at many Atlantis schools. Reporters found that students were swimming less, drilling more. And many schools eliminated enrichment programs in water polo, diving, and synchronized swimming in order to focus on the basics.

Responding to protests that a multiple-choice exam was not a valid test of swimming, Atlantis added an essay question: “In 250 words, describe what it feels like to swim a lap of the butterfly, with particular attention to the turn.”

Parents in some wealthy Atlantis communities threatened to boycott the high-stakes test. As one Atlantis parent put it, “Our children learned to swim at home. We want our children playing water polo and learning to dive, not wasting time on drill.”

Low-performing Atlantis students, many of them disadvantaged, seem to have benefited from multiple-choice exams, according to Atlantis. “Since the adoption of the multiple-choice format, drowning as a reason for nonattendance has been virtually eliminated,” the report asserts. School records confirm this; last year only a handful of students stopped coming to school because they had drowned, all of them students in the “below basic” classes. The dropout rate, however, remains high.

Every seemingly precise standardized- test result has an error range, called the standard error of measurement. Thus, for example, on a test with an error range of 4 points, a score of 72 means the student’s so-called true score lies somewhere between 68 and 76. And so, reliance on machine-scored, multiple-choice tests always raises questions about the reliability of the results. Detailed analysis of last year’s test results on Atlantis revealed that dozens of students passed at the “basic” swimmer level and earned diplomas even though they could barely keep their heads above water. Some certified lifeguards were denied diplomas because they scored at the “below basic” level. The testing company made errors in scoring exams, leading to more kids’ being denied promotion or graduation.

Atlantis may be imaginary, but everything I’ve imagined happening there is real here. Content standards are often bloated, and performance standards have been lowered in Arizona, California, New York, New Jersey, New Mexico, and other states in the face of high failure rates. States have ignored opportunity-to-learn standards, because it’s too painful to confront the fact that, for example, more than one-third of our high school students are taught science and math courses by adults who neither studied nor trained to teach those subjects. No imaginary Atlantis educators have been caught cheating, but real ones have been fired or indicted in Maryland, Texas, Connecticut, and other states.

Many schools in Atlantis eliminated enrichment programs in water polo, diving, and synchronized swimming—much to the dismay of parents in wealthy communities.

Most of our states rely on machine-scored, standardized tests, often resulting in cuts in “frills” like art, music, and physical education, and more drill. Twenty-four states have adopted a high-stakes graduation test, with more likely to follow suit. And here at home, test boycotts are a reality: Parents in well-to-do communities like Scarsdale, N.Y., and Berkeley, Calif., kept their children home on test days in the spring of 2001.

Because standardized tests are not precise instruments, so-called “false negatives” and “false positives” are also a fact of life. By Massachusetts’s own calculation, it’s statistically likely that about 3,000 of the 10th graders who failed the state math test in 1998 actually passed. Errors by testing companies penalized students in Arizona, Minnesota, Indiana, Massachusetts, and New York.

On Atlantis, and here in the United States, bad tests and the overreliance on test results are enemies of good standards. Here and on Atlantis, it’s students who suffer the most, not the politicians and other adults who make the policies. Atlantis is, of course, sinking slowly into the sea, meaning that its bitter arguments over educational standards and high-stakes tests will soon be over.

The rest of us should be so lucky.

John Merrow, the author of Choosing Excellence: “Good Enough” Schools Are Not Good Enough (Scarecrow Press, 2001), recently received a Peabody Award for his PBS documentary “School Sleuth: The Case of an Excellent School.” His next program, “Testing the Schools,” will appear on PBS’s “Frontline” Oct. 25.


Commenting has been disabled on edweek.org effective Sept. 8. Please visit our FAQ section for more details. To get in touch with us visit our contact page, follow us on social media, or submit a Letter to the Editor.


Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Teaching Webinar
What’s Next for Teaching and Learning? Key Trends for the New School Year
The past 18 months changed the face of education forever, leaving teachers, students, and families to adapt to unprecedented challenges in teaching and learning. As we enter the third school year affected by the pandemic—and
Content provided by Instructure
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Curriculum Webinar
How Data and Digital Curriculum Can Drive Personalized Instruction
As we return from an abnormal year, it’s an educator’s top priority to make sure the lessons learned under adversity positively impact students during the new school year. Digital curriculum has emerged from the pandemic
Content provided by Kiddom
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Equity & Diversity Webinar
Leadership for Racial Equity in Schools and Beyond
While the COVID-19 pandemic continues to reveal systemic racial disparities in educational opportunity, there are revelations to which we can and must respond. Through conscientious efforts, using an intentional focus on race, school leaders can
Content provided by Corwin

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Education Judge's Temporary Order Allows Iowa Schools to Mandate Masks
A federal judge ordered the state to immediately halt enforcement of a law that prevents school boards from ordering masks to be worn.
4 min read
Iowa Gov. Kim Reynolds speaks to reporters following a news conference, Thursday, Aug. 19, 2021, in West Des Moines, Iowa. Reynolds lashed out at President Joe Biden Thursday after he ordered his education secretary to explore possible legal action against states that have blocked school mask mandates and other public health measures meant to protect students against COVID-19. Reynolds, a Republican, has signed a bill into law that prohibits school officials from requiring masks, raising concerns as delta variant virus cases climb across the state and schools resume classes soon. (AP Photo/Charlie Neibergall)
Education Hurricane Ida Deals New Blow to Louisiana Schools Struggling to Reopen
The opening of the school year offered teachers a chance to fully assess the pandemic's effects, only to have students forced out again.
8 min read
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021. Louisiana students, who were back in class after a year and a half of COVID-19 disruptions kept many of them at home, are now missing school again after Hurricane Ida. A quarter-million public school students statewide have no school to report to, though top educators are promising a return is, at most, weeks away, not months.
Six-year-old Mary-Louise Lacobon sits on a fallen tree beside the remnants of her family's home destroyed by Hurricane Ida, in Dulac, La., on Sept. 4, 2021.
John Locher/AP
Education Massachusetts National Guard to Help With Busing Students to School
250 guard personnel will be available to serve as drivers of school transport vans, as districts nationwide struggle to hire enough drivers.
1 min read
Massachusetts National Guard soldiers help with logistics in this Friday, April 17, 2020 file photo, at a food distribution site outside City Hall, in Chelsea, Mass. Mass. Gov. Charlie Baker on Monday, Sept. 13, 2021, activated the state's National Guard to help with busing students to school as districts across the country struggle to hire enough drivers.
Massachusetts National Guard soldiers help with logistics in this Friday, April 17, 2020 file photo, at a food distribution site outside City Hall, in Chelsea, Mass.
Michael Dwyer/AP
Education FDA: ‘Very, Very Hopeful’ COVID Shots Will Be Ready for Younger Kids This Year
Dr. Peter Marks said he is hopeful that COVID-19 vaccinations for 5- to 11-year-olds will be underway by year’s end. Maybe sooner.
4 min read
Dr. Peter Marks, director of the Center for Biologics Evaluation and Research in the Food and Drug Administration, testifies during a Senate health, education, labor, and pensions hearing to examine an update from federal officials on efforts to combat COVID-19 on Capitol Hill in Washington on May 11, 2021. On Friday, Sept. 10, 2021, Marks urged parents to be patient, saying the agency will rapidly evaluate vaccines for 5- to 11-year-olds as soon as it gets the needed data.
Dr. Peter Marks, director of the Center for Biologics Evaluation and Research in the Food and Drug Administration, testifies during a Senate health, education, labor, and pensions hearing to examine an update from federal officials on efforts to combat COVID-19 on Capitol Hill in Washington on May 11, 2021.
Jim Lo Scalzo/AP