Robert J. Sternberg often writes about a lecture-style psychology course he took as a college freshman in which he got a C. “There is a famous Sternberg in psychology,” the professor told him at the time, “and it looks like there won’t be another.”
To Mr. Sternberg, the vignette illustrates that conventional assessments don’t measure all the abilities students need to succeed in life.
A nationally known psychologist, he has spent much of his career designing new measures that might more accurately capture the full range of students’ intellectual potential at the university level.
Now, a team of Yale University researchers is using the same ideas to rethink the tests that schools use to identify pupils for gifted and talented programs in elementary schools.
The team’s Aurora Battery, named for the colorful spectrums created by the northern and southern lights, is being translated and tested with tens of thousands of 9- to 12-year-olds, not only in the United States, but also in England, India, Kuwait, Russia, Saudi Arabia, Spain, and other countries.
If the preliminary results from those tests are borne out, its developers say, the new assessment could yield a very different pool of gifted students—one that includes a higher proportion of students from traditionally underrepresented minority groups than is often the case now.
“This test has the potential to capture a more diverse population of students with a more varied and better-qualified array of skills,” said Elena L. Grigorenko, a psychology professor and the leader of the Yale study team in New Haven, Conn.
Yale University researchers are pilot-testing an assessment for identifying gifted and talented children that taps intellectual skills other than those captured by traditional intelligence tests. The new tests include questions, such as the one below, designed to measure students’ creativity.
SOURCE: Child Study Center, Yale University
The new battery is based on Mr. Sternberg’s definition of “successful intelligence,” which holds that people who succeed in the real world possess a combination of practical, creative, and analytical skills.
Traditional intelligence tests, these researchers say, measure only a narrow subset: memory and analytical skills. Also known as “g” for general intellectual ability, those skills come in handy for comparing and contrasting, analyzing, judging, and classifying, and they are the kinds of abilities that teachers tend to value and emphasize in the classroom.
If people who score high on such measures succeed later on in life—and studies show that they often do—it’s partly because the educational system is geared to reward their particular mental skills, Mr. Sternberg said.
Validating Results
Yet plenty of people succeed without ever fitting that pattern—people like Virgin Airlines founder Richard Branson or filmmaker Steven Spielberg, both of whom were high school dropouts.
With the Aurora assessments, though, scholars hope to get a read on the skills that make the Bransons and Spielbergs of the world successful, as well as the academic skills that intelligence tests have traditionally measured.
In its entirety, Aurora is a comprehensive battery that includes a group-administered paper-and-pencil test, a parent interview, a scale for teacher rating of students, and some observation items. The paper-and-pencil test gauges creativity, for instance, by asking students to imagine what objects might say to one another if they could talk, or to generate a story plot to fit an abstract illustration on a children’s-book cover.
A question assessing students’ practical skills with numbers directs test-takers to draw a line mapping the shortest route between a friend’s house and a movie theater.
With the data now coming in from around the world, researchers hope to validate the tests for widespread use. They are comparing children’s Aurora scores with those they get on more-traditional screening tests for gifted programs, such as the Cognitive Abilities Tests or the Wechsler Intelligence Scales for Children, and looking at grades and teacher ratings of students.
Ms. Grigorenko said the early results show that children really do run the gamut in performance along different intellectual dimensions.
“They can be high on analytical skills and low on creative or practical,” she said. “The second thing is that you tend to close the achievement gap when you pay attention to kids’ skills other than analytical and memory skills.”
A similar pattern of results is emerging, meanwhile, from Project Kaleidoscope, an experiment that Mr. Sternberg initiated at Tufts University, in Medford, Mass., where he is now the dean of the school of arts and sciences.
Starting in the 2006-07 academic year, the school supplemented its freshman application process with optional essay questions designed to assess applicants’ wisdom and creative potential. Applicants could choose to write an essay on “confessions of a middle school bully,” for example, or imagine what would have happened had Rosa Parks not helped ignite the civil rights movement by declining to give up her seat on the bus.
Admissions Affected?
Contrary to concerns that the extra questions would deter students, the number of applicants rose that year. In the spring, Tufts admitted 30 percent more black students and 15 percent more Hispanic students than it had the previous year.
Average SAT scores for incoming freshmen increased, too, according to Mr. Sternberg. First-semester grades and survey results also show that the admitted students fared as well, academically and socially, as their predecessors had the year before.
“The lesson I’ve learned in Kaleidoscope is that the kinds of questions you ask predicate the answer you get,” said Lee A. Coffin, the dean of undergraduate admissions at Tufts. “If you just ask, ‘How did you spend your summer?’ you get some terrific answers and you get some terrible ones, but you don’t know that creativity is teased out in that question.”
Mr. Coffin said it was harder to tell, though, whether the increasing racial and ethnic diversity of the freshman class was due to Kaleidoscope or to Tufts’ efforts to boost financial aid and move to a need-blind admission process.
The Kaleidoscope project followed on the heels of the Rainbow Project, a set of tests for creative and practical thinking that Mr. Sternberg piloted with support from the College Board, the New York City-based organization that sponsors the SAT and Advanced Placement programs.
In 2001, the Rainbow tests were given to 1,013 high school students and college freshmen from 15 schools. Researchers compared the results with students’ SAT scores and examined students’ grades in high school and in their first year of college. (“Researchers Call SAT Alternative Better Predictor of College Success,” Jan. 29, 2003.)
Not a Replacement
Compared with the SAT alone, Mr. Sternberg found, the Rainbow tests doubled the accuracy with which researchers were able to predict students’ first-year grades. Score gaps between students of different racial and ethnic groups were also smaller on the broader test than they were on the SAT.
Mr. Sternberg and his research partners got similar results a few years earlier when they administered a practical-skills test to students applying to the graduate business school at the University of Michigan. He’s also testing efforts to embed Rainbow-style questions in the Advanced Placement tests the College Board offers in psychology, physics, and statistics.
“We don’t see these kinds of tests as a replacement for the analytical tests, because those abilities do matter for success in school,” said Mr. Sternberg. “So the mistake is not in using the analytical tests. It’s in overusing them.”
Mr. Sternberg’s ideas are not without their critics in academia. One is Linda S. Gottfredson, an education professor at the University of Delaware, in Newark. She contends that practical and creative abilities are really skills or achievements, rather than independent intelligences.
“What we find is that bright people have an edge in virtually everything,” she said. “No one has demonstrated that there is more than one highly general mental ability.”
‘Long Haul’
Whether the newer, broader tests will win a permanent place in schools or in college-admissions processes is also open to debate.
The College Board pulled the plug on the Rainbow Project two years ago, choosing instead to work on validating its own writing test, according to Alana Klein, a spokeswoman for the board.
She said the board got pushback on the Rainbow tests from admissions officers at some of its member institutions. At Tufts, the Kaleidoscope questions remain on admissions applications for now. Mr. Coffin said the university will track results for a few more years before deciding whether to make the questions permanent.
Mr. Sternberg’s ideas, like the theory of “multiple intelligences” that Harvard University’s Howard Gardner espouses, attract strong supporters at the K-12 level, though.
The Park School, a small, progressive private school in Buffalo, N.Y., twice invited Kaleidoscope researchers to administer the Aurora battery to students, even though results can’t yet be shared with the school, students, or parents.
“We wanted to encourage parents to value creative and practical intelligence,” said Donald Grace, the head of school. “We know we’re in it for the long haul.”
