Assessment

NAEP Tech., Engineering Test Brings Interactivity

By Liana Loewus — May 13, 2014 4 min read
  • Save to favorites
  • Print

Engineering is still far from a curricular staple in most K-12 schools, but a new national assessment aims to measure how well students can tackle the kinds of problem-solving used by engineers in their day-to-day work.

Nearly 20,000 8th graders participated earlier this year in the first formal administration of the interactive, computer-based test, which looks at applied technology and engineering skills. Results are not expected until the summer of 2016 from the new exam—part of the suite of tests included in the National Assessment of Educational Progress, known as “the nation’s report card.”

The eventual plan is for the Technology and Engineering Literacy, or TEL, exam, to be rolled out to students in grades 4, 8, and 12.

“What we’re looking at is not, are you able to solve the problem, but how do you go about solving the problem? What do you do first? What do you do second? ... Are you able to do it efficiently?” Lonnie Smith, an assessment specialist at the Educational Testing Service, said during a presentation about the new exam at a STEM education conference in Washington last month.

“We’re trying to assess not whether students can arrive at a solution, which in my opinion is not that telling of their abilities,” added Mr. Smith, who worked on the development of the NAEP exam. “But what is interesting is how do they get there?”

The assessment comes as the first results of a global exam on creative problem-solving were issued in April, part of the Program for International Student Assessment, or PISA. U.S. students scored above the average of the 44 countries and education systems that took the test.

Fixing a Water Pump

The new NAEP exam leads students through 10- to 30-minute computer-based modules in which they confront a problem, such as a broken water pump. The test-taker then interacts with characters in the module and uses a range of supplied resources to troubleshoot and find a solution.

The assessment covers student understanding in three domains: technology’s impact on society; engineering design and the basic principles of using technology in everyday life; and information and communications technology, such as mobile devices, computer software, and networking systems.

Mr. Smith from the ETS noted that video-game designers were brought in to assist with the development of the TEL assessment.

He explained that, just as a reading test assesses comprehension and not whether students know facts on the topic they’re asked to read about, the TEL exam assesses students’ problem-solving skills, not whether they know particulars of engineering or technology.

It’s the same kind of problem-solving students do in their own lives, said Peggy G. Carr, the associate commissioner of the National Center for Education Statistics, the federal agency that administers NAEP, who also spoke about the new exam on April 23 at the STEM education conference, which was organized by U.S. News & World Report.

For instance, if a student’s bike breaks down on the way to school, he or she needs to figure out how to fix the problem with the resources available, Ms. Carr explained.

A summary from the NCES says that in addition to the scenario-based tasks, the exam relies on short-answer and multiple-choice questions to measure students’ knowledge and skills. Since much of what students learn about how and why technology is used, and how to problem-solve around it, happens outside the classroom, the test will also include a questionnaire asking students where they have practiced particular skills.

Students ‘Loved It’

The NCES has released one sample module to the public so far. In the module, which is available on the agency’s website, students are asked to play the role of an engineer and figure out why a remote community’s water pump isn’t working. They watch a tutorial on how a water pump functions, ask a community member questions to determine if there has been a water shortage, and use a repair manual to help figure out what needs to be fixed.

Mr. Smith emphasized that the task is authentic.

“We did research on community organizations in Southeast Asia that use this technology,” he said.

Melisse May, the community-engagement director for the Greater Cincinnati STEM Collaborative, who attended the presentation and has a background in engineering, said she thought the module was “remarkably accurate.”

However, in addition to testing literacy and problem-solving, Ms. May said, the problem also seems to test “affinity for mechanics. If you’ve not ever seen or used a pump, that’s a lot of data to assimilate.”

Stacy Klein Gardner, the chair of the K-12 and precollege division of the American Society for Engineering Education, who was not associated with the assessment, responded by email after reviewing the released materials that “some of this is just good logic skills. ... I am particularly curious as to how they plan to assess engineering design in a test like this (with multiple choice?) when engineering design should be an open-ended task with no single best answer.”

The ability to communicate, analyze information, and collaborate, though, she said, is “extremely important in engineering.”

Leaving such questions aside, another panelist at the Washington conference on science, technology, engineering, and math said the exam has been a hit with students.

“The students loved it,” said William Ward, a senior research scientist at the NCES who also worked on the new assessment. “The 8th grade students—within the 60-minute testing session, they were engaged.”

According to reports from the exam administrators, “students who were not selected [to take the test] were coming to the door wanting to take it,” said Ms. Carr. “This is something we do not see with paper-and-pencil assessments.”

Related Tags:

A version of this article appeared in the May 14, 2014 edition of Education Week as New NAEP Tech., Engineering Exam Targets Problem Solving

Events

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
A Whole Child Approach to Supporting Positive Student Behavior 
To improve student behavior, it’s important to look at the root causes. Social-emotional learning may play a preventative role.

A whole child approach can proactively support positive student behaviors.

Join this webinar to learn how.
Content provided by Panorama
Recruitment & Retention Live Online Discussion A Seat at the Table: Why Retaining Education Leaders of Color Is Key for Student Success
Today, in the United States roughly 53 percent of our public school students are young people of color, while approximately 80 percent of the educators who lead their classrooms, schools, and districts are white. Racial
Jobs January 2022 Virtual Career Fair for Teachers and K-12 Staff
Find teaching jobs and other jobs in K-12 education at the EdWeek Top School Jobs virtual career fair.

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Assessment Whitepaper
Report: Now is the Time to Reimagine Assessments
In New Meridian’s latest whitepaper, Founder and CEO Arthur VanderVeen identifies five principles to boldly re-imagine assessment and cre...
Content provided by New Meridian
Assessment State Test Results Are In. Are They Useless?
While states, districts, and schools pore over data from spring 2021 tests, experts urge caution over how to interpret and use the results.
9 min read
FILE - In this Jan. 17, 2016 file photo, a sign is seen at the entrance to a hall for a college test preparation class in Bethesda, Md. The $380 million test coaching industry is facing competition from free or low-cost alternatives in what their founders hope will make the process of applying to college more equitable. Such innovations are also raising questions about the relevance and the fairness of relying on standardized tests in admissions process.
A sign is posted at the entrance to a hall for a test-preparation class. Assessment experts say educators should use data from spring 2021 tests with caution.
Alex Brandon/AP
Assessment Data Young Adolescents' Scores Trended to Historic Lows on National Tests. And That's Before COVID Hit
The past decade saw unprecedented declines in the National Assessment of Educational Progress's longitudinal study.
3 min read
Assessment Long a Testing Bastion, Florida Plans to End 'Outdated' Year-End Exams
Florida Gov. Ron DeSantis said the state will shift to "progress monitoring" starting in the 2022-23 school year.
5 min read
Florida Governor Ron DeSantis speaks at the opening of a monoclonal antibody site in Pembroke Pines, Fla., on Aug. 18, 2021.
Florida Gov. Ron DeSantis said he believes a new testing regimen is needed to replace the Florida Standards Assessment, which has been given since 2015.
Marta Lavandier/AP