Teaching Profession

Analysis Raises Questions About Rigor of Teacher Tests

By Stephen Sawchuk — January 31, 2012 8 min read

The average scores of graduating teacher-candidates on state-required licensing exams are uniformly higher, often significantly, than the passing scores states set for such exams, according to an Education Week analysis of preliminary data from a half-dozen states.

The pattern appears across subjects, grade levels, and test instruments supplied by a variety of vendors, the new data show, raising questions about the rigor and utility of current licensing tests.

There are, in essence, two main ways to interpret the findings. Some observers say the data suggest most states set low passing marks, screening out only candidates with the very lowest level of content knowledge.

“If there’s not a lot of variation in the performance of graduates by institution, it could mean that education seems to set a lower bar for institutions than other professions,” said Dan Goldhaber, a research professor at the University of Washington Bothell, who has studied teacher-licensing tests.

The other point of view holds that most current teacher examinations are better thought of as a minimum screen, and should be considered alongside professional-entry mechanisms as a whole.

Beyond the Mean

The average scaled score of 2009-10 teacher education program completers from four states on the Praxis II Mathematics Content Knowledge exam were higher than the national average.

BRIC ARCHIVE

SOURCES: Educational Testing Service; Education Week

The tests “are not designed to assess anything a candidate learns in a credential program,” said Teri Clark, the director of professional services for the California Commission on Teacher Credentialing, the state’s teacher-standards board.

She noted that California requires prospective educators to pass the tests typically before they complete their pedagogy work. “It doesn’t make a lot of sense to prepare a person to teach the subject matter if they don’t already have the subject-matter knowledge,” she said.

The battery of tests varies by state. Some require basic-skills exams before entering a program; others require educators to pass tests measuring content or pedagogy knowledge before completing their preparation, as in California; still others require them before recommending a candidate for a license.

Where to Cut Off

The 2008 rewrite of the federal Higher Education Act establishes much more detailed licensure test reporting requirements than its predecessor.

Among other requirements, Title II of the law compels states to include both passing rates on the tests and candidates’ average scaled scores. Scaled scores are, in essence, candidate performance translated from the raw number of correct answers to the scale on which the test is graded.

Though the U.S. Department of Education has not yet posted the state-generated “report cards” containing that information, Education Week obtained preliminary data from higher education institutions’ reports and directly from state officials.

The states reviewed, selected at random, were California, Georgia, Indiana, Maryland, South Carolina, and Virginia.

States’ average scaled scores for program completers in 2009-10 were compared with passing scores for each test obtained from test vendors and state websites.

Among the findings:

• Virginia’s teacher graduates’ average score on an elementary-content test, at 172, was nearly 30 points higher than the state cutoff score of 143.

• Among the tests California requires for licensure, only one consistently seemed to pose any significant challenge for graduates: the Reading Instruction Competence Assessment, required of elementary and special education teachers and updated in 2009.

• Of Georgia’s tests, an infant/early-childhood teacher test had the smallest disparity between program completers’ average scaled score and the state passing mark, at 13 points. By contrast, a music test and a Spanish test in Georgia show average scaled scores that were 41 and 38 points higher, respectively, than the cutoff scores on those tests.

• The cutoff score on a popular secondary math test in South Carolina and Indiana is set well below both the average scaled score of the state’s program completers and the national average of all U.S. candidates taking the exam. (See chart, right.)

Data in Context

High overall passing rates on licensing tests have been well documented since the federal government started collecting information on the tests in 1998.

According to the Education Department, the overall passing rate on licensing exams by graduates of traditional teacher-preparation programs in 2008-09 was 96 percent. The high rates overall are, in part, a reflection that many states have made passing the tests an institutional requirement.

Some technical differences exist between the Praxis series, developed by the Princeton, N.J.-based Educational Testing Service and used in many states, and licensing tests developed exclusively for states, as is the case in California and Georgia.

Still, regardless of the test, the overall pattern indicates that states have set the passing bar significantly below the mean in many cases—even for those teaching at the high school level, where teachers on average have stronger academic qualifications than their colleagues in the lower grades.

There is no objectively correct passing mark on such tests; determinations are made by state panels that weigh demographic, educational, and political factors. Where the state ultimately sets the cutoff point matters because it helps determine the supply of teachers.

Overall, the data echo a 2010 report from the ETS that, in general, found a wide disparity in mean performance on licensing tests between teacher-candidates who passed the exams and the small proportion who failed them.

The Title II data do have limitations. For one, it is hard to tell from the preliminary data just how “difficult” it is, relatively speaking, to pass the tests. Researchers typically express that relationship in terms of standard deviations from the average scaled score. To get at that information requires the spread of scores on the test, which states are not required to include in their report cards.

Another wrinkle: Most states permit teachers to take certification tests multiple times. It is not clear from the state-generated data how that policy affects the scaled scores.

The reports also show, overall, that teachers at the beginning of their preparation programs had lower average scores and passed the tests less often than those who had completed all program requirements. But even among those test-takers, average passing rates tended to be above 80 percent.

Value Debated

Researchers continue to debate the utility of current teacher-licensing exams and whether they adequately measure the range of skills that new teachers should master. The University of Washington’s Mr. Goldhaber, the author of a 2007 paper on such exams, found that teachers’ performance on them bore a weak or inconsistent relationship to how their students performed in class.

“I think [the Title II] results don’t tell you much about the workforce as a whole,” Mr. Goldhaber said.

But he added that such information does raise questions about the cost-effectiveness of the current teacher-testing systems. “If studying for these exams doesn’t make you a better teacher, you have a very expensive system to weed out a tiny percentage of the workforce,” he said.

At their most basic level, the results appear to stand in stark contrast to those of other professions that include tests as one part of licensing, such as medicine or law. Consider, for example, the California bar examination, which in its most recent administration was passed by only 54 percent of candidates.

Yet experts note that merely raising cutoff scores may not lead to a more highly trained workforce that is better able to teach.

“What gets tested on the teacher-licensure tests has little to do with teacher education programs at all. They’re not the materials you learn for teaching,” said Drew H. Gitomer, a professor at the Rutgers Graduate School of Education, in New Brunswick, N.J., and a former researcher at ETS. “The bigger question is what are the right measures and quality controls we want to have at licensure, and also beyond licensure.”

South Carolina’s deputy superintendent for school effectiveness, Mark A. Bounds, said the state education department is considering recommending that cutoff scores on roughly a dozen teacher-licensing assessments be raised. But overall, he said, the state is working to improve teaching quality through a number of initiatives, such as linking “value added” data on students back to the programs that produced their teachers.

Internal analyses show that only 77 of every 100 new teachers in the state advance from an initial to a professional license, which they must do through an induction program within their first three years on the job, he said.

“I don’t want to diminish testing, but I don’t want people to believe that if we only raised the Praxis scores, our teaching force would significantly improve,” Mr. Bounds said. “It’s one gateway to move through, as are induction and formal evaluation.”

Ms. Clark of California’s professional-standards board said that performance assessments of student-teachers, which are also required in the Golden State, are better aligned with teacher-preparation courses. Those tests, she said, are selected and scored locally by institutions, and programs use the information to counsel struggling individuals out of the profession. But the results are not reported as part of Title II.

Twenty-four states are at various stages of piloting and implementing a similar test that proponents hope will prove to be a more valid assessment of teaching ability.

Still other critics believe that teacher-licensing exams should be strengthened in ways other than raising the passing scores or integrating performance tasks.

“Direct questions should be raised about how the public is to be assured that any teacher subject test is at the college level,” said Sandra Stotsky, a professor of education reform at the University of Arkansas. “Cut scores can be raised, but the test can still be a weak test.”

Coverage of policy efforts to improve the teaching profession is supported by a grant from the Joyce Foundation, at www.joycefdn.org/Programs/Education.
A version of this article appeared in the February 01, 2012 edition of Education Week as Teachers Pass License Tests at High Rates

Events

School & District Management Live Event Education Week Leadership Symposium
Education Week's Premier Leadership Event for K12 School & District Leaders.
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Law & Courts Webinar
The Future of Criminal Justice Reform: A Sphere Education Initiative Conversation
America’s criminal justice system is in crisis and calls for reform are dominating the national debate. Join Cato’s Sphere Education Initiative and Education Week for a webinar on criminal justice and policing featuring the nation’s
Content provided by Cato Institute
This content is provided by our sponsor. It is not written by and does not necessarily reflect the views of Education Week's editorial staff.
Sponsor
Student Well-Being Webinar
Equity, Care and Connection: New SEL Tools and Practices to Support Students and Adults
As school districts plan to welcome students back into buildings for the upcoming school year, this is the perfect time to take a hard look at both our practices and our systems to build a
Content provided by Panorama Education

EdWeek Top School Jobs

Teacher Jobs
Search over ten thousand teaching jobs nationwide — elementary, middle, high school and more.
View Jobs
Principal Jobs
Find hundreds of jobs for principals, assistant principals, and other school leadership roles.
View Jobs
Administrator Jobs
Over a thousand district-level jobs: superintendents, directors, more.
View Jobs
Support Staff Jobs
Search thousands of jobs, from paraprofessionals to counselors and more.
View Jobs

Read Next

Teaching Profession Juliana Urtubey, an Elementary Special Educator, Is the 2021 National Teacher of the Year
Known as Ms. Earth for her work with school gardens, Urtubey is a National Board-certified teacher in Las Vegas.
4 min read
Juliana Urtubey
Juliana Urtubey
Courtesy Photo
Teaching Profession 4 Ways Districts Are Giving Teachers More Flexibility in Their Jobs
After a year-plus of pandemic schooling, some experts are seeing momentum for district leaders to reimagine what teaching can look like.
11 min read
Teacher working at home in front of camera.
Getty
Teaching Profession Why Teachers Leave—or Don't: A Look at the Numbers
New EdWeek survey results reveal why teachers consider leaving the profession, and how the pandemic has changed their decisionmaking.
6 min read
v40 32 Teacher Retention INTRO DATA
Stephanie Shafer for Education Week<br/>
Teaching Profession We Asked Teachers How They Want to Be Appreciated. Here's What They Said
All they need is respect, independence, a break, and a heartfelt word of thanks after a difficult year.
3 min read
Image shows a teacher in a classroom.
skynesher/E+