In a preview of where the SAT could falter in its bid to gain new ground in the K-12 world, the U.S. Department of Education has decided that it can’t fully approve Connecticut’s use of the college-admissions exam as a high school achievement test.
The department wants Connecticut to submit “substantial” additional evidence showing that the SAT meets all requirements in federal law when it’s used to measure achievement, rather than for its original purposes as a college-admissions test.
Among the areas where the SAT fell short in the federal review were how well it reflects Connecticut’s academic standards and whether students with disabilities and those learning English will get adequate accommodations on the test.
Connecticut is one of nine states planning to use the newly redesigned SAT in 2018-19 to measure student achievement in high school. That’s part of a big, yearslong shift in testing, in which states are increasingly using the SAT or the ACT instead of traditional tests.
All states have to submit their tests in grades 3-8 and high school for approval to the Education Department in a process known as “peer review.” The experts who review the assessments look for evidence that the testing plans comply with federal requirements that tests are well-designed, reliable, valid, and appropriate for the way that states plan to use them.
Connecticut’s results are worth noting because it’s the first state to complete the federal peer-review process with the new version of the SAT. The findings offer a glimpse into the areas that could prove bumpy for states if they opt to use the SAT as a test of high school achievement.
The peer-review findings don’t mean that Connecticut can’t go ahead and use the SAT. But the state does have to submit information to satisfy the reviewers, and until they’re satisfied, the federal Education Department can attach strings to some of Connecticut’s federal funding.
More Evidence Needed
Connecticut education department spokesman Peter Yazbak said the state is confident that it will receive a higher rating once it works with the College Board to submit the additional evidence requested by reviewers.
If the SAT had passed with flying colors, the peer reviewers would have sent a letter saying that it “meets all requirements” in federal law. That’s rare, though. Typically, states get letters saying their tests “substantially meet” the requirements. Then they have to work over a period of months—and sometimes years—to supply more evidence to satisfy the reviewers.
But there’s a lower rating states can get, too: a letter saying their test only “partially meets” federal requirements. That’s what happened to Connecticut with the SAT.
In an Aug. 14 letter to state education Commissioner Dianna R. Wentzell, Frank T. Brogan, the Education Department’s assistant secretary for elementary and secondary education, said Connecticut only partially meets federal requirements. The state will have to provide “substantial additional information” to show that using the SAT for achievement in high school passes federal muster, Brogan wrote.
(Wisconsin and Wyoming also got the lower “partially meets” rating last year when they went through peer review with the ACT as their high school test. Wisconsin is still working to get a higher rating. Wyoming has since changed its assessment lineup and no longer uses the ACT as its achievement yardstick for accountability.)
Alignment, Accommodations
The list of additional evidence needed for Connecticut is long, and explores areas at the heart of key questions about the SAT. One request addresses the issue of fairness: It seeks details about the guidelines item-writers used to ensure that the questions they wrote didn’t put some students at a disadvantage based on factors like race or socioeconomic status.
Other demands for evidence pose additional questions, such as how well the SAT reflects each state’s academic standards. Since the SAT was designed as a national test, for college admissions, is it really well-matched to each state’s specific academic expectations? That’s still being debated. The experts refer to this matching as “alignment to standards.” If the SAT isn’t well-aligned to a state’s standards—in Connecticut’s case, the Common Core State Standards—how valid are scores that declare students “proficient” (or not) in the material covered in the standards?
The federal peer reviewers told Connecticut it didn’t provide adequate evidence that the SAT is appropriately aligned to its standards, especially in math. Their notes say the College Board submitted its own alignment study for the review, but no independent study—as required by federal law—was included.
Connecticut, however, did commission its own study, in 2016. And Yazbak, from the state education department, said it was submitted for review. The state “believes that the SAT is adequately aligned to our state standards for the purpose for which we use it: to get an efficient and reliable estimate of a student’s overall achievement,” Yazbak said in an email.
The study found “very solid” alignment in English, with 71 percent of state standards matching content on the SAT. In math, it found only a 43 percent matchup. The researchers wrote that the material on the SAT “may not be as deep or broad” as the expectations in Connecticut’s academic standards, and they worried that teachers “may begin to limit their instruction” to topics on the SAT.
The Education Department also wants to know whether students with disabilities and those learning English get appropriate accommodations on the SAT and all the same benefits other students get.
Connecticut was dinged in these areas, too. Peer reviewers want to see evidence proving the SAT is “fair across all student groups in the design, development, and analysis of its assessments.” They also want Connecticut to prove it “enhances the accessibility” of the SAT by providing appropriate accommodations.
The peer reviewers said in their notes that they had no evidence “on the types and frequency of accommodation approval requests.” They went on to ask: “How many students automatically qualify and get approved? How many students do not qualify automatically and get approved or not approved? How is the decision made?”
At one point, the peer reviewers said in their notes that the College Board “did not provide any information to ensure that appropriate accommodations are available for English-learners.”
Hitting a Sore Spot
Those issues have been a sore spot for the College Board and the ACT as their tests are increasingly used by states to measure high school achievement.
In some cases, the College Board and ACT haven’t let students use the accommodations they’ve received in school and on state tests. That puts students in a bind if their state requires them to take one of those college-admissions exams for accountability: Students can insist on using unapproved accommodations on the exam and receive scores that can’t be certified for use in college admissions. Or they can take the exam without their usual supports and risk getting a lower score.
The College Board revised its policies in early 2017, allowing most students to get automatic approval for the same accommodations on the test that are included in their IEPs, or individualized education programs. It also provided, for the first time, supports for English-learners, such as getting test instructions read to them in their native language.
But those issues surfaced anyway in the peer review of the new SAT in Connecticut. The reviewers told Connecticut that it needs to prove that all students get the same “benefits” from participating in the test, including “college-reportable scores.”
Yazbak said Connecticut automatically grants students the accommodations on the SAT that are listed in their IEPs or Section 504 learning plans and allows English-learners a suite of supports, including extended time on the tests. It shared that information with federal officials, Yazbak said, and “remains hopeful” it will be acknowledged as the process proceeds.
In an email to Education Week, the College Board noted that the federal Education Department did not decide the SAT didn’t meet accountability requirements. It only asked for more information, which is “understandable.” The College Board will support Connecticut as it provides that information, the company said.
