K-12 school districts across the country are facing pressure to quickly adopt AI. Pedro A. Nogura and Ayisha Irfan have spent a combined five decades in the systems where these educational and technical decisions are made. Together, they have compiled a set of questions they believe school districts, colleges, and universities should ask before signing or renewing contracts with AI companies.
Learning and Cognition:
Learning requires iteration, scaffolding, and sitting with difficulty. Vendors must explain if their tool requires that work or skips it.
- Does the tool compel students to use higher-order skills such as problem-solving and critical thinking? What specific design choices reflect that intent?
- Does the tool ensure that students continue to acquire the foundational knowledge in history, geography, science, mathematics, and current events that higher-order thinking depends on?
- How well do students perform on tasks when the tool is not available? Can they explain what they have learned in oral and written form without it?
- Does the tool measure whether students retain knowledge over time, and does it include a means to check for genuine understanding rather than just task completion?
- What is the vendor's theory of how learning happens for students? Is it grounded in cognitive science? What specific metrics has the vendor used to measure the tool's impact on student cognition over time?
- How does the tool handle student errors and wrong answers? Does it simply correct them or does it scaffold student learning toward understanding the answer?
Teaching:
Vendors must treat teachers as equal partners, supporting their curriculum and craft.
- Does the tool support teachers in using creative pedagogical strategies to increase student engagement?
- Does the tool allow teachers to differentiate support for students and create opportunities to intervene for those who are struggling academically?
- What training does the vendor provide to teachers? Is it pedagogical training or product training? What guardrails exist in the tool for moments when students are using it without teacher supervision?
- How does the tool integrate with the school's existing curriculum? Does it require the curriculum to be reshaped around the tool?
Equity and Access:
Education leaders should understand who the tool is designed to serve—and who it might leave behind.
- What evidence does the vendor have for how the tool performs across different student populations, including students with IEPs and students whose first language is not English? Who is the tool designed to work best for? Are those performance metrics documented?
- When students are using the tool outside the classroom, does the tool's design assume scaffolding support at home?
Vendor Accountability:
Universities and school districts should not accept blind promises of educational transformation from multibillion-dollar tech companies, nor should they let their classrooms turn into testing grounds for unvetted software.
- Were teachers, students, and families consulted in the original design of this product?
- Is the AI model grounded in vetted educational content? What safeguards exist against the tool generating inaccurate, biased, and inappropriate content?
- What independent research has been conducted on the tool's learning outcomes and by whom?
- What channels exist for teachers, students, and families to provide ongoing feedback on the tool after deployment, and what is the vendor’s documented process for incorporating that feedback into product revisions?
- What is the vendor's policy on student data? What is collected, where is it stored, who has access to it, how is it used to train models, and what happens when the contract ends?
- What are the vendor's contractual commitments on data-breach notification, including the maximum number of days between breach detection and notification to the district and to affected families?
