Evaluating AI EdTech Tools for Schools

A sound evaluation covers six areas. Accuracy: test the tool on your own curriculum content, including hard and ambiguous cases, and count errors. Generative tools can state wrong answers confidently, so ask how the vendor reduces errors and how teachers can see and correct outputs. Privacy and security: in the US, student records are protected by FERPA, and COPPA applies to services collecting data from children under 13; many states add their own student privacy laws. Ask what data is collected, where it is stored, how long it is kept, whether it trains models, and which third-party AI providers receive it. Many districts use data privacy agreements, including templates from the Student Data Privacy Consortium. Accessibility: ask for an accessibility conformance report (often a VPAT) against WCAG 2.1 AA, then check it yourself with a screen reader and keyboard. Bias: check whether the tool performs worse for English learners, students with particular dialects, or students with disabilities, and whether automated scoring or flagging could fall unevenly on some groups. Cost: include licences, training, integration with the learning management system, and the price after any free period ends. Evidence: the Every Student Succeeds Act defines four evidence tiers: strong (well-designed randomized studies), moderate, promising, and 'demonstrates a rationale'. Many AI products offer only tier 4 evidence or vendor case studies, so a local pilot matters. Common misconceptions: that a well-known brand guarantees compliance, that 'AI-powered' means more effective, and that high engagement proves learning. Time on app and satisfaction surveys are not measured learning gains. The US Department of Education's 2023 report on AI in teaching stresses keeping humans in the loop, a useful test: can teachers override, review and explain what the tool does?