Evaluating AI EdTech Tools for Schools Quiz

Answers and explanations appear as you take the quiz.

1. Under the Every Student Succeeds Act, which evidence tier is based on well-designed randomized studies?
   • Demonstrates a rationale
   • Promising
   • Moderate
   • Strong
2. Which question best tests a vendor's privacy practices?
   • How many awards has the product won?
   • Is student data used to train models, and which third parties receive it?
   • What color themes are available?
   • How fast is the login page?
3. What is the recommended way to check accessibility claims?
   • Read the vendor's VPAT and test with a screen reader and keyboard yourself
   • Trust the marketing page
   • Ask students if they like the colors
   • Check the price
4. Why is high student engagement not enough to justify a purchase?
   • Engagement is illegal to measure
   • Engaged students always score lower
   • Engagement and time on app are not the same as measured learning gains
   • Engagement data cannot be exported
5. Why should a school rerun its accuracy test set after a vendor update?
   • Outputs can change when the underlying model changes
   • Test sets expire legally
   • Updates always improve accuracy
   • FERPA requires monthly testing
6. What makes a pilot more trustworthy?
   • Letting the vendor choose the outcome after results come in
   • Using only the most enthusiastic teacher
   • Measuring satisfaction only
   • Defining the outcome in advance and using a comparison group
7. Which bias check does the guide recommend?
   • Whether the logo is inclusive
   • Whether the tool performs worse for English learners or students with certain dialects
   • Whether the tool has a dark mode
   • Whether it supports emoji
8. What does the US Department of Education's 2023 AI report stress that makes a useful evaluation test?
   • Replacing teachers fully
   • Buying the newest model
   • Keeping humans in the loop so teachers can review and override
   • Avoiding all data collection