AI for Medical Students and USMLE Prep

USMLE Step 1 became pass/fail in January 2022, but it still has to be passed, and students prepare for it with established resources: question banks such as UWorld; NBME self-assessments; First Aid for the USMLE Step 1; Pathoma, Boards and Beyond and Sketchy; and community Anki decks such as AnKing. AI fits alongside these, not in place of them. A 2023 study by Kung and colleagues in PLOS Digital Health reported that ChatGPT performed at or near the passing threshold on USMLE-style questions, and later models have scored higher. Scoring well on multiple-choice questions does not mean every explanation a model writes is correct, or that the questions it writes are exam-quality. AI is useful for: explaining concepts in several ways, including analogies and step-by-step reasoning; walking through why each wrong option is wrong; turning notes into flashcards for active recall; and drafting study schedules around your exam date and weak areas. AI-made questions have typical error modes: two defensible answers; a wrong keyed answer; outdated or non-US guidelines; vignettes that name the diagnosis outright; cues such as the longest option being correct; and invented facts, like the wrong enzyme in a pathway or impossible lab values. Models also tend to be sycophantic. If you push back, they may abandon a correct answer. Some habits reduce the risk. Check claims against trusted resources. Ask the model to justify every distractor. Use AI questions for low-stakes drilling. Rely on NBME self-assessments to judge whether you are ready. Before pasting commercial question-bank content into a chatbot, check the provider's terms of use. A common misconception is that because AI can pass the USMLE, its questions are as good as the exam's. Another is that AI can replace a curated question bank written and reviewed by experts.