概述
They are not validated diagnostic devices. Their safest use is as a structured second-opinion brainstorm that a clinician checks against the patient, and doctors who understand both their strengths and their failure modes get the most out of them.
深入探討
Much of the evidence comes from case vignettes. In a 2023 JAMA research letter, Kanjee and colleagues tested GPT-4 on 70 difficult cases from the New England Journal of Medicine clinicopathological conferences. The final diagnosis appeared in the model's differential in about 64 percent of cases and was its top diagnosis in about 39 percent. A randomized trial by Goh and colleagues, published in JAMA Network Open in 2024, gave 50 physicians either GPT-4 plus usual resources or usual resources alone for structured diagnostic cases. Access to GPT-4 did not significantly improve physicians' diagnostic reasoning scores. GPT-4 working alone, however, scored substantially higher than the physicians who used only usual resources. One interpretation is that physicians did not know how to use the tool well, or did not trust it when it disagreed with them. Research systems such as Google's AMIE have also been tested on diagnostic dialogue and differential generation, with strong results in controlled studies. These results need careful reading. Published case series may be in a model's training data. Vignettes arrive already cleaned up, with the relevant findings chosen and summarized, while real patients bring incomplete, contradictory and unstated information. Scoring "correct diagnosis somewhere in the list" rewards long lists, which can prompt extra testing. The common misconception is that benchmark accuracy equals bedside accuracy. It does not. Known failure modes include confident but wrong reasoning, invented references or lab thresholds, anchoring on whatever diagnosis the user hints at, over-weighting common textbook presentations, and missing time-critical conditions when the description leaves out vital signs. Privacy is a separate issue. Entering identifiable patient information into a consumer chatbot without a business associate agreement can breach HIPAA, so clinicians should use tools their organization has approved.
戰略影響
配裝選擇
應用級設計決定了人工智慧是否能改善實際結果。
團隊與工作流程
良好的工作流程整合可以創造使用者值得信賴的生產力效益。
風險與安全
範圍明確的用例可以減少變更疲勞和實施風險。
The Future of Using ChatGPT for Differential Diagnosis
Diagnostic AI is moving from general chatbots toward tools built into EHRs and evidence platforms, where they can see structured data and cite sources. That may reduce some errors, but it adds the risk that clinicians defer to a confident suggestion. The Goh trial points to the open question: how clinicians and models should work together, not just how accurate the model is alone. Prospective studies with real patients and real outcomes are still scarce compared with vignette benchmarks. Until more exist, the defensible position is that these tools can widen a differential and prompt reconsideration. The clinician still owns the diagnosis.
現實世界的實施
A hospitalist types a de-identified summary of fever, rash, eosinophilia and a new anticonvulsant into her health system's approved AI tool and asks for can't-miss diagnoses. DRESS appears on the list, and she reviews the medication timeline.
A resident asks the model which findings would best tell apart his top three diagnoses for acute dyspnea, then uses the answer to plan a focused exam and tests, not to settle on a diagnosis.
A primary care physician who suspects a viral illness asks the model to argue against her leading diagnosis and list what she might be missing. This is a deliberate check on anchoring.
A clinician gets two different ranked lists after asking the same question twice with slightly different wording, and takes that as a sign the model's ordering is not a probability estimate.
風險與防護欄
將損壞的流程自動化可能會加劇現有問題。
團隊可能會過度自動化並消除所需的人工判斷。
如果不持續評估輸出，品質可能會出現偏差。
實施路線圖
繪製目前工作流程並確定摩擦最大的步驟。
在完全自動化之前定義人工檢查點。
對使用者進行提示、升級路徑和品質標準的訓練。
追蹤任務級結果以確認持續價值。
不斷探索
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Using ChatGPT for Differential Diagnosis quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
常見問題
What is Using ChatGPT for Differential Diagnosis?
Large language models such as GPT-4 can produce plausible differential diagnosis lists from a case description, and in published studies they often include the correct diagnosis. They are not validated diagnostic devices. Their safest use is as a structured second-opinion brainstorm that a clinician checks against the patient, and doctors who understand both their strengths and their failure modes get the most out of them.
In Kanjee and colleagues' 2023 test of GPT-4 on NEJM clinicopathological cases, about how often was the final diagnosis anywhere in the model's differential?
The final diagnosis was in the differential in about 64 percent of cases and was the top diagnosis in about 39 percent.
What was the main result of the 2024 Goh et al. randomized trial?
Physicians with GPT-4 did not significantly outperform physicians with usual resources, yet the model alone scored higher. That points to how people use the tool.
Why can vignette benchmarks overstate how well a model would do with real patients?
Pre-organized findings and possible training data contamination both make vignettes easier than messy real encounters.
A clinician asks, "Could this be lupus?" at the start of her prompt. Which failure mode does this invite?
Models tend to agree with the framing they are given, so hinting at a diagnosis pulls the differential toward it.
Why is scoring "correct diagnosis anywhere in the list" a flawed measure?
A longer list is more likely to contain the answer, but long differentials in practice can lead to extra and unnecessary workups.
繼續學習
相關指南
為此主題精選的更多指南