Kedis ku xidhan hagaha · Adag Heerka

Isticmaalka ChatGPT ee Kediska Cilad-sheegashada Kala Duwan

Waxay tijaabisaa waxa daraasadaha Kanjee iyo Goh ay ka heleen ogaanshaha GPT-4, sababta vignette bartilmaameedyada saxnaanta sariiraha ee sariirta, iyo sida ku xidhidhisku u saameeyaan soo-jeedinta.

Wadooyinka haga ee la xidhiidhaChatGPT ee Baadhista Kala Duwan
Su'aal 1 ee 8

In Kanjee and colleagues' 2023 test of GPT-4 on NEJM clinicopathological cases, about how often was the final diagnosis anywhere in the model's differential?

Horudhac su'aal kasta8 su'aalo

Jawaabaha iyo sharraxaadaha waxay soo baxayaan markaad qaadanayso kediska.

  1. In Kanjee and colleagues' 2023 test of GPT-4 on NEJM clinicopathological cases, about how often was the final diagnosis anywhere in the model's differential?
    • About 39 percent
    • About 64 percent
    • About 92 percent
    • About 15 percent
  2. What was the main result of the 2024 Goh et al. randomized trial?
    • Physicians given GPT-4 scored far higher than those without it
    • GPT-4 alone performed worse than every physician
    • Giving physicians GPT-4 did not significantly improve their diagnostic reasoning scores, although GPT-4 alone scored higher
    • The trial was stopped for patient harm
  3. Why can vignette benchmarks overstate how well a model would do with real patients?
    • Vignettes arrive with the relevant findings already chosen and cleaned up, and published cases may be in training data
    • Vignettes are always written in another language
    • Real patients have fewer symptoms than vignettes
    • Models cannot read case descriptions longer than a paragraph
  4. A clinician asks, "Could this be lupus?" at the start of her prompt. Which failure mode does this invite?
    • Invented laboratory reference ranges
    • Refusing to answer medical questions
    • Losing the conversation history
    • Sycophantic anchoring toward the diagnosis the user suggested
  5. Why is scoring "correct diagnosis anywhere in the list" a flawed measure?
    • It rewards long lists, which can prompt unnecessary testing
    • It ignores the patient's age
    • It only works for pediatric cases
    • It requires the model to cite references
  6. Getting two differently ordered lists from nearly identical prompts shows what about the model's ranking?
    • The model is broken and should not be used
    • The second answer is always the correct one
    • The ranking is not a calibrated probability estimate and shifts with wording
    • The model is searching the live internet
  7. What is the main privacy concern with pasting a patient's chart into a consumer chatbot?
    • The chatbot will refuse clinical questions
    • Entering identifiable information without a business associate agreement can breach HIPAA
    • Consumer chatbots cannot read lab values
    • It automatically bills the patient's insurer
  8. Which prompt request best helps a clinician act on a model's differential?
    • "Give me the single correct diagnosis"
    • "Confirm my diagnosis is right"
    • "Write the discharge summary"
    • "List supporting and contradicting features for each item and the test that best separates the top candidates"