What happened
Bioengineer.org reports on a randomized clinical vignette study in which 44 internal medicine teaching faculty reviewed four cases with AI-generated e-consultation advice. Participants rated the advice similarly whether they were told it came from a human specialist or an AI system.
Bioengineer.org reports that researchers at the University of Pittsburgh School of Medicine recruited 44 internal medicine teaching faculty from one large academic medical center. Participants reviewed four hypothetical clinical vignettes and answered management questions before and after reading electronic consultation advice generated by OpenEvidence, described in the report as a medically specialized system.
According to the report, participants were randomly told that the advice had been written either by a specialist attending physician or by an AI system, although all of the advice was AI-generated. Ratings across perceived quality, completeness, actionability, clarity, appropriateness, trustworthiness, and evidence-based practice averaged between 4.2 and 4.7 on a five-point scale, with no statistically significant difference associated with the label.
Bioengineer.org says that 37.5 percent of participants initially selected the management option later recommended by the AI advice, compared with 81.8 percent after reading it. The reported odds ratio was 8.43, with a 95 percent of 5.00 to 14.28 and a p-value below 0.001. The source reports no meaningful labeling effect, with an odds ratio of 1.18 and a confidence interval of 0.51 to 2.75.
The supplied report identifies the study as published in the Journal of General Internal Medicine and provides a DOI, but this evaluation relies on Bioengineer.org’s account and does not independently verify the paper, methods, or results. The source does not document OpenEvidence’s access conditions or pricing.
Source details: bioengineer.org ↗
Why it matters
The reported findings suggest that experienced physicians may act on AI consultation advice even when they know it is machine-generated. That could expand access to specialist-like guidance, but it also increases the importance of validating the system’s recommendations, monitoring automation , and defining responsibility when advice is wrong. The study measured persuasion and adoption, not clinical accuracy or patient outcomes.
The reported adoption rate is consequential because clinical decision-support systems can influence management choices even when clinicians know the advice came from AI. If recommendations are accurate and well-supported, the report says such systems could help distribute specialist knowledge and reduce variation in care, particularly where specialist access is limited.
The same finding creates a safety concern. Bioengineer.org notes that the study did not test whether the recommendations were objectively correct for the vignettes, nor did it measure patient outcomes. High acceptance therefore demonstrates influence, not clinical reliability. A system that is persuasive but wrong could create automation and shift quality-assurance demands toward validation, monitoring, and accountable clinical oversight.
The evidence is limited by the reported sample of 44 faculty members at one academic center and by the use of hypothetical cases. The source does not establish that the results generalize to community clinicians, other specialties, live patient care, or different AI consultation systems.
Interactive Mechanism: How It Actually Works
Explore the underlying technology behind this development interactively.
Which component of an AI application is the machine-learning model itself?
What to watch next
The report says future research should test whether the pattern holds with real patients, in other clinical settings, and under real time pressures. Replication, independent evaluation of recommendation accuracy, and evidence about oversight and clinician training will be important before the findings can support broader clinical conclusions.
The report says the researchers are calling for studies of the factors that influence adoption of AI-generated recommendations and for curricula that help faculty and trainees use consultation tools appropriately.
Useful follow-up evidence would include independent replication, prospective studies involving real clinical workflows, direct measurement of recommendation accuracy, and results showing whether AI-supported decisions improve or harm patient outcomes. The supplied source does not provide that evidence yet.
Access, pricing, regulatory status, and deployment requirements for OpenEvidence are not documented in the source, so the study should not be treated as evidence of general availability or a ready-to-use clinical product.