Medical benchmark finds an LLM logic cliff

LogiMed-RoB tested 10 state-of-the-art LLMs on 14,820 queries from 860 randomized controlled trials and found end-to-end consistency collapsing to 45.13%, even when the top model reached 98.88% on atomic checks. The LogiMed-RoB preprint also reports an 18.63–40.05% evidence-reasoning gap and blind-guess rates up to 48.28%, despite high-quality retrieval. For clinical AI teams, that makes GPT-Rosalind’s life-science promise a reason to gate evidence workflows, not trust final-label accuracy alone.