Medical Hallucinations | Xi Zhang

🎤 Talk at Glasgow AI4BioMed Lab!

Wed, 17 Jun 2026 00:00:00 +0000

💡 Talk Title

“CCS: Clinical Consensus Selection for Radiology Report Generation”

🖇️ Abstract

Radiology report generation (RRG) is commonly framed as a single-path task, where a multimodal large language model (MLLM) produces one decoded report as the final output. While progress has largely come from scaling training data, model capacity, and retrieval mechanisms, improving report quality at inference time remains underexplored. We observe that fixed radiology MLLMs often generate clinically stronger reports elsewhere in their candidate pool than the one selected by default decoding, suggesting that inference-time decision making is an overlooked bottleneck. To address this, we propose Clinical Consensus Selection (CCS), a decoder-agnostic inference-time selection framework that samples multiple candidate reports and selects the one with the highest clinical consensus across the rollout pool. CCS combines text-based utilities with a radiology-adapted utility computed by an image–report-trained multimodal embedder, measuring agreement beyond surface-level textual similarity. Across three datasets and multiple radiology MLLMs, CCS consistently improves inference-time performance over single-path decoding and generic Best-of-N baselines, with especially clear gains on clinical metrics. Further analysis shows that image-grounded utility forms a selection axis distinct from textual consensus, and that substantial headroom remains for improving RRG at inference time.

🔗 Project Website: https://x-izhang.github.io/CCS/

📺 Slides

📍 Glasgow AI4BioMed Lab — An interdisciplinary research lab focusing on AI applications in biomedical sciences, based in Glasgow.

📅

When: Wednesday, June 17, 2026 at 2pm
Where: F121

See you there!

🎉 Paper Accepted — See You at ACL 2026!

Mon, 06 Apr 2026 00:00:00 +0000

“CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding” has been accepted to ACL 2026!

📍 ACL 2026 — The 64th Annual Meeting of the Association for Computational Linguistics will take place in San Diego, California, July 2026

See you there!

🪧 Poster

CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding

Mon, 06 Apr 2026 00:00:00 +0000

For the latest updates and details, visit the CCD Project Website.

Why CCD? 🔬

Radiology MLLMs remain vulnerable to prompt-induced hallucinations when clinical sections contain counterfactual details or ambiguous guidance. The figure above contrasts baseline predictions with CCD-enabled outputs across report generation and question answering tasks. Red highlights mark unsupported findings that can compromise patient care, while blue text reflects misleading prompt context the model must resist.

Clinical Contrastive Decoding (CCD) mitigates these risks at inference time. Rather than retraining models or relying on retrieval corpora, CCD injects trustworthy, image-grounded signals distilled from specialist expert models. The result is a decoding policy that maintains fluency yet stays faithful to the radiograph.

CCD at a Glance ⚙️

Clinical Contrastive Decoding (CCD) is a plug-and-play inference framework designed to reduce medical hallucinations in radiology MLLMs. It introduces structured clinical supervision from expert models (e.g., DenseNet or MedSigLIP, or others) at decoding time, without modifying model weights or requiring external retrieval.

Given a chest radiograph, the expert model predicts symptom-level probabilities across 14 CheXpert categories. CCD integrates this signal through a dual-stage logit refinement strategy:

Symptom-grounded Contrastive Decoding (SCD): constructs an anchor prompt using high-confidence findings (e.g., “Atelectasis, Cardiomegaly”) and generates a contrastive logits path conditioned on this prompt. The final logits are a weighted interpolation between the anchor-conditioned and original paths, encouraging the model to mention supported findings and suppress false negatives.
Expert-informed Contrastive Decoding (ECD): transforms expert probabilities into token-level logit biases via log-odds conversion. These biases are injected into the logits from the first stage, softly penalising unsupported findings and reducing false positives.

This two-stage mechanism enables CCD to progressively guide generation with both symbolic supervision (via anchor prompts) and probabilistic constraints (via confidence scores), achieving robust improvements in both report generation and VQA. CCD is fully model-agnostic and integrates seamlessly with state-of-the-art radiology MLLMs such as MAIRA-2, Libra, LLaVA-Rad, and LLaVA-Med.

Unlike prior contrastive decoding approaches that rely on perturbed visual or textual inputs, CCD leverages clinically grounded signals from expert models to provide task-specific and symptom-level control during generation.

Key Contributions ✨

Empirical Insight: We conduct a systematic analysis of prompt-induced hallucinations in radiology MLLMs, revealing that noisy clinical sections (such as irrelevant or contradictory clinical details) can trigger unsupported findings across multiple datasets.
Inference-time Framework: We introduce Clinical Contrastive Decoding (CCD) — a dual-stage inference-time strategy that leverages expert-derived labels as anchor prompts and performs probabilistic logit adjustments, requiring no retraining or architectural modifications.
Consistent Gains: Extensive experiments on MIMIC-CXR, IU-Xray, and CheXpert Plus demonstrate that CCD achieves up to +17% improvement in RadGraph-F1 with SOTA radiology MLLMs (MAIRA-2), along with higher VQA accuracy, all without altering model weights or structures.

Evaluation Highlights 📈

Achieves up to +17% RadGraph-F1 on MIMIC-CXR when attached to state-of-the-art report generators.
Reduces mention-level hallucinations on IU X-Ray while maintaining BLEU / ROUGE scores, confirming that CCD improves factuality without harming surface-level metrics.

Ablation Studies 🧪

Ablations show both contrastive stages matter: removing the anchor projection (SCD) or the logit sharpening (ECD) degrades factual metrics by ~5–8%.

My Musings ⏳

As I reflect on the development and impact of Clinical Contrastive Decoding (CCD),

It’s better to be roughly right than precisely wrong.
— Carveth Read, Logic: Deductive and Inductive

This quote perfectly captures the spirit of our approach. In the high-stakes field of radiology, the cost of hallucinations can be severe. By integrating expert-derived signals at inference time, CCD prioritises clinical accuracy over mere linguistic fluency. It’s a reminder that in medical AI, being approximately correct and grounded in reality is far more valuable than generating fluent but misleading text.

BibTeX 📚

@article{zhang2025ccd,
 title={CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding},
 author={Zhang, Xi and Meng, Zaiqiao and Lever, Jake and Ho, Edmond SL},
 journal={arXiv preprint arXiv:2509.23379},
 year={2025}
}

🎤 Talk at Glasgow AI4BioMed Lab!

Wed, 28 Jan 2026 00:00:00 +0000

💡 Talk Title

“CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding”

🖇️ Abstract

Radiology multimodal large language models (MLLMs) often generate clinically unsupported descriptions, posing serious risks in medical applications. We introduce Clinical Contrastive Decoding (CCD), a training-free inference framework that integrates structured clinical signals from radiology expert models to mitigate hallucinations. CCD refines token-level logits during generation through a dual-stage contrastive mechanism, enhancing clinical fidelity without modifying the base MLLM. On the MIMIC-CXR dataset,CCD yields up to 17% improvement in RadGraph-F1, providing a lightweight solution for bridging expert models and MLLMs in radiology.

🔗 Project Website: https://x-izhang.github.io/CCD/

📺 Slides

📍 Glasgow AI4BioMed Lab — An interdisciplinary research lab focusing on AI applications in biomedical sciences, based in Glasgow.

📅

When: Wednesday, January 28, 2026 at 2pm
Where: F121

See you there!

🎤 Invited Talk at DICTA 2025!

Tue, 25 Nov 2025 00:00:00 +0000

💡 Talk Title

Multimodal Medical Models: Cross-modal Alignment and Consistency

🖇️ Abstract

This talk provides an overview of recent developments in medical vision–language modelling and examines key sources of misalignment that lead to hallucinations. Emerging strategies to enhance visual, semantic, and temporal consistency will also be discussed, highlighting pathways toward safer and more trustworthy clinical AI systems.

📺 Slides

📍 DICTA 2025 — The 26th International Conference on Digital Image Computing: Techniques and Applications will take place in Adelaide, Australia, from 3–5 December 2025.

🏥 MedAI-CHAS — A workshop focusing on Challenges, Hallucinations, and Solutions for Advancing Clinical Utility in Medical AI.

See you there!