Apr 2026 · Agentic AI in the Wild workshop · ICLR 2026
When a Team of AI Models Is Unsure
SCoOP: Semantic Consistent Opinion Pooling for uncertainty in multi-VLM systems
Chung-En (Johnny) Yu · Brian Jalaian · Nathaniel D. Bastian
My Ph.D. student Johnny Yu presented SCoOP as a poster at the ICLR 2026 workshop on Agentic AI in the Wild: From Hallucinations to Reliable Autonomy. It is the first step in our line of work on multi-agent uncertainty, which continues with CUSP.
Systems that combine several vision-language models can still hallucinate, and no single model's confidence tells you when. SCoOP is a training-free way to measure how unsure the whole system is, so it can catch hallucinations and abstain.
The problem
More AI systems now combine several vision-language models (VLMs), each answering the same question about an image. Individually confident models can still disagree, or agree on something wrong, and there was no simple way to turn their outputs into one trustworthy measure of how sure the system is.
How it works
- Treat each VLM as a probabilistic expert, and sample several answers from each.
- Map all of the answers into a shared semantic space, so answers that mean the same thing count as the same opinion.
- Pool the experts' opinions into one system-level uncertainty, and use it to flag likely hallucinations or abstain.
SCoOP needs no training and adds only microseconds of overhead on top of the models themselves.
Results
- On ScienceQA, SCoOP detects hallucinations with an AUROC of 0.866.
- Used to decide when to abstain, it reaches an AURAC of 0.907, well above the baselines.