Stanford AI Lab and Berkeley Lab Collaborate with NVIDIA to Propose New Model Verification Framework
Stanford AI Lab and Berkeley Sky Computing Lab have teamed up with NVIDIA to propose a new model verification framework "LLM-as-a-Verifier" to replace the traditional "LLM-as-a-Judge" AI evaluation mechanism. The core innovation lies in no longer using discretized scoring, but instead reading the probability distribution of the model at various scoring levels to calculate continuous reward values; simultaneously, each output is judged repeatedly and averaged to reduce random error. The verification process is divided into three independent dimensions: task satisfaction, format correctness, and error signals. In tests, the accuracy of Gemini 2.5 Flash as a verifier was 74.7%, significantly higher than the traditional Judge's 57.0%, and improved to 77.4% after 16 repetitions.
The paper team tested on Terminal-Bench 2, showing that GPT-5.4's random selection success rate was 81.8%, which increased to 86.4% after selection by the Verifier; on SWE-Bench Verified, the combined results of Claude Opus 4.5, Opus 4.6, and Gemini 3 Flash also rose from 76.1% to 77.8%. This framework is now open-sourced, providing a new standard for multi-model collaboration and self-checking.
Source: Public Information
ABAB AI Insight
Verifier框架的出现意味着AI评估逻辑从“裁判分数制”转向“概率信号制”。传统Judge仅给出离散标签，无法刻画模型在边界答案间的差异；Verifier则直接读取信念分布，让对错之间的置信度成为反馈主体。这让AI能理解“正确的概率”，而非单纯模仿人类判决。结构上，这相当于把评估从语言判断转为统计推理。
在更广的技术链条中，这建立了“AI自洽系统”的雏形——生成、比较、验证均可由模型完成，形成闭环学习生态。这样的多运行与自动优选机制，统计上可突破单次模型性能上限。它把AI的进步单位从模型规模转移到评估机制本身，使AI系统具备自我修正与自我选优的能力。
从历史视角看，这是AI发展由“扩张阶段”迈向“循环阶段”的标志事件。前者依赖参数累积提升性能，后者通过结构设计提升质量密度。Verifier代表AI评价体系的制度化转折：算法开始拥有判断自身输出真伪的独立能力——这意味着智能的评估权首次从人交还给模型本身。