arXiv — cs.AI preprintsInternational12 October 2026
What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)
This is an official announcement record
Firsthand records what arXiv — cs.AI preprints announced and links to the original. The wording below is theirs, not ours.
arXiv:2608.06202v2 Announce Type: replace-cross Abstract: Large language model (LLM) benchmark evaluations are routinely used to support claims about model safety, reliability, and deployment readiness. Yet most evaluations rely on a single access modality (model APIs), perform a single run per prompt, and report accuracy as the primary outcome metric, without accounting for conditions such as web search that may have effects on model behavior in deployment. We audit these assumptions for one of the most widely-used LLMs, comparing two modalities, ChatGPT's chat UI and OpenAI's API, with and w
Read the official announcement
Opens arxiv.org
More from arXiv — cs.AI preprints
- Faithfulness Without Anchors: A Position Paper on the Structural Misalignments of Explainable AI Faithfulness Metrics for Feature Importance12 October 2026
- EliteTable: Self-Improving Program Search for Executable Control Policies12 October 2026
- Equal-Budget Monte Carlo Tree Search versus Randomized-Restart List Scheduling for DAG Workflows: A Controlled Equal-Budget Computational Study12 October 2026
- Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning12 October 2026
- SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire12 October 2026
This content is for informational purposes only and is not professional advice. Specifications, prices, plan tiers, and features change frequently and may differ from what is shown here; verify current details on the manufacturer's or company's official page before purchasing. Ratings are based on analysis of published documentation, not independent lab testing.