arXiv — cs.AI preprintsInternational19 October 2026
FormInv: A Measurement Protocol for Semantic Invariance in Mathematical Reasoning Benchmarks
This is an official announcement record
Firsthand records what arXiv — cs.AI preprints announced and links to the original. The wording below is theirs, not ours.
arXiv:2605.29001v2 Announce Type: replace-cross Abstract: A paraphrase-quality audit of MathCheck (ICLR 2025) detected 4 semantically incorrect paraphrases in 129 groups (3.1%); removing them drops GPT-4o from rank 2 to rank 4 and elevates Claude Haiku and DeepSeek V3 above it. Cross-model unanimity found these errors automatically for under $10; in our own dataset the same protocol found that 47% of auto-generated connective-variation paraphrases were semantically incorrect. That flaw compounds a deeper measurement gap. On the theorems every model answers correctly in canonical form, a semant
Read the official announcement
Opens arxiv.org
More from arXiv — cs.AI preprints
- Structured Authority Injection: Persistent Human Control for LLM Agents via Typed MCP Overrides19 October 2026
- Rethinking User Encoders: Routing Diffusion States for Scenario-Adaptive User Representation19 October 2026
- From Events to Reasoning: Hierarchical Rule Composition over Explicit States19 October 2026
- ECHOES: Seeing Beyond the Current Image via Recursive History Compression and Privileged On-Policy Distillation19 October 2026
- Sensitivity Analysis of Decision Rules in an Agent-Based Agro-Ecological Model19 October 2026
This content is for informational purposes only and is not professional advice. Specifications, prices, plan tiers, and features change frequently and may differ from what is shown here; verify current details on the manufacturer's or company's official page before purchasing. Ratings are based on analysis of published documentation, not independent lab testing.