Tracelight benchmark report
June 2, 2026
How Tracelight performs against Anthropic and OpenAI on complex spreadsheet modelling and error checking, measured across 80 real Excel models.
Complex spreadsheet modelling
The modelling benchmark poses 240 prompts across the same 80 files, in three difficulty tiers: simple (avg. 980 cells to populate), medium (avg. 1,700) and hard (avg. 4,200). Agents receive the partially built model and instructions; we score exact cell-to-cell formula correctness against the reference build — a numerically correct hardcode scores zero, the formula logic must be right.
Tracelight Agent scored 75%, ahead of GPT-5.5 (68%) and Opus 4.8 (66%). The headline metric only scores the requested cells — it doesn't penalise an agent for damaging the rest of the workbook, and that flatters GPT-5.5. Auditing the delivered files, GPT-5.5 modified ~65% more out-of-scope cells than Tracelight - editing ranges the prompt never asked for. Tracelight's edits go through validated changesets that check formula health, flag unintended hardcodes and preserve surrounding structure and formatting, so completed sections drop into the model without collateral damage to the parts that already worked.
Anthropic
OpenAI
TracelightAnthropicOpenAITracelightError checking
We planted 480 real spreadsheet errors — broken links, off-by-one ranges, wrong-line-item references, hardcode overrides, sign flips — into 80 complex Excel models (avg. 18.8 sheets, 156k cells) and asked each system to audit the workbooks read-only. Tracelight Model Review found 68% of the planted errors, versus 37% for Opus 4.8 and 31% for GPT-5.5, both run at high effort via Anthropic's model-review skill for fair comparison.
The reports themselves differ as much as the recall numbers. Opus produces the longest audits — over 25 findings per model — but the bulk of them are stylistic and hygiene observations: hardcoded constants, missing documentation, formatting conventions, hedged "worth reviewing" notes. GPT-5.5 is more restrained but catches the fewest real errors. Tracelight's findings were roughly twice as precise as Opus's — a Tracelight finding was about twice as likely to correspond to an actual planted error — because it concentrates on logical faults: wrong precedents, broken roll-forwards, inconsistent formulas, bad aggregation ranges. The severity labels tell the same story: half of Opus's findings were mid-tier "warnings" that shift verification work onto the reader, while Tracelight's skewed toward concrete, high-severity diagnoses with exact cell references. In practice that's the difference between an actionable checklist and a wall of observations you must re-audit yourself.
AnthropicOpenAITracelightAnthropicOpenAITracelight