Post-mortems — what Grail Atlas got wrong
We keep this list because trust is earned, not asserted. Every substantive mistake we've made is here, with what happened, how we caught it, and what we changed.
Want to report something we got wrong? See the acceptable-use channel.
2026-05-19
Brand tier misclassification: Vacheron + Jaeger-LeCoultre
What happened
Our initial brand-tier table placed Vacheron Constantin and Jaeger-LeCoultre in 'luxury' alongside Omega and Tudor. This is wrong — Vacheron is the third member of the holy trinity (Patek / AP / Vacheron) and JLC is one of the few brands that manufactures movements for the holy trinity. Misclassification understated their values in the engine's factor table.
How we found out
Caught by the fifth advisory-panel watch-domain review. We hadn't done a deep enough domain-expert pass on the tier table before shipping the factor-aware valuation.
What we changed
Vacheron Constantin + Jaeger-LeCoultre moved to top-luxury. TAG Heuer + Hublot demoted to enthusiast (also flagged by the panel — neither is treated as a luxury reference in the trade today). Missing modern independents (Moser, MB&F, De Bethune, Greubel Forsey, Voutilainen, Akrivia) added to top-luxury. Committed 2026-05-19; brand-classifier tests updated; published.
2026-05-19
Momentum band too tight: ±3% stable range on thin comps
What happened
The initial market-momentum implementation flagged a ±3% slope as 'firming' or 'softening' regardless of how thin the comp set was. On a 5-comp recent window, ±3% is well below the noise floor — we were calling normal market variance 'momentum'.
How we found out
Watch-domain panel review caught it; we'd validated against synthetic fixtures (which had artificially-tight spreads) rather than realistic auction-archive variance.
What we changed
Required ≥8 comps per half-window. Widened stable band to ±5%. Switched to a trimmed median (drop top + bottom decile). Added regime detection. The engine now returns 'insufficient-data' rather than fabricating a confident slope.
2026-05-19
Backtest harness validated against engine-generated fixtures
What happened
The valuation-backtest harness was being exercised against fixtures the engine itself had produced — a circular check that would have passed even if the engine were broken.
How we found out
Engineering panel surfaced the circularity in the fifth-panel review. We'd built the harness for methodology testing without thinking about how it would catch a regression.
What we changed
Replaced with a curated corpus of comp panels + holdout pairs hand-authored to be realistic for each reference's market. Added a CI gate that fails when band coverage drops below 55%, median MAPE exceeds 10%, or deal-tier alignment drops below 50%. Floors set at current engine performance minus a margin — these are regression bounds, not target accuracy claims.
2026-05-19
Risk wording was binary on the first draft
What happened
The initial risk-assessment engine had a single 'flagged / not-flagged' boolean. A 'flagged' listing in copy read uncomfortably close to an accusation.
How we found out
Brett's own discomfort during a code review surfaced it; multiple external reviewers later confirmed the legal exposure.
What we changed
Replaced with a graded scale (clear / caution / elevated / high) plus factual non-accusatory signals ('the seller's account is 12 days old', not 'the seller is suspicious'). Code-enforced ban on the words 'fraud', 'scammer', 'crook', 'thief' in any published string.
2026-05-19
Trademark-exposure tagline in draft marketing copy
What happened
Drafted internal marketing copy that referred to the project using a specific competitor-trademarked tagline. This is the kind of language that draws an early cease-and-desist before launch.
How we found out
Caught during legal review of the policy drafts.
What we changed
Code-enforced banned-phrase test on production copy. The specific phrase stays documented internally so the team knows what's banned, and never appears in any public string.