LEGEND· 2012

Postmortem Puzzle · Hard · +75 XP · ~5 min

LEGEND · 2012 · Knight Capital
Class: trading systems
Loss: $440M / 45 min

Knight Capital — pick the root cause

Casualty stat: $440M lost in 45 min · firm bankrupt

Read the redacted report. Pick the most likely root cause.

Incident report — redacted

BACKGROUND
• Knight ran 8 servers in their order-routing tier.
• A new router 'SMARS' was being deployed under repurposed feature flag X.
• Flag X had previously enabled 'Power Peg' — code dormant since 2003.
• Deploy: copy new binary to all 8 servers, restart.

TIMELINE
• T-1 day: deploy script runs. Reports success.
• T+0 09:30:00: market open. SMARS goes live on flag X.
• T+0 09:30:01: orders begin executing at unexpected sizes and prices.
• T+0 09:30:30: ops notices anomalous volume.
• T+0 09:45:00: team isolates one server as the source; takes it offline.
• Loss as of disable: $440M.

OBSERVATIONS
• 7 of 8 servers ran SMARS correctly.
• 1 server ran the old Power Peg code under flag X.
• Flag X on that server reached the dormant code path.

Most likely root cause

Playing anonymously. Sign in to save XP, streak, and badges.