fail.ticker.io / methodology & faq

Methodology & FAQ

There is no single true count of AI incidents — only databases with different editorial bars. This page explains exactly where every number on this site comes from, what the definitions are, and what the known gaps look like.

Definitions

We follow the OECD definition. An AI incident is an event where the development or use of an AI system caused actual harm — to people, property, rights, infrastructure, or society. An AI hazard is the near-miss variant: an event that plausibly could have caused such harm but didn't (yet). The wire labels each entry as one or the other.

Sources

SourceWhat it isPowersLicense
OECD AI Incidents Monitor Automated pipeline mining global news (Event Registry) with LLM classification; 16,000+ events since 2020, ~30 new per day. Totals, yearly/monthly charts, harm-type chart, the wire, FAILCON Attribution + linkback
AI Incident Database (AIID) Human-curated incidents with full report trails, run by the Responsible AI Collaborative; weekly public snapshots. Curated count, per-incident case links CC BY-SA 4.0
MIT AI Risk Repository — Incident Tracker MIT FutureTech's classification of AIID incidents by risk domain, severity across ten harm categories, and EU AI Act risk level. Hall of Fail, risk-domain chart CC BY 4.0

The three sources interlock: MIT classifies AIID's incidents (same IDs), and OECD records cross-reference AIID where cases overlap. Where a number could come from more than one source, the chart or tile says which one it uses.

Pipeline & cadence

A scheduled pipeline pulls the OECD and MIT feeds daily and the AIID snapshot weekly, rebuilds the aggregate JSON files served at /data/stats.json, /data/wire.json and /data/hall.json, and republishes the site. The "updated" stamp on every page is the last successful run. The site is static — what you see is the data as of that stamp.

FAILCON

FAILCON is our 5-level activity meter: the trailing 30-day count of documented incidents and hazards in the OECD feed, bucketed against thresholds calibrated on the last 24 months of data. It measures documentation volume, not danger — a quiet month in the news is not a safe month in deployment. Level 5 is the quietest this decade gets; level 1 means the trailing month is heavier than almost any month on record.

Known gaps (read before quoting)

  • All three sources skew towards English-language media; incidents outside it are undercounted.
  • The OECD feed is LLM-classified news — single events occasionally split into several records, or merge.
  • "Documented" lags "happened" by days; recent counts always drift upward after the fact.
  • Severity scores in the Hall of Fail are model-assigned by MIT and can change on re-review.
  • Counts are floors, not ceilings: most AI failures never make the news at all.

FAQ

What counts as an AI incident?

An event where an AI system's development or use caused actual harm (OECD definition). A hazard is the near-miss variant. Both appear on the wire, labelled separately.

Why do your numbers differ from other trackers?

Editorial bars differ: OECD's automated monitor lists 16,000+ events, AIID's human curation ~1,600. We show both and label which is which. Every count is a floor.

How often is the data updated?

OECD and MIT feeds daily, the AIID snapshot weekly. The "updated" stamp is the last successful pipeline run.

Can I reuse the data?

Yes — the JSON feeds are open. Attribute the upstream sources (AIID, CC BY-SA 4.0; MIT AI Risk Repository, CC BY 4.0; OECD AIM) and link back here.

I found an incident you're missing. Where do I report it?

Report it upstream to the AI Incident Database submission form — it will flow into this tracker (and every other one built on AIID) once accepted.

Is this site anti-AI?

No — it's built by people who ship software with AI every day. Uncounted failure modes get repeated; counting them honestly is basic engineering hygiene.