ai-claim-provenance.md

AI Claim-Provenance Engine

A provenance engine for AI's claim stream — model releases, benchmark scores, vendor metrics — extracted into a ledger of claims, each tiered by source reliability and pinned to a hash-verified snapshot. It renders a brief that points at receipts and never adjudicates: a claim with no re-verifiable snapshot does not render.

PythonPostgreSQLClaims ledgerHash-verified snapshot receiptsAdmiralty source tieringLiteLLMKubernetesZero-network serve

What it is

The AI industry ships claims faster than anyone can check them — a model beats a benchmark, a vendor leads a category, a score appears with no referee. This is a pipeline that treats each of those as a claim to be pinned to its source, not believed. It ingests the model-news stream and the independent-eval feeds, extracts every assertion into a subject–predicate–object triple, tiers it by how reliable the source is and how many others corroborate it, and stores the raw source as a hash-verified snapshot. The output is a brief of claim-cards — the claim, its source rating, whether it was scored on an independent benchmark or self-reported, and a one-tap pointer to the receipt. It never tells you the claim is true. It tells you exactly who said it and what backs it.

The invariants are the product

The rules are structural, not aspirational — each is enforced on the serve path, not promised in a README:

  • Never adjudicate. No truth labels, no verified / debunked verdicts. A card serves the claim triple, the source rating, and the verifiability class — independently benchmarked versus vendor-administered — as raw data. Corroboration raises a claim's visibility; it never blesses it.
  • A receipt that can't be re-verified is no receipt. Every source is snapshotted and re-hashed on read. A snapshot that is absent or tampered fails integrity and its evidence line is dropped — and a claim left with no verified receipt does not render at all. Provenance is checked at serve time, not asserted once at write time.
  • Grounded or rejected. A claim the model proposes but cannot ground in the source text is refused at the extraction gate. Nothing reaches the ledger on the model's word alone.
  • The funnel is logged, not hidden. Every proposed claim that doesn't survive — ungrounded, unresolved, un-snapshotted — is written to a rejection ledger with the exact stage it died at. The pipeline's failures are as inspectable as its output.
  • Source tier served raw. Each claim carries an Admiralty rating — a reliability letter times a corroborating-source count — and independent-benchmark claims are structurally distinguished from vendor-administered ones. The reader tiers; the engine doesn't.
  • Zero network on serve. The brief renders from the local database and snapshot store only — no third-party request on the path that serves you the page.

How it's built

fetch → snapshot → extract → resolve → store → render, every stage real code. A self-hosted LLM gateway does the extraction; everything downstream is deterministic. Ingest sources are hostile-by-default, so TLS on that path is non-negotiable — the only override is a custom CA bundle, never an off switch. It runs unattended in-cluster on a schedule, lays a dated brief onto a persistent volume, and re-verifies every receipt against the snapshot store on the way out.