Skip to content

Provenance

Provenance answers two questions about a fact: how did it get into the store, and how did it get into this response. Limma reports both, and neither is a score.

How it reached this response: provenance

On the full fact form, provenance is one of two words:

  • addressed — the fact was returned by a coordinate lookup. The set it came from is enumerable and its size is knowable, so a completeness claim is available for it.
  • explored — the fact was returned by a ranked lane (word overlap, embeddings, the passage windows). It may be the best thing in the store; what it can never carry is a completeness claim.

r = client.post("/v1/public/facts", json={
    "workspace_id": WS, "question": "revenue", "mode": "enumerate",
    "coordinate": {"attribute": "revenue", "period": "FY2023"}})
fact = r.json()["facts"][0]
print("provenance:", fact["provenance"])
print("attestation_method:", fact["attestation_method"], "| byte_verified:", fact["byte_verified"])
print("status:", fact["status"], "| claim_level:", fact["claim_level"])
provenance: addressed
attestation_method: row | byte_verified: True
status: observed | claim_level: aggregate

r = client.post("/v1/public/facts", json={
    "workspace_id": WS, "question": "revenue 2023", "mode": "explore", "limit": 3})
lanes = {(f["kind"], f["attribute"], f["provenance"]) for f in r.json()["facts"]}
for kind, attribute, provenance in sorted(lanes, key=str):
    print(kind, "|", attribute, "|", provenance)
claim | headcount | explored
claim | net_income | explored
claim | revenue | explored

The default is explored, deliberately and in the safe direction: a lane that does not say where its fact came from has not earned a completeness claim.

How it reached the store

  • status — observed (passed the extraction confirmation gate) or conjectured (extracted but not promoted; held back rather than dropped, so you can decide).
  • attestation_method and byte_verified — whether and how the printed token was located in the document; see a receipt.
  • claim_level — aggregate (a total) or line_item (a component); aggregate_role / part_of / parts when the document's own arithmetic proved the row's place.

Declared versus extracted

A document's facts are extracted: a reader recovered the structure prose does not declare — the row label, the column period, the "in millions" line — and every one of those steps can be wrong. A connected table's facts are declared: the schema states which column is the entity, the period and the value, so nothing is inferred. POST /v1/public/connect (limma_connect) is that path; its mapping is stated rather than guessed on purpose, and a heuristic deciding that rev_amt means revenue would be a guess wearing the authority of a declaration.

Today each claim row carries a grade per axis in the store — declared, derived, inferred, or "" when the writer recorded nothing — written once, when the row is created. A reader maps "" (and anything it does not recognise) down to inferred: the cautious direction, so a completeness claim never inherits authority from a column nobody populated. On stores filled by today's extractors the empty grade is the common case, so what a reader sees for most rows is that downgrade, not a recorded grade. Those grades are not on a delivered numeric fact, and a numeric fact carries no per-row confidence.

Relation rows do carry both. Every relation row — on limma_facts' relation lane and in limma_search's matches — has provenance_kind ∈ {declared, extracted} and a confidence, on the row and on each of its sources. A row pulled through limma_connect with a relation_mapping — or, from a document-store export, a document_mapping — is declared with confidence 1.0 and a structured receipt per slot. extracted is the kind a reader's rows carry, and two things write them: a document ingested as being about one entity, into a workspace whose pack declares relations, is read for those relations by a model (coverage is declared only for what was actually read, so a document nothing could read leaves its subject not_recorded, never none); and a criterion stores each profile's judgment as a row. Both carry a span receipt that limma_region re-reads. limma_search's provenance block counts {declared, extracted} over the rows it delivered.

Confidence, today

Numeric and prose facts are delivered without a confidence number; relation rows carry the one above. What exists for an answer is on limma_ask: its warrant's verification_level is the trust contract (certified_* / corroborated / sourced / llm_reasoned / refused), and warrant.calibrated_error_rate, when present, gives the sourced tier's measured wrong-rate with its interval and the dataset it was measured on. See the verb reference.