Version, Evidence, and Claim Ledgers

Conceptual map

1. V-01.01Source hierarchy
2. V-01.02Version records
3. V-01.03Claim units
4. V-01.04Assumption tables
5. V-01.05Residual conclusions

1. Freeze the object before evaluating it

An audit conclusion applies to a particular manuscript, code tree, data snapshot, and execution environment. A title or filename does not identify that object.

Let an audit object be \(O=(M,C,D,E)\): manuscript bytes, code bytes, data or data manifest, and environment specification. A version record contains immutable digests, acquisition dates, repository revision, dirty-tree status, and the command that constructs released artifacts. The record also distinguishes source inputs from generated outputs.

For example, the SHA-256 digest of the byte string “claim-v1” followed by a newline is 6887fd7e30f48ee45eea7690830d4e041ffa919af8d28af1062a96d8d5e1b763. One edited byte produces a different identifier. The digest gives an integrity check; it supplies no assessment of the sentence encoded by those bytes.

2. Evidence is a typed dependency graph

Represent the release as a directed graph whose nodes include claims, assumptions, lemmas, tables, scripts, source data, generated data, and artifacts. An edge must name its relation: “uses,” “proves,” “generates,” “tests,” or “contradicts.” Reachability from a claim to a source node records provenance, while proof checking evaluates whether the intervening edges are valid.

3. Residual conclusions preserve uncertainty

Useful statuses include verified under listed assumptions, numerically reproduced, contradicted by a witness, unsupported by available evidence, blocked by unavailable data, and outside scope. “Pass” should identify the gate passed. A computation can reproduce a printed table while the estimand remains unidentified; a proof can be valid while its premise is empirically unsupported.

A ledger update propagates through dependents. If an assumption, dataset, or function changes, every reachable claim becomes review-due until its supporting path is rechecked. This invalidation rule prevents an unchanged PDF table from retaining a stale verified label after its generating source changed.

4. Mutable and missing inputs bound the conclusion

A mutable URL needs an acquisition timestamp and local digest. A Git revision identifies committed content but excludes uncommitted modifications unless the dirty state and patch are captured. Large data governed by access restrictions may use a manifest with provider version, query, schema, row count, and permitted checksum. A digest collision is theoretically possible; the integrity claim is conditional on the selected hash function and threat model. Empty artifacts, missing submodules, line-ending conversion, generated timestamps, and nondeterministic archive order can also change bytes without changing intended semantics.

5. Implementation, exercises, and sources

Create the ledger before substantive review. Hash every available input, record unavailable inputs explicitly, and extract the released claims into units small enough to receive separate conclusions. Build dependency edges from build rules and manual proof reading. On revision, compare bytes and semantics, then invalidate the transitive dependents of changed nodes.

Exercises

1. Recompute the digest in Section 1 and show that removing the newline changes it.
2. Draw an evidence graph for one theorem and one generated table.
3. Give a release change that preserves every source digest but changes the execution result.

1. Roger D. Peng (2011), “Reproducible Research in Computational Science,” Science 334, 1226–1227.Reproducibility and computational evidence.
2. Geir Kjetil Sandve, Anton Nekrutenko, James Taylor, and Eivind Hovig (2013), “Ten Simple Rules for Reproducible Computational Research,” PLoS Computational Biology 9, e1003285.Versioned inputs and recorded workflows.
3. Philip B. Stark (2018), “Before Reproducibility Must Come Preproducibility,” Nature 557, 613.Specified evidence and analysis before rerunning.