Lux: [settling in] Last episode we opened the appendices — four myths busted. Today we go deeper. One specific drawer: the reproducibility layer.
Hex: Why does a theory paper need a reproducibility layer?
Lux: Because every number in the paper comes from code. And the code comes with a darkroom. Same negatives, same chemicals — same prints. Every time.
Hex: [leaning forward] Walk me through it.
Lux: Gladly. This is how the emergence calculus framework in the Six Birds project keeps itself honest.
🎵 *[Theme — clean pulse]*
Lux: Step one. The smoke test. One Python script — run it from the repository root — and it regenerates eight artifact directories from scratch.
Hex: Eight directories. What's in them?
Lux: [counting] DPI exhibit — confirms total variation contracts under coarse-graining. Clock budget exhibit — records the maintenance cost versus drift tradeoff. Enablement birth exhibit — shows the closure defect triggering a richer description. Constraints and cones exhibit. No global time exhibit. The no-signalling toy. And then the robustness sweep — results and summary across seeds zero through nine.
Hex: And each directory produces what, exactly?
Lux: A metadata file. JSON format. Every numerical result the paper cites from that exhibit, stored in one place. If the DPI exhibit says total variation drops by a certain amount — that number lives in the metadata file, and the paper reads it from there.
Hex: What about the robustness sweep? That's two files, not one.
Lux: Right. A results CSV — row per seed, column per metric — and a summary JSON that aggregates across all ten seeds. No-signalling distance in Z: zero across every seed. No-signalling distance in X: machine epsilon — two-point-two times ten-to-the-minus-sixteen — across every seed. If any metric drifts beyond that, the sweep catches it.
Hex: [tapping the desk] So the paper isn't quoting numbers typed by hand.
Lux: Never. Which brings us to step two — the paper tables script. A second command reads those metadata files and generates LaTeX tables. The tables are written to a docs directory, and the manuscript includes them automatically. Change the code, re-run the two scripts, and every table in the paper updates to match the new results.
Hex: Like a contact sheet in a darkroom. You develop the negatives, and the contact sheet indexes what you've got.
Lux: Exactly. And the artifact manifest — the table that lists which artifacts exist — is itself auto-generated at build time. A bill of materials for the paper's numerical claims.
Hex: [slowly] So if an artifact is missing, the manifest shows it.
Lux: Immediately visible. No hiding.
🎵 *[Transition — steady beat]*
Hex: What stops someone from changing the code and generating garbage that still passes?
Lux: [sitting forward] Contract checks. Null gates. Every regeneration runs a suite of sanity tests alongside the artifacts.
Hex: Give me examples.
Lux: Four. First — set sigma to zero in the large-eddy simulation. The commutator should be exactly zero. If it's not, something broke. Second — set the averaging parameter to zero. The mismatch should vanish. Third — set lambda to one in the dephasing model. Idempotence should be exact. Fourth — run the quantum DPI check. Zero violations beyond tolerance.
Hex: These sound like unit tests.
Lux: Closer to regression certificates. They don't generate new evidence — they certify that the implemented audits haven't drifted. If you refactor the clock budget code and the null gate still passes, you haven't broken the audit. If the null gate fails, you've introduced a bug.
Hex: And how many artifacts does the physics suite produce?
Lux: Over twenty. Manifest, smoke core data, quantum check summaries, DPI histograms, closure idempotence plots, route mismatch tables, failure mode breakdowns. Each one traces back to the script that generated it — the generation map spells out every link.
Hex: [nodding] The darkroom chemicals are still fresh.
Lux: Or they're not, and you know immediately.
🎵 *[Transition — crisp snap]*
Lux: [spreading hands] Step three. Run bundles.
Hex: What's a run bundle?
Lux: Every experiment produces a self-describing package. Inside: the config file that defines the experiment, a metrics JSON with every quantitative result, a provenance JSON recording the environment, and at least one plot or table.
Hex: And the config files?
Lux: Six canonical configurations for the geometry pipeline. Grid plane — a twenty-five-by-twenty-five random walk. Sphere point cloud — five hundred points with nearest-neighbor graphs. Sierpinski gasket — level five, lazy random walk. Anisotropic grid with a directional gate. Holonomy demo. And a Pythagoras random-walk grid.
Hex: [carefully] And every run bundle is timestamped?
Lux: Timestamped with the git SHA. You know exactly which code version produced which result. Each bundle is a labeled film canister — date, exposure, lens, all written on the label.
Hex: So you can always go back to the exact state of the code that generated a specific result.
Lux: Always. And the configs themselves are deterministic. Explicit random seeds. Sorted iteration order to avoid platform-dependent nondeterminism. The only expected variation across machines is floating-point rounding from different linear algebra libraries — and that stays at machine epsilon.
Hex: So reproducibility has a precision floor.
Lux: [nodding] Floating-point hardware sets it. Not the framework.
Hex: And for the paper itself?
Lux: The project commits reference run packs — frozen snapshots of the key results. The paper cites those frozen packs, not the latest regeneration. That way the manuscript stays stable even while the code evolves.
Hex: [leaning back] The negatives in the archive. Even if you rebuild the darkroom, the archive prints stay consistent.
🎵 *[Transition — warm pad]*
Lux: Step four. Vendoring.
Hex: Vendoring — that's a software term, right? Copying dependencies into your own repository?
Lux: [carefully] Same idea, different material. The paper never includes raw experimental output directly. Instead, a vendoring step copies figures and tables from the run bundles into tracked paper directories. And for every vendored artifact — every figure, every table — the system writes a provenance file.
Hex: What's in the provenance file?
Lux: The originating run folder. The source file within that folder. The output path in the paper directory. A hash of the artifact. And a timestamp. Five pieces of traceability for every single figure.
Hex: So you can trace any figure in the paper back to the exact experiment that produced it.
Lux: The exact experiment, the exact code version, the exact data. That's the stamp on the back of every gallery print. You don't just see the photograph — you know which negative it came from, which chemical bath, which session.
Hex: [half-smiling] And then there's an evidence map?
Lux: One more layer. A table that maps every plot and every quantitative table in the manuscript to its generating script, its dataset keys, and the run bundle used.
Hex: Like a library card catalogue for figures.
Lux: [pointing] Exactly. Want to know where Figure Three came from? The evidence map tells you the script, the data, the run bundle, and the vendored path. Plus the project provides umbrella commands — type one line, regenerate the entire evidence suite for background cosmology or large-scale structure. Type another, aggregate every metric across all experiments into a single flattened table.
Hex: So the whole chain is — data, experiment, run bundle, vendored artifact, paper figure, evidence map.
Lux: [nodding] Five links. All traceable. All reproducible.
Hex: That is the most infrastructure I've ever heard of for a set of theory papers.
Lux: And it's not just one paper. The laboratory paper has its smoke test and eight artifact directories. The physics calculus paper has its own check script — Lean proofs plus over twenty artifacts. The geometry pipeline paper has config-driven runs with committed reference packs. The cosmology paper has vendoring with provenance YAML. Each paper has its own reproducibility layer, and they all follow the same pattern.
Hex: [raising eyebrows] Same pattern across different papers?
Lux: Same philosophy. Code produces artifacts. Artifacts produce tables and figures. Tables and figures enter the paper through a tracked, auditable pipeline. Every link documented.
Hex: The darkroom has multiple rooms.
Lux: But one set of principles. And that's the point. The darkroom isn't decoration. It's the reason you can trust the prints. Not because the author says the prints are accurate — because you can walk into the darkroom, load the same negatives, mix the same chemicals, and develop the same photographs yourself.
Hex: Verification, not reputation. Again.
Lux: Again. Same principle from the appendices episode, now with a full mechanical pipeline behind it.
Hex: [folding arms] What's next?
Lux: Next episode — holonomy obstruction. The mathematical reason there's no global time in these systems. From infrastructure back to structure.
Hex: From the darkroom to the geometry.
Lux: From the darkroom to the geometry.
🎵 *[Outro theme]*