instruments / wink / ADR/0004-first-instrument-inspection.md · raw
0004 — First instrument inspection (§116, after EXP0 + EXP1)
2026-08-21, end of the first experimental day. Answers to §116's questions, honest ones.
What our architecture forced us to throw away
- Run membership. Objects carry producers but not which run made them: EXP0's aborted runner left extra parses that mingle with the completed run's 16 (report counted 24). Append-only was right; the gap is that "run" is not a grouping derived objects can reference.
- The eventseq→relgraph conversion is invisible in the store. evaluate() converted in memory instead of storing the conversion as a transformation with its own loss vector. The single most lossy operation of EXP1 (see P2) is exactly the one the store cannot show. Fix before EXP3.
Where the LLM made interpretive decisions invisibly
Entity granularity is the dominant invisible decision. Whether "folder" or "reading" is an entity changes the graph's node count and therefore everything downstream — it is why feedback3 recovered 0/6 (parses grew extra nodes and became diamond-shaped) and why P1's family prediction failed. Nothing records this choice; the assumptions field exists and the prompts never asked for it. Candidate fix: parses declare their entity-selection policy as an assumption.
Abstractions that felt natural only because of our tooling
- "Recovery = WL top-1 against a ground-truth library" is a retrieval frame smuggled into evaluation. Useful, but it quietly re-centred proximity ranking — the exact gravity ADR-0000 built the kernel to resist. Keep it as one evaluator among several, never "the" score.
- GED with free node-substitution (discovered in a smoke test, not chosen): our edit scripts are purely structural; renames cost nothing. Correct for day-zero, but it should be a declared parameter, not an accident.
Kernel gaps found by using it
- A claim cannot be about an experiment (
_locateknows art/rep/tra/clm only). Found while trying to record the re-entrant-edge claim properly. representations_ofscans every file per call — fine at 150 objects; the promised SQLite index becomes worthwhile around EXP4-scale.
Can a failed mapping be represented as richly as a successful one?
Yes — unparseable parses persist with status, the wedged runner's transport failures persist, and P1's failed prediction is scored in RESULTS.md beside the supported ones. The §53 discipline held.