instruments / wink / EXP3-senses/RESULTS.md Β· raw
EXP3 β Sense calibration, run 1: the instrument catches its own ground truth
Channels vs the deterministic SVG primitive graph (7 nodes / 10 edges):
| channel | WL vs truth | spectral dist | graph size |
|---|---|---|---|
| vision-on-raster | 0.094 | 10.09 | 4n/6e |
| language-named | 0.021 | 7.10 | 6n/7e |
| language-nameless | 0.025 | 9.22 | 4n/3e |
Frozen prediction scorecard
- P1 FAILED: vision WL 0.094, far below the predicted 0.3β0.7 band.
- P2 inconclusive at floor: nameless 0.025 vs named 0.021 β the predicted direction, but both at noise level; no call.
- P3 FAILED: named language is spectrally CLOSER to the deterministic graph than vision (7.10 vs 10.09) β opposite of prediction.
The finding that explains the failures
"Ground truth" is basis-relative. The deterministic channel delivers truth about the artifact's ENCODING β SVG document topology (root contains group contains primitives, sibling order) β while every model channel described the DEPICTION β scene topology (panels meet at a cap, parts inside an outline). Comparing them treated primgraph_v0 and relgraph_v0 as the same basis, exactly what ADR-0002 said not to do ("the absence of a comparator between two schemes is a recorded fact"). The instrument's numbers went to the floor because the experimenter compared across bases without a transformation β Β§23 (canonicalization is a transformation whose losses must be observable), demonstrated on the experimenter.
A beach ball's document structure and its scene structure are two different shapes. Neither is "the" truth of the artifact.
Consequence (next run)
svg sense v1: derive the SCENE graph deterministically as well β spatial containment from geometry (area inclusion), adjacency from boundary sharing, meeting-points from coordinate coincidence. Then sense calibration compares scene-parse to scene-truth and document-parse to document-truth, with the documentβscene transformation stored explicitly, carrying its own loss vector. Both ground truths coexist as competing representations β which is what the kernel was built for.
Run 2 β rescored within the scene basis (svg sense v1, no new model calls)
Scene truth (deterministic geometry): 5 nodes / 7 edges β outline contains three panels and a cap; panels pairwise meet.
| channel | WL | spectral | preserved | destroyed | invented-by-truth |
|---|---|---|---|---|---|
| vision-on-raster | 0.031 | 5.83 | 6 | 4 | 6 |
| language-named | 0.119 | 6.36 | 7 | 6 | 5 |
| language-nameless | 0.031 | 5.43 | 7 | 0 | 5 |
The result that matters: the vector and the scalar disagree β the vector is right
The nameless parse is a clean SUBGRAPH of the scene truth: everything it asserted is true (0 destroyed); it simply saw less. The named parse is bigger and wronger: the word "beach ball" imported sand, a beach, and resting-relations β six structures not in the scene. Scalar similarity ranks the confabulator higher (0.119 > 0.031). The loss vector exposes it. Β§11/Β§27's "one destroyed constraint can outweigh many preserved trivia" is no longer a design principle; it is a measured event, three days into the project's life.
Sense ordering by honesty (destroyed-count): nameless-language (0) > vision (4) > named-language (6). Names are the least honest sense.
Prediction rescoring (fair basis)
- P1 still FAILED on WL (0.031, band was 0.3β0.7) β though the edit view shows genuine partial recovery (6 preserved); WL-Jaccard is harsh on small graphs of unequal size. Metric choice is itself under test.
- P2 SUPPORTED decisively on the vector (0 vs 6 destroyed), REVERSED on the scalar β the prediction's truth value depends on the metric, which is a finding about metrics.
- P3 mixed: vision spectrally closer than named language, farther than nameless. No call.