a place for friends (8)

instruments / wink / EXP3-senses/RESULTS.md Β· raw

EXP3 β€” Sense calibration, run 1: the instrument catches its own ground truth

Channels vs the deterministic SVG primitive graph (7 nodes / 10 edges):

channelWL vs truthspectral distgraph size
vision-on-raster0.09410.094n/6e
language-named0.0217.106n/7e
language-nameless0.0259.224n/3e

Frozen prediction scorecard

  • P1 FAILED: vision WL 0.094, far below the predicted 0.3–0.7 band.
  • P2 inconclusive at floor: nameless 0.025 vs named 0.021 β€” the predicted direction, but both at noise level; no call.
  • P3 FAILED: named language is spectrally CLOSER to the deterministic graph than vision (7.10 vs 10.09) β€” opposite of prediction.

The finding that explains the failures

"Ground truth" is basis-relative. The deterministic channel delivers truth about the artifact's ENCODING β€” SVG document topology (root contains group contains primitives, sibling order) β€” while every model channel described the DEPICTION β€” scene topology (panels meet at a cap, parts inside an outline). Comparing them treated primgraph_v0 and relgraph_v0 as the same basis, exactly what ADR-0002 said not to do ("the absence of a comparator between two schemes is a recorded fact"). The instrument's numbers went to the floor because the experimenter compared across bases without a transformation β€” Β§23 (canonicalization is a transformation whose losses must be observable), demonstrated on the experimenter.

A beach ball's document structure and its scene structure are two different shapes. Neither is "the" truth of the artifact.

Consequence (next run)

svg sense v1: derive the SCENE graph deterministically as well β€” spatial containment from geometry (area inclusion), adjacency from boundary sharing, meeting-points from coordinate coincidence. Then sense calibration compares scene-parse to scene-truth and document-parse to document-truth, with the document↔scene transformation stored explicitly, carrying its own loss vector. Both ground truths coexist as competing representations β€” which is what the kernel was built for.


Run 2 β€” rescored within the scene basis (svg sense v1, no new model calls)

Scene truth (deterministic geometry): 5 nodes / 7 edges β€” outline contains three panels and a cap; panels pairwise meet.

channelWLspectralpreserveddestroyedinvented-by-truth
vision-on-raster0.0315.83646
language-named0.1196.36765
language-nameless0.0315.43705

The result that matters: the vector and the scalar disagree β€” the vector is right

The nameless parse is a clean SUBGRAPH of the scene truth: everything it asserted is true (0 destroyed); it simply saw less. The named parse is bigger and wronger: the word "beach ball" imported sand, a beach, and resting-relations β€” six structures not in the scene. Scalar similarity ranks the confabulator higher (0.119 > 0.031). The loss vector exposes it. Β§11/Β§27's "one destroyed constraint can outweigh many preserved trivia" is no longer a design principle; it is a measured event, three days into the project's life.

Sense ordering by honesty (destroyed-count): nameless-language (0) > vision (4) > named-language (6). Names are the least honest sense.

Prediction rescoring (fair basis)

  • P1 still FAILED on WL (0.031, band was 0.3–0.7) β€” though the edit view shows genuine partial recovery (6 preserved); WL-Jaccard is harsh on small graphs of unequal size. Metric choice is itself under test.
  • P2 SUPPORTED decisively on the vector (0 vs 6 destroyed), REVERSED on the scalar β€” the prediction's truth value depends on the metric, which is a finding about metrics.
  • P3 mixed: vision spectrally closer than named language, farther than nameless. No call.

back to wink