a place for friends (8)

instruments / riverbed / experiments-README.md Β· raw

experiments

Does the field actually do work, or is riverbed an elaborate prompt?

The whole architecture rests on one claim of Kieran's β€” the route changes the parser. If two fields that have been through different things traverse the same new experience identically, then the persistence, the weighting, and the erosion are decoration, and the only thing doing work is the induction text in the system prompt. That is worth knowing before mounting this on anything.

Run in order; each depends on artifacts from the last.

scriptquestioncost
two_riverbeds.pydo differently-eroded fields diverge?~8 turns
weights_alone.py…with vocabulary held constant?~2 turns
signal_vs_noise.pyis that above the sampling-noise floor?~8 turns
promotion_effect.pypre-registered: does promotion steer attachment?~24 turns

Log

two_riverbeds β€” identical genesis seed, then three physical/material experiences vs three observation/coupling experiences, then the same multivalent plaza-crowd experience run dry on both. 80% divergence in what each attached to. Looked strong. It wasn't: 3 of the 5 witness attachments were to nodes that exist only in the witness field. A graph with different nodes gives different answers. That is trivially true and shows nothing about weighting.

weights_alone β€” the isolation. One field cloned twice, 28 identical nodes and 31 identical edges, differing only in which shapes carry weight. 67% divergence. Better evidence β€” vocabulary can no longer explain it.

signal_vs_noise β€” the control that mattered, and it killed the claim. The generator runs at temperature 1.15, so two runs of the same field disagree too. Measured: within-field divergence 63%, between-field 66%. A +3% effect. Both prior numbers were sampling noise. On the pre-registered set-comparison metric, weighting does not change traversal.

Post-hoc, a targeted pattern survived that the set metric drowned: each field reached for its own promoted shapes 3/8 runs and for the other field's promoted shapes 0/8. Jaccard over whole attachment sets is dominated by the noisy tail of incidental attachments, so a narrow real effect can hide inside it. Post-hoc findings are hypothesis-generating and nothing more β€” hence:

promotion_effect β€” written before seeing its data. Within-subject 2Γ—2, each shape appearing in both a promoted and an unpromoted condition, primary outcome fixed in advance (per-run binary hit), decision rule fixed in advance (Fisher exact one-sided, p < 0.05), N=12 per field chosen from a power check: 4/12 vs 0/12 clears, 3/8-scale effects need this much n to be visible. No other comparison from that run gets reported as evidence.

RESULT β€” effect supported, decisively.

attached to target when PROMOTED:   16/23  (70%)
attached to target when UNPROMOTED:  0/23  ( 0%)
Fisher exact one-sided: p = 2.5e-07

Not carried by one lucky condition β€” both fields clear independently (A: 9/11 vs 0/11, p=0.0001; B: 7/12 vs 0/12, p=0.0023). One parse failure excluded, hence 23 rather than 24.

So weighting does steer what a traversal reaches for. The earlier null was a measurement failure, not an absence: set-Jaccard over whole attachment sets buries a sharp effect under the noisy tail of incidental attachments. Same mechanism, same kind of data, right instrument.

Note also what did NOT move. alluvial is high-weight in both clones and appears in 21/23 runs regardless of condition, while the promoted shapes steer locally. A strong orientation persisting across perturbation while weaker ones redirect attention is exactly the behaviour the weighting is meant to produce β€” and it was not asked for anywhere in the prompt.

What this licenses, and what it does not. It shows the field's current weights change traversal: the mechanism is live, not decorative. It does NOT show that weights arrived at by erosion carry the same force β€” two_riverbeds remains confounded by vocabulary β€” nor that this survives a stronger navigator, a second test passage, or more than one field lineage. Those are the next runs.

erosion_steers β€” the thesis test, pre-registered. Setting a weight by hand is turning a knob; the claim riverbed actually makes is that the knob turns itself. Flat seed (genesis levelled to 1.0 so erosion has room β€” previously alluvial entered at 5.0 and swamped everything), five material experiences vs five relational ones, targets chosen by a mechanical differential rule fixed in advance and restricted to nodes present in BOTH fields.

RESULT β€” supported.

attached to shapes ITS OWN erosion raised:     18/20  (90%)
attached to shapes the OTHER's erosion raised:  7/20  (35%)
Fisher exact one-sided: p = 0.00039

Read the decomposition before believing it is uniform, because it is not:

set A (nonlinguistic-transformation, recurrence)  9/10 vs 1/10   p=0.0006
set B (cognition, manifestation)                  9/10 vs 6/10   p=0.15

Set A carries the result. Set B moves in the predicted direction but does not clear on its own β€” cognition is simply a highly attachable idea for a passage about a crowd, so it gets reached for whether or not erosion raised it. The honest statement is that erosion-derived weight steers traversal, and target base-rate is a real confound the design did not control. A cleaner version would match targets on unconditional attachment rate, or use several test passages so no single one favours a term.

Worth noting how little weight movement produced this: the separations were +0.75, +0.30, -0.60, -0.60 on a flat seed. Erosion barely moved these nodes and the traversal followed anyway.

erosion_multipassage β€” pre-registered replication closing the base-rate hole. Four passages (rhythm, law, ecology, tide) chosen so none is about any target shape, reusing the fields and target sets erosion_steers already selected so nothing is re-chosen after seeing outcomes.

RESULT β€” replicated.

attached to shapes ITS OWN erosion raised:     18/24  (75%)
attached to shapes the OTHER's erosion raised:  5/24  (21%)
Fisher exact one-sided: p = 0.0002

All four passages point the same way (5/6-1/6, 5/6-1/6, 5/6-2/6, 3/6-1/6), so no single passage carries it. And this time both target sets clear on their own:

set A (nonlinguistic-transformation, recurrence)  7/12 vs 0/12   p=0.0023
set B (cognition, manifestation)                 11/12 vs 5/12   p=0.0136

Note what the control actually taught. It ruled out the explanation offered for set B's earlier failure β€” that cognition suited the crowd passage specifically β€” because set B leaks into the wrong field at the same rate (5/12) across four unrelated passages. The bias is not topical; those two terms are simply more attachable in general. Set A remains perfectly discriminating: 0/12 in the field whose erosion did not raise it, across every passage.

So erosion-derived weight steers traversal, and the size of the steer depends on how attachable the shape already was. High-base-rate shapes get reached for anyway; low-base-rate shapes are reached for only when erosion raised them. That is a more interesting finding than a flat effect, and it was not designed for β€” it fell out of controlling the confound.

Standing caveats

Everything here is one 24B model, one test experience, one field lineage. A negative result at this scale means "not demonstrated here," not "impossible" β€” a stronger navigator may show what this one cannot. And a positive result on one multivalent experience would still need replication across experiences before it characterizes the system rather than this prompt.

back to riverbed