Codebee / Future research agenda
Questions worth
testing.
A research backlog built from experiments, failed hypotheses, and questions that remain open.
The next stage asks which observations survive changes in model, scale, task, and experimental design. Negative and inconclusive results are useful outcomes.
This is a proposed agenda. Transformer-scale studies, learned-adapter benefits, and a GPU research lab remain subject to experimental review and compute approval.
The hypothesis queue
Open a direction.
Follow the next test.
The mission names below identify research threads. Each separates its starting evidence from the experiment still needed.
01Mission 9.26Weight geometryProposed campaign
Does signed cancellation observed in small experiments transfer to real transformer weights?
Starting point
The proposed campaign is motivated by owner-reported small-scale cancellation observations. Reviewed neighboring studies establish finite synthetic interaction effects and important control corrections; transfer to learned transformer updates remains unverified.
Next test
Compare signed combinations, overshoot, averaging, and matched controls across architectures, parameter scales, adapter ranks, datasets, and tasks. Begin with cheap screening before longer jobs.
Measure: new-task gain, retained-task performance, effective update size, uncertainty, and compute used.
Related interaction evidence ↗02Mission 9.16-G / follow-upBehavioral geometryFollow-up proposed
Are acquired behaviors localized in low-rank weight updates, or distributed across the network?
Starting point
A Qwen checkpoint study recovered a deliberately constructed rank-one edit. Base-to-Instruct differences were broadly distributed. Controlled forward-pass probes included failed predictions; this does not establish how a learned skill is stored.
Next test
Study genuinely trained adapters on benign task behaviors. Compare layer, rank, and random-direction controls across model families, with an independent inference implementation and held-out evaluation.
Measure: behavior change, task selectivity, layer dependence, and capability retention.
Read the checkpoint study summary ↗03Mission 9.21Spectral GooseProposed study
Can spectral LoRA modifications reduce interference while preserving previously learned capabilities?
Starting point
Spectral structure and acquisition–retention trade-offs motivate the question. The reviewed record does not establish a spectral-LoRA advantage or general protection against forgetting.
Next test
Compare spectral interventions against ordinary LoRA and norm-, rank-, and parameter-budget-matched controls, starting from a shared checkpoint with separate training, selection, and test data.
Measure: interference, retained-task loss, new-task performance, and sensitivity to rank and scale.
Related adaptation design review ↗04Mission 9.10 / follow-upSkill GatewayOpen question
Why did gateway access fail to consistently improve tool reuse, and can training change that?
Starting point
Earlier gains were not consistently attributable to gateway use. Finding a tool, calling it correctly, and completing the task are distinct outcomes. Retrieval improvements alone do not resolve that chain.
Next test
Compare base and adapted models with gateway access held constant, plus matched wording and gateway-absent controls. Log actual exposure and execution before attributing an outcome to discovery.
Measure: correct tool selection, valid calls, end-to-end task correctness, failures, and total cost.
Read the tool-reuse research summary ↗05Mission 9.15-C / replicationArtifact commitmentReplication needed
Does the artifact-writing intervention replicate across models and unfamiliar tasks?
Starting point
An initial factorial showed a prompting signal with independent validity review pending. A separate 144-attempt replication remained inconclusive and included incorrect and invalid published artifacts.
Next test
Replicate across model families and unfamiliar task banks, holding prompts and permissions fixed. Grade actual persisted outputs independently, including wrong values, invalid schemas, missing files, and unintended writes.
Measure: strict correct completion and harm, with writing rate reported separately.
Read the initial and replication results ↗06Context & memoryContext dehydrationProposed study
Can self-authored compressed representations preserve useful reasoning and execution state?
Starting point
Compact state and persistent experience are research directions. A small representation or cheaper lookup is not, by itself, evidence that task-relevant meaning and execution state survived.
Next test
Compare full context, self-authored compression, a fixed summary baseline, and deliberately incomplete summaries. Test resumed tasks, stale evidence, source changes, and permission boundaries.
Measure: resumed-task correctness, state recovery, omitted facts, unsafe carryover, token use, and latency.
Related memory and reuse questions ↗07Learned tool-use behaviorLoRA habit acquisitionTraining study proposed
Can verified AgentKit behaviors become model habits that transfer beyond external instructions?
Starting point
Verified experience, sufficiency judgment, and correct artifact delivery are candidate learning targets. The reviewed records do not establish a transferable AgentKit habit learned in model weights.
Next test
First verify checkpoint identity, real optimizer steps, and save–reload parity. Then compare base and adapted models with identical tools and guidance, reserved task families, and regression checks.
Measure: held-out behavior, evidence sufficiency, transfer, base-task retention, and training cost.
Read the current training gates ↗First proposed major campaign
Weight geometry.
Then the boundary
of the effect.
Start by testing whether cancellation observations survive controlled changes in model architecture, parameter scale, LoRA rank, training data, and evaluation tasks.
A result that disappears under a matched control narrows the hypothesis. A result that replicates motivates the next experiment. Neither outcome requires a positive headline.
Geometry findings could inform where adapters are applied; adaptation failures could reveal which geometric hypotheses need revision. These connections remain questions to test.
Updated priorities / 30 October 2026
Carry the limits
into the next test.
These are proposed follow-ups, not completed runs or approved compute allocations.
- 01
Reconcile the Goldfish record
Resolve source and runtime identities, verify adapter update evidence, and evaluate a separately versioned, leak-free dataset before drawing unseen-grid conclusions.
- 02
Audit the retrieval study
Review scoring, bank independence, placebo controls and power; freeze the design before any separately authorized model run.
- 03
Probe geometry and preservation
Retain the failed shared-direction replication, validate against independent references, and test whether controlled damaging edits are detected.
- 04
Stress-test retained evidence
Test corrupted evidence, multi-step composition and recovery beyond the small reference families; consistent records do not establish identity by themselves.
- 05
Verify resources before scheduling
Check allocation, quotas, storage and access before planning GPU work. An application or planning document is not an award.
Priorities derive from the 29–30 October consolidation and later small controlled tests. Training and model runs require their own authorization and review.
Updated priorities / 31 October 2026
Make the next
comparison count.
These are proposed research directions. The presence of a notebook or a CPU smoke test does not establish that a larger run has been approved or completed.
- 01
Compare equal information
Test active-query selection against conventional baselines with identical source information, measured training and storage costs, and genuinely unseen targets.
- 02
Challenge recovery
Use independent trusted evidence, coherent corruption and checkpoints to test what can be recovered and when abstention is necessary.
- 03
Scale the lizard fairly
Compare the physical analogy with strong exact and trained baselines, include the two failed predictions, and separate analogy from novel capability.
- 04
Test real tool use
Move beyond synthetic registry labels to actual authorized tool executions and independently verified outcomes.
- 05
Freeze before confirmation
Record task splits, seeds, costs, code and stopping rules before confirmation; request separate authorization for any new training.
Explore the interactive repair illustration → · Read the 31 October findings →
Proposed scientific operating protocol
Resolve uncertainty.
Keep the evidence.
A research lab should inherit its previous findings, including failures and corrections.
- 01
Register the hypothesis
Connect existing missions, negative findings, competing explanations, and the smallest test that could reject the claim.
- 02
Screen cheaply
Use small experiments to eliminate weak directions before committing longer compute jobs.
- 03
Confirm independently
Freeze controls, multiple seeds, held-out evaluation, parameter budgets, and stopping rules before inspecting the final test.
- 04
Retain evidence and artifacts
Keep results, failures, code identities, checkpoints, confidence, and reproduction instructions together.
- 05
Propose the next informative test
A proposed local research swarm would cross-reference findings and suggest follow-up missions, with review before execution.
Designed for scheduled compute
Any future GPU campaign would need approved resources, checkpointing, and restartable jobs. The proposal is designed to work within finite allocations; this page does not announce an awarded allocation or an operating supercomputer laboratory.
Agenda supplied by the program owner; current evidence checked against selected local reports on 29 October 2026. The 9.26 and 9.21 names are roadmap labels rather than verified completed-study records in this review. Evidence sources and scope →
Discuss the research ↗