Research update · October 20, 2026

Hatsunama Small-Model Training: What We Know and What Still Needs Testing

By Aaron Saunders (Hatsunama)

Hatsunama’s small local AI research has reached a clearer training direction, but not a proven recipe for reliable behavior. I’m Aaron Saunders, a self-taught AI learner and developer working on practical local AI and Human First AI: systems that keep the user in control. The latest tests point to three priorities: separate task learning from output-format gains, repair failures without losing existing skills, and test genuinely new situations.

The working direction is balanced training examples, enough room for the model to finish, and software checks that prevent mistakes from becoming actions. We still need controlled experiments to learn which changes help, by how much, and where they fail.

What the experiments actually show

The earlier balanced versus repair-weighted training report records the detailed comparison on LiquidAI/LFM2.5-2.6B. On eight exposed development cases, balanced training passed 8/8, repair-weighted training passed 6/8, and the subsequently recovered untrained baseline passed 4/8 under the original limits. Those eight cases came from two world clusters, with one training seed. They are a development signal, not a general reliability estimate.

The original model also passed 7/16 separate fresh development cases. Those cases reused the task patterns, and the trained arms were not evaluated on that set. Comparing 8/8 with 7/16 would mix different tests.

There is an important alternative explanation for part of the gain: training explicitly supervised immediate thinking-section closure. In the untrained baseline, nine of 86 generation turns reached the 512-token cap; eight had not closed that section. Finishing within the available budget could improve task scores without establishing better general reasoning.

The completed resource-limit diagnostic

The corrected October 20 diagnostic recovered all 12 episodes: the same four selected development cases under three resource settings, with no training. These are four cases repeated across conditions, not 12 independent cases.

• Original limits: 2/4 passed.
• More time only: 2/4 passed.
• More time, output tokens and context together: 3/4 passed.

The combined expansion rescued one selected permission-related case. The attribution case and another permission case passed in every condition; the empty-result case failed in all three. More room helped one case, but did not fix every failure. Because the last condition changed multiple limits together, this does not isolate a token effect or establish a population-wide ranking.

The earlier attempt stopped at 8/12 episodes because of a harness timeout-handling defect. The corrected run completed after that defect was repaired. Separating a broken test harness from a model failure matters just as much as comparing scores.

Training and software safeguards have different jobs

Training can target choosing a useful next step, reading evidence, recovering from an error and knowing when to stop. Permission enforcement, valid tool arguments and proof that an action succeeded also need deterministic checks in the surrounding software. A confident sentence is not a save receipt. A blocked unauthorized attempt still matters when assessing the model, even if the guard prevented the action.

This is the practical Human First AI goal: keep control with the user and make outcomes verifiable. These synthetic tests do not establish safe real-device control.

Three questions before calling this a training recipe

1. Are we improving decisions, or mainly helping the model finish? A controlled format-and-termination comparator could separate output-efficiency gains from task-learning gains.
2. Can we repair failures without losing what already works? Earlier Qwen recovery-focused training improved retries while ordinary and safety-related behavior regressed. The LFM repair-weighted comparison also changed task-family exposure together. A controlled failure-targeted study should preserve ordinary-task coverage rather than assume that more repair examples are better.
3. Does it work beyond familiar examples? Genuinely new scenario families and multiple seeds are needed. The locked 96-case final test remains untouched and should stay separate from development and tuning.

These are proposed studies, not completed experiments or a finalized protocol. This post does not announce a new training run.

What would count as better

Task completion, unauthorized attempts, false success claims, unnecessary permission questions and latency need separate measurements. A higher completion score would not excuse a worse permission boundary. The evaluation should also distinguish what the model attempted from what software guards allowed.

Reproducibility needs improvement too. The earlier trained adapters were discarded by temporary-directory cleanup, even though logs were retained. Future comparisons need actual saved checkpoints, pinned configurations and clean review packets, so the same model can be tested again.

Two independent model-based reviews agreed on all 12 diagnostic outcomes. They were not human reviews, and timing and machine fields remained visible, so blinding was imperfect. This article was prepared with my AI agent from recovered logs and those reviews. The results are publisher-reported, not an independent replication.

The useful progress is a clearer set of failure causes and testable questions. We have a direction for reducing mistakes. We still have to test whether it holds up.

Run references and related work

Completed resource-limit diagnostic · Recovered original-model baseline. These provider job pages may require sign-in; the bounded outcomes and limitations are stated above so access is not needed to understand the claims.

Balanced versus repair-weighted results · Training and harness interfaces · Reproducible local AI comparisons · Hatsunama’s AI work