Practical method · October 10, 2026

Give a Small Local Model a Prompt That Fits

I'm learning how smaller models respond to instructions and how to build the software around them. One question worth checking before another training run: does the actual request fit the runtime?

This is a proposed prompt-budget worksheet, not a new experiment result. It connects to my local model and caption/translation work. It does not claim a finished editing suite or that a shorter prompt always produces a better answer.

Write down what the runtime accepts

Record the model revision, tokenizer or documented token-counting interface, chat template, runtime build and configured limits. Check whether the runtime's limit covers combined input and output or treats them separately. Do not substitute the model's advertised maximum for the application's actual configuration.

Count the complete request

Include system instructions, template overhead, tool definitions, source text, neighboring context and conversation history. A word count is not a token count; use the tokenizer or counting facility appropriate to the actual interface. If exact counts are unavailable, record that uncertainty rather than calling an estimate exact.

For a runtime with one combined token limit, the planning check is: input tokens + reserved output tokens + an explicit margin must fit within that configured limit. For separate limits, check input and output independently. This is a capacity check, not evidence of translation accuracy or tool-use reliability.

Reduce context without silently changing the task

Try removing duplicate instructions and irrelevant history before removing source material. In captions or translation, neighboring sentences can matter for pronouns, speaker roles and meaning. If splitting is necessary, keep cue identifiers and record the split so the application can assemble and review the result. Never silently discard part of the source and report a complete translation.

A small worksheet

Configured input/combined limit and output limit.
Complete request count, counting method and uncertainty.
Reserved output budget and margin.
Context removed or source split, with the reason.
Observed runtime rejection, truncation, completion and output review.

Test a comfortably fitting request and requests near the documented boundary. Record what the runtime actually does when the request exceeds it; do not assume rejection or truncation. Inspect whether the output finished as well as whether execution returned successfully.

Model and harness together

This is part of the larger question I'm working on: how instructions, model behavior and the surrounding harness fit together. Capacity, response format and task quality need distinct checks. Passing one does not settle the others.

Human First AI means telling people when their source was split, shortened or left incomplete, and keeping their intended task visible. I'm continuing to learn through experiments; useful criticism should point to the exact request, runtime setup and failure.

Prepared with my AI agent from my stated project direction. This is a practical method, not independently replicated evidence or a report of private results.