oh myllm

A local model can work for hours.
If the harness is built for it.

View on GitHub Read the thesis

THE THESIS

The model is not going to get smarter. The harness will.

A single road through the system.

MODEL PATH

Local, LAN, or cloud.
One interface.

llama-server runs as a managed child process and speaks OpenAI-compatible HTTP. Providers change. The runtime path does not.

EVENT STREAM

The UI renders.
The core decides.

Desktop, CLI, and eval consume the same ordered events. Business logic stays out of the interface, so every surface tells the same truth.

TOOL CONTRACT

Fewer tools.
Stronger calls.

JSON schemas compile to GBNF. Every output has a token budget. Malformed calls are repaired before the main model pays for another turn.

tool {
  grammar: constrained
  output:  budgeted
  repair:  deterministic
}

ORDERED MEMORY

SQLite remembers.
Sequence decides.

A session is an ordered event log. Sequence is authority. Time is display. Replay, recovery, and measurement begin from the same source of truth.

Long work needs more than a long prompt.

Reliability is a chain. Break one link and a two-hour run becomes an expensive story about almost finishing.

CONTEXT IS A BUDGET

Keep the prefix still.

System, core tools, project, then history. Anything that changes is appended. Stable prefixes preserve the KV cache and avoid a full prefill.

REPAIR BEFORE REPROMPT

Cheap fixes go first.

Deterministic repair, then a utility model, then the main model. A 30B turn is too expensive to spend on a comma the runtime can fix itself.

VERIFY THE CLAIM

Done is an exit code.

Tests, build commands, and task-specific checks decide completion. The model can propose success. The verifier gets the final word.

Every token must earn its place.

Files, tool output, history, and project state compete for a finite window. oh my llm accounts for the cost, pins what cannot move, and compacts what can.

system → core tools → project → history

MEASURE, DO NOT ASSUME

The harness is an ablation bench.

Every reliability mechanism can be turned off and measured. Trajectories are stored. False success is reported. Better is a result, not a mood.

Signal What it catches Why it matters

pass@1 First-run task completion Reliability without cherry-picking

false_success The agent says done while verification fails The most dangerous agent failure

trajectory Prompts, tool calls, repairs, and outcomes Evidence for every architectural claim

REPORTS / Read the measurements

OPEN SOURCE / MIT / BUILT IN PUBLIC

Ride farther.

Build the harness that lets local models finish the work.

github.com/fthsrbst/oh-my-llm