lmstudio-ollama-mcp

Claude Code for local models. A single binary that bridges LM Studio · Ollama · llama.cpp as coding agents — with hardware-aware parallel sub-agents, intelligent routing, and zero data leaving your machine.

Introduction

lmstudio-ollama-mcp brings the agentic loop from Claude Code / Codex to on-device models. Instead of paying per token and sending code to the cloud, you run a local GGUF (Gemma, Qwen, etc.) via LM Studio, Ollama, or llama.cpp. A frontier model is optional: when configured, it only plans while local models execute.

Installation

Prerequisites

• Node.js >=18
• One local runtime: LM Studio (port 1234) · Ollama (11434) · llama.cpp (8080)
• ~6–10GB free for a 7–12B Q4 model

Install

npm install -g lmstudio-ollama-mcp
# aliases keep muscle memory:
forge --help
forgecode --help
# one-off without install:
npx lmstudio-ollama-mcp doctor

Quickstart

git clone https://github.com/your-org/your-project && cd your-project
lmstudio-ollama-mcp init        # creates lmstudio-ollama-mcp.json
lmstudio-ollama-mcp doctor      # verifies providers + hardware
lmstudio-ollama-mcp models      # lists local GGUFs
lmstudio-ollama-mcp "list the codebase structure and suggest 3 small improvements"

Hybrid mode (optional frontier):

export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
lmstudio-ollama-mcp "design and implement a rate-limiter middleware"

First run checklist

1. Start a runtime — LM Studio: Developer → Local Server → Start. Ollama: ollama serve. llama.cpp: ./llama-server -m model.gguf --port 8080
2. Load a model — LM Studio: pick Gemma 12B QAT or Qwen3 27B. Ollama: ollama pull qwen2.5-coder:7b
3. Verify — lmstudio-ollama-mcp doctor should show ● lmstudio available + models.
4. Try a small task — lmstudio-ollama-mcp "add JSDoc to src/utils/logger.ts" (routed locally).
5. Try parallel — lmstudio-ollama-mcp --parallel 4 "implement auth + tests + docs"

Providers

All providers speak OpenAI-compatible Chat Completions with tools. The bridge normalizes reasoning_content (Gemma/Qwen) and handles tool calling consistently.

LM Studio

Developer → Local Server → Start. Forge detects via GET /v1/models; if the server is down but models exist on disk (~/.lmstudio/models), doctor still lists them but chat will fail until the server is started. Load a model with good tool-calling: Gemma 12B QAT (recommended on 16GB) or Qwen3 27B IQ2_S.

~/.lmstudio/bin/lms ls
~/.lmstudio/bin/lms load google/gemma-4-12b-qat --yes
~/.lmstudio/bin/lms server start
curl -s http://localhost:1234/v1/models | jq .data[].id

Ollama

Uses native /api/tags for rich metadata (size, quantization). Run ollama serve then pull a coder model.

ollama serve &
ollama pull qwen2.5-coder:7b
ollama pull gemma3:12b
curl -s http://localhost:11434/api/tags | jq .models[].name

llama.cpp

Any GGUF via llama-server. The bridge checks /health and /v1/models.

./llama-server -m models/gemma-12b-q4.gguf --port 8080 --ctx-size 8192
curl -s http://localhost:8080/health
lmstudio-ollama-mcp --provider llamacpp --model hf.co/model "explain this repo"

Frontier (hybrid)

When OPENAI_API_KEY or ANTHROPIC_API_KEY is set, a frontier provider is auto-registered. In auto strategy, frontier plans (decomposition, hard reasoning) while local models execute small tasks. Cost drops ~90% vs. cloud-only.

Custom endpoints

Any OpenAI-compatible endpoint works (e.g., vLLM, local Mistral server, remote LM Studio):

{
  "providers": {
    "my-remote": { "type": "openai", "baseUrl": "http://192.168.1.10:1234/v1", "apiKey": "not-needed", "enabled": true }
  }
}

Routing & model selection

ModelRouter (src/providers/router.ts:14) classifies every prompt:

preferLocalFor defaults to ["lint","format","test","search","summarize","explain"] — always local even in frontier-first.

{
  "router": {
    "strategy": "auto",
    "frontierProvider": "openai",
    "frontierModel": "gpt-4o-mini",
    "thresholds": {
      "smallTaskMaxTokens": 2000,
      "preferLocalFor": ["lint","format","test","search","summarize","explain"]
    }
  }
}

Override per run: --model lmstudio:gemma-3-12b or --provider ollama --model qwen2.5-coder:14b.

Hardware-aware scheduling

detectHardware() (src/hardware/detector.ts:12) reads os.cpus(), os.totalmem(), Apple Silicon flag. recommendParallelism():

cpuLimit = floor(cores * overcommit) - 1
memLimit = floor((totalGb*1024 - 2048) / perAgentMb)  // default perAgentMb=1200
unifiedBonus = isAppleSilicon ? 1 : 0
suggested = min(cpuLimit, memLimit) + unifiedBonus
// clamp: 1..8 default, up to 12 on 32GB/12c, up to 16 on 64GB

Scheduler (src/hardware/scheduler.ts:17) is a bounded p-limit queue. Sub-agents run via scheduler.runAll(tasks) preserving order, respecting maxParallel. CLI override: --parallel 4 or --no-parallel.

Orchestrator & sub-agents

Orchestrator (src/core/orchestrator.ts:28) decomposes a goal into 2–6 SubTasks via a planner LLM (frontier if available, else local). Prompt asks for [{"id":"t1","title":"...","prompt":"...","kind":"code|test|search"}] JSON. Tasks with dependsOn are batched topologically.

lmstudio-ollama-mcp --parallel 4 "implement auth module, write tests, update docs"
# planner → 3 sub-tasks:
# t1 Explore & plan [search → local]
# t2 Implement      [code   → local or frontier]
# t3 Verify         [test   → local]
# each sub-agent: Agent(provider, model, ToolExecutor) → 18 tool turns max → synthesis

Agent loop & tools

Agent (src/core/agent.ts:22) runs a 25-turn OpenAI tool loop: provider.chat(messages, tools) → execute tool calls → append tool messages → repeat until no tool_calls.

System prompt enforces: read before edit, small diffs, verify with bash, concise summary.

Configuration

Resolution: DEFAULT < ~/.lmstudio-ollama-mcp/config.json < ./lmstudio-ollama-mcp.json < env vars. Legacy ~/.forgecode/config.json / forgecode.json still read for compat (new path wins).

lmstudio-ollama-mcp config --show   # resolved JSON
lmstudio-ollama-mcp config --path   # file locations
lmstudio-ollama-mcp init            # scaffold lmstudio-ollama-mcp.json

Reference

{
  "version": 1,
  "providers": {
    "lmstudio": { "type": "lmstudio", "baseUrl": "http://localhost:1234/v1", "enabled": true },
    "ollama":   { "type": "ollama",   "baseUrl": "http://localhost:11434",      "enabled": true },
    "llamacpp": { "type": "llamacpp", "baseUrl": "http://localhost:8080",       "enabled": true },
    "openai":   { "type": "openai",   "baseUrl": "https://api.openai.com/v1",   "apiKey": "sk-..." }
  },
  "router": {
    "strategy": "auto",                       // auto | local-first | frontier-first | local-only
    "frontierProvider": "openai",
    "frontierModel": "gpt-4o-mini",
    "thresholds": {
      "smallTaskMaxTokens": 2000,
      "preferLocalFor": ["lint","format","test","search","summarize","explain"]
    }
  },
  "hardware": {
    "maxParallelAgents": 4,                   // auto if omitted
    "maxMemoryPerAgentMb": 1200,
    "cpuOvercommit": 1                        // 0.5..2
  },
  "permissions": {
    "allowBash": true,
    "allowWriteOutsideWorkspace": false,
    "allowNetwork": true
  }
}

Strategies

• auto (recommended) — trivial/small → local, medium/large → frontier if available else local
• local-first — only medium/large go to frontier
• local-only — never call frontier (air-gapped)
• frontier-first — always prefer frontier

CLI reference

Flags: --model <provider:model> (e.g. lmstudio:gemma-3-12b), --provider <id>, --parallel <n>, --no-parallel, --verbose, -v/--version

lmstudio-ollama-mcp --model ollama:qwen2.5-coder:14b "explain auth flow"
lmstudio-ollama-mcp --provider lmstudio --model gemma-3-12b "fix test in tests/tools.test.ts"
lmstudio-ollama-mcp --no-parallel "small typo fix"
lmstudio-ollama-mcp --parallel 8 "migrate Jest to Vitest"

MCP

Model Context Protocol support is on the roadmap. Current OpenAI-compatible providers already speak the same tool surface; a future mcp command will expose local models as MCP tools for other agents:

# planned
lmstudio-ollama-mcp mcp serve --port 3000
# exposes: lmstudio-ollama-mcp tools as MCP over stdio / SSE

Troubleshooting

No provider available

Run lmstudio-ollama-mcp doctor. Common fixes:

• LM Studio: Developer → Local Server → Start. Ensure a model is loaded (check lms ps).
• Ollama: ollama serve in another terminal, then ollama pull qwen2.5-coder:7b.
• llama.cpp: ./llama-server -m model.gguf --port 8080 --ctx-size 8192

Model lists but chat fails

LM Studio filesystem scan can find models even when the server is off. Start the server before chatting.

Slow inference

Use a smaller quantized model (Q4) on 8–16GB. Reduce --parallel or set hardware.maxParallelAgents: 2 to lower contention.

Reasoning models truncated

Gemma/Qwen emit reasoning_content. The bridge already merges it when content is empty, but request at least 1024 max_tokens (default is 2048).

Path escapes workspace

Set permissions.allowWriteOutsideWorkspace: true in config (not recommended).

Architecture

CLI (commander) → Router (classify) → Orchestrator (decompose → Scheduler) → Agent (tool loop) → Provider (OpenAI-compatible)
                ↓                    ↓                    ↓                     ↓
            Config (Zod)      Hardware (cores/RAM)   Planner LLM         ToolExecutor (fs/glob/grep/bash)

All code in src/ with Zod schemas, Vitest suites, and TypeScript strict mode.

MIT · github.com/fthsrbst/lmstudio-ollama-mcp · npm: lmstudio-ollama-mcp · alias forge