Yes, you can run a language model for your game's NPCs entirely on the player's machine, with no cloud calls and no per-conversation bill. In 2026 the small models are good enough and the tooling is mature enough that plenty of shipped and in-development games do exactly this. It is not free, though. The work moves from paying a cloud provider to building and maintaining the plumbing yourself.
This guide walks through the stack, the model choice, and the parts that are genuinely hard, then tells you honestly when rolling your own is the right call and when middleware saves you the trouble.
The stack, end to end
Five pieces sit between a player's question and an NPC's answer:
- A model. A small language model that fits the player's memory budget.
- A runtime. The code that runs the model locally, such as llama.cpp, Ollama, LM Studio, or an in-engine runtime like Unity Sentis.
- A bridge. How your engine talks to the model, usually a local HTTP endpoint or in-process bindings.
- Context and memory. What you feed the model each turn: character sheet, world state, and the relevant slice of history.
- A validation layer. The check that keeps output in character and inside your rules before it reaches the game.
Miss the last two and you get a tech demo. Ship the last two and you get a game.
Choosing a model
- Size, quality, and VRAM are a triangle. Smaller models run on more machines but reason less well. In 2026, models in the few-billion-parameter range run on mid-range GPUs and are dramatically better than the same size two years ago.
- Quantization makes it fit. Quantized formats such as GGUF shrink a model to consumer VRAM for a modest quality cost. This is how a capable model runs alongside your renderer.
- Pick a family, then test. General small models from the Llama, Mistral, Qwen, and Phi families are common starting points. Benchmarks do not predict how a model handles your specific characters, so test a few on your actual prompts.
- Budget for the player, not for you. Your minimum-spec machine sets the ceiling. A model that needs more VRAM than your renderer can spare is not an option, however good it is.
Running it
- llama.cpp is the low-level workhorse. It runs quantized models on CPU and GPU across platforms, and many higher-level tools wrap it.
- Ollama and LM Studio make local models easy to pull and serve behind a local endpoint, which is convenient during development.
- Unity Sentis runs neural networks inside the Unity runtime itself, keeping everything in-process and shippable without a separate server.
- The bridge is usually a local HTTP or socket call: your engine sends the current context, the model returns text, your game applies it. Keep the round-trip off the main thread so a slow generation never stalls a frame.
The hard parts, where projects stall
- Memory and state. The model has no idea what happened last session. You have to summarize and store player history, relationships, and world state, then feed back the relevant slice each turn. This is retrieval, and it is most of the work.
- Output validation. A raw model will happily contradict your lore, break character, or invent canon. You need a layer that checks every output against your rules and rejects or retries anything that violates them.
- Structure. Free text is hard for a game to act on. Constrain the model to structured output, with typed fields and enumerated actions, so your engine can apply results safely.
- Pacing and safety. An NPC that responds to anything invites players to try everything. Decide what the model is allowed to do, and gate the rest.
Build or buy?
Rolling your own gives you total control and no vendor lock-in. It also means building a retrieval system, a validation layer, a structured-output contract, and a safety model, then maintaining them as models change. That is a real engineering commitment, and it is the part teams underestimate.
Middleware exists to hand you those hard parts. LoreWeaver Director, for example, runs on-device and does the state-reading, structured decisions, and constraint validation for you, so your team writes the game rather than the plumbing. It is in beta today. If your requirements are unusual or you want to own every layer, build it. If you want the outcome without the scaffolding, that is what a runtime is for.
FAQ
Do I need a GPU to run a local LLM for NPCs? A GPU helps a lot. Small quantized models can run on CPU, slowly. For responsive dialogue in a shipping game, a mid-range GPU is the practical floor, and you budget its VRAM against your renderer.
Can local NPCs work fully offline? Yes, and that is the main appeal. Once the model ships with the game, there is no network dependency and no per-conversation cost.
Which engines can run a local LLM? Unreal, Unity, and Godot can all call a local model, either through an in-process runtime like Unity Sentis or a local endpoint served by llama.cpp or Ollama.
Is running a local LLM cheaper than a cloud NPC platform? After launch, usually yes, because there is no per-conversation fee. The cost moves to the player's hardware and to your engineering time up front.
How do I stop a local model from breaking my game's lore? With a validation layer that checks each output against your rules before the game applies it. This is the single most important piece and the easiest to underestimate. We cover it in how to stop AI from breaking your game's lore.
The short version
Running a local LLM for NPCs is very doable in 2026, and for many teams it is the right call. Just budget for the parts that are not the model: memory, validation, and structure. If you would rather skip that build, Director does it on-device out of the box, and Architect is free to structure the world it runs on.