System prompt
A system prompt is the standing instruction a model reads before your conversation: who it should be, what it may do, and how to answer — set by the app or by you, not by the model.
Every chat you have with a model starts with text you usually never see. Before your first message, the application slips in a block of instructions — “You are a helpful coding assistant. Answer in the language of the user. Never invent file paths.” — and the model treats it as the rules of the room. That block is the system prompt, and it shapes tone, format and refusals far more than any single question does.
Locally you own it. Runtimes like Ollama and LM Studio let you set a system prompt per model or per chat, and a good one is often worth more than a bigger model: tell a small model exactly what role it plays and what output you expect, and its answers tighten up noticeably. It is also where privacy rules live — “only use the provided documents” — and where you pin down the language a model keeps drifting away from.
The cost is tokens. The system prompt is sent with every request and sits in the context window for the whole conversation, so a two-page instruction sheet eats context and KV-cache memory before you have typed a word. Keep it as short as it can be while still being unambiguous.
Where you see it on the radar
The context length the fit engine prices includes everything in the window — your system prompt, retrieved passages and the conversation itself. If your instructions are long, pick a longer context in the finder and see what that does to memory.
Check the fit at your context length →