Field manual

Local AI, plainly explained.

Every term the radar uses, answered in one breath first and a minute of plain language second — no hype, no prior knowledge assumed. Each entry ends where the concept does real work on this site.

18 terms and growing

Also in:Español Português

01Context WindowThe context window is how much text a model can consider at once, measured in tokens — and holding that text costs memory through the KV cache.02DistillationDistillation trains a small model to imitate a larger one's answers — the reason many of the best small models exist, and why they carry a bigger model's habits, good and bad.03Fine-TuningFine-tuning continues a model's training on your own examples so it adopts a style, format or specialty — it changes the weights, where a prompt merely instructs them.04GGUFGGUF is the single-file format local AI runtimes load: a model's weights, usually quantized, packed together with everything needed to run it.05InferenceInference is a model answering — using its already-trained weights to produce output. Training is how those weights were learned; inference is every use after that.06KV CacheThe KV cache is the working memory a model fills while reading your conversation — it grows with every token, and at long contexts it can rival the model itself in size.07Local RuntimeA local runtime is the app that actually runs models on your machine — it loads a GGUF file, drives your GPU, and gives you a chat window or an API.08Mixture of Experts (MoE)A Mixture-of-Experts model keeps many specialist sub-networks but activates only a few per token — big-model knowledge at small-model speed, with big-model memory still required.09OffloadingOffloading splits a model between graphics memory and ordinary RAM when it does not fully fit — the model runs, just noticeably slower.10Parameters (7B, 70B)Parameters are a model's learned weights, counted in billions — the B in 7B. More parameters mean more memory, and only sometimes better answers.11QuantizationQuantization stores a model's weights at lower precision — for example 4-bit instead of 16-bit — cutting memory to roughly a quarter at a small quality cost.12RAG (retrieval-augmented generation)RAG lets a model answer from your own documents by fetching the relevant passages first and placing them into the prompt — knowledge without retraining anything.13SafetensorsSafetensors is the format models are originally published in: full-precision weights across many files, built for data-center GPUs and training — not for laptops.14TemperatureTemperature is the dial that sets how adventurous a model's word choices are: low values make it predictable and repetitive, high values creative and more error-prone.15TokenA token is the unit AI models actually read and write — a word fragment of a few characters, with roughly 750 English words fitting into 1,000 tokens.16Trending ScoreThe trending score is Hugging Face's own measure of which models the community is engaging with right now — a momentum signal, not a quality grade.17Unified MemoryUnified memory is one pool of RAM shared by the processor and the graphics cores — the reason Apple Silicon Macs can run models that would need an enormous graphics card elsewhere.18VRAM vs. System RAMVRAM is the memory on your graphics card — the space a model must mostly fit into to run fast. System RAM is the bigger, slower fallback.