llama.cpp slots and KV cache
What your flags actually do
Set --ctx-size, --parallel and --kv-unified, and watch how the cache gets split. One shared pool, or one fixed part per slot.
--ctx-size (the whole pool)
--parallel (slot count)
--kv-unified
What is in one request
Text
Images
How many images
what one request uses still free in that slot request does not fit
Boot line you get
Where the workload numbers come from
Text was tokenised on a running llama.cpp server with the LFM2 tokenizer: 700 words of prose came back as 761 tokens, so 1.087 tokens per word. The word counts behind the buttons are 35 for a chat message, 500 for a page, 5,000 for a ten-page document and 9,000 for an hour of transcript.
Image costs were measured against LFM2.5-VL-1.6B by sending real images and reading the prompt length: 256 x 256 costs 67 tokens, and 512, 768 and 1024 square all cost 259 tokens because the model resizes them to the same grid. A 1920 x 1080 frame costs 2,311 tokens, since it gets tiled instead of resized. Image cost is therefore not proportional to pixels, and it changes from model to model. Treat these as one model's numbers, not a law.
What this shows. --ctx-size sizes the whole pool, and the server divides it by the slot count unless the cache stays unified. --parallel sets the slot count; its default is auto, which resolves to 4 slots on the build measured here (b10011). --kv-unified keeps one shared pool, and it is on by default only while the slot count is auto. Read your own boot line to confirm what your build picked.