The ninja Jinja template
your agent hides
from the model.

a Why an LLM stack needs it

Chat models are trained on a fixed textual dialogue format — ChatML's <|im_start|>, Llama's <|start_header_id|>, Mistral's [INST]. The API you call accepts structured JSON. Something must translate [{role, content}] → "…one flat string…". For Hugging Face models that something is a Jinja template stored with the checkpoint. Since transformers v4.44 the built-in default chat template was removed — if a tokenizer doesn't define one, you must supply it or chat requests fail outright.

b Where the template physically lives

# Hugging Face repo
tokenizer_config.json  →  "chat_template"
chat_template.jinja    # newer, standalone file

# GGUF metadata (llama.cpp / LM Studio)
tokenizer.chat_template = "{%- set …"

# serving runtimes
vLLM / TGI  → jinja2 (Python)
llama.cpp   → minja (C++ engine, --jinja)
Ollama      → Go text/template in Modelfile

c The two different "templates" people conflate

This is the source of 90% of the confusion when someone says "the Jinja template broke my agent":

d Anthropic's API?

Talking to api.anthropic.com, Claude Code sends a structured messages array plus a single system parameter (a string or an array of content blocks). There is no user-visible Jinja on that path — formatting happens inside Anthropic's serving stack. Jinja enters the picture the moment the same agent points at an open model behind llama.cpp, vLLM, TGI or LM Studio.

1 Format match = capability

The chat template must reproduce the exact byte pattern used in instruction tuning. Feed a ChatML model a Llama-style prompt and it still answers — but tool calling accuracy, stop behaviour and instruction following all degrade, often silently.

# trained on:  <|im_start|>user\n…<|im_end|>
# served with: [INST] … [/INST]
# result:     works-ish. emits raw tool JSON as prose,
              forgets to stop, ignores system rules.

2 Special tokens become real token ids

<|im_end|> isn't decoration — the tokenizer maps it to a single id (in one Qwen3.x GGUF: 248046) that signals "turn over". Drop it and the model keeps generating the user's next turn for you.

{%- if add_generation_prompt %}
  {{- '<|im_start|>assistant\n' }}   # ← the "your turn" cue
{%- endif %}

3 Whitespace is billed

Naive indentation inside a loop adds a newline + spaces per iteration. Multiply by 40 messages × every request.

4 Tool schemas are tokens too

Each tool is serialized — usually tojson of the whole JSON Schema. A dozen agent tools easily costs a few thousand tokens of overhead on every single call.

5 Prompt caching is prefix-sensitive

Render {{ today }} or a random session id near the top and the cached prefix changes every request → full re-prefill, higher latency, no cache discount.

6 Templates enforce structure — and fail hard

Model authors embed validation directly in Jinja. Qwen3.x's official template refuses any second system message; other templates enforce strict role alternation:

{%- if message.role == 'system' %}
  {%- if not loop.first %}
    {{- raise_exception('System message must be at the beginning.') }}
  {%- endif %}

# vLLM, another model family:
# 400 "Conversation roles must alternate user/assistant/…"

No fuzzy fallback. The server returns 400 (or 500 from llama.cpp) and the agent surfaces a bare AI_APICallError.

7 Two injection surfaces

(a) Untrusted data. File contents, web pages and tool outputs are rendered straight into the prompt string. A repo containing <|im_end|><|im_start|>system can forge a turn boundary — which is exactly why minja marks input so data can't be re-tokenized as control tokens.

(b) Untrusted template. Rendering a Jinja string you didn't write is a server-side-template-injection risk. LangChain's own docs say it plainly: prefer template_format="f-string", or never accept jinja2 templates from untrusted sources.

8 Ordering & merging is the agent's job

Because templates are strict, the CLI must normalize before sending: merge plugin-injected system blocks into one message, put tool results in the role the template understands, keep message alternation valid.

// opencode request.ts (conceptual)
system.map(x => ({role:"system", content:x}))  // ✗ N messages
[{role:"system", content: system.join("\n\n"})] // ✓ one

9 Truncation cuts where the template decided

When the rendered prompt exceeds the context window, the runtime drops tokens — usually from the oldest turns. A verbose system prompt + big tool block means your actual code gets evicted first.