# ValueError: The tokenizer's chat_template is not compatible with the model's expected format. Expected 'llama' format, got 'chatml'.

- **ID:** `llm/chat-template-mismatch-hf`
- **Domain:** llm
- **Category:** config_error
- **Verification:** ai_generated
- **Fix Rate:** 92%

## Root Cause

The Hugging Face tokenizer's chat_template (e.g., ChatML for Mistral) does not match the model's expected conversation format (e.g., Llama's [INST] tags), causing incorrect tokenization of system and user messages.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| transformers 4.36.0 | active | — | — |
| Mistral 7B Instruct v0.2 | active | — | — |
| Llama 2 13B Chat | active | — | — |
| Zephyr 7B Beta | active | — | — |

## Workarounds

1. **Set the correct chat_template from the model's official repository. For Llama 2: `tokenizer.chat_template = "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% endif %}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<s>[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' </s>' }}{% endif %}{% endfor %}"`.** (95% success)
   ```
   Set the correct chat_template from the model's official repository. For Llama 2: `tokenizer.chat_template = "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% endif %}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<s>[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' </s>' }}{% endif %}{% endfor %}"`.
   ```
2. **Load the tokenizer with `use_fast=True` and pass `model_max_length` explicitly to avoid truncation issues: `AutoTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.2', use_fast=True, model_max_length=8192)`.** (85% success)
   ```
   Load the tokenizer with `use_fast=True` and pass `model_max_length` explicitly to avoid truncation issues: `AutoTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.2', use_fast=True, model_max_length=8192)`.
   ```

## Dead Ends

- **** — Setting chat_template to None falls back to the default template, which may still be incorrect for the model, leading to garbled prompts. (70% fail)
- **** — Manual formatting is error-prone and may omit special tokens (e.g., <s>, </s>) that the model requires, causing poor generation or errors. (50% fail)
- **** — Using an incompatible tokenizer can introduce vocabulary mismatches and unknown token IDs, breaking the model entirely. (80% fail)