{
  "id": "llm/chat-template-mismatch-hf",
  "signature": "ValueError: The tokenizer's chat_template is not compatible with the model's expected format. Expected 'llama' format, got 'chatml'.",
  "signature_zh": "ValueError：分词器的 chat_template 与模型期望的格式不兼容。期望 'llama' 格式，得到 'chatml'。",
  "regex": "ValueError: The tokenizer's chat_template is not compatible with the model's expected format\\. Expected '\\w+' format, got '\\w+'\\.",
  "domain": "llm",
  "category": "config_error",
  "subcategory": null,
  "root_cause": "The Hugging Face tokenizer's chat_template (e.g., ChatML for Mistral) does not match the model's expected conversation format (e.g., Llama's [INST] tags), causing incorrect tokenization of system and user messages.",
  "root_cause_type": "generic",
  "root_cause_zh": "Hugging Face 分词器的 chat_template（例如 Mistral 的 ChatML）与模型期望的对话格式（例如 Llama 的 [INST] 标签）不匹配，导致系统消息和用户消息的 token 化错误。",
  "versions": [
    {
      "version": "transformers 4.36.0",
      "introduced": null,
      "deprecated": null,
      "removed": null,
      "behavior_change": null,
      "status": "active"
    },
    {
      "version": "Mistral 7B Instruct v0.2",
      "introduced": null,
      "deprecated": null,
      "removed": null,
      "behavior_change": null,
      "status": "active"
    },
    {
      "version": "Llama 2 13B Chat",
      "introduced": null,
      "deprecated": null,
      "removed": null,
      "behavior_change": null,
      "status": "active"
    },
    {
      "version": "Zephyr 7B Beta",
      "introduced": null,
      "deprecated": null,
      "removed": null,
      "behavior_change": null,
      "status": "active"
    }
  ],
  "os_specific": {},
  "dead_ends": [
    {
      "action": "",
      "why_fails": "Setting chat_template to None falls back to the default template, which may still be incorrect for the model, leading to garbled prompts.",
      "fail_rate": 0.7,
      "condition": "",
      "sources": []
    },
    {
      "action": "",
      "why_fails": "Manual formatting is error-prone and may omit special tokens (e.g., <s>, </s>) that the model requires, causing poor generation or errors.",
      "fail_rate": 0.5,
      "condition": "",
      "sources": []
    },
    {
      "action": "",
      "why_fails": "Using an incompatible tokenizer can introduce vocabulary mismatches and unknown token IDs, breaking the model entirely.",
      "fail_rate": 0.8,
      "condition": "",
      "sources": []
    }
  ],
  "workarounds": [
    {
      "action": "Set the correct chat_template from the model's official repository. For Llama 2: `tokenizer.chat_template = \"{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% endif %}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<s>[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' </s>' }}{% endif %}{% endfor %}\"`.",
      "success_rate": 0.95,
      "how": "Set the correct chat_template from the model's official repository. For Llama 2: `tokenizer.chat_template = \"{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% endif %}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<s>[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' </s>' }}{% endif %}{% endfor %}\"`.",
      "condition": "",
      "sources": []
    },
    {
      "action": "Load the tokenizer with `use_fast=True` and pass `model_max_length` explicitly to avoid truncation issues: `AutoTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.2', use_fast=True, model_max_length=8192)`.",
      "success_rate": 0.85,
      "how": "Load the tokenizer with `use_fast=True` and pass `model_max_length` explicitly to avoid truncation issues: `AutoTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.2', use_fast=True, model_max_length=8192)`.",
      "condition": "",
      "sources": []
    }
  ],
  "workarounds_zh": [
    "从模型的官方仓库设置正确的 chat_template。对于 Llama 2：`tokenizer.chat_template = \"{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% endif %}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<s>[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' </s>' }}{% endif %}{% endfor %}\"`。",
    "使用 `use_fast=True` 加载分词器，并显式传递 `model_max_length` 以避免截断问题：`AutoTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.2', use_fast=True, model_max_length=8192)`。"
  ],
  "transition_graph": {
    "leads_to": [],
    "preceded_by": [],
    "frequently_confused_with": []
  },
  "official_doc_url": "https://huggingface.co/docs/transformers/main/en/chat_templating",
  "official_doc_section": null,
  "error_code": null,
  "verification_tier": "ai_generated",
  "confidence": 0.9,
  "fix_success_rate": 0.92,
  "resolvable": "true",
  "first_seen": "2024-01-10",
  "last_confirmed": "2024-06-01",
  "last_updated": "2024-06-01",
  "evidence_count": 1,
  "tags": [],
  "locale": "en",
  "aliases": []
}