# ValueError: You have to provide either 'max_length' or 'padding' and 'truncation' to use padding side 'left' with batch size > 1

- **ID:** `llm/huggingface-tokenizer-padding-side-mismatch`
- **Domain:** llm
- **Category:** config_error
- **Verification:** ai_generated
- **Fix Rate:** 85%

## Root Cause

Hugging Face tokenizer requires explicit padding and truncation configuration when using left padding (common for decoder-only LLMs) in batched inference, otherwise it cannot determine the sequence length alignment.

## Version Compatibility

| Version | Status | Introduced | Deprecated |
|---------|--------|------------|------------|
| transformers 4.30.0 | active | — | — |
| transformers 4.31.0 | active | — | — |
| transformers 4.35.0 | active | — | — |
| Python 3.9 | active | — | — |
| Python 3.10 | active | — | — |
| Python 3.11 | active | — | — |

## Workarounds

1. **Configure tokenizer with explicit padding=True, truncation=True, and max_length=model_max_length before tokenizing batched inputs:

tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b-hf')
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = 'left'
encoded = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors='pt')** (95% success)
   ```
   Configure tokenizer with explicit padding=True, truncation=True, and max_length=model_max_length before tokenizing batched inputs:

tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b-hf')
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = 'left'
encoded = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors='pt')
   ```
2. **Set tokenizer.pad_token_id explicitly and use DataCollatorWithPadding from transformers for dynamic batching:

from transformers import DataCollatorWithPadding
data_collator = DataCollatorWithPadding(tokenizer, padding='max_length', max_length=512)** (90% success)
   ```
   Set tokenizer.pad_token_id explicitly and use DataCollatorWithPadding from transformers for dynamic batching:

from transformers import DataCollatorWithPadding
data_collator = DataCollatorWithPadding(tokenizer, padding='max_length', max_length=512)
   ```

## Dead Ends

- **** — Decoder-only models (e.g., LLaMA, GPT) expect left padding for causal attention masking; right padding causes misalignment and incorrect generation. (70% fail)
- **** — Bypasses the error but prevents batching, leading to poor throughput and increased latency in production. (50% fail)