llm prompt_engineering ai_generated partial

LLM ignores system prompt instructions when user message or context is very long

ID: llm/system-prompt-ignored-long-context

Also available as: JSON · Markdown
65%Fix Rate
80%Confidence
3Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

LLMs exhibit 'lost in the middle' effect where instructions at the start of context are weakened by large amounts of content. System prompts lose effectiveness when conversation history or RAG context grows large.

generic

Workarounds

  1. 90% success Repeat critical instructions at the end of the prompt, close to the expected output 
    Place key constraints in both system prompt AND as the final user message: 'Remember: output JSON only'
  2. 92% success Use structured output (response_format/json_schema) to enforce format compliance 
    response_format={'type': 'json_schema', 'json_schema': schema}  # model-enforced structure
  3. 85% success Chunk long context and process iteratively instead of single mega-prompt 
    Split 100K context into 10K chunks, process each with full instructions, then aggregate results

Dead Ends

Common approaches that don't work:

  1. Put all instructions in the system prompt and trust they'll be followed 82% fail

    System prompt influence degrades as context length grows. At 50K+ tokens of content, earlier instructions can be effectively forgotten.

  2. Increase context window to fit more instructions 75% fail

    Larger context windows amplify the 'lost in the middle' problem. More tokens = more dilution of instructions.