llm security ai_generated partial

System prompt leaked via prompt injection — user input not sanitized

ID: llm/prompt-injection-system-prompt-leak

Also available as: JSON · Markdown
60%Fix Rate
85%Confidence
4Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

Adversarial user input can leak system prompts or override behavior.

generic

Workarounds

  1. 75% success Use a separate system message role and enforce input/output guardrails 
    Use role:system for instructions + apply output filtering

    Sources: https://platform.openai.com/docs/guides/safety-best-practices

  2. 70% success Implement layered defense: input sanitization + output filtering + canary tokens 
    Add unique canary string to system prompt; monitor outputs for its presence

    Sources: https://owasp.org/www-project-top-10-for-large-language-model-applications/

Dead Ends

Common approaches that don't work:

  1. Filter exact phrase ignore previous instructions 85% fail

    Trivially bypassed by rephrasing, unicode tricks, or multilingual variants

  2. Put system prompt at the end of context 70% fail

    LLMs attend to all positions; placement alone does not prevent extraction