llm data_error ai_generated true

ValueError: Cannot create Document from empty text. Node content is None or empty string.

ID: llm/llamaindex-empty-node-from-parser

Also available as: JSON · Markdown · 中文
92%Fix Rate
88%Confidence
1Evidence
2024-01-10First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
llama-index==0.10.43 active — — —
llama-index-core==0.10.43 active — — —
pypdf==4.2.0 active — — —

Root Cause

LlamaIndex document parser received a file or chunk with no extractable text, often from binary files (PDFs without OCR, images) or empty markdown sections.

generic

中文

LlamaIndex 文档解析器接收到无可提取文本的文件或块，通常来自二进制文件（无 OCR 的 PDF、图像）或空的 Markdown 部分。

Official Documentation

https://docs.llamaindex.ai/en/stable/module_guides/loading/documents/

Workarounds

  1. 90% success Add a filter to skip files with no text before parsing: `if not file_content.strip(): continue` in the ingestion loop. 
    Add a filter to skip files with no text before parsing: `if not file_content.strip(): continue` in the ingestion loop.
  2. 95% success Use a try-except block to catch the ValueError and log the file path for manual review: `try: doc = parser.parse(file); except ValueError as e: logger.warning(f'Skipping {file}: {e}')` 
    Use a try-except block to catch the ValueError and log the file path for manual review: `try: doc = parser.parse(file); except ValueError as e: logger.warning(f'Skipping {file}: {e}')`
  3. 85% success For PDFs, enable OCR with `pypdf` or `pdfplumber` to extract text from scanned documents: `pip install pypdf[ocr]` and set `parser = SimpleDirectoryReader(required_exts=['.pdf'], file_extractor={'.pdf': PDFReader(ocr=True)})` 
    For PDFs, enable OCR with `pypdf` or `pdfplumber` to extract text from scanned documents: `pip install pypdf[ocr]` and set `parser = SimpleDirectoryReader(required_exts=['.pdf'], file_extractor={'.pdf': PDFReader(ocr=True)})`

中文步骤

  1. Add a filter to skip files with no text before parsing: `if not file_content.strip(): continue` in the ingestion loop.
  2. Use a try-except block to catch the ValueError and log the file path for manual review: `try: doc = parser.parse(file); except ValueError as e: logger.warning(f'Skipping {file}: {e}')`
  3. For PDFs, enable OCR with `pypdf` or `pdfplumber` to extract text from scanned documents: `pip install pypdf[ocr]` and set `parser = SimpleDirectoryReader(required_exts=['.pdf'], file_extractor={'.pdf': PDFReader(ocr=True)})`

Dead Ends

Common approaches that don't work:

  1. 70% fail

    The pipeline may silently drop valid data if the error is not caught and logged; the empty node is still created but breaks downstream indexing.

  2. 95% fail

    Empty text is not a chunk size issue; the source file itself has no extractable content.

  3. 80% fail

    Conversion may still produce empty files for binary inputs; the root cause is lack of extractable text, not format.