ROSBAG2_COMPRESSION_001 ros2 data_error ai_generated true

[rosbag2_compression] Compression ratio exceeded threshold: 10.5 (max 5.0). Bag file may be corrupted.

ID: ros2/rosbag2-compression-ratio-exceeded

Also available as: JSON · Markdown · 中文
88%Fix Rate
84%Confidence
1Evidence
2024-04-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Humble active — — —
Iron active — — —
Jazzy active — — —
Rolling active — — —

Root Cause

The compression algorithm (e.g., zstd) produced a compressed file larger than the uncompressed data, often due to incompressible data (e.g., random noise from sensors like lidar or radar) or a misconfigured compression level.

generic

中文

压缩算法（例如 zstd）生成的压缩文件大于未压缩数据，通常是由于不可压缩的数据（例如来自激光雷达或雷达等传感器的随机噪声）或配置错误的压缩级别。

Official Documentation

https://github.com/ros2/rosbag2/blob/humble/rosbag2_compression/README.md

Workarounds

  1. 90% success Increase the compression ratio threshold in the storage options: ros2 bag record -o my_bag --compression-mode file --compression-format zstd --compression-queue-size 100 --max-compression-ratio 20.0 /topic1 /topic2 
    Increase the compression ratio threshold in the storage options: ros2 bag record -o my_bag --compression-mode file --compression-format zstd --compression-queue-size 100 --max-compression-ratio 20.0 /topic1 /topic2
  2. 85% success Filter out noisy topics (e.g., raw sensor data) from recording to avoid incompressible data: ros2 bag record -o my_bag --exclude '/noisy_lidar/.*' /important_topic 
    Filter out noisy topics (e.g., raw sensor data) from recording to avoid incompressible data: ros2 bag record -o my_bag --exclude '/noisy_lidar/.*' /important_topic
  3. 75% success Preprocess sensor data to reduce noise (e.g., downsampling or filtering) before recording, using a separate node that publishes filtered messages. 
    Preprocess sensor data to reduce noise (e.g., downsampling or filtering) before recording, using a separate node that publishes filtered messages.

中文步骤

  1. Increase the compression ratio threshold in the storage options: ros2 bag record -o my_bag --compression-mode file --compression-format zstd --compression-queue-size 100 --max-compression-ratio 20.0 /topic1 /topic2
  2. Filter out noisy topics (e.g., raw sensor data) from recording to avoid incompressible data: ros2 bag record -o my_bag --exclude '/noisy_lidar/.*' /important_topic
  3. Preprocess sensor data to reduce noise (e.g., downsampling or filtering) before recording, using a separate node that publishes filtered messages.

Dead Ends

Common approaches that don't work:

  1. Increase the compression level to 22 (max for zstd) 70% fail

    Higher compression levels can make the ratio worse for incompressible data, as overhead increases.

  2. Disable compression entirely and store bags uncompressed 50% fail

    This solves the warning but defeats the purpose of using compression for storage efficiency.

  3. Change the compression format to 'lz4' 75% fail

    LZ4 is faster but has even worse compression ratios for incompressible data and may still trigger the warning.