No explicit error; silent data corruption data protocol_error ai_generated partial

Protobuf deserialization silently drops unknown oneof fields causing data loss

ID: data/protobuf-oneof-unknown-field

Also available as: JSON · Markdown · 中文
80%Fix Rate
83%Confidence
1Evidence
2023-11-30First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Protocol Buffers 3.21.0 active — — —
gRPC 1.58.0 active — — —

Root Cause

When a Protobuf message with a oneof field is deserialized by an older version that doesn't know the new oneof variant, the entire oneof is set to None, losing all other known fields in that oneof.

generic

中文

当包含 oneof 字段的 Protobuf 消息被不知道新 oneof 变体的旧版本反序列化时，整个 oneof 被设置为 None，丢失该 oneof 中所有其他已知字段。

Official Documentation

https://protobuf.dev/programming-guides/proto3/#oneof

Workarounds

  1. 85% success Add a catch-all 'bytes unknown_oneof' field to the message to preserve raw bytes of unknown oneof variants, then manually parse. 
    Add a catch-all 'bytes unknown_oneof' field to the message to preserve raw bytes of unknown oneof variants, then manually parse.
  2. 90% success Ensure all consumers are updated to the latest proto definition before deploying new oneof variants. 
    Ensure all consumers are updated to the latest proto definition before deploying new oneof variants.

中文步骤

  1. Add a catch-all 'bytes unknown_oneof' field to the message to preserve raw bytes of unknown oneof variants, then manually parse.
  2. Ensure all consumers are updated to the latest proto definition before deploying new oneof variants.

Dead Ends

Common approaches that don't work:

  1. Adding a new field outside the oneof to carry the new variant 50% fail

    This changes the message structure and may cause confusion, and the oneof still gets cleared on unknown variants.

  2. Using 'optional' keyword for oneof fields 70% fail

    The 'optional' keyword does not prevent the oneof from being cleared when an unknown variant is encountered.