data serialization_error ai_generated true

Protobuf deserialization produces wrong values when field numbers are reused across different message types

ID: data/protobuf-field-number-collision

Also available as: JSON · Markdown · 中文
90%Fix Rate
84%Confidence
1Evidence
2023-09-18First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Protocol Buffers 3.21.0 active — — —
protoc 24.0 active — — —
gRPC 1.57.0 active — — —

Root Cause

Protobuf wire format uses field numbers only, not names, so reusing field numbers across different message types in the same file can cause field values to be assigned to the wrong fields during deserialization.

generic

中文

Protobuf线格式仅使用字段编号，而非名称，因此在同一文件中跨不同消息类型重复使用字段编号可能导致反序列化期间字段值被分配给错误的字段。

Official Documentation

https://protobuf.dev/programming-guides/encoding/#fields

Workarounds

  1. 95% success Ensure all message types use unique field numbers across the entire .proto file: message User { int32 id = 1; } message Product { int32 product_id = 1; } // Change to use distinct numbers 
    Ensure all message types use unique field numbers across the entire .proto file: message User { int32 id = 1; } message Product { int32 product_id = 1; } // Change to use distinct numbers
  2. 85% success Use a migration script to rewrite existing serialized data with corrected field numbers: for each message, decode using old schema, then re-encode with new schema mapping old field numbers to new ones 
    Use a migration script to rewrite existing serialized data with corrected field numbers: for each message, decode using old schema, then re-encode with new schema mapping old field numbers to new ones

中文步骤

  1. Ensure all message types use unique field numbers across the entire .proto file: message User { int32 id = 1; } message Product { int32 product_id = 1; } // Change to use distinct numbers
  2. Use a migration script to rewrite existing serialized data with corrected field numbers: for each message, decode using old schema, then re-encode with new schema mapping old field numbers to new ones

Dead Ends

Common approaches that don't work:

  1. Adding deprecated annotations to conflicting fields 80% fail

    Adding fields with new numbers does not fix the existing field number collision issue.

  2. Using 'reserved' keyword for conflicting field numbers 75% fail

    Reserved prevents new use but does not fix existing serialized data with wrong field numbers.