data serialization_error ai_generated true

当联合字段将null作为第一个元素而非最后一个时，Avro反序列化失败

Avro deserialization fails when union field has null as first element instead of last

ID: data/avro-union-null-ordering

其他格式: JSON · Markdown 中文 · English
80%修复率
83%置信度
1证据数
2023-11-05首次发现

版本兼容性

版本状态引入弃用备注
Apache Avro 1.11.0 active — — —
Confluent Schema Registry 7.4.0 active — — —
Kafka 3.5.0 active — — —

根因分析

某些Avro库期望null是联合类型中的第一个元素（例如['null', 'string']），而其他库期望它在最后，导致架构兼容性问题。

English

Some Avro libraries expect null to be the first element in a union type (e.g., ['null', 'string']), while others expect it last, causing schema compatibility issues.

generic

官方文档

https://avro.apache.org/docs/1.11.0/spec.html#Unions

解决方案

  1. Ensure all Avro schemas use the same union ordering convention: always put null first: {"type": ["null", "string"]}
  2. Use a custom deserializer that reorders union types: GenericDatumReader<GenericRecord> reader = new GenericDatumReader<>(writerSchema, readerSchema) { @Override protected Object read(Object old, Decoder in) throws IOException { return super.read(old, in); } };

无效尝试

常见但无效的做法:

  1. Setting compatibility to NONE in schema registry 80% 失败

    Changing schema registry compatibility type does not fix the union ordering issue.

  2. Modifying the data to include null values in a different order 75% 失败

    The null position is determined by the schema, not the data payload.