data type_error ai_generated true

Avro反序列化失败：联合字段期望特定分支但收到null

Avro deserialization fails: union field expects a specific branch but got null

ID: data/avro-record-schema-union-mismatch

其他格式: JSON · Markdown 中文 · English
85%修复率
84%置信度
1证据数
2023-02-14首次发现

版本兼容性

版本状态引入弃用备注
Apache Avro 1.11.1 active — — —
Confluent Avro Serializer 7.4 active — — —
Python avro 1.11.0 active — — —

根因分析

Avro模式定义了一个联合字段（例如，["null", "string"]），但由于模式演变或数据损坏，数据包含null值而模式期望非null分支，或相反。

English

An Avro schema defines a union field (e.g., ["null", "string"]) but the data contains a null value when the schema expects a non-null branch, or vice versa, due to schema evolution or data corruption.

generic

官方文档

https://avro.apache.org/docs/current/spec.html#Unions

解决方案

  1. 更新Avro模式，将null作为联合的第一个分支：将'["string"]'改为'["null", "string"]'并将默认值设为null。
  2. 使用自定义反序列化器优雅地处理null值，将其映射为默认字符串：if val is None: val = 'default'。

无效尝试

常见但无效的做法:

  1. Adding a default value of null to the union field in the schema 70% 失败

    The default value applies during schema evolution but does not fix existing data that has mismatched branches.

  2. Changing the entire field type to a single type (e.g., string) to avoid unions 90% 失败

    This breaks compatibility with existing data and requires re-encoding all messages.