data schema_error ai_generated true

Avro 反序列化失败：写入器和读取器模式之间联合类型顺序不匹配

Avro deserialization fails: union type ordering mismatch between writer and reader schema

ID: data/avro-union-type-ordering-mismatch

其他格式: JSON · Markdown 中文 · English
90%修复率
88%置信度
1证据数
2023-11-05首次发现

版本兼容性

版本状态引入弃用备注
avro 1.11.3 active — — —
confluent-kafka-avro 7.5.0 active — — —
fastavro 1.9.4 active — — —

根因分析

Avro 联合类型对顺序敏感；如果写入器模式的联合类型顺序与读取器模式不同（例如 ["null", "string"] 与 ["string", "null"]），反序列化会因索引不匹配而失败。

English

Avro unions are order-sensitive; if the writer schema has union types in a different order than the reader schema (e.g., ["null", "string"] vs ["string", "null"]), deserialization fails with an index mismatch.

generic

官方文档

https://avro.apache.org/docs/current/spec.html#Unions

解决方案

  1. Ensure union types are always in alphabetical order: e.g., use ["null", "string"] not ["string", "null"]. In Avro schema definition: {"name": "field", "type": ["null", "string"]}. This is a best practice that prevents ordering issues.
  2. When reading, specify the writer schema explicitly: reader = fastavro.reader(fo, writer_schema=writer_schema). This bypasses the reader schema's union ordering.

无效尝试

常见但无效的做法:

  1. 80% 失败

    Defaults in Avro apply to the field itself, not to union type ordering. The index mismatch still occurs regardless of defaults.

  2. 70% 失败

    Compatibility checks validate schema evolution rules but do not enforce union type ordering consistency across different schema versions.