AVRO_SCHEMA_EVOLUTION_TYPE_MISMATCH data schema_error ai_generated partial

Avro模式演变在字段类型从int变为long时失败

Avro schema evolution fails when field type changes from int to long

ID: data/avro-schema-evolution-field-type-change

其他格式: JSON · Markdown 中文 · English
75%修复率
88%置信度
1证据数
2023-08-20首次发现

版本兼容性

版本状态引入弃用备注
Apache Avro 1.11.0 active — — —
Confluent Schema Registry 7.5.0 active — — —

根因分析

Avro模式演变不允许将字段类型从int改为long而没有自定义转换，因为这违反了向后兼容性规则。

English

Avro schema evolution does not allow changing a field's type from int to long without a custom conversion, as it violates backward compatibility rules.

generic

官方文档

https://avro.apache.org/docs/current/spec.html#Schema+Evolution

解决方案

  1. Define a new field with the target type (e.g., 'new_field' as long) and deprecate the old field. Update producers to write to the new field and consumers to read from both.
  2. Use a custom logical type conversion in the schema registry to handle int-to-long promotion, e.g., by setting `avro.schema.conversion` property.

无效尝试

常见但无效的做法:

  1. 80% 失败

    Simply adding a default value to the field does not resolve the type change issue because Avro checks the type compatibility first.

  2. 90% 失败

    Deleting and recreating the schema entirely breaks existing data streams and requires reprocessing all historical data.