KAFKA-ERR-100 data config_error ai_generated true

Kafka JSON反序列化因未配置模式注册表而失败，报错'未知模式ID'

Kafka JSON deserialization fails with 'Unknown schema ID' when schema registry is not configured

ID: data/kafka-json-deserialization-with-schema-registry

其他格式: JSON · Markdown 中文 · English
95%修复率
87%置信度
1证据数
2023-04-20首次发现

版本兼容性

版本状态引入弃用备注
Confluent Platform 7.4 active — — —
kafka-python 2.0.2 active — — —
Apache Kafka 3.5 active — — —

根因分析

Kafka生产者启用了模式注册表写入JSON数据，但消费者未配置模式注册表，或者模式ID不在注册表中。

English

Kafka producer writes JSON data with schema registry enabled, but the consumer does not have schema registry configured or the schema ID is not present in the registry.

generic

官方文档

https://docs.confluent.io/platform/current/schema-registry/index.html

解决方案

  1. 配置消费者使用相同的模式注册表URL：在消费者属性中添加'schema.registry.url=http://localhost:8081'。
  2. 如果模式注册表不可用，使用忽略模式ID的自定义反序列化器：使用kafka.deserializers.string_deserializer处理原始JSON。

无效尝试

常见但无效的做法:

  1. Disabling schema registry on the producer side 80% 失败

    Producer may have already written messages with schema IDs; disabling registry only affects new messages, not existing ones.

  2. Manually editing the Kafka topic to remove schema IDs from message headers 90% 失败

    Editing message headers is complex and risky; schema IDs are embedded in the message payload, not just headers.