kafka runtime_error ai_generated true

org.apache.kafka.common.errors.TransactionTimeoutException: The transaction coordinator has aborted the transaction due to timeout

ID: kafka/transaction-abort-timeout

Also available as: JSON · Markdown · 中文
80%Fix Rate
86%Confidence
1Evidence
2024-05-12First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Kafka 3.6.0 active — — —
Kafka 3.7.1 active — — —

Root Cause

A Kafka transaction exceeded the configured max.transaction.timeout.ms, causing the coordinator to abort it to free resources.

generic

中文

Kafka事务超过了配置的max.transaction.timeout.ms，导致协调器中止事务以释放资源。

Official Documentation

https://kafka.apache.org/documentation/#transaction_config

Workarounds

  1. 85% success Increase `max.transaction.timeout.ms` in broker config (e.g., to 300000) and restart broker, then set producer's `transaction.timeout.ms=120000`. 
    Increase `max.transaction.timeout.ms` in broker config (e.g., to 300000) and restart broker, then set producer's `transaction.timeout.ms=120000`.
  2. 80% success Optimize transaction logic to complete within the default 60000ms by batching fewer records or reducing external calls. 
    Optimize transaction logic to complete within the default 60000ms by batching fewer records or reducing external calls.
  3. 88% success Example broker config: `transaction.max.timeout.ms=300000` in server.properties, then producer config: `props.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, 120000);` 
    Example broker config: `transaction.max.timeout.ms=300000` in server.properties, then producer config: `props.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, 120000);`

中文步骤

  1. Increase `max.transaction.timeout.ms` in broker config (e.g., to 300000) and restart broker, then set producer's `transaction.timeout.ms=120000`.
  2. Optimize transaction logic to complete within the default 60000ms by batching fewer records or reducing external calls.
  3. Example broker config: `transaction.max.timeout.ms=300000` in server.properties, then producer config: `props.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, 120000);`

Dead Ends

Common approaches that don't work:

  1. 75% fail

    The broker's max.transaction.timeout.ms overrides the producer setting, so the timeout still applies.

  2. 60% fail

    It is not a fix; it changes the application's semantics and may cause duplicate writes.