kafka runtime_error ai_generated partial

org.apache.kafka.common.errors.TransactionAbortableException: The transaction has been aborted by the coordinator

ID: kafka/transaction-abortable

Also available as: JSON · Markdown · 中文

75%Fix Rate

82%Confidence

1Evidence

2024-01-10First Seen

Version Compatibility

Version	Status	Introduced	Deprecated	Notes
Kafka 3.4.0	active	—	—	—
Kafka 3.5.0	active	—	—	—
Kafka 3.6.0	active	—	—	—

Root Cause

Transactional coordinator aborted the transaction due to a timeout, fencing, or coordinator failover.

generic

中文

事务协调器因超时、围栏或协调器故障转移而中止事务。

Official Documentation

https://kafka.apache.org/documentation/#transactionalconfigs

Workarounds

75% success Retry the transaction with a new transactional.id after aborting the current one. Code example: producer.initTransactions(); try { producer.beginTransaction(); // send messages producer.commitTransaction(); } catch (TransactionAbortableException e) { producer.abortTransaction(); // Reinitialize with a new transactional.id properties.put("transactional.id", "new-txn-id-" + System.currentTimeMillis()); producer.close(); producer = new KafkaProducer<>(properties); producer.initTransactions(); }
```
Retry the transaction with a new transactional.id after aborting the current one.
Code example:
producer.initTransactions();
try {
  producer.beginTransaction();
  // send messages
  producer.commitTransaction();
} catch (TransactionAbortableException e) {
  producer.abortTransaction();
  // Reinitialize with a new transactional.id
  properties.put("transactional.id", "new-txn-id-" + System.currentTimeMillis());
  producer.close();
  producer = new KafkaProducer<>(properties);
  producer.initTransactions();
}
```
70% success Increase transaction coordinator timeout and ensure stable network. Command to check coordinator: kafka-transactions --bootstrap-server localhost:9092 --describe --transactional-id my-txn-id Set in producer config: properties.put("transaction.timeout.ms", "60000"); properties.put("max.block.ms", "120000");
```
Increase transaction coordinator timeout and ensure stable network.
Command to check coordinator:
kafka-transactions --bootstrap-server localhost:9092 --describe --transactional-id my-txn-id
Set in producer config:
properties.put("transaction.timeout.ms", "60000");
properties.put("max.block.ms", "120000");
```

中文步骤

Retry the transaction with a new transactional.id after aborting the current one.
Code example:
producer.initTransactions();
try {
  producer.beginTransaction();
  // send messages
  producer.commitTransaction();
} catch (TransactionAbortableException e) {
  producer.abortTransaction();
  // Reinitialize with a new transactional.id
  properties.put("transactional.id", "new-txn-id-" + System.currentTimeMillis());
  producer.close();
  producer = new KafkaProducer<>(properties);
  producer.initTransactions();
}

Increase transaction coordinator timeout and ensure stable network.
Command to check coordinator:
kafka-transactions --bootstrap-server localhost:9092 --describe --transactional-id my-txn-id
Set in producer config:
properties.put("transaction.timeout.ms", "60000");
properties.put("max.block.ms", "120000");

Dead Ends

Common approaches that don't work:

Increase transaction.timeout.ms to 3600000 50% fail
Longer timeout may mask underlying issues like coordinator instability or network partitions.
Disable idempotent producer 90% fail
Transactions require idempotent producer; disabling it causes different errors.
Restart all brokers 80% fail
Restarting brokers does not clear the aborted transaction state; coordinator still has the abort marker.