kafka runtime_error ai_generated partial

org.apache.kafka.common.errors.TransactionalIdExpiredException: The transactional id has expired and is now fenced

ID: kafka/transactional-id-expired

Also available as: JSON · Markdown · 中文

92%Fix Rate

88%Confidence

1Evidence

2024-06-10First Seen

Version Compatibility

Version	Status	Introduced	Deprecated	Notes
3.6.0	active	—	—	—
3.7.1	active	—	—	—
3.8.0	active	—	—	—

Root Cause

The transactional.id has not been used within the transaction.max.timeout.ms window, causing the coordinator to fence it.

generic

中文

事务 ID 在 transaction.max.timeout.ms 窗口内未被使用，导致协调器将其隔离。

Official Documentation

https://kafka.apache.org/documentation/#brokerconfigs_transaction.max.timeout.ms

Workarounds

95% success Use a new transactional.id for the producer, e.g., in Java: props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "new-app-id-" + System.currentTimeMillis()); Then initTransactions() and begin transaction.
```
Use a new transactional.id for the producer, e.g., in Java:
props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "new-app-id-" + System.currentTimeMillis());
Then initTransactions() and begin transaction.
```
90% success Increase transaction.max.timeout.ms on the broker: kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --alter --add-config transaction.max.timeout.ms=900000 And ensure the producer uses a lower transaction.timeout.ms.
```
Increase transaction.max.timeout.ms on the broker:
kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --alter --add-config transaction.max.timeout.ms=900000
And ensure the producer uses a lower transaction.timeout.ms.
```

中文步骤

Use a new transactional.id for the producer, e.g., in Java:
props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "new-app-id-" + System.currentTimeMillis());
Then initTransactions() and begin transaction.

Increase transaction.max.timeout.ms on the broker:
kafka-configs.sh --bootstrap-server localhost:9092 --entity-type brokers --entity-default --alter --add-config transaction.max.timeout.ms=900000
And ensure the producer uses a lower transaction.timeout.ms.

Dead Ends

Common approaches that don't work:

98% fail
Restarting the producer does not recover the expired transactional.id; it remains fenced.
80% fail
Increasing transaction.timeout.ms on the producer alone does not retroactively prevent expiration.