How to Acknowledge Current Offset in Spring Kafka for Manual Commit

Kafka commit offset

It depends on how you commit:

On commitSync assuming you commit each offset separately like in your example, if offset 1 is not committed it will retry to commit that offset until it succeeds or until it encounters nonRetryable failure.
One drawback of that mechanism is that the application is blocked until the broker responds and therefor limit the application throughput.

On commitAsync on the other hand, there is no retry in case it failed to commit offset 1. The reason for that is that by the time the broker responds get back to your Kafka client there may be later commit that was already successful.
Imagine on your example that commit of message with offset 1 was failed due to temporary network issue. Meanwhile the application already processed successfully message with offset 2. If we now retry previous failed commit it might succeed to commit offset 1 after 2 was already committed. In a case of rebalance, this will cause more duplicated.

Consequences of no commit/acknowledge to Kafka after consuming a message (with auto commit disabled)

The listener will not re-consume the message unless the consumer is restarted, or a rebalance occurs.

Kafka maintains 2 pointers for each consumer/partition; the position (last read record) and the committed offset. When a consumer starts, or a rebalance occurs, the position is set to the committed offset. The position is only changed by fetching new records, or if a seek is performed on the consumer for that partition.

Related Topics