What I would like to achieve:
I would like to identify the retried message using Spring Kafka when producing the retried message.
Background:
I am a Spring Kafka producer.
I produce messages that I put in a kafka broker.
Unfortunately, the kafka cluster is flaky, but not to a point where it is completely down. Usually, couple of retries will send the message successfully to Kafka.
The different consumers of the topic have the requirement to identify a message that has been retried.
Meaning, when they get a message I managed to produce without problems, no need for anything else.
However, if the message has been produced, but went thought the retry mechanism, I would like to identify / tag those a such.
Here is what I tried:
retries (defaults to Integer.MAX_VALUE): the maximum number of attempts to publish the messagedelivery.timeout.ms (defaults to 120,000): the maximum time to wait for a message to be acknowledged before considering it failedretry.backoff.ms (defaults to 100): the time to wait before retryingretry.backoff.max.ms (defaults to 1,000): the maximum delay between consecutive retries
With this from the official doc, I went to use this construct:
Properties props = new Properties(); // other properties props.put(RETRIES_CONFIG, 20); props.put(RETRY_BACKOFF_MS_CONFIG, "500"); props.put(DELIVERY_TIMEOUT_MS_CONFIG, "5000");[...]props.put(BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers()); props.put(KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); props.put(VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); KafkaProducer<String, String> producer = new KafkaProducer<>(props);
While this would configure the retry, it does not distinguish a retried message.
Question:
How to distinguish, identify, tag that a message has been retried?