I worked with SpringBoot with Java 21 ,Apache Camel and connection with sftp.
I have a problem with calling re-delivery when I have a connection problem, I think. I have several sftp configurations. Every now and then when I have a connection problem, for example once an hour or once every few hours, an IOException is thrown with the message "input stream closed". The problem is that it happens rarely, but I already have an error in the logs and send a message about the problem. It seems to be temporary, so I want to retry 3 times if it fails then handle the error. I have the scheduler set to every 5 seconds so every 5 seconds a connection is created and terminated (because disconnect=true). When I spent some time digging in the Camel libraries, I noticed that the process looks like this
- scheduler starts a new task
- checks if there is a connection to the sftp server, if not, it establishes a connection
- moves to listing and calls the ls method to check the data
- throws an error here
SftpConsumerpublic synchronized SftpRemoteFile[] listFiles(String path) throws GenericFileOperationFailedException { LOG.trace("Listing remote files from path {}", path); if (ObjectHelper.isEmpty(path)) { path = "."; } try { Vector<?> files = this.channel.ls(path); return (SftpRemoteFile[])files.stream().map((f) -> { return new SftpRemoteFileJCraft((ChannelSftp.LsEntry)f); }).toArray((x$0) -> { return new SftpRemoteFileJCraft[x$0]; }); } catch (SftpException var3) { throw new GenericFileOperationFailedException("Cannot list directory: " + path, var3); } }if there is a disconnection or some network problem, anastpail earlier, Camel can handle it, but when the files are listed, it crashes.GenericFileOperationFailedException->SftpExcetpion->IOException is generated, such a wrapper.
The problem is that when I define onException for IOException or Exception in general, it will fall into the method I want, but it completely skips redelivery. Even if I set 2-3 and the delay time, it does not take it into account
@Override public void configure() { onException(IOException.class) .maximumRedeliveries(3) .redeliveryDelay(3000) .logHandled(true) .log("Redeliveries for IOException finished") .bean(consumerExceptionHandler, "handleIOException") .handled(true) .end(); onException(Exception.class) .logHandled(true) .bean(consumerExceptionHandler, "handleException") .handled(true) .end(); from("sftp://user@host:22/directory?password=secret&disconnect=true") .routeId("sftpRouteWithRetry") .log("Starting SFTP route to fetch files") .to(myQueue) .log("File downloaded successfully to /local/directory") .end();}When I checked deep.In ScheduledPoolStrategy there is all invoke and there is catching exception and rollback which return boolean
ScheduledPoolStrategy private void doRun() { if (this.isSuspended()) { LOG.trace("Cannot start to poll: {} as its suspended", this.getEndpoint()); } else { if (this.backoffMultiplier > 0 && this.idleCounter >= (long)(this.backoffIdleThreshold > 0 ? this.backoffIdleThreshold : Integer.MAX_VALUE) || this.errorCounter >= (long)(this.backoffErrorThreshold > 0 ? this.backoffErrorThreshold : Integer.MAX_VALUE)) { if (this.backoffCounter++< this.backoffMultiplier) { if (this.idleCounter > 0L) { LOG.debug("doRun() backoff due subsequent {} idles (backoff at {}/{})", new Object[]{this.idleCounter, this.backoffCounter, this.backoffMultiplier}); } else { LOG.debug("doRun() backoff due subsequent {} errors (backoff at {}/{})", new Object[]{this.errorCounter, this.backoffCounter, this.backoffMultiplier}); } return; } this.idleCounter = 0L; this.errorCounter = 0L; this.backoffCounter = 0; this.successCounter = 0L; LOG.trace("doRun() backoff finished, resetting counters."); } long count = this.counter.incrementAndGet(); boolean stopFire = this.repeatCount > 0L && count > this.repeatCount; if (stopFire) { LOG.debug("Cancelling {} scheduler as repeat count limit reached after {} counts.", this.getEndpoint(), this.repeatCount); this.scheduler.unscheduleTask(); } else { int retryCounter = -1; boolean done = false; Throwable cause = null; int polledMessages = 0; while(!done) { try { cause = null; done = true; if (this.isPollAllowed()) { if (retryCounter == -1) { LOG.trace("Starting to poll: {}", this.getEndpoint()); } else { LOG.debug("Retrying attempt {} to poll: {}", retryCounter, this.getEndpoint()); } this.polling = true; try { boolean begin = this.pollStrategy.begin(this, this.getEndpoint()); if (begin) {++retryCounter; polledMessages = this.poll(); LOG.trace("Polled {} messages", polledMessages); if (polledMessages == 0 && this.isSendEmptyMessageWhenIdle()) { this.processEmptyMessage(); } this.pollStrategy.commit(this, this.getEndpoint(), polledMessages); if (polledMessages > 0 && this.isGreedy()) { done = false; retryCounter = -1; LOG.trace("Greedy polling after processing {} messages", polledMessages); this.errorCounter = 0L; this.lastError = null; this.lastErrorDetails = null; this.firstPollDone = true; } } else { LOG.debug("Cannot begin polling as pollStrategy returned false: {}", this.pollStrategy); } } finally { this.polling = false; } } LOG.trace("Finished polling: {}", this.getEndpoint()); `} catch (Exception var17)` { Exception e = var17; try { boolean retry = this.pollStrategy.rollback(this, this.getEndpoint(), retryCounter, e); if (retry) { done = false; } else { cause = e; done = true; } } catch (Exception var16) { cause = var16; done = true; } } if (cause != null && this.isRunAllowed()) { try {this.getExceptionHandler().handleException("Failed polling endpoint: " + this.getEndpoint() +". Will try again at next poll", cause); } catch (Exception var15) { LOG.warn("Error handling exception. This exception will be ignored.", var15); } } } if (cause != null) { this.idleCounter = 0L; this.successCounter = 0L;++this.errorCounter; this.lastError = cause; if (cause instanceof HttpResponseAware) { int code = ((HttpResponseAware)cause).getHttpResponseCode(); if (code > 0) { this.addLastErrorDetail("http.response.code", code); } } } else { this.idleCounter = polledMessages == 0 ? ++this.idleCounter : 0L;++this.successCounter; this.errorCounter = 0L; this.lastError = null; this.lastErrorDetails = null; } this.firstPollDone = true; LOG.trace("doRun() done with idleCounter={}, successCounter={}, errorCounter={}", new Object[]{this.idleCounter, this.successCounter, this.errorCounter}); } } }one of the most important points I think is here
boolean retry = this.pollStrategy.rollback(this, this.getEndpoint(), retryCounter, e);This influences whether the handler is run later or not.
this.getExceptionHandler().handleException("Failed polling endpoint: " + this.getEndpoint() +". Will try again at next poll", cause);But the above rollback always tries to force a disconnect and return a default value of false from the super() call
RemoteFilePollingConsumerPollStrategypublic boolean rollback(Consumer consumer, Endpoint endpoint, int retryCounter, Exception e) throws Exception { if (consumer instanceof RemoteFileConsumer<?> rfc) { if (rfc.isRunAllowed()) { if (this.log.isWarnEnabled()) { this.log.warn("Trying to recover by force disconnecting from remote server and re-connecting at next poll: {}", rfc.remoteServer()); } try { rfc.forceDisconnect(); } catch (Exception var7) { if (this.log.isDebugEnabled()) { this.log.debug("Error occurred during force disconnecting from: {}. This exception will be ignored.", rfc.remoteServer(), var7); } } } } return super.rollback(consumer, endpoint, retryCounter, e); }Reading the documentation about "reprovisioning" I came across an explanation that it was created for temporary problems, e.g. with the database or IO operations, where retrying usually solves the situation. Based on this I do not understand why in this case I cannot make another attempt to list, i.e. connect and fetch the data?
My expectations are based on catching the error in this case IOException and the possibility to retry the process to try to solve the problem only after that there should be a redirect to the handler.
It seems that RemoteFilePollingConsumerPollStrategy imposes a rigid behavior and I don't really know how I can override it to make it useful or maybe I don't understand something with this flow but when I throw exception from my Route it works as I expect, it was repeated 3 times and then it was processed by custom method