I have an API client class built using _client = new RestSharp.RestClient(...);
with methods of this form:
public async Task<TResponse> PostAsync<TRequest, TResponse>(string resource, TRequest payload) where TRequest : class{ var request = new RestRequest(resource); request.AddJsonBody(payload); var response = await PolicyFactory.RestSharpPolicy<B2BResponse<TResponse>>(_logger).ExecuteAsync( async token => await _client.ExecuteAsync<B2BResponse<TResponse>>(request, Method.Post, token)); LogAndRaiseErrors(request, response); return response.Data.Data;}
Here's the LogAndRaiseErrors method:
protected void LogAndRaiseErrors(RestRequest request, RestResponse response){ if (response.StatusCode != HttpStatusCode.OK) { _logger.LogError("API Error:{response}", SerializeResponse(response)); throw new BigCommerceException(response.Content); } _logger.LogDebug("B2B API Call:\n{response}", SerializeResponse(response));}
I've had a look at the Polly documentation, but it is a bit sparse.
How would I construct the ResiliencePipeline used in the PostAsync
method to achieve the following primary goals:
- Read the
response.Headers.FirstOrDefault(h=>h.Name?.ToLower() == "retry-after")?.Value
and delay for the specified seconds, which the existing code does not do at all. - I Believe retry-after should affect all threads which may be calling the given API? add Circuit Breaker?
Any pointers on secondary goals would be great:
- Log every request (currently done in LogAndRaiseErrorsMethod)
- Log when retries occur including the delay specified in the header.
- Move the
LogAndRaiseErrors(...)
into the Policy so the policy can deal with logging and any exceptions - Add Policy to retry for network failures/timeouts etc.
- I suspect we need some jitter (is that what that's for?) so not all threads pile on the API at once?
UPDATE here's what I tried:
I think I have retry working correctly, but not circuit breaker.I moved pipeline creation to the constructor so there is only one pipeline (logging shows it's called once)
public ApiClient(..., ILogger<B2BClient> logger){ ... _client = client; _resiliencePipeline = PolicyFactory.GetRestSharpPolicy(_logger);}public async Task<TResponse> GetAsync<TResponse>(string resource){...}public async Task<TResponse> PostAsync<TRequest, TResponse>(string resource, TRequest payload) where TRequest : class{ var request = new RestRequest(resource); request.AddJsonBody(payload); var response = await _resiliencePipeline.ExecuteAsync( async token => await _client.ExecuteAsync<BigCommerceResponse<TResponse>>(request, Method.Post, token)); LogAndRaiseErrors(request, response); return response.Data.Data;}
Here's my pipeline creation, but I never see any Circuit Breaker log entries, but I do see a lot of Retry log entries for Retry.
public static class PolicyFactory{ public static ResiliencePipeline<RestResponse> GetRestSharpPolicy(ILogger logger) { logger.LogInformation("Building ResiliencePipeline"); return new ResiliencePipelineBuilder<RestResponse>() .AddCircuitBreaker(new CircuitBreakerStrategyOptions<RestResponse> { FailureRatio = 0, ShouldHandle = new PredicateBuilder<RestResponse>() .HandleResult(static result => result.StatusCode == HttpStatusCode.TooManyRequests), OnOpened = args => { logger.LogWarning("Circuit Breaker Opened on {StatusCode} for {Duration}s ({ResponseUri})", args.Outcome.Result.StatusCode, args.BreakDuration.TotalSeconds, args.Outcome.Result.ResponseUri); return ValueTask.CompletedTask; }, OnClosed = args => { logger.LogWarning("Circuit Breaker Closed on {StatusCode} ({ResponseUri})", args.Outcome.Result.StatusCode, args.Outcome.Result.ResponseUri); return ValueTask.CompletedTask; } }) .AddRetry(new RetryStrategyOptions<RestResponse> { ShouldHandle = new PredicateBuilder<RestResponse>() .HandleResult(static result => result.StatusCode == HttpStatusCode.TooManyRequests), DelayGenerator = delayArgs => { var retryAfter = delayArgs.Outcome.Result.Headers.FirstOrDefault(h => h.Name?.ToLower() == "retry-after")?.Value.ToString(); return int.TryParse(retryAfter, out var seconds) ? new ValueTask<TimeSpan?>(TimeSpan.FromSeconds(seconds)) : new ValueTask<TimeSpan?>(TimeSpan.FromSeconds(0.5)); }, MaxRetryAttempts = 5, OnRetry = args => { logger.LogWarning("Retry Attempt:{Attempt} Delay:{Delay}", args.AttemptNumber, args.RetryDelay); return ValueTask.CompletedTask; } }) .Build(); }}