Scala Functional retries
Systems tend to fail for various reasons and handling these failures due to intermittent failures is very important in order for system to recover from failure and heal by retrying.
In this article, I have tried to capture on how to implement a functional retry mechanism in Scala composed with retry strategies.
Following are the important elements for retry:
Exception Handling
Categorising exception into retry-able and non-retry-able. When applications fails due to exception like OOM, recovering JVM is going to be nearly impossible, in those cases retry mechanisms is not worthy.
When system fails with 5XX error when attempting to make external service calls, retries will be crucial to give it another try. As application developer we need to classify exceptions into allowable exceptions and disallowed and retry upon those failures.
Exception Processor
Once application throws exception, application developer must need an exception processor to process exception based on exception. For instance when and making HTTP service call if in retry client has to be reinitialised or retry on different endpoint or release existing resources on filesystem, processor handles them.
Retry policy
Wrapping up all of these into a builder with elements like type of retry mechanism.
Retry strategies: LinerBackOffStrategy, ExponentialBackOffStrategy, FixedDelayBackOffStrategy
Jitter: Jitter (randomised delay) to prevent successive collisions, to avoid multiple micro-services retrying at same time. Jitters adds finite delay to next retry attempt.
Max_Attempts: Maximum number of attempts before exhausting.
Duration: Duration until next retry.
Retry Handler
Retry handler executes task and in case of exception, processes exception and validates if its retryable exception, if yes, then it increments retry attempt and processes exception, which can be block of code to recreate your connection pool etc.
If exception is fatal, then retrying would not be worthy hence system degrades gracefully.
That is it! Thank you for reading.
