Resilence libraries

Content:

  1. Introduction
  2. Retry
  3. Rate limit
  4. Time limit

Introduction:

Many may ask: what is a resilience library?

A resilence library is a library providing common functionality to make distributed applications more resilient to the problems that sometimes happen in distributed environments.

Common functionality include:

Resilience libraries are very similar. Probably becaus ethey have inspired each other.

We will look at 3 resilence libraries:

resilience4j was first released in 2017. It is open source under Apache license. It is a relative complex library. It is split in modules each supporting one specific functionality. It can be used in different ways - a straightforward way that works with any type of Java application and also a special Spring Boot way - and the two are totally different in nature.

polly was first released in 2016. It is open source under BSD 3 license. It is very focused on but therefore also very well integrated with the .NET async model. Microsoft has created some extensions Microsoft.Extensions.Resilience under MIT license.

pyresilience is brand new - from 2026. It is open source under MIT license. It openly states that it is inspired by resilience4j. It is very pythonic though and is much easier to use than libraries for other languages.

Retry:

This is for when your application is interacting with some service where transient failure may happen due to network glitches or a node in the service cluster crashing. Your application should retry the interaction to see if it can be successfully completed due to network being up again or successful failover has happened within service cluster.

It is desirable to be able to specify max number of retries and a time interval between retries.

RetryTest.java: 

package res;

import java.time.Duration;
import java.util.Random;

import io.github.resilience4j.core.functions.CheckedSupplier;
import io.github.resilience4j.retry.Retry;
import io.github.resilience4j.retry.RetryConfig;
import io.github.resilience4j.retry.RetryRegistry;

public class RetryTest {
    private static final Random rng = new Random();
    public static String test() throws Exception {
        if(rng.nextDouble() < 0.25) {
            System.out.println("Exception");
            throw new Exception("Somewhat expected");
        }
        return "OK";
    }
    public static void main(String[] args) throws Throwable {
        RetryConfig config = RetryConfig.custom()
                .maxAttempts(5)
                .waitDuration(Duration.ofMillis(100))
                .build();
        RetryRegistry registry = RetryRegistry.of(config);
        Retry retry = registry.retry("demo");
        CheckedSupplier<String> f = Retry.decorateCheckedSupplier(retry, RetryTest::test);
        for(int i = 0; i < 10; i++) {
            String res = f.get();
            System.out.printf("%d : %s\n", i, res);
        }
    }
}

RetryTest.java: 

package res.retry;

import java.util.Random;

import org.springframework.stereotype.Component;

import io.github.resilience4j.retry.annotation.Retry;

@Component
public class RetryTest {
    private final Random rng = new Random();
    @Retry(name = "demo")
    public String test() throws Exception {
        if(rng.nextDouble() < 0.25) {
            System.out.println("Exception");
            throw new Exception("Somewhat expected");
        }
        return "OK";
    }
}

RetryTestDriver.java: 

package res.retry;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class RetryTestDriver {
    @Autowired
    private RetryTest retry;
    public void test() throws Exception {
        for(int i = 0; i < 10; i++) {
            String res = retry.test();
            System.out.printf("%d : %s\n", i, res);
        }
    }
}

RetryTestMain.java: 

package res.retry;

import org.springframework.context.annotation.AnnotationConfigApplicationContext;

// -Dorg.slf4j.simpleLogger.defaultLogLevel=DEBUG
public class RetryTestMain {
    public static void main(String[] args) throws Throwable {
        AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext();
        ctx.scan("res.retry");
        ctx.refresh();
        RetryTestDriver d = ctx.getBean(RetryTestDriver.class);
        d.test();
        ctx.close();
    }
}

Spring config:

GeneralRetryConfig.java: 

package res.retry;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;

@Configuration
@ComponentScan("io.github.resilience4j.spring6.retry")
@EnableAspectJAutoProxy
public class GeneralRetryConfig {
}

SpecificRetryConfig.java: 

package res.retry;

import java.time.Duration;

import org.springframework.stereotype.Component;

import io.github.resilience4j.spring6.retry.configure.RetryConfigurationProperties;

@Component("retryConfigurationProperties")
public class SpecificRetryConfig extends RetryConfigurationProperties {
    public SpecificRetryConfig() {
        super();
        InstanceProperties ip = new InstanceProperties();
        ip.setMaxAttempts(5);
        ip.setWaitDuration(Duration.ofMillis(100));
        getInstances().put("demo", ip);
    }
}

Spring Boot config:

application.properties: 

resilience4j.retry.instances.demo.maxAttempts = 5
resilience4j.retry.instances.demo.waitDuration = 100ms

application.yaml: 

resilience4j:
    retry:
        instances:
            demo:
                maxAttempts: 5
                waitDuration: 100ms

RetryTest.cs: 

using System;

using Polly;
using Polly.Retry;

namespace RetryTest
{
    public class Program
    {
        private static readonly Random rng = new Random();
        public static String Test()
        {
            if(rng.NextDouble() < 0.25)
            {
                Console.WriteLine("Exception");
                throw new Exception("Somewhat expected");
            }
            return "OK";
        }
        public static void Main(string[] args)
        {
            ResiliencePipeline pl = new ResiliencePipelineBuilder().AddRetry(new RetryStrategyOptions
            {
                MaxRetryAttempts = 5,
                Delay = TimeSpan.FromMilliseconds(100)
            }).Build();
            for (int i = 0; i < 10; i++)
            {
                String res = pl.Execute(Test);
                Console.WriteLine("{0} : {1}", i, res);
            }
        }
    }
}

retry_test.py: 

from random import random

from pyresilience import resilient, RetryConfig

rng = random()

@resilient(retry=RetryConfig(max_attempts=5, delay=0.1))
def test():
    if random() < 0.25:
        print('Exception')
        raise Exception('Somewhat expected')
    return "OK"

for i in range(10):
    res = test()
    print('%d : %s' % (i, res))

Rate limit:

This is for when your application is interacting with some service that is limited in the volume it can handle and exceeding that limiot may have negative consequences beyond just longer response time. Your application need to limit the rate of interactions.

So basically one need to specify a max number of interactions per time unit or a max number of concurrent interactions.

There are two distinct flavors of implementation: implementations that block when reaching the limit and implementations that return error or throw exception when reaching the limit to let the caller decide what to do.

RateLimitTest.java: 

package res;

import java.time.Duration;

import io.github.resilience4j.core.functions.CheckedSupplier;
import io.github.resilience4j.ratelimiter.RateLimiter;
import io.github.resilience4j.ratelimiter.RateLimiterConfig;
import io.github.resilience4j.ratelimiter.RateLimiterRegistry;

public class RateLimitTest {
    public static String test() {
        return "OK";
    }
    public static void main(String[] args) throws Throwable {
        RateLimiterConfig config = RateLimiterConfig.custom()
                                                    .limitForPeriod(2)
                                                    .limitRefreshPeriod(Duration.ofSeconds(1))
                                                    .build();
        RateLimiterRegistry registry = RateLimiterRegistry.of(config);
        RateLimiter lim = registry.rateLimiter("demo");
        CheckedSupplier<String> f = RateLimiter.decorateCheckedSupplier(lim, RateLimitTest::test);
        for(int i = 0; i < 10; i++) {
            String res = f.get();
            System.out.printf("%d : %s\n", System.currentTimeMillis(), res);
        }
    }
}

RateLimitTest.java: 

package res.ratelimit;

import org.springframework.stereotype.Component;

import io.github.resilience4j.ratelimiter.annotation.RateLimiter;

@Component
public class RateLimitTest {
    @RateLimiter(name = "demo")
    public String test() {
        return "OK";
    }
}

RateLimitTestDriver.java: 

package res.ratelimit;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

import io.github.resilience4j.ratelimiter.RequestNotPermitted;

@Component
public class RateLimitTestDriver {
    @Autowired
    private RateLimitTest lim;
    public void test() throws Exception {
        for(int i = 0; i < 10; i++) {
            boolean done = false;
            while(!done) {
                try {
                    String res = lim.test();
                    System.out.printf("%s : %s\n", System.currentTimeMillis(), res);
                    done = true;
                } catch(RequestNotPermitted ex) {
                    Thread.sleep(1000);
                }
            }
        }
    }
}

RateLimitTestMain.java: 

package res.ratelimit;

import org.springframework.context.annotation.AnnotationConfigApplicationContext;

// -Dorg.slf4j.simpleLogger.defaultLogLevel=DEBUG
public class RateLimitTestMain {
    public static void main(String[] args) throws Throwable {
        AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext();
        ctx.scan("res.ratelimit");
        ctx.refresh();
        RateLimitTestDriver d = ctx.getBean(RateLimitTestDriver.class);
        d.test();
        ctx.close();
    }
}

Spring config:

GeneralRateLimitConfig.java: 

package res.ratelimit;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;

@Configuration
@ComponentScan("io.github.resilience4j.spring6.ratelimiter")
@EnableAspectJAutoProxy
public class GeneralRateLimitConfig {
}

SpecificRateLimitConfig.java: 

package res.ratelimit;

import java.time.Duration;

import org.springframework.stereotype.Component;

import io.github.resilience4j.spring6.ratelimiter.configure.RateLimiterConfigurationProperties;

@Component("rateLimiterConfigurationProperties")
public class SpecificRateLimitConfig extends RateLimiterConfigurationProperties {
    public SpecificRateLimitConfig() {
        super();
        InstanceProperties ip = new InstanceProperties();
        ip.setLimitForPeriod(2);
        ip.setLimitRefreshPeriod(Duration.ofSeconds(1));
        getInstances().put("demo", ip);
    }
}

Spring Boot config:

application.properties: 

resilience4j.ratelimiter.instances.demo.limitForPeriod = 2
resilience4j.ratelimiter.instances.demo.limitRefreshPeriod = 1s

application.yaml: 

resilience4j:
    ratelimiter:
        instances:
            demo:
                limitForPeriod: 2
                limitRefreshPeriod: 1s

RateLimitTest.cs: 

using System;
using System.Threading;
using System.Threading.RateLimiting;

using Polly;
using Polly.RateLimiting;

namespace RateLimitTest
{
    public class Program
    {
        public static String Test()
        {
            return "OK";
        }
        public static void Main(string[] args)
        {
            ResiliencePipeline pl = new ResiliencePipelineBuilder().AddRateLimiter(new SlidingWindowRateLimiter(new SlidingWindowRateLimiterOptions
            {
                PermitLimit = 2,
                SegmentsPerWindow = 4,
                Window = TimeSpan.FromSeconds(1),

            })).Build();
            for (int i = 0; i < 10; i++)
            {
                bool done = false;
                while(!done)
                {
                    try
                    {
                        String res = pl.Execute(Test);
                        Console.WriteLine("{0} : {1}", DateTime.Now.Ticks, res);
                        done = true;
                    }
                    catch (RateLimiterRejectedException ex)
                    {
                        Thread.Sleep(1000);
                    }
                }
            }
        }
    }
}

rate_limit_test.py: 

from time import time, sleep

from pyresilience import resilient, RateLimiterConfig, RateLimitExceededError

@resilient(rate_limiter=RateLimiterConfig(max_calls=2, period=1.0))
def test():
    return "OK"

for i in range(10):
    done = False
    while not done:
        try:
            res = test()
            done = True
        except RateLimitExceededError:
            sleep(1.0)
    print('%.3f : %s' % (time(), res))

Time limit:

This is for when your application is using a service that may be slow to respond and your application is being used by another service that cannot wait for slow responses. Your application need to return an error/exception to the caller if the called servce is too slow. This is easy if the interaction with the potential slow service has a timeout parameter - but if it does not then something is needed.

Obviously the timeout must be specified.

TimeLimitTest.java: 

package res;

import java.time.Duration;
import java.util.Random;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeoutException;

import io.github.resilience4j.timelimiter.TimeLimiter;
import io.github.resilience4j.timelimiter.TimeLimiterConfig;

public class TimeLimitTest {
    private static final Random rng = new Random();
    public static String test() {
        if(rng.nextDouble() < 0.25) {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
        return "OK";
    }
    public static void main(String[] args) throws Throwable {
        TimeLimiterConfig config = TimeLimiterConfig.custom()
                                                    .timeoutDuration(Duration.ofMillis(100))
                                                    .cancelRunningFuture(true)
                                                    .build();
        TimeLimiter lim = TimeLimiter.of("demo", config);
        //
        System.out.println("Sync:");
        for(int i = 0; i < 10; i++) {
            try {
                String res = lim.executeFutureSupplier(() -> CompletableFuture.supplyAsync(TimeLimitTest::test));
                System.out.printf("%d : %s\n", i, res);
            } catch(TimeoutException ex) {
                System.out.printf("%d : %s\n", i, ex.getMessage());
            }
        }
        //
        System.out.println("Async:");
        ScheduledExecutorService es = Executors.newScheduledThreadPool(10);
        @SuppressWarnings("unchecked")
        CompletableFuture<String>[] f = new CompletableFuture[10];
        for(int i = 0; i < 10; i++) {
            f[i] = lim.executeCompletionStage(es, () -> CompletableFuture.supplyAsync(TimeLimitTest::test))
                      .exceptionally(ex -> ex.getCause().getMessage())
                      .toCompletableFuture();
        }
        for(int i = 0; i < 10; i++) {
            String res = f[i].get();
            System.out.printf("%d : %s\n", i, res);
        }
        es.shutdown();
    }
}

TimeLimitTest.java: 

package res.timelimit;

import java.util.Random;
import java.util.concurrent.CompletableFuture;

import org.springframework.stereotype.Component;

import io.github.resilience4j.timelimiter.annotation.TimeLimiter;

@Component
public class TimeLimitTest {
    private final Random rng = new Random();
    @TimeLimiter(name = "demo")
    public CompletableFuture<String> test() {
        if(rng.nextDouble() < 0.25) {
            return CompletableFuture.supplyAsync(() -> { try { Thread.sleep(1000); } catch (InterruptedException e) { }; return "OK"; });
        } else {
            return CompletableFuture.supplyAsync(() -> { return "OK"; });
        }
    }
}

TimeLimitTestDriver.java: 

package res.timelimit;

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;

@Component
public class TimeLimitTestDriver {
    @Autowired
    private TimeLimitTest lim;
    public void test() throws Exception {
        ScheduledExecutorService es = Executors.newScheduledThreadPool(10);
        @SuppressWarnings("unchecked")
        CompletableFuture<String>[] f = new CompletableFuture[10];
        for(int i = 0; i < 10; i++) {
            f[i] = lim.test();
        }
        for(int i = 0; i < 10; i++) {
            String res = f[i].exceptionally(t -> t.getCause().getMessage()).get();
            System.out.printf("%d : %s\n", i, res);
        }
        es.shutdown();

    }
}

TimeLimitTestMain.java: 

package res.timelimit;

import org.springframework.context.annotation.AnnotationConfigApplicationContext;

// -Dorg.slf4j.simpleLogger.defaultLogLevel=DEBUG
public class TimeLimitTestMain {
    public static void main(String[] args) throws Throwable {
        AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext();
        ctx.scan("res.timelimit");
        ctx.refresh();
        TimeLimitTestDriver d = ctx.getBean(TimeLimitTestDriver.class);
        d.test();
        ctx.close();
    }
}

Spring config:

GeneralTimeLimitConfig.java: 

package res.timelimit;

import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;

@Configuration
@ComponentScan("io.github.resilience4j.spring6.timelimiter")
@EnableAspectJAutoProxy
public class GeneralTimeLimitConfig {
}

SpecificTimeLimitConfig.java: 

package res.timelimit;

import java.time.Duration;

import org.springframework.stereotype.Component;

import io.github.resilience4j.spring6.timelimiter.configure.TimeLimiterConfigurationProperties;

@Component("timeLimiterConfigurationProperties")
public class SpecificTimeLimitConfig extends TimeLimiterConfigurationProperties {
    public SpecificTimeLimitConfig() {
        super();
        InstanceProperties ip = new InstanceProperties();
        ip.setTimeoutDuration(Duration.ofMillis(100));
        getInstances().put("demo", ip);
    }
}

Spring Boot config:

application.properties: 

resilience4j.timelimiter.instances.demo.timeoutDuration = 100ms

application.yaml: 

resilience4j:
    timelimiter:
        instances:
            demo:
                timeoutDuration: 100ms

TimeLimitTest.cs: 

using System;
using System.Threading;
using System.Threading.Tasks;

using Polly;
using Polly.Timeout;

namespace TimeLimitTest
{
    public class Program
    {
        private static readonly Random rng = new Random();
        public static String Test()
        {
            if (rng.NextDouble() < 0.25)
            {
                Thread.Sleep(1000);
            }
            return "OK";
        }
        public async static Task<String> Test2(CancellationToken ct)
        {
            if (rng.NextDouble() < 0.25)
            {
                await Task.Delay(1000, ct);
            }
            return "OK";
        }
        public static void Main(string[] args)
        {
            ResiliencePipeline pl = new ResiliencePipelineBuilder().AddTimeout(new TimeoutStrategyOptions
            {
                Timeout = TimeSpan.FromMilliseconds(100)
            }).Build();
            // sync doesn't work because Thread.Sleep does not take CancellationToken
            for (int i = 0; i < 10; i++)
            {
                try
                {
                    String res = pl.Execute(Test);
                    Console.WriteLine("{0} : {1}", i, res);
                }
                catch (TimeoutRejectedException ex)
                {
                    Console.WriteLine("{0} : {1}", i, ex.Message);
                }
            }
            // async
            for (int i = 0; i < 10; i++)
            {
                try
                {
                    String res = pl.ExecuteAsync(ct => new ValueTask<string>(Test2(ct))).Result;
                    Console.WriteLine("{0} : {1}", i, res);
                }
                catch (TimeoutRejectedException ex)
                {
                    Console.WriteLine("{0} : {1}", i, ex.Message);
                }
            }
        }
    }
}

time_limit_test.py: 

from time import sleep
from random import random

from pyresilience import resilient, TimeoutConfig, ResilienceTimeoutError

@resilient(timeout=TimeoutConfig(seconds=0.1))
def test():
    if random() < 0.25:
        sleep(1.0)
    return "OK"

for i in range(10):
    try:
        res = test()
    except ResilienceTimeoutError as ex:
        res = str(ex)
    print('%d : %s' % (i, res))

Article history:

Version Date Description
1.0 June 11th 2026 Initial version

Other articles:

See list of all articles here

Comments:

Please send comments to Arne Vajhøj