Overview
In this tutorial, I would like to demo Circuit Breaker Pattern, one of the Microservice Design Patterns for designing highly resilient Microservices using a library called resilience4j along with Spring Boot.
This article assumes you are familiar with Retry Pattern – Microservice Design Patterns.
Need For Resiliency
Microservices are distributed in nature. When you work with distributed systems, always remember this number one rule – anything could happen. We might be dealing with network issues, service unavailability, application slowness etc. An issue with one system might affect another system behavior/performance. Dealing with any such unexpected failures/network issues could be difficult to solve.
Ability of the system to recover from such failures and remain functional makes the system more resilient. It also avoids any cascading failures to the downstream services.
Circuit Breaker Pattern
In Microservice architecture, when there are multiple services (A, B, C & D), one service (A) might depend on the other service (B) which in turn might depend on C and so on. Sometimes due to some issue, Service D might not respond as expected. Service D might have thrown some exception like OutOfMemory Error or Internal Server Error. Such exceptions are cascaded to the downstream services which might result in poor user experience as shown below.
We have Retry Pattern in which we could retry couple of times until we get the proper response. The problem with this retry approach is – if it was an intermittent network issue, then it makes sense in retrying! What if it was an app issue?
Let’s consider below architecture in which Service B depends on Service C which has an issue. It is not behaving correctly. With this Retry Pattern, Service B will be sending multiple requests again and again to Service C assuming it will work. When there are hundreds of concurrent requests are sent to Service B which in turn would be bombarding Service C with Retry requests!
This is where Circuit Breaker Pattern helps us! When the requests to Service B are continuously failing, what is the point for sending the requests to Service C? Circuit Breaker simply skips the calls to Service C and goes with the fall back method / default values instead for certain duration which is configurable. That is, it does not bombard Service C continuously. It gives Service C sometime to recover from failure. Circuit Breaker Pattern retries after some time and so on.
Circuit Breaker Pattern – State
Circuit Breaker maintains some states and its behavior would change depends on the state.
| State | Description |
|---|---|
| CLOSED | Dependent service (Service C) is up. Requests for Service C is allowed. |
| OPEN | Dependent service is unavailable / Error rate is beyond threshold. Requests for Service C is skipped |
| HALF_OPEN | Once the state becomes OPEN, We do wait for sometime in the OPEN state. After certain duration, the state becomes HALF_OPEN. During this period, we do send some requests to Service C to check if we still get the proper response. If the failure rate is below the threshold, the state would become CLOSED. If the failure rate is above the threshold, then the state becomes OPEN once again. This cycle continues. |
Sample Application
Let’s consider this simple application to explain this Circuit Breaker Pattern.
- We have multiple Microservices as shown above
- Product service acts as product catalog and responsible for providing product information
- Product service depends on the rating service.
- Rating service maintains product reviews and ratings. It is notorious for being slow and throwing random errors.
- Whenever we look at the product details, product service sends the request to the rating service to get the reviews for the product.
- We have other services like account-service, order-service and payment-service etc which is not relevant to this article discussion.
- Product service is a core service without which the user can not start the order workflow.
Project Set Up
Lets first create a Spring Boot project with these dependencies.
We also need this dependency.
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-spring-boot3</artifactId>
<version>...</version>
</dependency>This will be a multi-module maven project as shown below.
If the user tries to see a product, let’s say product id 1, then the product-service is expected to respond like this by fetching the ratings as well.
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 4.5,
"reviews": [
{
"userFirstname": "vins",
"userLastname": "guru",
"productId": 1,
"rating": 5,
"comment": "excellent"
},
{
"userFirstname": "marshall",
"userLastname": "mathers",
"productId": 1,
"rating": 4,
"comment": "decent"
}
]
}
}Common-DTO
As we have couple of services which are going to share the DTOs among them, Lets keep them as a separate module. This module will contain below classes.
- Review
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ReviewDto {
private String userFirstname;
private String userLastname;
private int productId;
private int rating;
private String comment;
}- Product Rating
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ProductRatingDto {
private double avgRating;
private List<ReviewDto> reviews;
}- Product
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ProductDto {
private int productId;
private String description;
private double price;
private ProductRatingDto productRating;
}Rating Service
This service is responsible for maintaining all the product reviews. To keep things simple, I am going to use a simple Map as data base here.
- Service class
@Service
public class RatingService {
private Map<Integer, ProductRatingDto> map;
@PostConstruct
private void init(){
// product 1
ProductRatingDto ratingDto1 = ProductRatingDto.of(4.5,
List.of(
ReviewDto.of("vins", "guru", 1, 5, "excellent"),
ReviewDto.of("marshall", "mathers", 1, 4, "decent")
)
);
// product 2
ProductRatingDto ratingDto2 = ProductRatingDto.of(4,
List.of(
ReviewDto.of("slim", "shady", 2, 5, "best"),
ReviewDto.of("fifty", "cent", 2, 3, "")
)
);
// map as db
this.map = Map.of(
1, ratingDto1,
2, ratingDto2
);
}
public ProductRatingDto getRatingForProduct(int productId) {
return this.map.getOrDefault(productId, new ProductRatingDto());
}
}- Controller
- If you take a look at our controller, It seems to fail at random.
- It can produce valid response in certain cases
- It responds with Internal server error in certain cases
@RestController
@RequestMapping("ratings")
public class RatingController {
@Autowired
private RatingService ratingService;
@GetMapping("{prodId}")
public ResponseEntity<ProductRatingDto> getRating(@PathVariable int prodId) throws InterruptedException {
ProductRatingDto productRatingDto = this.ratingService.getRatingForProduct(prodId);
return this.failRandomly(productRatingDto);
}
private ResponseEntity<ProductRatingDto> failRandomly(ProductRatingDto productRatingDto) throws InterruptedException {
// simulate delay
Thread.sleep(100);
// simulate failure
int random = ThreadLocalRandom.current().nextInt(1, 4);
if(random < 3)
return ResponseEntity.status(500).build();
return ResponseEntity.ok(productRatingDto);
}
}Product Service
Product service is responsible for providing list of products based on the user search criteria. It is one of the core services which should be up & responsive even under critical load. If it is down, it will have a severe impact on the revenue. Since this service depends on rating-service, we do not want any network issues or rating-service unavailability affect this product-service. This is where resilience4j library comes into picture.
- Configuration
- I first create a configuration for resilience4j as shown below.
- We can have multiple service configuration as shown below.
- I use the COUNT_BASED sliding window in which we track the given number of requests.
- failureRateThreshold: I also set the failure threshold as 60 which means if the 60 requests are failing in the 100 requests, keep the circuit breaker in CLOSED state.
- waitDurationInOpenState: When the Circuit breaker is in OPEN state, wait for 10 seconds before sending any more requests to rating service.
- You can have default configuration and use that for all the services, override values for specific services as shown below.
resilience4j.circuitbreaker:
configs:
default:
slidingWindowType: COUNT_BASED
slidingWindowSize: 100
permittedNumberOfCallsInHalfOpenState: 10
waitDurationInOpenState: 10
failureRateThreshold: 60
recordExceptions:
- org.springframework.web.client.HttpServerErrorException
instances:
ratingService:
baseConfig: default
someOtherService:
registerHealthIndicator: true
slidingWindowSize: 10
permittedNumberOfCallsInHalfOpenState: 3
slidingWindowType: TIME_BASED
minimumNumberOfCalls: 20
waitDurationInOpenState: 50s
failureRateThreshold: 50
eventConsumerBufferSize: 10- Product entity
@Data
@AllArgsConstructor(staticName = "of")
public class Product {
private int productId;
private String description;
private double price;
}- This product-service acts as a client for the rating-service.
- @CircuitBreaker indicates that resilience4j will apply circuit breaker logic for this method execution.
- name=ratingService indicates that resilience4j will use the configuration for ratingService in the yaml.
- fallbackMethod is used when the main method fails for some reason
@Service
@AllArgsConstructor
public class RatingServiceClient {
private static final Logger log = LoggerFactory.getLogger(RatingServiceClient.class);
private final RestClient client;
private final ExecutorService executorService;
@CircuitBreaker(name = "ratingService", fallbackMethod = "onError")
public CompletionStage<ProductRatingDto> getProductRatingDto(int productId) {
return CompletableFuture.supplyAsync(() -> this.getRating(productId), executorService);
}
private ProductRatingDto getRating(int productId){
return this.client.get()
.uri("{productId}", productId)
.retrieve()
.body(ProductRatingDto.class);
}
private CompletionStage<ProductRatingDto> onError(int productId, Throwable throwable) {
log.error("error: {}", throwable.getMessage());
return CompletableFuture.completedFuture(ProductRatingDto.of(0, Collections.emptyList()));
}
}- Product service
@Service
@AllArgsConstructor
public class ProductService {
private final RatingServiceClient ratingServiceClient;
private final ExecutorService executorService;
// assume this would be DB in real life
private Map<Integer, Product> db;
@PostConstruct
private void init(){
this.db = Map.of(
1, Product.of(1, "Blood On The Dance Floor", 12.45),
2, Product.of(2, "The Eminem Show", 12.12)
);
}
public CompletableFuture<ProductDto> getProductDto(int productId){
// assuming this is a DB call
var product = CompletableFuture.supplyAsync(() -> this.db.get(productId), executorService);
var rating = this.ratingServiceClient.getProductRatingDto(productId);
return product.thenCombine(rating, (p, r) -> ProductDto.of(productId, p.getDescription(), p.getPrice(), r));
}
}- Controller
@RestController
@RequestMapping("product")
public class ProductController {
@Autowired
private ProductService productService;
@GetMapping("{productId}")
public CompletableFuture<ProductDto> getProduct(@PathVariable int productId){
return this.productService.getProductDto(productId);
}
}Circuit Breaker Pattern – Demo
All the services are ready. Start both product-service and rating-service. Let’s access below end point.
http://localhost:8080/product/1- Case 1: When the rating-service works perfectly fine.
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 4.5,
"reviews": [
{
"userFirstname": "vins",
"userLastname": "guru",
"productId": 1,
"rating": 5,
"comment": "excellent"
},
{
"userFirstname": "marshall",
"userLastname": "mathers",
"productId": 1,
"rating": 4,
"comment": "decent"
}
]
}
}- Case 2: When the rating-service responds with bad request response
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 0.0,
"reviews": []
}
}Performance Test
The real benefit of adding Circuit Breaker is NOT the functionality, but the Performance! To demo that I have created a simple JMeter test script with 100 concurrent users which will be sending requests to product-service for 1 min.
Note: We can also enable both Retry and Circuit Breaker as shown below.
@Retry(name = "ratingService", fallbackMethod = "onError")
@CircuitBreaker(name = "ratingService", fallbackMethod = "onError")
public CompletionStage<ProductRatingDto> getProductRatingDto(int productId) {
return CompletableFuture.supplyAsync(() -> this.getRating(productId), executorService);
}- Case 1: Retry only. No Circuit Breaker
- Case 2: Circuit Breaker only. No Retry.
- Case 3: Circuit Breaker + Retry
Results:
From the results, Circuit Breaker clearly helps us with increasing performance as we skip unnecessary calls as it would fail anyway!
| Test | Average Response Time (Seconds) | Throughput (Requests/Second) |
|---|---|---|
| Retry | 3.153 | 27.6 |
| Circuit Breaker | 0.478 | 186.2 |
| Retry + Circuit Breaker |
0.527 | 169.2 |
Summary
We were able to successfully demonstrate the Circuit Breaker Patter which is one of the resilient Microservice Design Patterns. Read more about other Resilient Microservice Design Patterns.
- Timeout Pattern – Microservice Design Patterns
- Retry Pattern – Microservice Design Patterns
- Bulkhead Pattern – Microservice Design Patterns
- Rate Limiter Pattern – Microservice Design Patterns
The source code is available here.
Happy learning 🙂
