Bulkhead Pattern with Spring Boot

Overview

In this tutorial, I would like to demo Bulkhead Pattern, one of the Microservice Design Patterns for designing highly resilient Microservices using a library resilience4j along with Spring Boot.

Need For Resiliency

Microservices are distributed in nature. When you work with distributed systems, always remember this number one rule – anything could happen. We might be dealing with network issues, service unavailability, application slowness etc. An issue with one system might affect another system behavior/performance. Dealing with any such unexpected failures/network issues could be difficult to solve.

Ability of the system to recover from such failures and remain functional makes the system more resilient. It also avoids any cascading failures to the downstream services.

Bulkhead Pattern

A ship is split into small multiple compartments using Bulkheads. Bulkheads are used to seal parts of the ship to prevent entire ship from sinking in case of flood. Similarly failures should be expected when we design software. The application should be split into multiple components and resources should be isolated in such a way that failure of one component is not affecting the other.

 

For example: Lets assume that there are 2 services A and B. Service A has very limited resources (say 5 threads). It can process only 5 concurrent requests with its available threads. Service A has 2 sets of APIs as shown below.

• /a/b – this depends on Service B which is slow sometimes.
• /a – this does not depend on Service B

When there are multiple concurrent requests to Service A, say 10, 5 of them are for endpoint /a/b and 5 of them are for endpoint /a, there is a chance that Service A might use all its threads to work on the requests for /a/b and block all the 5 threads. Even though the remaining requests are for /a which does not have any other service dependency, Service A does not have free threads to work on the requests (resource exhaustion)! This behavior will affect overall performance of the application and might cause poor user experience. Service B slowness indirectly affects Service A performance as well.

Bulkhead Pattern helps us to allocate limit the resources which can be used for specific services. So that resource exhaustion can be reduced. For example, in the above scenario, we limit the max number of threads which can be used for /a/b is 2. So, we have always some resources to process both endpoints.

Sample Application

Let’s consider this simple application to explain Bulkhead Pattern.

• We have 2 Microservices as shown above.
• Product service acts as product catalog and responsible for providing product information.
• It has 2 endpoints
  • /products
  • /product/{product-id}
• /products endpoint gives all the products information
• /product/{product-id} endpoint gives specific product information with more details including ratings and reviews etc.
  • This endpoint depends on rating-service.
• Rating service maintains product reviews and ratings. It is notorious for being slow due to the huge amount of data it has.

Now let’s see how to apply Bulkhead Pattern in this architecture.

Project Set Up

Let’s first create a Spring Boot project with these dependencies.

We also need this dependency.

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-boot2</artifactId>
    <version>...</version>
</dependency>

This will be a multi-module maven project as shown below.

• common-dto
• product-service
• rating-service

If the user tries to see a product, let’s say product id 1, then the product-service is expected to respond like this by fetching the ratings as well.

{
    "productId": 1,
    "description": "Blood On The Dance Floor",
    "price": 12.45,
    "productRating": {
        "avgRating": 4.5,
        "reviews": [
            {
                "userFirstname": "vins",
                "userLastname": "guru",
                "productId": 1,
                "rating": 5,
                "comment": "excellent"
            },
            {
                "userFirstname": "marshall",
                "userLastname": "mathers",
                "productId": 1,
                "rating": 4,
                "comment": "decent"
            }
        ]
    }
}

Common-DTO

As we have couple of services which are going to share the DTOs among them, Lets keep them as a separate module. This module will contain below classes.

• Review

@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ReviewDto {

    private String userFirstname;
    private String userLastname;
    private int productId;
    private int rating;
    private String comment;

}

• Product Rating

@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ProductRatingDto {

    private double avgRating;
    private List<ReviewDto> reviews;

}

• Product

@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ProductDto {

    private int productId;
    private String description;
    private double price;
    private ProductRatingDto productRating;

}

Rating Service

This service is responsible for maintaining all the product reviews. To keep things simple, I am going to use a simple Map as data base here.

• Service class

@Service
public class RatingService {

    private Map<Integer, ProductRatingDto> map;

    @PostConstruct
    private void init(){

        // product 1
        ProductRatingDto ratingDto1 = ProductRatingDto.of(4.5,
                List.of(
                        ReviewDto.of("vins", "guru", 1, 5, "excellent"),
                        ReviewDto.of("marshall", "mathers", 1, 4, "decent")
                )
        );

        // product 2
        ProductRatingDto ratingDto2 = ProductRatingDto.of(4,
                List.of(
                        ReviewDto.of("slim", "shady", 2, 5, "best"),
                        ReviewDto.of("fifty", "cent", 2, 3, "")
                )
        );

        // map as db
        this.map = Map.of(
                1, ratingDto1,
                2, ratingDto2
        );

    }

    public ProductRatingDto getRatingForProduct(int productId) {
        return this.map.getOrDefault(productId, new ProductRatingDto());
    }

}

• Controller
  • Lets simulate some slowness using thread.sleep

@RestController
@RequestMapping("ratings")
public class RatingController {

    @Autowired
    private RatingService ratingService;

    @GetMapping("{prodId}")
    public ProductRatingDto getRating(@PathVariable int prodId) throws InterruptedException {
        Thread.sleep(3000);
        return this.ratingService.getRatingForProduct(prodId);
    }

}

Product Service

Product service is responsible for providing list of products based on the user search criteria. It is one of the core services which should be up & responsive even under critical load. If it is down, it will have a severe impact on the revenue. Since this service depends on rating-service which could be occasionally slow, we do not want to block all the resources (threads) of this product-service. This is where resilience4j library comes into picture.

• Configuration
  • I first set the max threads for the tomcat to be 15. (This is just to simulate resource exhaustion).
  • The I create a configuration for resilience4j as shown below.
  • maxConcurrentCalls: max number of concurrent calls allowed to rating-service.
  • maxWaitDuration: any additional requests will wait for the given duration. Otherwise it will go with default/fallback method.

server:
  tomcat:
    threads:
      max: 15
resilience4j.bulkhead:
  instances:
    ratingService:
      maxConcurrentCalls: 10
      maxWaitDuration: 10ms

• Product entity

@Data
@AllArgsConstructor(staticName = "of")
public class Product {

    private int productId;
    private String description;
    private double price;

}

• This product-service acts as a client for the rating-service.
  • @Bulkhead indicates that resilience4j will apply Bulkhead logic for this method execution.
  • name=ratingService indicates that resilience4j will use the configuration for ratingService in the yaml.
  • fallbackMethod is used when the main method fails for some reason

@Service
public class RatingServiceClient {

    private final RestTemplate restTemplate = new RestTemplate();

    @Value("${rating.service.endpoint}")
    private String ratingService;

    @Bulkhead(name = "ratingService", fallbackMethod = "getDefault")
    public ProductRatingDto getProductRatingDto(int productId){
        return this.restTemplate.getForEntity(this.ratingService + productId, ProductRatingDto.class)
                .getBody();
    }

    public ProductRatingDto getDefault(int productId, Throwable throwable){
        return ProductRatingDto.of(0, Collections.emptyList());
    }

}

• Product service

@Service
public class ProductService {

    private Map<Integer, Product> map;

    @Autowired
    private RatingServiceClient ratingServiceClient;

    @PostConstruct
    private void init(){
        this.map = Map.of(
                1, Product.of(1, "Blood On The Dance Floor", 12.45),
                2, Product.of(2, "The Eminem Show", 12.12)
        );
    }

    public ProductDto getProductDto(int productId){
        ProductRatingDto ratingDto = this.ratingServiceClient.getProductRatingDto(1);
        Product product = this.map.get(productId);
        return ProductDto.of(productId, product.getDescription(), product.getPrice(), ratingDto);
    }
    
    public List<ProductDto> getAllProducts(){
        return this.map.values()
                .stream()
                .map(product -> ProductDto.of(product.getProductId(), product.getDescription(), product.getPrice(), ProductRatingDto.of(0, Collections.emptyList())))
                .collect(Collectors.toList());
    }

}

• Controller
  • Lets simulate some slowness for the products endpoint as well.

@RestController
@RequestMapping
public class ProductController {

    @Autowired
    private ProductService productService;

    @GetMapping("product/{productId}")
    public ProductDto getProduct(@PathVariable int productId){
        return this.productService.getProductDto(productId);
    }

    @GetMapping("products")
    public List<ProductDto> getAllProducts() throws InterruptedException {
        Thread.sleep(50);
        return this.productService.getAllProducts();
    }

}

Bulkhead Pattern – Demo

All the services are ready. I start both product-service and rating-service.

I have created a simple JMeter test script with 20 concurrent users which will be continuously accessing these endpoints for 30 seconds.

• 15 users will be accessing this below endpoint

http://localhost:8080/product/1

• 5 users will be accessing this below endpoint

http://localhost:8080/products

I ran the the performance test and I see the results as given below.

Case 1:  When Bulkhead Pattern is NOT enabled

Case 2:  When Bulkhead Pattern is enabled

If we look at the results above, limiting resources for the rating-service with Bulkhead Pattern indirectly increases the throughput of the /products endpoint.

Summary

We were able to successfully demonstrate the Bulkhead Pattern which is one of the resilient Microservice Design Patterns.

Read more about other Resilient Microservice Design Patterns.

• Timeout Pattern – Microservice Design Patterns
• Retry Pattern – Microservice Design Patterns
• Circuit Breaker Pattern – Microservice Design Patterns
• Rate Limiter Pattern – Microservice Design Patterns

The source code is available here.

Happy learning 🙂