Overview:
In this tutorial, I would like to demo Timeout Pattern, one of the Microservice Design Patterns for designing highly resilient Microservices.
Need For Resiliency:
Microservices are distributed in nature. When you work with distributed systems, always remember this number one rule – anything could happen. We might be dealing with network issues, service unavailability, application slowness etc. An issue with one system might affect another system behavior/performance. Dealing with any such unexpected failures/network issues could be difficult to solve.
Ability of the system to recover from such failures and remain functional makes the system more resilient. It also avoids any cascading failures to the downstream services.
Timeout Pattern:
We do experience intermittent application slowness once in a while for no obvious reasons. It could have happened to all of us even for applications like google.com. In Microservice architecture, when there are multiple services (A, B, C & D), one service (A) might depend on the other service (B) which in turn might depend on C and so on. Sometimes due to some network issue, Service D might not respond as expected. This slowness could affect the downstream services – all the way up to Service A & block the threads in the individual services.
As it is not uncommon issue, It is better to take this service slowness/unavailability issues into consideration while designing your Microservices by setting a timeout for any network call. So that we could have the core services working as expected & responsive even when the dependent services are NOT available.
| Advantages |
|---|
| Make the core services work always even when the dependent services are not available |
| We do not want to wait indefinitely |
| We do not want to block any threads |
| To handle network related issues and make the system remain functional using some cached responses. |
Sample Application:
Lets consider this simple application to explain this timeout pattern.
- We have multiple microservices as shown above
- Product service acts as product catalog and responsible for providing product information
- Product service depends on the rating service.
- Rating service maintains product reviews and ratings. It is notorious for being slow due to the huge amount of data it has.
- Whenever we look at the product details, product service sends the request to the rating service to get the reviews for the product.
- We have other services like account-service, order-service and payment-service etc which is not relevant to this article discussion.
- Product service is a core service without which the user can not start the order workflow.
Project Set Up:
Lets first create a Spring Boot project with these dependencies.
We also need this dependency.
<dependency>
<groupId>io.github.resilience4j</groupId>
<artifactId>resilience4j-spring-boot2</artifactId>
<version>1.6.1</version>
</dependency>This will be a multi-module maven project as shown below.
If the user tries to see a product, let’s say product id 1, then the product-service is expected to respond like this by fetching the ratings as well.
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 4.5,
"reviews": [
{
"userFirstname": "vins",
"userLastname": "guru",
"productId": 1,
"rating": 5,
"comment": "excellent"
},
{
"userFirstname": "marshall",
"userLastname": "mathers",
"productId": 1,
"rating": 4,
"comment": "decent"
}
]
}
}Common-DTO:
As we have couple of services which are going to share the DTOs among them, Lets keep them as a separate module. This module will contain below classes.
- Review
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ReviewDto {
private String userFirstname;
private String userLastname;
private int productId;
private int rating;
private String comment;
}- Product Rating
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ProductRatingDto {
private double avgRating;
private List<ReviewDto> reviews;
}- Product
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ProductDto {
private int productId;
private String description;
private double price;
private ProductRatingDto productRating;
}Rating Service:
This service is responsible for maintaining all the product reviews. To keep things simple, I am going to use a simple Map as data base here.
- Service class
@Service
public class RatingService {
private Map<Integer, ProductRatingDto> map;
@PostConstruct
private void init(){
// product 1
ProductRatingDto ratingDto1 = ProductRatingDto.of(4.5,
List.of(
ReviewDto.of("vins", "guru", 1, 5, "excellent"),
ReviewDto.of("marshall", "mathers", 1, 4, "decent")
)
);
// product 2
ProductRatingDto ratingDto2 = ProductRatingDto.of(4,
List.of(
ReviewDto.of("slim", "shady", 2, 5, "best"),
ReviewDto.of("fifty", "cent", 2, 3, "")
)
);
// map as db
this.map = Map.of(
1, ratingDto1,
2, ratingDto2
);
}
public ProductRatingDto getRatingForProduct(int productId) {
return this.map.getOrDefault(productId, new ProductRatingDto());
}
}- Controller
- Simulating random slowness with Thread.sleep
@RestController @RequestMapping("ratings") public class RatingController { @Autowired private RatingService ratingService; @GetMapping("{prodId}") public ProductRatingDto getRating(@PathVariable int prodId) throws InterruptedException { Thread.sleep(ThreadLocalRandom.current().nextInt(10, 10000)); return this.ratingService.getRatingForProduct(prodId); } }
Product Service:
Product service is responsible for providing list of products based on the user search criteria. It is one of the core services which should be up & responsive even under critical load. If it is down, it will have a severe impact on the revenue. Since this service depends on rating-service, we do not want any network issues or rating-service unavailability affect this product-service. This is where resilience4j library comes into picture.
- Configuration
- I first create a configuration for resilience4j as shown below. Here we explicitly set the timeout as 3 seconds. We can add multiple services with specific time out.
resilience4j.timelimiter:
instances:
ratingService:
timeoutDuration: 3s
cancelRunningFuture: true
someOtherService:
timeoutDuration: 1s
cancelRunningFuture: false
---
rating:
service:
endpoint: http://localhost:7070/ratings/- Product entity
@Data
@AllArgsConstructor(staticName = "of")
public class Product {
private int productId;
private String description;
private double price;
}- This product-service acts as a client for the rating-service.
- @TimeLimiter indicates that resilience4j will apply timeout for this method execution.
- name=ratingService indicates that resilience4j will use the configuration for ratingService in the yaml.
- fallbackMethod is used when the main method fails for some reason.
@Service
public class RatingServiceClient {
private final RestTemplate restTemplate = new RestTemplate();
@Value("${rating.service.endpoint}")
private String ratingService;
@TimeLimiter(name = "ratingService", fallbackMethod = "getDefault")
public CompletionStage<ProductRatingDto> getProductRatingDto(int productId){
Supplier<ProductRatingDto> supplier = () ->
this.restTemplate.getForEntity(this.ratingService + productId, ProductRatingDto.class)
.getBody();
return CompletableFuture.supplyAsync(supplier);
}
private CompletionStage<ProductRatingDto> getDefault(int productId, Throwable throwable){
return CompletableFuture.supplyAsync(() -> ProductRatingDto.of(0, Collections.emptyList()));
}
}- Product service
@Service
public class ProductService {
private Map<Integer, Product> map;
@Autowired
private RatingServiceClient ratingServiceClient;
@PostConstruct
private void init(){
this.map = Map.of(
1, Product.of(1, "Blood On The Dance Floor", 12.45),
2, Product.of(2, "The Eminem Show", 12.12)
);
}
public CompletionStage<ProductDto> getProductDto(int productId){
return this.ratingServiceClient.getProductRatingDto(1)
.thenApply(productRatingDto -> {
Product product = this.map.get(productId);
return ProductDto.of(productId, product.getDescription(), product.getPrice(), productRatingDto);
});
}
}- Controller
@RestController
@RequestMapping("product")
public class ProductController {
@Autowired
private ProductService productService;
@GetMapping("{productId}")
public CompletionStage<ProductDto> getProduct(@PathVariable int productId){
return this.productService.getProductDto(productId);
}
}Demo:
All the services are ready. Start both product-service and rating-service. Lets access below end point.
http://localhost:8080/product/1- Case 1: When the rating-service works perfectly fine and responds within 3 seconds.
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 4.5,
"reviews": [
{
"userFirstname": "vins",
"userLastname": "guru",
"productId": 1,
"rating": 5,
"comment": "excellent"
},
{
"userFirstname": "marshall",
"userLastname": "mathers",
"productId": 1,
"rating": 4,
"comment": "decent"
}
]
}
}- Case 2: When the rating-service takes time to respond (more than 3 seconds).
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 0.0,
"reviews": []
}
}- Case 3: When the rating-service is down.
- Resilience4j calls the fall back method when the service is not available.
{
"productId": 1,
"description": "Blood On The Dance Floor",
"price": 12.45,
"productRating": {
"avgRating": 0.0,
"reviews": []
}
}Here the rating and reviews are empty. But that is ok as it is not critical. The product itself is not available then we will have very bad user experience and could impact the revenue.
Pros:
- In this approach, we do not block any threads indefinitely in the product service
- Any unexpected events during network call will get timed out within 3 seconds.
- Core services are not affected because of the poor performance of the dependent services.
Cons:
- We still the block the threads for 3 seconds. It could still be a problem for applications which receive many concurrent requests.
- We do the timeout only in the product-service. rating-service could still be processing the request.
Summary:
Timeout Pattern is one of the simplest Microservice Design Patterns for designing resilient Microservices. Introducing timeout solves the network related issues partially. Not fully. There are other design patterns which could handle this better along with timeout pattern.
Read more about other Resilient Microservice Design Patterns.
- Retry Pattern – Microservice Design Patterns
- Circuit Breaker Pattern – Microservice Design Patterns
- Bulkhead Pattern – Microservice Design Patterns
- Rate Limiter Pattern – Microservice Design Patterns
The source code is available here.
Happy learning 🙂
