gRPC On Kubernetes With Linkerd

vIns

4 years ago

Overview:

In this tutorial, I would like to show you gRPC On Kubernetes with Linkerd – to demonstrate how we could load balance the gRPC requests on Kubernetes.

If you are new to gRPC, take a look at these articles.

gRPC On Kubernetes:

gRPC is a great choice for client/server application development or inter-Microservices communication. It depends on HTTP/2 by default which maintains a persistent connection between the client and the server. Even though gRPC provides very good performance, load balancing gRPC requests becomes a problem because of the persistent connection.

To understand the issue better, Let’s consider a simple client / server application. We have 3 instances of the server application (Pods) running on a Kubernetes cluster. When the client sends a request to the Kubernetes service we can see that the load is never distributed across all the Pods. It will always go to 1 single pod. Kubernetes service is connection oriented. So it can NOT balance the gRPC requests.

Lets see how we could solve this problem by using Linkerd Service Mesh.

Sample Application:

Lets consider an application in which the user wants to find the squares up to N number.

We will have 2 services.

grpc-compute-service: It can find the square of a given number.
aggregator-service: As the user (browser) wants all the squares up to N number, aggregator-service will send multiple requests from 1 to N to grpc-compute-service.

For ex: If we send a request for 3 to the aggregator-service, we will get result as shown here. The hostname field will contain the value of the actual pod instance which processed the request.

[
   {
      "input":1,
      "result":1,
      "hostname":"hostname"
   },
   {
      "input":2,
      "result":4,
      "hostname":"hostname"
   },
   {
      "input":3,
      "result":9,
      "hostname":"hostname"
   }
]

Project Setup:

I create a Spring Boot multi module maven project as shown here. (For the project dependencies, please check the Github link at the end of this article).

My proto file looks like this.

message Input {
  int32 number = 1;
}

message Output {
  int32 number = 1;
  int32 result = 2;
  string host = 3;
}

service SquareService {
  rpc findSquareUnary(Input) returns (Output) {};
}

gRPC Service:

@GrpcService
public class SquareService extends SquareServiceGrpc.SquareServiceImplBase {

    @Override
    public void findSquareUnary(Input request, StreamObserver<Output> responseObserver) {
        var number = request.getNumber();
        Output output = Output.newBuilder()
                .setNumber(number)
                .setResult(number * number)
                .setHost(getHostName())
                .build();
        responseObserver.onNext(output);
        responseObserver.onCompleted();
    }

    private String getHostName(){
        try {
            return InetAddress.getLocalHost().getHostName();
        } catch (UnknownHostException e) {
            e.printStackTrace();
        }
        return null;
    }

}

Aggregator:

@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ResultDto {

    private int input;
    private int result;
    private String hostname;

}

service – it has to send N requests to the backend grpc service as shown here.

@Service
public class GrpcSquareService {

    @GrpcClient("square")
    private SquareServiceGrpc.SquareServiceBlockingStub blockingStub;

    public Flux<ResultDto> getSquareResponseUnary(int number){
        return Flux.range(1, number)
                .map(i -> Input.newBuilder().setNumber(i).build())
                .map(i -> this.blockingStub.findSquareUnary(i))
                .map(o -> ResultDto.of(o.getNumber(), o.getResult(), o.getHost()))
                .subscribeOn(Schedulers.boundedElastic());
    }

}

rest controller – to receive the requests from the browser.

@RestController
@RequestMapping("grpc")
public class SquareServiceController {

    @Autowired
    private GrpcSquareService service;

    @GetMapping("/{number}")
    public Flux<ResultDto> getResponseUnary(@PathVariable int number){
        return this.service.getSquareResponseUnary(number);
    }

}

gRPC On Kubernetes – Deployment:

Once the above set up is done, I dockerized the applications. I built 2 docker images and I have pushed them into docker hub. (You can directly pull them to play with this.)

vinsdocker/square-app
vinsdocker/aggregator-app

Now lets deploy this app on a Kubernetes cluster.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: square-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: square-app
  template:
    metadata:
      labels:
        app: square-app
    spec:
      containers:
        - name: square-app
          image: vinsdocker/square-app
---
apiVersion: v1
kind: Service
metadata:
  name: square-service
spec:
  selector:
    app: square-app
  ports:
    - port: 6565
      protocol: TCP
      targetPort: 6565
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: aggregator-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: aggregator-app
  template:
    metadata:
      labels:
        app: aggregator-app
    spec:
      containers:
        - name: aggregator-app
          image: vinsdocker/aggregator-app
          env:
            - name: GRPC_CLIENT_SQUARE_ADDRESS
              value: static://square-service:6565
---
apiVersion: v1
kind: Service
metadata:
  name: aggregator-service
spec:
  selector:
    app: aggregator-app
  ports:
    - port: 8080
      protocol: TCP
      targetPort: 8080

Save the above content in a deployment.yaml file. Run this command to deploy.

kubectl apply -f deployment.yaml

As you see we are running 3 instances of the square app.

Lets expose the aggregator service outside Kubernetes cluster to access.

kubectl port-forward service/aggregator-service 8080

Lets send a request.

curl http://localhost:8080/grpc/5

This is the output I get. (Send the curl request multiple times. We will still see same output)

[
   {
      "input":1,
      "result":1,
      "hostname":"square-app-7cfffb48d9-r77vg"
   },
   {
      "input":2,
      "result":4,
      "hostname":"square-app-7cfffb48d9-r77vg"
   },
   {
      "input":3,
      "result":9,
      "hostname":"square-app-7cfffb48d9-r77vg"
   },
   {
      "input":4,
      "result":16,
      "hostname":"square-app-7cfffb48d9-r77vg"
   },
   {
      "input":5,
      "result":25,
      "hostname":"square-app-7cfffb48d9-r77vg"
   }
]

The gRPC requests are always routed to the same pod!

gRPC On Kubernetes With Linkerd:

Linkerd is a light weight service mesh. It is gRPC aware and it could load balance gRPC requests just out of the box without any change in the app / deployment yaml.

Installing Linkerd is just a 5 mins work. Detailed steps are here.

Once you have Linkerd setup in your cluster, Run this command.

linkerd inject deployment.yaml| kubectl apply -f -

Now if we send the same CURL request, we could see the requests are being balanced automatically by Linkerd proxy side car containers.

[
   {
      "input":1,
      "result":1,
      "hostname":"square-app-6b8449f5b4-hdbj5"
   },
   {
      "input":2,
      "result":4,
      "hostname":"square-app-6b8449f5b4-857px"
   },
   {
      "input":3,
      "result":9,
      "hostname":"square-app-6b8449f5b4-lvht9"
   },
   {
      "input":4,
      "result":16,
      "hostname":"square-app-6b8449f5b4-857px"
   },
   {
      "input":5,
      "result":25,
      "hostname":"square-app-6b8449f5b4-hdbj5"
   }
]

Linkerd adds lightweight side-car proxy containers for every pod in our app as shown below which distribute the gRPC requests.

Summary:

We were able to successfully demonstrate load balancing gRPC Requests On Kubernetes with Linkerd.