Overview:
In this tutorial, I would like to show you gRPC On Kubernetes with Linkerd – to demonstrate how we could load balance the gRPC requests on Kubernetes.
If you are new to gRPC, take a look at these articles.
gRPC On Kubernetes:
gRPC is a great choice for client/server application development or inter-Microservices communication. It depends on HTTP/2 by default which maintains a persistent connection between the client and the server. Even though gRPC provides very good performance, load balancing gRPC requests becomes a problem because of the persistent connection.
To understand the issue better, Let’s consider a simple client / server application. We have 3 instances of the server application (Pods) running on a Kubernetes cluster. When the client sends a request to the Kubernetes service we can see that the load is never distributed across all the Pods. It will always go to 1 single pod. Kubernetes service is connection oriented. So it can NOT balance the gRPC requests.
Lets see how we could solve this problem by using Linkerd Service Mesh.
Sample Application:
Lets consider an application in which the user wants to find the squares up to N number.
We will have 2 services.
- grpc-compute-service: It can find the square of a given number.
- aggregator-service: As the user (browser) wants all the squares up to N number, aggregator-service will send multiple requests from 1 to N to grpc-compute-service.
For ex: If we send a request for 3 to the aggregator-service, we will get result as shown here. The hostname field will contain the value of the actual pod instance which processed the request.
[
{
"input":1,
"result":1,
"hostname":"hostname"
},
{
"input":2,
"result":4,
"hostname":"hostname"
},
{
"input":3,
"result":9,
"hostname":"hostname"
}
]Project Setup:
- I create a Spring Boot multi module maven project as shown here. (For the project dependencies, please check the Github link at the end of this article).
- My proto file looks like this.
message Input {
int32 number = 1;
}
message Output {
int32 number = 1;
int32 result = 2;
string host = 3;
}
service SquareService {
rpc findSquareUnary(Input) returns (Output) {};
}gRPC Service:
@GrpcService
public class SquareService extends SquareServiceGrpc.SquareServiceImplBase {
@Override
public void findSquareUnary(Input request, StreamObserver<Output> responseObserver) {
var number = request.getNumber();
Output output = Output.newBuilder()
.setNumber(number)
.setResult(number * number)
.setHost(getHostName())
.build();
responseObserver.onNext(output);
responseObserver.onCompleted();
}
private String getHostName(){
try {
return InetAddress.getLocalHost().getHostName();
} catch (UnknownHostException e) {
e.printStackTrace();
}
return null;
}
}Aggregator:
- dto
@Data
@NoArgsConstructor
@AllArgsConstructor(staticName = "of")
public class ResultDto {
private int input;
private int result;
private String hostname;
}- service – it has to send N requests to the backend grpc service as shown here.
@Service
public class GrpcSquareService {
@GrpcClient("square")
private SquareServiceGrpc.SquareServiceBlockingStub blockingStub;
public Flux<ResultDto> getSquareResponseUnary(int number){
return Flux.range(1, number)
.map(i -> Input.newBuilder().setNumber(i).build())
.map(i -> this.blockingStub.findSquareUnary(i))
.map(o -> ResultDto.of(o.getNumber(), o.getResult(), o.getHost()))
.subscribeOn(Schedulers.boundedElastic());
}
}- rest controller – to receive the requests from the browser.
@RestController
@RequestMapping("grpc")
public class SquareServiceController {
@Autowired
private GrpcSquareService service;
@GetMapping("/{number}")
public Flux<ResultDto> getResponseUnary(@PathVariable int number){
return this.service.getSquareResponseUnary(number);
}
}gRPC On Kubernetes – Deployment:
Once the above set up is done, I dockerized the applications. I built 2 docker images and I have pushed them into docker hub. (You can directly pull them to play with this.)
- vinsdocker/square-app
- vinsdocker/aggregator-app
Now lets deploy this app on a Kubernetes cluster.
apiVersion: apps/v1
kind: Deployment
metadata:
name: square-app
spec:
replicas: 3
selector:
matchLabels:
app: square-app
template:
metadata:
labels:
app: square-app
spec:
containers:
- name: square-app
image: vinsdocker/square-app
---
apiVersion: v1
kind: Service
metadata:
name: square-service
spec:
selector:
app: square-app
ports:
- port: 6565
protocol: TCP
targetPort: 6565
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: aggregator-app
spec:
replicas: 1
selector:
matchLabels:
app: aggregator-app
template:
metadata:
labels:
app: aggregator-app
spec:
containers:
- name: aggregator-app
image: vinsdocker/aggregator-app
env:
- name: GRPC_CLIENT_SQUARE_ADDRESS
value: static://square-service:6565
---
apiVersion: v1
kind: Service
metadata:
name: aggregator-service
spec:
selector:
app: aggregator-app
ports:
- port: 8080
protocol: TCP
targetPort: 8080Save the above content in a deployment.yaml file. Run this command to deploy.
kubectl apply -f deployment.yamlAs you see we are running 3 instances of the square app.
Lets expose the aggregator service outside Kubernetes cluster to access.
kubectl port-forward service/aggregator-service 8080Lets send a request.
curl http://localhost:8080/grpc/5This is the output I get. (Send the curl request multiple times. We will still see same output)
[
{
"input":1,
"result":1,
"hostname":"square-app-7cfffb48d9-r77vg"
},
{
"input":2,
"result":4,
"hostname":"square-app-7cfffb48d9-r77vg"
},
{
"input":3,
"result":9,
"hostname":"square-app-7cfffb48d9-r77vg"
},
{
"input":4,
"result":16,
"hostname":"square-app-7cfffb48d9-r77vg"
},
{
"input":5,
"result":25,
"hostname":"square-app-7cfffb48d9-r77vg"
}
]The gRPC requests are always routed to the same pod!
gRPC On Kubernetes With Linkerd:
Linkerd is a light weight service mesh. It is gRPC aware and it could load balance gRPC requests just out of the box without any change in the app / deployment yaml.
- Installing Linkerd is just a 5 mins work. Detailed steps are here.
Once you have Linkerd setup in your cluster, Run this command.
linkerd inject deployment.yaml| kubectl apply -f -Now if we send the same CURL request, we could see the requests are being balanced automatically by Linkerd proxy side car containers.
[
{
"input":1,
"result":1,
"hostname":"square-app-6b8449f5b4-hdbj5"
},
{
"input":2,
"result":4,
"hostname":"square-app-6b8449f5b4-857px"
},
{
"input":3,
"result":9,
"hostname":"square-app-6b8449f5b4-lvht9"
},
{
"input":4,
"result":16,
"hostname":"square-app-6b8449f5b4-857px"
},
{
"input":5,
"result":25,
"hostname":"square-app-6b8449f5b4-hdbj5"
}
]Linkerd adds lightweight side-car proxy containers for every pod in our app as shown below which distribute the gRPC requests.
Summary:
We were able to successfully demonstrate load balancing gRPC Requests On Kubernetes with Linkerd.
Read more about gRPC / Kubernetes.
The source code is available here.
Happy learning 🙂
