Design Patterns for Microservice-To-Microservice Communication

In my last blog, I talked about Design Patterns for Microservices. Now, I want to deep dive more into the most important pattern in Microservice Architecture and that is Inter-Communication between the Microservices. I still remember when we used to develop monolithic application, communication used be a tough task. In that world, we had to carefully design relations between database
tables and map with object models. Now, in Microservice world, we have broken them down into separate services and that creates a mesh around it to communicate with each other. Lets talk about what all the communication styles and patterns have evolved so far to resolve this.
Many Architects have divided Inter-Communication between microservices into Synchronous and Asynchronous interaction. Let's take one by one.

Synchronous -

              When we say synchronous, it means Client makes a request to Server and waits for its response. The thread will be blocked until it receives communication back. The most relevant protocol to implement synchronous communication is HTTP.  HTTP can be implemented by REST or SOAP. Recently, REST is picking up rapidly for microservices and winning over the SOAP.  But, for me both are good to use. Now let's talk about different flows/use-cases in Synchronous style. What are the issues we would face and how to resolve them.
  1. Let's start with a simple one. You need a Service A calling Service B and waiting for response for live data. This will be a good candidate to implement synchronous style as there are not many downstream services involved. You would not need to implement any complex design pattern for this use case except Load Balancing if using multiple instances.

2. Now, let's make it little more complicated. Service A is making calls to multiple downstream services like Service B, Service C, Service D for live data. 
  • Service B, Service C and Service D all have to be called sequentially - This kind of scenario will be there when Services are dependent on each other to retrieve data or the functionality has a sequence of events to be executed through these services.
  • Service B, Service C and Service D can be called in parallel - This kind of scenario will be there when Services are independent of each other or Service A may be doing Orchestrator role.
This scenario brings the complexity while doing the communication. Let's discuss one by one.
  a.    Tight coupling - Service A will have tight coupling with each Service B,          C, D. It has to know each service endpoint, credentials.
         Solution Service Discovery Pattern is used to solve these kind of issues. It helps to decouple the consumer and producer app by providing lookup feature. Services B, C and D can register themselves as services. Service Discovery can be implemented server side as well as client side. For Server side, we have AWS ALB, NGINX tools which accept the requests from client, discover the service and route the request to the identified location. For Client side, we have Spring Eureka discovery service. The real benefit I do see using Eureka is that it caches the available services information at client side. So even if Eureka Server is down for sometime, it doesn't become single point of failure. Other than Eureka, there are other service discovery tools like etcd and consul are also used widely.
        b.    Distributed Systems - If Service B, C and D have multiple instances, Then it needs to know how to do the load balancing.
               Solution - Load Balancing generally goes hand-in-hand with Service Discovery. For Server side load balancer, AWS ALB can be used and for client side, Ribbon can be used along with Eureka both can do the same. 
        c.    Authenticating/Filtering/Handling Protocols - If Service B, C and D needs to be secured and  need authentication, need to filter through only certain requests for these services and if  Service A and other services understand different protocols.
                Solution - API Gateway Pattern helps to resolve these issues. It can handle authentication, filtering and can convert protocols from AMQP to HTTP or others. It can also help enabling  observability metrics like distributed logging, monitoring, distributed tracing, etc... Apigee, Zuul, Kong are some of the known tools which can be used for the same. Please note. This pattern i will suggest if Service B, C and D are part of managed APIs else its overkill to have API Gateway. Read further down for Service Mesh for other alternate solution.
       d.   Handling Failures - If any of the Services B, C or D is down and if Service A can still serve the request of client with some of the features, it has to be designed accordingly. Other problem is let's suppose if Service B is down and all the requests are still making call to Service B and exhausting the resources as its not responding. It can make whole system down and Service A will not be able to send requests to C and D as well.
             Solution Circuit Breaker and Bulkhead Pattern helps to address these concerns. Circuit Breaker Pattern identifies if a downstream service is down for a certain time and trips the circuit to avoid sending calls to it. It retries to check again after a defined period if service got up and close the circuit to continue the calls to it. This really helps to avoid network clogging and exhausting the resource consumption. Bulkhead helps               to isolate the resources used for a service and that helps to avoid the cascading of failures. Spring cloud Hystrix does the same job. It applies both Circuit Breaker and Bulkhead Patterns.
       e.   Microservice-To-Microservice Network Communication -  API Gateway  is generally used to for managed APIs where it handles the Requests from UIs or Other consumers and make downstream calls to multiple microservices and respond back. But when a microservice wants to call to another microservice in the same group, API Gateway is overkill and not meant for that purpose. So it ends up that individual microservice takes the responsibility to make network communication, do security authentication, handle timeouts, handle failures, load balancing, service discovery, monitoring, logging. Its a too much overhead for a microservice.
             Solution - Service Mesh Pattern helps you to handle these kind of NFRs. It can offload all the Network Functions we discussed above. With that, microservice will not call directly to other microservice but go through this Service Mesh and it will handle the communication with all features. The beauty of this Pattern is that now you can concentrate on writing business logic in any language Java, NodeJS, Python without  worrying if these languages have support to implement all network functions or not. Istio and Linkerd are picking up to address these requirements. The only thing i don't like about Istio is that it is limited to Kubernetes as of now.   

Asynchronous -

           When we talk about Asynchronous, it means Client makes call a request to Server and receive the acknowledgment of the request received and forget about it. Server will process the request and completes it. Let's now talk about when you would need asynchronous style. If you have an application which is Read heavy, synchronous style might be a good fit especially when it needs the live data. However, when you have Write heavy transactions, and you can't afford losing data records, you may want to choose for asynchronous. Because, if a downstream system is down and you are keep sending synchronous calls to it, you will lose the requests and business transactions. Thumb rule is never ever use Async for live data read and never ever use Sync for business critical write transactions until unless you need the data immediately after write. You need to choose between Availability of the data records and Strong Consistency of the data.
 There are different ways we can implement the asynchronous style :
           1.  Messaging - In this approach, Producer will send the messages to a Message Broker and Consumer can listen to the message broker to receive the message and process accordingly. In this also, there are 2 patterns; one-to-one and one-to-many.  We talked some of the complexity which Synchronous style brings but some of them are eliminated by default in Messaging style. For example, Service Discovery becomes irrelevant as Consumer and Producer both  talks to Message Broker only. Load Balancing is handled by scaling up messaging system. Failure handling is in-built by Message Broker mostly. Rabbitmq, ActiveMQ, Kafka are best known solutions in cloud platform for messaging.  
         2. Event Driven - Even Driven looks similar to Messaging but it solves different purpose. Instead of sending messages, it will send Event details to the Message Broker along with the payload. Consumers will identify if what is the event and how to react on it. This gives more loose coupling. There are different types of payload can be passed.    
  • Full payload - This will have all the data related to the event which is required by consumer to take further action. But this makes it more tightly coupled
  • Resource URL - This will be just a URL to a resource that represents the event.
  • Only Event - No Payload will be sent. Consumer would know based on on the event name how to retrieve relevant data from other sources like DB or queues 
Image title

There are other styles like Choreography style, which is used to rollback your transactions in case of failures. But i personally don't like that. It is too complicated to be implemented. I have seen that too working well with Synchronous style only. 
That's all for this blog. Let me know your experience on Microservice-To-Microservice communication.

No comments: