Hexagonal Architecture for Java

1. Overview

Hexagonal Architecture is a style which talks about layering your objects in such a way that isolates your core logic from outside elements. Core logic is the piece specific to your business and outside elements are like integration points  e.g DBs, external APIs, UIs and others. It divides software into inside and outside parts. Inside part contains Core Business logic and Domain layer (explained in Layered Architecture). Outside part consists UI, database, messaging and other stuff. Inside and Outside part communicate with each other via ports and adapters.

2. Benefits

• Software developed using this architecture is independent of the channels and can support multiple channels
• Easy to swap out the inbound and outbound integration points
• Testing of the software becomes easy because we can mock integration points easily

3. Implementation in Java

As explained above, Hexagonal architecture is more around ports and adapters. In Java, Interfacesimplement the ports and Implementation class works as adapters. So, we will take a simple example with Spring Boot application and see how this style can be applied on this app.

In this application, we have  the functionality to create/view Employee Details. Core Business logic is in EmployeeService and the domain is an Employee. So, these will be considered as inside parts.

So now, this application can expose this functionality via REST or Messaging. Hence, we have created   EmployeeControllerAdapter to expose REST endpoints, which implement EmployeeUIPort.

As part of business logic, EmployeeService also needs to call the DB which is again an integration point (outside part) so we have created EmployeeRepositoryPort and EmployeeServiceAdapter implements this port.

So, we do see how EmployeeService has used the EmployeeUIPort port to expose its service and EmployeeRepositoryPort to interact with the DB. Also, EmployeeControllerAdapter and EmployeeServiceAdapter helps to integrate with REST APIs and DB.

4. Summary

To summarize it, Hexagonal Architecture is an approach to divide the application into inside and outside part. They are connected through port (exposed by inside) and adapters (implemented by outside). So, by applying this approach, the core use case code remains intact and can serve to multiple channels supporting different protocols. It also helps to make application tested easily. However, I would suggest  not to implement this architecture fully for whole application, but use interfaces and adapters selectively.

As always, the code of all examples above can be found over on GitHub.

Rajesh Bhojwani December 25, 2018

Read more ...

Spring Boot Elasticsearch

Overview

In my last blog, I talked about how we can use Spring-Data-Elasticsearch project to connect with Elasticsearch Engine and do all CRUD operations. However, I did also mention that this project is not updated to be compatible with latest version of elasticsearch engine. So, In this blog, I am going to cover how to interact with latest version of elasticsearch engine using Transport Client library. I am going to use Spring Boot as Client application and then add dependencies for other required libraries.

Pre-requisites

• Jdk 1.8
• Maven
• Elasticsearch engine download 5.x or 6.x ( I will explain the steps how to download)
• Eclipse or VSD as IDE

Setup Elasticsearch 

Step 1 - Go to elastic official website https://www.elastic.co/downloads/past-releases

Step 2 - Select Elasticsearch in drop down and then version as 5.5.0 and click on Download button.

Step 3 - It will give you options if you want to download as Zip, TAR, RPM. You can choose as per your convenience. I selected the Zip format as using it on windows. Step 4 - unzip the downloaded content and go to  bin  folder. There will be a file named elasticsearch.bat.

Step 5 - Run this file on windows OS through command prompt and it will start the elasticsearch engine for you. Once started, it will start listening to port 9200. So the url will be http://localhost:9200/ Also, 9300 port is exposed as cluster node. 9200 port is for REST communication and can be used by Java or any other language and 9300 is for Elasticsearch Cluster Node communication with each other as well Java can also connect to this cluster node using Transport protocol.

Step 6 - Test to verify if it started properly by launching the url using curl command. You can use PowerShell on windows.

curl http://localhost:9200/

Once the search engine starts up, let's try to test some of the REST apis it provides to interact with engine.

Launch -  http://localhost:9200/users/employee/1  using Postman or curl with POST method. Input should be in JSON format.

Response will show that it has created "users" as index name, "employee" as type and document has been created with id as "1".

Now, To view the content of the document, we can call another REST api end point  http://localhost:9200/users/employee/1 using GET method. This will result the document information as below:

Now, if we want to search a document by any particular field, you need to add _search as path in the REST API url. So launch        curl -XGET 'http://localhost:9200/users/employee/_search' 

This will search for all the documents for index "users" and type "employee". Now, if you want to search for a particular field, need to add Query match criteria as part of JSON.

 curl -XGET 'http://localhost:9200/users/employee/_search'

-H 'Content-Type: application/json' -d

' {"query": { "match": {"name" : "Rajesh" } }}' 

This will search for a document having field 'name' as 'Rajesh'. 

Now, we have seen how Elasticsearch REST APIs works to create, retrieve and other operations for the documents; let's try to understand how we can connect to elasticsearch engine from Applications code. Either, we can make call to these REST apis directly from the code or we can use Transport Client provided by Elasticsearch. Let's develop an Spring Boot application to showcase all the CRUD operations.

Develop Spring Boot Application

Maven Dependencies 

Other than spring boot jars, It requires elasticsearch, transport client and log4j jars.

Configuration -

As we will be using Transport client to connect to elasticsearch engine, we need to give url path for the cluster node of engine. so i have put properties in application.properties file for the host and port of the url.

Domain -

Create a domain class named User. JSON input will be mapped to this User object. This will be used to create user document associated with the index and type. 

Configuration -

Java configuration file has been created to create Transport client which connects to elasticsearch cluster node. It is also loading the value of host and port from environment configured through application.properties. 

Controller -

UserController is created to showcase below features -

1.     Create an Index named "users" and Type "employee". It will create a document storing user information. Id for a document can be passed as input in JSON or if it is not passed, elasticsearch will generate its own Id. Client has a method called prepareIndex() which builds the document object and store against the index and type. This method is a POST method call where User information will be passed as JSON.

2. View the user information based on "id" passed. Client has prepareGet() method to retrieve information based on index, type and id. It will return the User Information in JSON format. 

3. View the user information based on a field name. I have used matchQuery() here to search by "name" field and return User information. However, there are many different types of query() available with QueryBuilders class . e.g rangeQuery() to search a field value in particular range like age between 10 to 20 years. There is a wildcardQuery() to search a field with wildcard. termQuery() is also available. You can play with all of these based on your need.

4. Update the document by searching it with Id and replace a field value. Client has a method called update(). It accepts UpdateRequest as input which builds the update query.

5. Last method is to showcase delete a document of an index and type. Client does have prepareDelete() method which accepts the index, type and id to delete the document. 

Full Code has been put on github - https://github.com/RajeshBhojwani/spring-boot-standalone-elasticsearch.git

Build Application

 Run mvn clean install   command to build the jar file.

Start Application

Run java -jar target/standalone-elasticsearch-0.0.1-SNAPSHOT.jar   command to start the Spring Boot application.

Test Application

Application will be running on http://localhost:8102   url. Now let's test couple of use cases we talked above.

1. Test Creating document.

     Launch  http://localhost:8102/rest/users/create as POST method either through curl or Postman

Input -

You will see the response showing "CREATED". 

2. To test if the document has been created, lets test the view functionality.

     launch -  http://localhost:8102/rest/users/view/1 with GET method

In response, you will see the User information for id with value "1".

3. You can view User information by name field as well by launching  http://localhost:8102/rest/users/view/name/Rajesh   This is passing "Rajesh" as "name" field value.

Similarly, update and delete features can be tested by launching  http://localhost:8102/rest/users/update/1  and  http://localhost:8102/rest/users/delete/1 

So these are all the features I have played around and sharing with you through this blog. As mentioned above, there are many more different types of queries you can explore with elasticsearch transport client.

Rajesh Bhojwani December 13, 2018

Read more ...

Elasticsearch with Spring-Data-Elasticsearch Application

Overview 

Elasticsearch is a real-time distributed and open source full-text search and analytics engine. It is document-based search platform with fast searching capabilities. It’s optimized for needle-in-haystack problems rather than consistency or atomicity.

Elasticserach is a huge topic so in this blog i am going to keep it very simple at 101 level. In this blog I will cover how to download Elasticsearch and set it up. Also, how you can use Spring Boot and Spring Data ElasticSearch projects to integrate with Elasticsearch engine.

Setup Elasticsearch

Elasticsearch latest version is 6.5.x as of today. However, I am going to download only 2.4.4 version only. The reason is that spring-data-elasticsearch project is compatible only with 2.4.4 version as of now. If you want to know more details on the compatibility, please check this link - Spring-Data-Elasticsearch---Spring-Boot---version-matrix

If you still want to use the latest Elasticsearch 5.x or 6.x, you would need to use Spring  with Transport Client or Node client apis directly instead of spring-data-elasticsearch. Follow below steps to download and setup elasticsearch engine.

Step 1 - Go to elastic official website https://www.elastic.co/downloads/past-releases

Step 2 - Select Elasticsearch in drop down and then version as 2.4.4 and click on Download button.

Step 3 - It will give you options if you want to download as Zip, TAR, RPM. You can choose as per your convenience. I selected the Zip format as using it on windows.

Step 4 - unzip the downloaded content and go to  bin  folder. There will be a file named elasticsearch.bat.

Step 5 - Run this file on windows OS through command prompt and it will start the elasticsearch engine for you. Once started, it will start listening to port 9200. So the url will be http://localhost:9200/ Also, 9300 port is exposed as cluster node. 9200 port is for REST communication and can be used by Java or any other language and 9300 is for Elasticsearch Cluster Node communication with each other as well Java can also connect to this cluster node using Transport protocol.

Step 6 - Test to verify if it started properly by launching the url using curl command. You can use PowerShell on windows.

curl http://localhost:9200/

Develop Spring Boot Application -

     Now, we will develop a Spring Boot application which will showcase ElasticsearchTemplate and ElasticsearchRepository way of accessing the Elasticsearch engine and do CRUD operations.

    Before we develop the application, let's first understand how ElasticsearchTemplate and ElasticsearchRepository works.

ElasticsearchTemplate - It is a Template class which implements the ElasticsearchOperations. It is more powerful than ElasticsearchRepository as it can do more than CRUD operations. It has operations to create, delete indexes, bulk upload. It can do aggregated search as well.

ElasticsearchRepository -  If we define an interface which extends the ElasticsearchRepository, which is provided by Spring data Elasticsearch, it will provide the CRUD operations automatically for that Document. e.g. UserRepository interface is defined below with "User" document by extending ElasticsearchRepository. Now all the find, save, delete, update default operations can be done on User document. If you don't know what is document in Elasticsearch, read below in this blog for more explanation. 

It extends the ElasticsearchCrudRepository which extends eventually the Repository interface. This repository interface is standard feature of Spring data. No need to provide implementation of this interface. You can write customized queries as well by using @Query annotation.

Both of these uses Transport client or Node Client

    Prerequisites

•   Java 1.8 installed
•   Eclipse or Visual Studio Code IDE
•   Maven

 Maven Dependencies   

Configuration -

   application.properties  needs to have spring data properties which is used by ElasticsearchTemplate and ElasticsearchRepository to connect the engine. I have used Transport client properties like cluster-nodes, index name to connect the elasticsearch engine.

Mappings

In Elasticsearch, Index is like a DB in RDBMS and Mappings/Type is similar to a table in RDBMS. Document is a collection of fields belonging to a type and resides in index. Spring data provides annotation like @Document to create document. Here, we define User as a document which has index as "my_index" and type as "user".

Develop Controllers -

First Controller is UserController. It will use UserDAOImpl to have ElasticserachTemplate interacting with Elasticsearch Engine. 

UserDAOImpl - This class initialize elasticsearchtemplate and use queryForList method to retrieve data.

Another Controller is UserRepositoryConroller. This is using UserRepository to interact with elasticsearch engine.

This Repository class extends the ElasticsearchRepository class which internally extends  ElasticsearchCrudRepository ->  PagingAndSortingRepository

You can find the full code at github link -https://github.com/RajeshBhojwani/spring-boot-elasticsearch.git

Build Application

  Application can be build using Maven commands.

 mvn clean install  will build the code and create elasticsearch-0.0.1-SNAPSHOT.jar  file.

Run the application

     java -jar  target/elasticsearch-0.0.1-SNAPSHOT.jar  will start the application. And application will listen to port 8102 as defined in application.properties file.

Test the application -

Test UserController flow which uses ElasticsearchTemplate.

Step 1 - Add new Users. Use this REST API url to add new User  http://localhost:8102/new 

Add Json data in Request body.

Step 2 - Check the response. You will see new User is created with userId generated which is Unique Id for this document. Output will be as below:

Step 3 - Retrieve all Users. Use   http://localhost:8102/all 

Test UserRepositoryController flow which uses ElasticsearchRepository.

Step 1 - Add new Users. Use this REST API url to add new User  http://localhost:8102/repo/new 

Add Json data in Request body as we did with earlier test case.

Step 2 - Check the response. You will see new User is created with userId generated which is Unique Id for this document.

There are many other apis i have put in the code which you can play around. It will showcase how ElasticsearchTemplate can work with indexes, How you can search by field name. With ElasticsearchRepository, you can search by userId as well.

There are many other ways we can connect to Elasticsearch Engine. I will just provide brief introduction about them here as it would require whole new blog to explain in detail.

Spring Boot with Transport Client - As explained above, if you want to use latest elasticsearch engine 5.x or 6.x, spring data elasticsearch would not support that as of now. But Elasticsearch provides RestHighLevelClient  class which works well with latest version and uses RestClient  class to configure the host and port for the cluster. It uses index() method with IndexRequest as input to create a document. It uses get() method with GetRequest as input to search a document. It does have update() and delete() method as well.

Nodejs - If you are not Java/Spring fan, then you have an option of Nodejs as well. What you need to do is download  elasticsearch and get-json module using npm. elasticsearch has Client module which can configure host and port of the elasticsearch server cluster. This Client now have methods like create() to create the index; index() method to put document in that index.

That's all for Elasticsearch integration with Spring data project. Please do let me know your views through comments. Happy to address your queries if I can.

Rajesh Bhojwani December 02, 2018

Read more ...

Spring Retry - Way To Handle Failures

In Microservice world, we do have services talking to each other. One of the known way of communication is Synchronous. However, in Cloud Computing world, the fact is that we cannot avoid Network glitches, temporary service down (due to restart or crash. not more than few seconds). So when clients need real time data and your downstream service is not responding momentarily, may impact the users so you would like to create retry mechanism. There are many solution options available in Java to try out. I am going to talk about Spring-Retry in this blog. We will build a small application and see how Spring Retry works. Before we start that, let's first understand few basics about Retry Pattern:

• Retry should be tried only if you think that it may suffice the requirement. You should not put it for each use case. This is something you don't build right from the begining but based on the learning while doing development or testing. For example, If you find while testing that when you hit a resource, it works one time but next time gives timeout error and works fine when hit again. After checking with downstream system, not able to find out any root cause or solution. So you might want to build a Retry feature to handle at your application side. But first attempt should be to fix at downstream side. Don't jump quickly to build solution at your end.
• Retry may cause resource clogging and make things even worse preventing application to recover so number of retries have to be limited. You should try to start with minimum count e.g. 3 and not going beyond 5 or so.
• Retry should not be done for each exception. It should be coded only for particular type of exception. For example, Instead of putting code around Exception.class, do it for SQLException.class.
• Retry can cause to have multiple threads trying to access same shared resource and locking can be a big issue. So exponential backoff algorithm has to be applied to continually increase the delay between retries until you reach the maximum limit.
• While applying Retry idempotency has to handled. Trigerring same request again, should not trigger double transaction in the system.

Now, let's build a simple Service showcasing how Spring-Retry helps to address Retry.

Pre-Requisites

• Spring Boot 2.1.x
• Gradle
• Visual Studio Code/Eclipse

Gradle Dependencies

Spring Retry uses Spring AOP internally to work. So it is also required to be added as dependency.

dependencies {
implementation('org.springframework.boot:spring-boot-starter-web')
implementation('org.springframework.retry:spring-retry')
implementation('org.springframework.boot:spring-boot-starter-aop')

testImplementation('org.springframework.boot:spring-boot-starter-test')
}

Enable Retry

Put @EnableRetry annotation on the SpringBoot main class.

1 2 3 4 5 6 7 8

@EnableRetry @SpringBootApplication public class DemoApplication { public static void main(String[] args) { SpringApplication.run(DemoApplication.class, args); } }

Put Retryable Logic in Service

@Retryable annotation has to be applied on a method which needs to have Retry logic. In this code, I have put counter variable to show in logs how many times it is trying the retry logic.

• You can configure on what exception it should trigger the retry.
• It can also define how many retry attempts it can do. Default is 3 if you don't define. 
• @Recover method will be called once all the retry attempts are exhausted and service still throws the Exception(SQLException in this case). Recover method should be handling the fallback mechanism for those requests.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

@Service public class BillingService { private static final Logger LOGGER = LoggerFactory.getLogger(BillingService.class); int counter =0; @Retryable(value = { SQLException.class }, maxAttempts = 3) public String simpleRetry() throws SQLException { counter++; LOGGER.info("Billing Service Failed "+ counter); throw new SQLException(); } @Recover public String recover(SQLException t){ LOGGER.info("Service recovering"); return "Service recovered from billing service failure."; } }

Create REST Endpoint To Test

This client is created just to hit the BillingService simpleRetry() method. 

1 2 3 4 5 6 7 8 9 10 11

@RestController @RequestMapping(value="/billing") public class BillingClientService { @Autowired private BillingService billingService; @GetMapping public String callRetryService() throws SQLException { return billingService.simpleRetry(); } }

Launch the URL And See the Logs

http://localhost:8080/billing

1 2 3 4

2018-11-16 09:59:51.399 INFO 17288 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Billing Service Failed 1 2018-11-16 09:59:52.401 INFO 17288 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Billing Service Failed 2 2018-11-16 09:59:53.401 INFO 17288 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Billing Service Failed 3 2018-11-16 09:59:53.402 INFO 17288 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Service recovering

The logs shows it has tried the simpleRetry method 3 times and then route to recover method.

Apply BackOff Policy

Now, as we discussed above that having back to back retry can cause locking of the resources. So we should add Backoff Policy to create a gap between retries. Change the BillingService simpleRetry method code as below:

1 2 3 4 5 6 7

@Retryable(value = { SQLException.class }, maxAttempts = 3, backoff = @Backoff(delay = 5000)) public String simpleRetry() throws SQLException { counter++; LOGGER.info("Billing Service Failed "+ counter); throw new SQLException(); }

Capture the logs showing 5 seconds gap in all retries.

1 2 3 4

2018-11-17 23:02:12.491 INFO 53392 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Billing Service Failed 1 2018-11-17 23:02:17.494 INFO 53392 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Billing Service Failed 2 2018-11-17 23:02:22.497 INFO 53392 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Billing Service Failed 3 2018-11-17 23:02:22.497 INFO 53392 --- [nio-8080-exec-1] c.e.springretrydemo.BillingService : Service recovering

So this is the way Spring Retry works.

To access the full Code click https://github.com/RajeshBhojwani/spring-retry

There are many other options for applying Retry pattern in Java. Some are as following:

• AWS SDK - This can be used only if you are using AWS related service and AWS API Gateway through AWS SDK apis.
• Failsafe - Failsafe is a lightweight, zero-dependency library for handling failures. It was designed to be as easy to use as possible, with a concise API for handling everyday use cases and the flexibility to handle everything else. 
• Java 8 Function Interface 

Thats all for this blog. Let me know what all libraries you are using in Microservice to handle failures and retry.

Rajesh Bhojwani November 19, 2018

Read more ...