Deployment Challenges and Solutions For PCF Platform

Many users have started PCF as platform to host their applications and microservices. More you use the platform and more you uncover the issues & challenges. So today, i am going to talk about Microservice Deployment issues on PCF platform.
I have been working with many teams who started breaking their monolith applications and convert into microservices. I have a team which has 20+ microservice to handle now. And they need to deploy these microservices to multiple environments as part of  SDLC e.g. Dev, System, Perf, QA, Prod, etc.. Now, for deploying a microservice to PCF, you need to use manifest.yml file. And, it is recommended to separate each manifest file for each environment. So for these 20+ microservices, they need to maintain 20*5 =100+ manifest files. Can you imagine, how hard it would be to handle so many files. One manual mistake can cause so much damage. Lets first talk about first why you need separate manifest.yml file for each environment.

1.     Each environment, your microservice would require diff configuration of the resources e.g. In Dev, you would need only 2 instances of the app with 512 gb memory but in Prod, you might  need 4 instances with 1gb memory.

2.     Each environment might be binding to a backing service with different plan e.g for Dev, you might need only basic/free plan of the DB service but in Prod, you would need to advance plan to with more resources so your service might be different. You might want to ensure putting the name of the service instance accordingly so that team understands which is free and which is chargeable. 

3.     There might be few env variables which you would like to setup while deploying the application and that may vary based on environment. e.g. spring. dev

    There might be many more reasons as well. But let's go back to how to resolve the number of growing manifest files.

PCF used to have a feature called Inheritance for manifest file where you can create a parent manifest yml file and inherit the common properties from there. But this has been deprecated and anyhow that doesn't resolve the number of manifest files you create; though it reduces the content of each file significantly.

Inheritance has been replaced by Variable Substitution. 
This feature helps to create a template manifest file, put the placeholders for variables and replace the values dynamically from external file.

1.     Lets start with an simple example. Here is a sample manifest file for a GO app.

- name: sample-app
  instances: ((noofinstances))
  memory: ((memory))
    GOPACKAGENAME: go_calls_ruby
  command: go_calls_ruby

2. This template file has put placeholder for variables for instances and memory. Lets create another file data.yml which is going to have those variable values.

noofinstances: 2
memory: 1G

3.  Now, push the application to PCF using cf push command and use --vars-file argument to pass the data.yml file.

cf push -f ~/workspace/manifest.yml --vars-file ~/workspace/data.yml

Values of 'noofinstances' and 'memory' from data.yml file will replace in manifest file. This shows how to use variable substitution feature. Now, we can use this solution to replace multiple manifest files to one.
Step 1.
Create a Template manifest file

- name: sample-app
  instances: ((noofinstances))
  memory: ((memory))
    - mysql
    - newrelic
  env: ((env))

Step 2.
Create a data.yml file which will have data for each environment. I would recommend to keep only all non-prod environment data in one file. For Prod, you should always have a separate manifest.yml and data.yml file.

//dev env
dev_noofinstances: 2
dev_memory: 512M
dev_env: dev

//system env
system_noofinstances: 3
system_memory: 1G
system_env: system

//perf env
perf_noofinstances: 4
perf_memory: 2G
perf_env: perf

//qa env
qa_noofinstances: 3
qa_memory: 1G
qa_env: qa

Step 3.
In CI/CD Pipeline like bamboo, jenkins, write an script. Most PCF deployments are configured through pipeline only. This script will take environment value as input as per pipeline standards and based on the env value like 'dev', 'system', 'perf' and 'qa', it will read this data.yml file and retrieve all the values of the related environment (Please notice, data.yml file variables have env value as prefix. Need to follow that standard). Script would create a temporary file <env>_data.yml file.


memory: 512M
env: dev

memory: 1G
env: system

Step 4.
 Now, as per earlier steps, just push the application to PCF using cf push command passing the data file dynamically through pipeline for each environment.

cf push -f manifest.yml --vars-file=<env>_data.yml

This solution has replaced 4 manifest files to 1 manifest file. You can even go one step further and club all related microservices' manifest data in one data.yml file and it can reduce the count further. But please note, keep the Prod data.yml file separate to avoid any manual mistakes being touched so frequently.

That's all for this blog. To understand more about the PCF and Microservice concepts, read these blogs.

No comments: