Recently, we did some great work on Continuous Delivery and thought we should share our learnings.  We implemented the solution of Continuous Integration & Auto Deployment (Continuous Delivery) for a major Australian customer in retail & ecommerce space.  The challenges we faced were to maintain several instances of the site for the retail application which has approximately 25 different frontend applications for each brand or a group, and over 30 ecommerce applications built using Magento for each brand.  To overcome this across the product line we used Ansible, which is a continuous integration and auto deployment tool.

Configuring Ansible with custom scripts is also easy, once a developer creates an Ansible script in .YML to configure server and auto deploy builds, then this can be added in a scheduler or used by any teams to setup system or deploy new build on fly. It helps saving not only up-front costs, but making it easier to react to change over time which can be done easily.

Ansible is a simple, powerful & agent less open source tool to write custom scripts for configuring the server and deploying application code to testing, staging, release & production. It gives you the power to deploy multi-tier applications reliably and consistently, all from one common framework. You can configure needed services as well as push application artifacts from one common system.

The typical process we had can be depicted as below

So now let’s get into each section mentioned in above depiction individually.

The above process can be divided into 2 steps basically:

–   Continuous Integration

–   Auto Deployment (Continuous Delivery)

CONTINOUS INTEGRATION

CHALLENGES WE FACED

– NEED TO SAVE CONFIGURATION TIME – If you are working on a project which has / requires multiple instances of the application to be created when ever required

– REQUIRED MULTIPLE INSTANCES WITH CONSISTENCY – Our development / QA teams needed to test on different instances on different servers or multiple instances on single server with consistency.

– REDUCE DEBUGGING TIME – The development team needed to spend time debugging on getting configuration right to start dev-ops which was taking out much time in starting development in an agile environment

– ROLLBACK / REBUILD – In case of unfortunate crash of any environment which hosts the application, setting up new ones would consume lot of time & immense patience

– REDUCE INTEGRATION FAILURE – Due to various servers on different setup of Linux or any other operating systems the general process of integration was failing and lot of time was required to getting this right

– CONSISTENT WORKFLOW WITH HISTORY MAINTAINCE – We needed all the systems to be consistent and at same time need the history of different integrations done on each instance on same server or different servers then source will help you go it

HOW WE ACHIEVED CONTINOUS INTEGRATION?

Please refer to the below representation to understand the flow of process.

–          In brief, the B2B & B2C application we were working on was developed and tested by various developers / testers working on Multiple servers (which includes 3 development server with multiple instances, 5 QA servers, 3 release servers & 1*(3) production environment on each load balancers), All these servers are with multiple instances for each ecommerce web application. Basic example of servers and list of task needed to perform is represented by the below image.

–  Custom GUI to run specific tasks or execute a particular playbook was setup, the playbooks also was run using scheduler wherever required

–  The history of integration logs is maintained such that at any point we could refer to the previous build and manually get back the previous build through the GUI playbook execution

–  We had created multiple Playbook tasks to do the following (Monitor Servers, Load Balance, Stop/Start service, Deploy Application)

• The Playbook can have below tasks for example,

– Install Server Dependencies (Common Libraries)

– Install Apache

– Install PHP

– Install Database (MySQL, PostgreSQL etc.)

– Install Git or other version controls

– Install Drush/Other required build libraries.

– Install Memcache / Redis, etc.

– Check Load Balance Servers.

– Maintain Scheduler Jobs.

– Maintain Configuration Logs.

• Configuration of the above tasks on each app servers play important role in reducing the integration and debug time on multiple development / test / release server instances for different purpose specific to business.
• We had written scripts using YML language and were executed using Ansible tool to deploy the code across these all server. So there is always a rollback or rebuild plan if even a test server is down.
• Each Playbook task can have multiple roles and each role can have the script to configure certain areas of the application like (Apache, MySQL, Linux, Nginx )

–          These Playbooks can be repeated to perform same operation on each server to get it up and running with consistency across servers.

AUTO DEPLOYMENT (CONTINOUS DELIVERY)

CHALLENGES WE FACED

– NEEDED CONSISTENT ENVIRONMENT ACROSS SERVERS – Any successful application which needs updates to go into the production environment in a frequent set of interval or whenever the need is there to fix a critical bug (hotfix) needs the Auto-Deployment in place

– FASTER DEPLOYMENT – Lot of time was spent in managing the deployment process each week to get the updates to the users. A common update needed to be pushed across various server which would take up whole day development of multiple developers and hotfix to single customer needed careful watch not to disturb other applications

– TIMELY ROLLOUT OF UPDATES – You can manually do it too, but you waste lot of time doing it or waiting for someone to do it. And this can cost time as well as business and is something no customer would want. A scheduler process to timely update the application was very much in need

– REPEATABLE & RELIABLE DEPLOYMENTS – Each and every team in the cycle of development of a product requires the updated system to do their process and having an auto deployment allows them to have updated application in ease and need not worry on configuring & deploying applications without worry about failures.

– EASY FALLBACK & ZERO DOWN TIME – The main advantage why we need an Auto Deployment in place is because it can save lot of time and the business will have zero down time as there is a quick fallback action on failure.

– VARIED RELEASE ACROSS SERVERS IN TIMEZONES – Updating releases on different environments, then you crib for auto deployment technique in place.

HOW WE ACHIEVED AUTO DEPLOYMENT (CONTINOUS DELIVERY)

We wrote a custom script using the YML language which includes configuration files, playbooks (variables, tasks, handlers).

The playbooks can be used to achieve the below features:

REPEATABLE & RELIABLE – Ansible allows you to write ‘playbooks’ that are descriptions of the desired state of your systems, which are usually kept in source control. Ansible then does the work of getting your playbooks to execute on schedule basic or manual. Playbooks make your day-to-day management repeatable and reliable.

SIMPLE TO WRITE & MAINTAIN – Playbooks are simple to write and maintain. Most users become productive with Ansible after only a few hours.  Ansible uses the same tools you likely already use on a daily basis and playbooks are written in an YML language so they are very easy to evolve and edit.

NO AGENT = MORE SECURE, MORE PERFORMANCE, LESS EFFORT – Thanks to its agentless design, Ansible can be introduced into your environment without any bootstrapping of remote systems or opening up additional ports.

Though Ansible doesn’t keep looking for commits on the GIT repo and automates the code pull / push as done by Jenkins , But scripts to schedule the process which will be done in an set time interval via cron process can we written which can take care of all customization required.

Our process does the below,

– Writing custom scripts using Ansible tool is one of the simplest and the easiest to get started with deployment. A lot of this is because it’s just SSH. It uses SSH to connect to servers and run the tasks in put in playbooks to setup the deployment.

– We have numerous of commits going in to the test & demo branches and once test is passed the commit is moved by the developer into the release branch.

– The set of commits on release branch is moved into a git tag based on the release version and these are stacked on for the Cron to pick it up.

– The ansible scheduler (cron) picks ups these stacked release tags from the manager and pushes the tags into nodes for execute the deployment playbook.

– The scheduler can be written to run on varied time zones such that you can deliver upgrade based on your client’s needs.

– Once the code is moved into the nodes, the script to process other build process of pushed in code is executed and finally we have an updated system in place always.

– If there is any failure in the playbook process steps, the changes are rolled back and notification is sent over with logging of particular data.

Beyond the obvious of installing packages and configuring services, custom scripts written using YML and executed using Ansible coordinates all the deployment.

• Connect to any server using SSH and perform the actions listed.
• We were able to write scripts to add API servers to the loadbancer pool.
• Gracefully upgrades services in a rolling fashion for zero-downtime deployments.
• Push / Pull code from GIT version control repositories.
• Manage code files and configurations settings.
• Manage logs of actions executed via scheduler / GUI.

We hope this has been useful to some of you, and if you have any questions, do drop us a comment or reach out to us for a discussion.

Image Source: Ansible Forums, Github