Best Practices – JMeter – Adding Performance Testing in CI / CD Pipeline

We already have seen Continuous Performance Testing process as part of these articles.

• JMeter – ANT – Jenkins Integration – Part 1
• JMeter – ANT – Jenkins Integration – Part 2
• JMeter – Maven – Jenkins Integration
• JMeter – Gradle – Jenkins – TBD

In this article, I would like to show how I have set up the Continuous Delivery Pipeline.

Continuous Integration:

As you might be already aware, CI is software development practice, where developers use version control system & integrate their code changes very often to the main branch and each change is built and verified. Main idea behind the process is to detect any integration related issues as early as possible.

Continuous Deployment / Delivery:

It is an extension to the above Continuous Integration process. It is a software development methodology where the release process is automated. As and when CI process PASSed, software is automatically promoted to the next higher environment – again a series of automated tests like smoke test, full regression, small-scale load test, full-scale load test etc are performed until it is ready for production push.

Build Pipeline:

I use Jenkins for CI/CD process. We have created different Jenkins jobs for each task to be performed.

• Build
• Integration Tests
• Deploy To QA
• Smoke Test
• Full Regression
• Small-scale load test
• Deploy To Staging / Pre-Prod
• Soak Test
• Spike Test ..etc

I have chained these jobs using Jobs upstream/downstream config in Jenkins.

Downstream Config:

Chain the next job to be built as part of Post-build actions. For ex: Integration Test should be performed after Build.

Upstream Config:

A job is constantly looking for its previous job to be built. For ex: Smoke Test on QA should be performed only if Deploy To QA is complete.

You might not need both config. Either of them should do.

 

Build Pipeline Dashboard:

I have installed this plugin in Jenkins to visualize the build pipeline. Create a new Build Pipeline View and configure the initial job as shown below.

 

 

click on this image to view this bigger

 

Now as soon as a new change is pushed to the main branch , a series of tests are performed automatically and software is continuously promoted to the next higher environments like QA / Pre-prod from Dev environment.

Pipeline Plugin:

Using jenkins pipeline plugin, we can build the whole pipeline using DSL. More information is here.

Parameters can also be passed at run time.

 

The real challenge is not in setting up these pipelines, instead they are in setting up the infrastructure setup – scaling out – load generators on the fly & monitoring the application under test and load generators.

Infrastructure Setup:

I use Docker, Rancher & AWS for the environment setup & Redis for the test data management.  Details about these setup are already available in TestAutomationGuru.

• JMeter – Distributed load testing using Docker
• JMeter – Distributed load testing using Docker + RancherOS in AWS
• JMeter – Redis – Test data management

 

Monitoring:

I use InfluxDB to collect the performance test results, load generators metrics and application under test server metrics. Grafana to visualize the metrics. Details about these setup are already available in TestAutomationGuru.

• JMeter – Real time metrics – Basic setup
• JMeter – Real time metrics – Advanced setup
• JMeter – Server performance monitoring using collectd
• JMX monitoring using collectd

Summary:

Once we set up these build pipeline, infrastructure for performance testing and monitoring tools to collect the performance metrics, whenever a new feature is committed to the main branch all these tests are performed sequentially! Any functional or potential performance related issues are found as early as possible. Overall time to push changes to production also greatly reduced.

 

Happy Testing & Subscribe 🙂