JMeter – Continuous Performance Testing – JMeter + Maven

I am going to shamelessly copy some content from the article on JMeter + Ant.

Goal:

To create a Continuous Delivery Pipeline which includes Performance Testing process to detect any performance related issues as early as possible.

Usually the full scale Performance Test will be done in the Staging/Pre-Production environment which could be identical to your Production environment. Code push to Staging happens after thorough QA functional/regression verification is done. So even if QA certifies the build, there is a chance that code might not work as expected at certain load which could be found only in Staging environment. This performance issue requires code change -> build -> QA verification once again which could postpone your Production push!

So, Can we fit the performance testing into the existing continuous development & testing process to detect the performance issues as early as possible?

Yes! We absolutely can! The idea here is to run the performance test in the lower environments [Dev/QA] (we might have to adjust the load accordingly) & to compare the results against some baseline metrics. Note that this is NOT going to replace the full scale performance test which we will do in the Staging environment.

Lets see how we can implement the above process in this post by using JMeter + Maven + Jenkins.

 

Create Performance Test Script:

1. Create some basic performance test script using JMeter.
2. Check below useful articles from TestAutomationGuru.
   1. Creating Modular JMeter Test Scripts.
   2. Ensure that the script can be run on any given environment like QA/PPE/Staging etc. [Check here]
3. I am going to consider a simple JMeter test which will login, search for a product and logout.
   1. I have used variables in the Thread Count, Ramp Up and Duration fields.
   2. At run time, we will pass values for those variables from Jenkins.
   3. To run the test in design mode/standalone mode, We use User Defined Variables as shown here. So, it takes the default values (like 1 user with 1 second ramp-up for 5 seconds test duration).
4. Ensure that the scripts runs fine.

Running JMeter test using Maven:

Maven is more of a Project management tool.  It is like ant – but follows a standard project structure to make our development life easier. We might not be able to create our own tasks as we did in Ant. But Maven can take care of setting up JMeter with all the dependencies to run our test.

• Maven – More Info.
• Maven – Intsall

I create a simple mvn project in my eclipse as shown here.

• Under src/test create a folder – jmeter
• Copy your jmeter jmx file & move to src/test/jmeter
• As part of JMeter v3.0, to create the new dashboard, ApacheJMeter.jar expects the following folders to be present in the bin location. (You can get them from your JMeter bin folder)
  • report-template
  • reportgenerator.reporties
• Keep them under src/test/resources
• Include below content in your pom.xml

 

• We use jmeter-maven-plugin for executing the JMeter tests. So you do not have to download and setup JMeter in the remote machines. As long as maven is installed, it is fine.
  • Check the configuration section of this plugin for the run time properties we pass to the JMeter test.
  • More info on this is here.
• From JMeter v3.0, JMeter creates a very nice HTML dashboard for the non-gui tests. As maven has a standard folder structure we can not create folders on our own at run time as we did using ant. However maven has an ant plugin which could be very useful for this purpose.

That is it!!

We can run our JMeter test using maven as shown below

mvn clean verify

You would see below output!

 

Creating HTML Report:

Using maven-antrun-plugin, we can create a custom folder & create the new HTML dashboard. Execute below command.

mvn pre-site

The HTML report & charts would be created under target\jmeter\results\dashboard.

 

Running Distributed Testing using Maven:

Under the configuration section of  jmeter-maven-plugin, add below section for JMeter distributed testing.

Adding JMeter External Plugins:

Looks like mvn-jmeter-plugin is broken for JMeter v3.0. So I use the jmeter-maven-plugin 1.10.1 for JMeter 2.13, Add the required dependencies as shown here. [I have added the dependency for Parameterized Controller.] [ This section will be updated for JMeter 3.0 when i am able to add the plugins]

Summary:

By using jmeter-maven-plugin, we are able to run JMeter tests – whenever we build new artifact, we can also make our JMeter test executed & get the immediate feedback on functional and performance behavior of the application.

 

Happy Testing & Subscribe 🙂