JMeter – Distributed Load Testing using Docker

vIns

8 years ago

Overview:

A single JMeter instance might not be able to generate enough load to stress test your application. As this site shows, one JMeter instance will be able to control many other remote JMeter instances and generate larger load on your application. JMeter uses Java RMI [Remote Method Invocation] to interact with objects in a distributed network.

JMeter master and slave communicate as shown in the below picture.

We need to open 2 ports for each Slave/Server.

Server_port=1099
server.rmi.localport=50000

Open a port in client machine for slaves to sends the results to master.

client.rmi.localport=60000

By running multiple instances of JMeter as server in multiple machines we can generate as much load as we need.

Docker:

What is the use of docker here?

Docker is a bit like a virtual machine. But unlike a virtual machine, rather than creating a whole virtual operating system, Docker allows applications to use the same Linux kernel as the system that they’re running on and only requires applications be shipped with things not already running on the host computer. This gives a significant performance boost and reduces the size of the application – source: opensource.com

Docker is a manager of Infrastructure. It will be able to package a software and all its dependencies to run as a container. You can deploy the software, packaged as a docker image, in any machine where docker is installed. It, kind of, separates the software from the hardware – so the developer can rest assured that the application will run on any machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code.

Docker’s role in JMeter Distributed Testing:

If we look at the above setup – to do distributed load testing – we need 1 master & we need N number of slaves to generate huge load. Each and every JMeter slave machine needs to have specific version of Java and JMeter installed. Specific ports should be opened and JMeter server should be running, ready and waiting for the master to send the instruction.

Setting up few machines manually might look easy. What if we have to do this for 50, 100, 1000 machines? Also imagine what will happen if we need to upgrade JMeter versions in all the machines in future!! That is where docker comes into picture.

We basically setup the whole infrastructure for JMeter distributed testing in a file called Dockerfile. Check these dockerfiles and read the comments to understand what each step does.

Dockerfile for JMeter Base:

In distributed testing, all the environment are expected to have same version of Java, JMeter, plugins etc. Only difference between the master and slave would be the ports which are exposed and process running. So, Lets create a Dockerfile which has all the common steps for both master and slave. Lets call it as jmbase image and we would need to do the followings to build our base image.

We need Java8 – so lets openjdk-8-jre slim version to keep size as less as possible
We might need few utilities like wget, unzip, telnet etc. So lets install them.
We need latest version of JMeter. Create a variable for version – so that maintenance would be easier in future.
Add a folder which contains all the plugins.
Add a folder which contains a sample test.

# Use Java 8 slim JRE
FROM openjdk:8-jre-slim
MAINTAINER TestAutomationGuru

# JMeter version
ARG JMETER_VERSION=3.3

# Install few utilities
RUN apt-get clean && \
    apt-get update && \
    apt-get -qy install \
                wget \
                telnet \
                iputils-ping \
                unzip

# Install JMeter
RUN   mkdir /jmeter \
      && cd /jmeter/ \
      && wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-$JMETER_VERSION.tgz \
      && tar -xzf apache-jmeter-$JMETER_VERSION.tgz \
      && rm apache-jmeter-$JMETER_VERSION.tgz

# ADD all the plugins
ADD jmeter-plugins/lib /jmeter/apache-jmeter-$JMETER_VERSION/lib

# ADD the sample test
ADD sample-test sample-test

# Set JMeter Home
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VERSION/

# Add JMeter to the Path
ENV PATH $JMETER_HOME/bin:$PATH

Dockerfile for JMeter Client / Master:

Master docker file should be inherited from the base image and should expose port 60000.

# Use vinsdocker base image
FROM vinsdocker/jmbase
MAINTAINER TestAutomationGuru

# Ports to be exposed from the container for JMeter Master
EXPOSE 60000

Dockerfile for JMeter Server / Slave:

Server docker file should be inherited from the base image and should expose port 1099 and 50000. jmeter-server should be running.

# Use vinsdocker base image
FROM vinsdocker/jmbase
MAINTAINER TestAutomationGuru

# Ports to be exposed from the container for JMeter Slaves/Server
EXPOSE 1099 50000

# Application to run on starting the container
ENTRYPOINT $JMETER_HOME/bin/jmeter-server \
                        -Dserver.rmi.localport=50000 \
                        -Dserver_port=1099

As you see in the above Dockerfile, if we need to change the Java / JMeter version / port, I just need to update the dockerfile and Docker will take care of the rest.

I have pushed these dockerfiles into docker hub under vinsdocker account. So anyone will be able to pull those files and set up the JMeter distributed testing infrastructure.

Ensure that docker is installed in your machine. Once it is installed, the rest is easy. You just need to follow the steps here.
Run below commands one by one.

sudo docker run -dit --name slave01 vinsdocker/jmserver /bin/bash
sudo docker run -dit --name slave02 vinsdocker/jmserver /bin/bash
sudo docker run -dit --name slave03 vinsdocker/jmserver /bin/bash

Docker will automatically pull the docker image I have uploaded and create 3 containers for JMeter server. If you need more containers, keep executing above command just by changing the container name.

Run the below command to create a container for JMeter master.

sudo docker run -dit --name master vinsdocker/jmmaster /bin/bash

Run below command to see all the running containers and ports opened etc.

sudo docker ps -a

Run the below command to get the list of ip addresses for these containers.

sudo docker inspect --format '{{ .Name }} => {{ .NetworkSettings.IPAddress }}' $(sudo docker ps -a -q)

I have included a sample-test which runs for 30 seconds with 5 concurrent users – in the docker image – which you could see inside the container. Path: /sample-test/sample-test.jmx

In case – you need to copy any files from the host to the docker container – You can issue below command. For ex: I copy the test into my JMeter master container. This command will copy my local jmeter test (docker-test.jmx) into the master container in this path: /jmeter/apache-jmeter-3.3/bin/docker-test.jmx

sudo docker exec -i master sh -c 'cat > /jmeter/apache-jmeter-3.3/bin/docker-test.jmx' < docker-test.jmx

Go inside the container with the below command and we can see if the file has been copied successfully.

sudo docker exec -it master /bin/bash

Lets run the test in master to see if it works fine [not in distributed mode]. Docker container will be able to run the JMeter test as it has all the softwares & dependencies to run the JMeter test.

jmeter -n -t sample-test/sample-test.jmx
Creating summariser <summary>
Created the tree successfully using sample-test/sample-test.jmx
Starting the test @ Thu Dec 21 17:14:59 UTC 2017 (1513876499683)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary +      1 in 00:00:01 =    1.5/s Avg:   265 Min:   265 Max:   265 Err:     0 (0.00%) Active: 1 Started: 1 Finished: 0
summary +    336 in 00:00:29 =   11.4/s Avg:   112 Min:    87 Max:   325 Err:     0 (0.00%) Active: 5 Started: 5 Finished: 0
summary =    337 in 00:00:30 =   11.2/s Avg:   113 Min:    87 Max:   325 Err:     0 (0.00%)
summary +      4 in 00:00:00 =  210.5/s Avg:    97 Min:    93 Max:   109 Err:     0 (0.00%) Active: 0 Started: 5 Finished: 5
summary =    341 in 00:00:30 =   11.3/s Avg:   113 Min:    87 Max:   325 Err:     0 (0.00%)
Tidying up ...    @ Thu Dec 21 17:15:30 UTC 2017 (1513876530127)
... end of run

That’s it. We are now ready for running our test in distributed using docker containers. We just need to append -R[slave01,slave02,slave03]

jmeter -n -t sample-test/sample-test.jmx -R172.17.0.5,172.17.0.6,172.17.0.7
Creating summariser <summary>
Created the tree successfully using sample-test/sample-test.jmx
Configuring remote engine: 172.17.0.5
Configuring remote engine: 172.17.0.6
Configuring remote engine: 172.17.0.7
Starting remote engines
Starting the test @ Thu Dec 21 17:01:48 UTC 2017 (1513875708955)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary +      4 in 00:00:11 =    0.4/s Avg:   182 Min:    98 Max:   232 Err:     0 (0.00%) Active: 15 Started: 15 Finished: 0
summary +   1021 in 00:00:20 =   51.5/s Avg:   111 Min:    85 Max:   283 Err:     0 (0.00%) Active: 0 Started: 15 Finished: 15
summary =   1025 in 00:00:30 =   33.7/s Avg:   111 Min:    85 Max:   283 Err:     0 (0.00%)
Tidying up remote @ Thu Dec 21 17:02:20 UTC 2017 (1513875740196)
... end of run

If you had noticed, we create all the containers in the same host. Ie, the JMeter and JMeter slaves are all running in the same machine. So the all the system resources would be shared by these containers.

Summary:

In this post, our aim was to use Docker to create the JMeter distributed testing infrastructure. If you had followed the above steps, you would have understood that creating the test infrastructure using docker is very easy and fast. We write the whole infrastructure in a file which can be version controlled. Then we create an instance (container) from the file. Docker ensures that the container has all the softwares and dependencies etc.

You might ask if it is ok to run multiple jmeter server instances in one machine to generate more load! No, It is not OK. It will not help at all. In fact, One instance of JMeter will be able to generate more load than running multiple instances of JMeter in the same host.

So why did we use docker and do all these?

As I said above, our aim here is to understand how docker works in JMeter testing. We can understand the real use of docker when we use AWS/digitalocean, cloud computing service providers, where you can create any number of VMs on demand. We will see that in the next post!

Happy Testing & Subscribe 🙂

Note: If you have any questions related to docker install, I request you to raise that in StackOverFlow.