Overview:
A single JMeter instance might not be able to generate enough load to stress test your application. As this site shows, one JMeter instance will be able to control many other remote JMeter instances and generate larger load on your application. JMeter uses Java RMI [Remote Method Invocation] to interact with objects in a distributed network.
JMeter master and slave communicate as shown in the below picture.
We need to open 2 ports for each Slave/Server.
Server_port=1099
server.rmi.localport=50000
Open a port in client machine for slaves to sends the results to master.
client.rmi.localport=60000
By running multiple instances of JMeter as server in multiple machines we can generate as much load as we need.
Docker:
What is the use of docker here?
Docker is a bit like a virtual machine. But unlike a virtual machine, rather than creating a whole virtual operating system, Docker allows applications to use the same Linux kernel as the system that they’re running on and only requires applications be shipped with things not already running on the host computer. This gives a significant performance boost and reduces the size of the application – source: opensource.com
Docker is a manager of Infrastructure. It will be able to package a software and all its dependencies to run as a container. You can deploy the software, packaged as a docker image, in any machine where docker is installed. It, kind of, separates the software from the hardware – soΒ the developer can rest assured that the application will run on any machine regardless of any customized settings that machine might have that could differ from the machine used for writing and testing the code.
Docker’s roleΒ in JMeter Distributed Testing:
If we look at the above setup – to do distributed load testing – we need 1 master & we need N number of slaves to generate huge load. Each and every JMeter slave machine needs to have specific version of Java and JMeter installed. Specific ports should be opened and JMeter server should be running, readyΒ and waiting for the master to send the instruction.
Setting up few machines manually might look easy. What if we have to do this for 50, 100, 1000 machines? Also imagine what will happen if we need to upgrade JMeter versions in all the machines in future!! That is where docker comes into picture.
We basically setup the whole infrastructure for JMeter distributed testing in a file called Dockerfile. Check these dockerfiles and read the comments to understand what each step does.
Dockerfile for JMeter Base:
In distributed testing, all the environment are expected to have same version of Java, JMeter, plugins etc. Only difference between the master and slave would be the ports which are exposed and process running. So, Lets create a Dockerfile which has all the common steps for both master and slave. Lets call it as jmbase image and we would need to do the followings to build our base image.
- We need Java8 – so lets openjdk-8-jre slim version to keep size as less as possible
- We might need few utilities like wget, unzip, telnet etc. So lets install them.
- We need latest version of JMeter. Create a variable for version – so thatΒ maintenance would be easier in future.
- Add a folder which contains all the plugins.
- Add a folder which contains a sample test.
# Use Java 8 slim JRE
FROM openjdk:8-jre-slim
MAINTAINER TestAutomationGuru
# JMeter version
ARG JMETER_VERSION=3.3
# Install few utilities
RUN apt-get clean && \
apt-get update && \
apt-get -qy install \
wget \
telnet \
iputils-ping \
unzip
# Install JMeter
RUN mkdir /jmeter \
&& cd /jmeter/ \
&& wget https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-$JMETER_VERSION.tgz \
&& tar -xzf apache-jmeter-$JMETER_VERSION.tgz \
&& rm apache-jmeter-$JMETER_VERSION.tgz
# ADD all the plugins
ADD jmeter-plugins/lib /jmeter/apache-jmeter-$JMETER_VERSION/lib
# ADD the sample test
ADD sample-test sample-test
# Set JMeter Home
ENV JMETER_HOME /jmeter/apache-jmeter-$JMETER_VERSION/
# Add JMeter to the Path
ENV PATH $JMETER_HOME/bin:$PATH
Dockerfile for JMeter Client / Master:
Master docker file should be inherited from the base image and should expose port 60000.
# Use vinsdocker base image
FROM vinsdocker/jmbase
MAINTAINER TestAutomationGuru
# Ports to be exposed from the container for JMeter Master
EXPOSE 60000
Dockerfile for JMeter Server / Slave:
ServerΒ docker file should be inherited from the base image and should expose port 1099 and 50000. jmeter-server should be running.
# Use vinsdocker base image
FROM vinsdocker/jmbase
MAINTAINER TestAutomationGuru
# Ports to be exposed from the container for JMeter Slaves/Server
EXPOSE 1099 50000
# Application to run on starting the container
ENTRYPOINT $JMETER_HOME/bin/jmeter-server \
-Dserver.rmi.localport=50000 \
-Dserver_port=1099
As you see in the above Dockerfile, if we need to change the Java / JMeter version / port, I just need to update the dockerfile and Docker will take care of the rest.
I have pushed these dockerfiles into docker hub under vinsdockerΒ account. So anyone will be able to pull those files and set up the JMeter distributed testing infrastructure.
- Ensure that docker is installed in your machine. Once it is installed, the rest is easy. You just need to follow the steps here.
- Run below commands one by one.
sudo docker run -dit --name slave01 vinsdocker/jmserver /bin/bash
sudo docker run -dit --name slave02 vinsdocker/jmserver /bin/bash
sudo docker run -dit --name slave03 vinsdocker/jmserver /bin/bash
Docker will automatically pull the docker image I have uploaded and create 3 containers for JMeter server. If you need more containers, keep executing above command just by changing the container name.
- Run the belowΒ command to create a container for JMeter master.
sudo docker run -dit --name master vinsdocker/jmmaster /bin/bash
- Run below command to seeΒ all the running containers and ports opened etc.
sudo docker ps -a
- Run the below command to get the list of ip addresses for these containers.
sudo docker inspect --format '{{ .Name }} => {{ .NetworkSettings.IPAddress }}' $(sudo docker ps -a -q)
- I have included a sample-test which runs for 30 seconds with 5 concurrent users – in the docker image – which you could see inside the container. Path:Β /sample-test/sample-test.jmx
- In case – you need to copy any files from the host to the docker container –Β You can issue below command. For ex: I copy the test into my JMeter master container. This command will copy my local jmeter test (docker-test.jmx) into the master container in this path:Β /jmeter/apache-jmeter-3.3/bin/docker-test.jmx
sudo docker exec -i master sh -c 'cat > /jmeter/apache-jmeter-3.3/bin/docker-test.jmx' < docker-test.jmx
- Go inside the container with the below command and we can see if the file has been copied successfully.
sudo docker exec -it master /bin/bash
- Lets run the test in master to see if it works fine [not in distributed mode]. Docker container will be able to run the JMeter test as it has all the softwares & dependencies to run the JMeter test.
jmeter -n -t sample-test/sample-test.jmx
Creating summariser <summary>
Created the tree successfully using sample-test/sample-test.jmx
Starting the test @ Thu Dec 21 17:14:59 UTC 2017 (1513876499683)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 1 in 00:00:01 = 1.5/s Avg: 265 Min: 265 Max: 265 Err: 0 (0.00%) Active: 1 Started: 1 Finished: 0
summary + 336 in 00:00:29 = 11.4/s Avg: 112 Min: 87 Max: 325 Err: 0 (0.00%) Active: 5 Started: 5 Finished: 0
summary = 337 in 00:00:30 = 11.2/s Avg: 113 Min: 87 Max: 325 Err: 0 (0.00%)
summary + 4 in 00:00:00 = 210.5/s Avg: 97 Min: 93 Max: 109 Err: 0 (0.00%) Active: 0 Started: 5 Finished: 5
summary = 341 in 00:00:30 = 11.3/s Avg: 113 Min: 87 Max: 325 Err: 0 (0.00%)
Tidying up ... @ Thu Dec 21 17:15:30 UTC 2017 (1513876530127)
... end of run
- That’s it. We are now ready for running our test in distributed using docker containers. We just need to append -R[slave01,slave02,slave03]
jmeter -n -t sample-test/sample-test.jmx -R172.17.0.5,172.17.0.6,172.17.0.7
Creating summariser <summary>
Created the tree successfully using sample-test/sample-test.jmx
Configuring remote engine: 172.17.0.5
Configuring remote engine: 172.17.0.6
Configuring remote engine: 172.17.0.7
Starting remote engines
Starting the test @ Thu Dec 21 17:01:48 UTC 2017 (1513875708955)
Remote engines have been started
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 4 in 00:00:11 = 0.4/s Avg: 182 Min: 98 Max: 232 Err: 0 (0.00%) Active: 15 Started: 15 Finished: 0
summary + 1021 in 00:00:20 = 51.5/s Avg: 111 Min: 85 Max: 283 Err: 0 (0.00%) Active: 0 Started: 15 Finished: 15
summary = 1025 in 00:00:30 = 33.7/s Avg: 111 Min: 85 Max: 283 Err: 0 (0.00%)
Tidying up remote @ Thu Dec 21 17:02:20 UTC 2017 (1513875740196)
... end of run
If you had noticed, we create all the containers in the same host. Ie, the JMeter and JMeter slaves are all running in the same machine. So the all the system resources would be shared by these containers.
Summary:
In this post, our aim was to use Docker to create the JMeter distributed testing infrastructure. If you had followed the above steps, you would have understood that creating the test infrastructure using docker is very easy and fast. We write the whole infrastructure in a file which can be version controlled. Then we create an instance (container) from the file. Docker ensures that the container has all the softwares and dependencies etc.
You might ask if it is ok to run multiple jmeter server instances in one machine to generate more load! No, It is not OK. It will not help at all. In fact, One instance of JMeter will be able to generate more load than running multiple instances of JMeter in the same host.
So why did we use docker and do all these?
As IΒ said above, Β our aim here is to understand how docker works in JMeter testing. We can understand the real use of docker when we use AWS/digitalocean, cloud computing service providers, where you can create any number of VMs on demand. We will see that in the next post!
Happy Testing & Subscribe π
Note: If you have any questions related to docker install, I request you to raise that in StackOverFlow.
Awesome, helpful article. However, providing the sample jmx file here would have been helpful. You can’t really make much out from the partial screenshot.
Sean, any simple jmx file should work here. From the screenshot, you can figure out what i have. A thread group (5 users , 120 sec duration) + debug sampler + a constant timer with 300ms delay.
Very nice artical but it will be benefited only when if I tried to implement π
Then you should implement soon π
Looks like you are using some 3rd party jmeter plugins in your test plan which might not be available in the docker container. create a new image yourself with required setup.
Nice Article. But how can we get the reports after the run ? From 3.0 on wards we have nice html reports after the test runs. Similarly where the .jtl file will get saved in this case.
This article just shows that we can use docker for distributed testing. Yours is a valid question. The result can be saved in the host machine. So the result file will be present in the host folder even after destroying the docker containers. We need to pass certain arguments while creating the container from the image. https://docs.docker.com/engine/tutorials/dockervolumes/
We would love to see docker compose version of it if you can provide. Cheers
It is already here – please check all articles under docker – http://www.testautomationguru.com/category/docker/
Ouch, Excellent!!
This document was a life saver. Nicely written with easy to follow step by step instructions. Thank you!
While I was able to complete the setup following these instructions and execute a small test using a script developed on version 2.13, the scripts that I created on Jmeter version 3.1 did not work. Can you please advise on how to upgrade the version to 3.1 in the docker file? I’m pretty new to Docker.
I have updated docker images with JMeter v3.3
Thanks. Much appreciated!
Hi, great post. One thing. Do you have info relating to running a master in a container on one host and then a slave on another container on another host?
Yes, check this one – http://www.testautomationguru.com/jmeter-distributed-load-testing-using-docker-rancheros-in-cloud/
Hi,
I am trying to run Jmeter server in EC2 environment but it exit without run. Could you please let me know what could be the issue?
[root@ip-172-31-31-168 ec2-user]# sudo docker run -dit -e LOCALIP=’172.31.31.168β -p 1099:1099 -p 50000:50000 vinsdocker/jmawsserver /bin/bash
38f2dde04b9130897f1c07b64df32713990d40c93daf4c204dc6efaa55472b4e
[root@ip-172-31-31-168 ec2-user]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
38f2dde04b91 vinsdocker/jmawsserver “/bin/sh -c ‘$JMETER⦔ 13 seconds ago Exited (0) 11 seconds ago admiring_northcutt
I would suggest you to check this http://www.testautomationguru.com/jmeter-distributed-load-testing-using-docker-in-aws/
Great Article!
I am getting the following error when trying to start the master container:
An error occurred:
No X11 DISPLAY variable was set, but this program performed an operation which requires it.
Any Suggestions
You need to run the test in the non-gui mode- so provide the option -n. For ex: jmeter -n -t test.jmx
This is really helpful .
QQ :
Is it possible to define how much load should run on each slave?
Yes. But it requires a little bit more work. But easily doable using kubernetes
Hi Vinoth,
Thank you for the post. Its really helpful. I am using jmeter version 5.1.1 in docker file and facing below issue. Please help.
Error : java.io.FileNotFoundException: rmi_keystore.jks (No such file or directory)
Thats a common issue with jmeter 4 or above. I have seen this. You need to update jmeter server properties. you need to follow the steps given here – http://jmeter.apache.org/usermanual/remote-test.html#setup_ssl
This is an fantastic article to work out..
I have tried to build the same image but im not getting the IPs for the containers for the Image i build. Welcome any input on this.