JMeter is my favorite & one of the best tools I have ever used. I am glad that you too would like to learn more about JMeter. If you are a beginner, then the below given information might be useful to you.
When I started learning JMeter first, to become good at it, I used to clarify JMeter related questions in StackOverFlow. While doing so, I noticed that most of the new users have some common questions & do some mistakes. Aim of this post is to share the best practices on using JMeter.
I assume you have JMeter installed, you know how to record, playback, meaning & basic usage of some important elements like Thread Group, HTTP Sampler etc
Tip #1: Exclude Static Files
- When you record web application HTTP requests using JMeter proxy server, Do not include the static files like Javascript/CSS/Image files etc.
- For example, in one of my applications, there are few images which are getting loaded randomly. If I record them, I will always test the 1 image which I had recorded in my test.
- Another example would be – Lets say your application refers to jquery.1.1.min.js when you recorded script. Later the developer uses new version of jQuery lib. If you had included this request in your recorded script, then your script will always download wrong version jQuery lib.
- So, You can use the JMeter suggested exclusion list while recording.
- You can safely remove them even if they are recorded somehow in your existing scripts.
- Once you have excluded, you will have only the actual HTTP requests for the App server.
- When you playback the script, Enable the below options in the HTTP request.
- Enabling this option makes JMeter to behave like a real browser. Once it receives the HTTP response, if it needs to download any static files as part of the response, JMeter will download all those static files as a browser would.
- This will also make your script look neat and clean.
Tip #2: Use Transaction/Simple Controller
- When you record the browser actions, try to group the HTTP requests which are sent as part of the user action.
- That is, each and every user action needs to be placed under a transaction controller if possible.
- For example, as part of one button click, your application might make multiple HTTP AJAX requests.
- Consider a Credit Card online application. Upon login, there could be multiple portlets in the page which makes different calls to get the relevant information for the portlet.
- For the user, It is just one login action. But the application makes more than 1 HTTP request to load the page. So, You should group them and place them under one Transaction Controller. Give a meaningful name for the controller.
- Enabling parent sample will give the total time of individual requests.
- You can also use Simple Controller to logically group the requests if you do not want to use Transaction Controller.
Tip #3: Use HTTP Request Defaults
- Sometimes you might want to run a script in different environments. That is, You record the script in QA environment & planning to run that in staging environment later when it is available.
- To make your script to work in any test environment, Once the script is recorded, you can safely remove all the server name/IP, port protocol details etc.
- Add a HTTP Request Defaults as shown below & update the default environment details in one place for all the HTTP Requests in your test.
- Now you need to update only in one place whenever you have to run the test in different test environments. No need to update each and every HTTP requests in your test plan.
- You can also add 1 HTTP Request Defaults for each environment as shown below. Based on the environment you want to run your test, enable only the corresponding HTTP Request Defaults.
- Having 1 HTTP Request Defaults for each environment approach might not be best. But it is convenient. You can quickly select 1 and disable others to run the test.
- Best approach would be to use properties. See here for more information.
- If you are in the beginner level, Do not get confused with property files etc. Using this tip will do.
Tip #4: Add Config Elements
- If you are testing a web based application, then you need to add
- HTTP Cookie Manager to maintain the user session for the subsequent requests after login
- HTTP Cache Manager if you download the static files.
- I remember I wasted almost an hour trying to understand why I was always getting the login page even after successful login when I created my first JMeter test. I was missing HTTP Cookie Manager. This cookie manager is responsible for maintaining the cookies set by the application like a browser does.
- You can use User Defined Variables to maintain simple test data like login credentials, any other test data you had provided as part of your action in the browser.
- Variable name: username
- Variable value: guru
- In the test plan, you use ${username} to retrieve the value ‘guru’
- You can use CSV Data Set Config if you need to run the same script for different sets of data. Check here.
Tip #5: Correlation
- This is very very important & bit confusing for new JMeter users.
- Correlation in JMeter is handling dynamic data which was generated by the previous requests & posting it as part of the subsequent requests.
- For example, your web application generates a lot of dynamic data like session id which is going to be completely unique for the session for the user.
- Based on the session id only the server will know if the user has already logged in or not.
[Image courtesy – guru99]
- From the above images, when the wrong the session id is sent, the server will redirect you to the login page. Because we forgot to correlate.
- To find out which information is requires correlation, check your HTTP requests very carefully. If any of the parameters value is not entered by you, it might require correlation – [not all]
- For ex: below authid was not entered by me in the application. Apparently the server created it. Browser uses the information when I submit next page. All the browsers know how to handle such information.
- Our JMeter is like a browser but it does NOT know how to handle this. We need to take care of this explicitly.
- ‘authid’ should have been there in Login response when we send the login request. I need to find the information from the View Results Tree listener and use in the next ‘User Info’ request to run the script successfully.
- Always use Regular Expression Extractor to extract such information.
- If the response is in XML or JSON format, do not get tempted to use XPath extractor or JSON extractor. They are very bad for your test. Regular Expression Extractor will work just fine.
- Why are they bad? Check here.
Tip #6: Thread Count Vs Loop Count
- Having 10 users and 1 loop count VS 1 user and 10 loop count are not same even if we have 10 requests in total. [It might look obvious to some of you. I have faced this question from new users]
- Above setup simulates 10 concurrent users on the server. Each user sends only one request. Event though server processed 10 requests, it was sent by 10 different users almost at the same time. It puts more load on the server.
- For this test plan, at any point, the server has only one user. As and when the request is done, JMeter sends next request for the server to process. Server processed 10 requests one by one. It is very less load on the server.
- Use Ramp Up Period wisely. We should gradually increase the user. At least 1 user / second. Do not try to add 100 users in 1 second unless there is a clear requirement to do so.
Tip #7: Listener
- View Results Tree is the best listener for creating and debugging test scripts in JMeter.
- View Results Tree is also the worst listener to have while running your test plan for actual performance test of the application. It consumes A LOT of memory. It is guaranteed to get Out Of Memory error if you have this listener while running your scripts for multiple concurrent users. So, just use them only when you design your test plan. Remove it later.
- I have seen people using many View Results Tree listener in their test plan. Adding 1 listener in the Test Plan level is fine. You do not need sampler level listener.
- I also have noticed people adding many different listeners in the test plan as shown here – assuming each and every listener writes different information in the result file. That is NOT true.
It does not matter if you want to generate a Aggregate Report or Summary Report or Graph Results or any report. JMeter writes a same set of information as result in xml / csv format for all the result files.
- So no need to add many listeners & write the results to disk. One result file will do. Based on the type of report you need (Say aggregate or graph or throughput over time etc), JMeter reads the file and shows the result accordingly.
- I personally do NOT add any listener in my test plan while doing the performance test.
- Tip# 9 will provide more information on this.
Tip #8: Assertion
- Lets say I have a login request. I run the test. JMeter shows the request is success.
- I run the same test after providing invalid credentials. JMeter will still say that request is success!!!
- If you are surprised (Many people ask this in StackOverFlow & other forums), Yes it is expected.
First ask yourself – What is wrong with the application in not letting you login when you provide invalid credentials? So why do we expect JMeter should fail the request? Showing error message that credentials are invalid – is functionally right. As long as It is functionally right, JMeter will not show the fail message. JMeter decides success/failure based on the HTTP Response Code.
- To make JMeter fail the request as you expect, You need to add Assertions.
- Add assertion wherever required. But do not add more assertion as it might affect the performance of the test. Each and every element in the test plan takes some time to execute & consumes some memory. As part of assertions JMeter needs to parse the whole HTTP response & decide success or failure. So use it wisely.
Tip #9: Running the Test
- You should ALWAYS RUN the test in NON-GUI mode. This is very important.
- Check here for the commands which can be used in non-gui mode. GUI mode consumes more memory.
- A simple non-gui command to run the JMeter test would be as shown below.
jmeter -n -t testplan.jmx -l result.jtl
- –l result.jtl –> this option is responsible for creating the JMeter result file in the current location of the test plan.
- It is better to save the results in CSV format as file size would be relatively less and fatser. Default format is XML.
jmeter -n -t testplan.jmx -l result.jtl -Djmeter.save.saveservice.output_format=csv
- Once the result is file created, Launch JMeter GUI. Add your favorite listener (Aggregate / Graph / Throughput over time etc) . Then open the result.jtl file to get the information.
Tip #10: Tweaking JVM
- It is good to have 64 bit OS to run JMeter test. 32 bit OS can not have more than 4GB RAM.
- There is no hard and fast rule for the max number of users you can go up to in one machine. It is completely depends on the machine configuration.
- Check your heap space set for JMeter. You can increase it – You can go up to 80% of your usable RAM.
JVM_ARGS="-Xms512m -Xmx512m" jmeter.bat
- You need to adjust heap space if you face Out Of Memory error.
- Check your CPU / Memory usage while the test is running or at max load. As a load generator, your machine should be running below 80% of CPU usage mostly. If not, then there is a chance that the results could be bogus.
- If the machine is not able to generate the load you were expecting, then go for distributed load testing. More info on this can be found here.
Happy Testing 🙂
