JMeter – How to create a Data + Keyword Driven Framework for Performance Testing

We might have heard/seen/created a Keyword driven framework for functional testing automation using QTP / Selenium WebDriver (TestAutomationGuru has one for WebDriver here). Have you ever heard of a keyword driven framework for JMeter for Performance Testing? I have not!

In this article, Lets see how to create a Data + Keyword driven framework for JMeter. (Data + Keyword driven frameworks are also called Hybrid Frameworks)

Data Driven Framework:

Creating a data driven framework is very easy in JMeter. You come up with a test plan for a business work flow; remove any hard coded test data and parameterize it to work for the data from a file / DB. Most of us would have done this using CSV Data Set Config test element in JMeter.

Keyword Driven Framework:

Keyword driven framework separates the test script implementation from the test case design. We create keywords for each and every business action / functionality of the application. By calling these actions/functions in a specific order, we execute the business workflow.

For Example:

Lets assume we have a web based application in which an user can register to create an account to book flight ticket, view the ticket details, edit, cancel etc. An already registered user can also do all these activities. If the user forgets the credentials, he can use ‘forgot password’ page to get the credentials.

Considering the above requirements, We could create different functions for below functionalities.

• Register New User
• Login
• Logout
• Book Ticket
• Edit Ticket
• View Ticket
• Cancel Ticket
• Forgot Password

All these functions should be created generic & data driven. For example – ‘Book Ticket’  is a high level keyword / a module – but JMeter script for this transaction should be created in a such a way that it can be data driven. Ex: It should accept any type of payment information like CC, Promocodes etc.

• Book Ticket
  • Accepts VISA, Master Card, American Express, Discover etc

We could come up with below test cases for the above requirement.

If we introduce the ‘Search Flight’ functionality in future in the application – By adding a Keyword for it, you come up with more test cases considering different combinations.

Creating Reusable Modules:

It has been explained in detail here.

This can be achieved by placing the required samples for each transaction/functionality inside a Simple / Transaction Controller & make a call to the controller using a Module Controller.

So I create a transaction controller for each business action and place them under Test Fragments as shown here.

Since any action can be called randomly using a ‘Keyword’, Lets place these Transaction Controllers under a Switch Controller. By passing the Transaction Controller name as the ‘Keyword’ or Switch Value, we can execute specific Transaction.

I rename Test Fragment & Switch Controller elements names as shown below & I also use a variable action for Switch Value.

 

Creating a Test Script:

We have created the reusable modules & we also know the business requirements. Now we need to come up with the list of test cases to be executed. Advantage of using a Keyword / Hybrid framework is we create the test cases to be executed in a spreadsheet / in a table format. Any non-technical user can understand this easily. If you need to create a new workflow – you just rearrange the keywords & update test data accordingly. You do not touch the JMeter script. (assuming JMeter script is well written & can handle any type of input).

So I create my test cases in a CSV file as shown here. ‘keywords’ are ‘;’ separated.

 

Creating a Keyword Execution Engine:

Thread Group will be the driver/keyword execution engine. So I add a Thread Group & a CSV Data Set Config to read the test cases from the CSV file.

Variable ‘keywords’ will have the list of all the keywords to be executed. I need to split them & store them in different variables – so that I can call these keywords one by one to complete the workflow.

keywords=Register New User;Book Ticket;Logout

I can use JMeter’s split function for this.

${__split(${keywords},keyword,;)}

The split function would create new variables as given below.

keyword_1=Register New User
keyword_2=Book Ticket
keyword_3=Logout

 

Add a ForEach Controller to iterate all the keywords for the test case & Module Controller to call the keyword via Switch Controller in the Test Fragment.

 

Execution:

Lets add a View Results Tree, execute & see – what is happening!!

 

JMeter reads the test case file & based on the keywords mentioned in the file, it executes the specific transaction in the given order. If you need more scenarios/different combinations, you need to rearrange the keywords / update test data accordingly. The whole business work flow is decided at run time based on keywords in the spreadsheet.

 

Source code:

I have uploaded the above test plan in GitHub for you to download.

 

Summary:

When I was learning JMeter, I was very curious If we could achieve something like this. I wanted to try & did this few years ago – at last sharing it now. [But mainly I used this for Functional automation testing].  Initially it takes significant amount of effort in creating the reusable library. Using a Hybrid framework with well designed scripts, We can create business workflow as we need in a spreadsheet without even touching the JMeter test script. Each and every framework has its own pros and cons. It is upto you to choose the best one for you!!

 

Happy Testing & Subscribe  🙂