Best Practices – Automated Smoke Test

In this post, I would like to show how I have implemented automated smoke test in my project. It is based on the Hybrid framework I had implemented. I would request you to read the post on Hybrid Framework first if you have not.

 

Problem Statement:

We follow agile methodology & we have a release once in every 4 weeks. We have many environments like Production(of course, yes), Pre-Production, UAT, QA1-2, Dev1-2 etc. New code is pushed to the QA environments twice a day!! Few days prior to the release, We get builds in multiple environments like QA, UAT & PPE on the same day for different purposes. We needed to spend sometime to ensure that the new code does not break any basic functionalities of the application before doing actual testing. We also need to ensure that connectivity to the external systems works fine. It is called Smoke Test.

1. A set of test cases are executed repeatedly as and when we get new code on an environment.
2. We need to execute same set of test cases in multiple environments whenever the code is pushed to the environment. (Even though it is same code, there are a lot of configurations which affect the application behaviors).
3. Doing smoke test manually once takes 15 mins. This effort can be saved as we do that many times a day.
4. Dependency on the QA team. Apparently Developer/someone else does not want to do smoke test. If no tester is available at the time of code push, developers do not get the smoke test results immediately.
5. Very boring activity. 🙁

 

Solution:

The only solution is to automate the smoke testing process. The existing Hybrid framework can easily work here by creating a separate suite file for smoketest.

1. Identify Test cases: We have more than 5000 automated test cases included in the automated regression suite which covers almost all the features of the application. As part of smoke test, We need to pick the test cases to cover that basic functionalities of the application.
2. Create a suite file: Once the test cases are identified, create a suite file (Say, smoketest.suite) with the test case numbers identified.
3. Execute:  Runner.exe in the framework,  expects 3 arguments.
   • Environment in which we need do the smoke test
   • Browser: IE is default
   • Suite: smoketest.suite

Now runner.exe invokes QTP & executes the identified test cases & creates the result file.

Jenkins Integration:

By using existing framework & creating a suite file, we now have the smoke test created which will work for the given environment. But still we have the dependency on the QA team members to invoke it. How can be it solved?   Jenkins

[Not sure how to integrate? Please check this guide]

We already keep all the automated test cases in Stash repository. Jenkins can pull everything & trigger the runner.exe by passing the required arguments in the remote slave machine.

I created a separate job for Smoke Test in Jenkins as given below.

 

As Jenkins is a web based application, anyone can access it by using its URL. Whenever there is a code push, someone in the team (does not have to be a QA guy anymore) – they click on the build button by selecting the environment to trigger the smoke test on the environment.

Runner.exe also creates a nice HTML result file as shown below with the test cases details executed. Jenkins sends the smoke test result to the entire team members. In case of issues, developers can look into that immediately.

 

Are you interested in sending out emails as shown here? check here for the details steps.

 

More Details on UFT/Jenkins:

This post only shows a very high level idea about the implementation. If you are interested in implementing something similar for your organization, Please refer to below posts of TestAutomationGuru.

• QTP/UFT – Jenkins Integration
• Passing Parameters to UFT test from Jenkins
• Github / SVN integration for UFT scripts

 

Summary:

Thus an automated smoke test process was implemented by using the existing Hybrid framework. This practice i had implemented had a huge impact in the team & saved a lot of time, gave us immediate feedback on the build quality and reduced the dependency among the team members.

 

Happy Testing 🙂