QTP/UFT – Display Test execution progress in Jenkins Console

We had seen the QTP/UFT integration & triggering QTP test scripts remotely using Jenkins in this article. (If you have not read that, I recommend you to read the article first).


In this article we will see how to display the QTP/UFT results in the Jenkins console.

I have more than 5000 automated test scripts which need to be executed very often remotely which takes more than 8 hours to execute. When I first used Jenkins to run my QTP test scripts in the remote slave machines, I had to wait for several hours to see the results. It was very annoying.

The setup was given below.

uft001

Solution:

  • Runner.vbs uses QTP’s Automation Object Model (COM API) to launch, open the test & run..etc.
  • Runner.vbs once launches QTP,  QTP runs as a separate process -information is not exchanged between Runner.vbs & QTP.
  • As Runner.vbs uses QTP’s COM API, it can access QTP’s Environment object using which the information can be exchanged.

Updated Runner.vbs:

We will modify the Runner.vbs slightly to display the results in the console.

'Create QTP object
Set QTP = CreateObject("QuickTest.Application")
QTP.Launch
QTP.Visible = TRUE

'Open QTP Test
QTP.Open "QTP Test Path", TRUE 'Set the QTP test path

'Create Environment Variables to pass the results from QTP to runner.vbs
QTP.Test.Environment.Value("JenkinsFlag") = "N"
QTP.Test.Environment.Value("JenkinsTestCaseDescription") = ""
QTP.Test.Environment.Value("JenkinsTestCaseResult") = ""

'Set Result location
Set qtpResultsOpt = CreateObject("QuickTest.RunResultsOptions")
qtpResultsOpt.ResultsLocation = "Result path" 'Set the results location

'Run QTP test
QTP.Test.Run qtpResultsOpt, FALSE

'Write the result in the console
While QTP.Test.isRunning
	If QTP.Test.Environment.Value("JenkinsFlag") = "Y" Then
		QTP.Test.Environment.Value("JenkinsFlag") = "N"

		' Show TC ID and Description
		WScript.StdOut.WriteLine QTP.Test.Environment.Value("JenkinsTestCaseNumber") & " : " & QTP.Test.Environment.Value("JenkinsTestCaseDescription") & " : "

		'Wait till the test is executed & result is updated
		While (QTP.Test.Environment.Value("JenkinsTestCaseResult") = "" AND QTP.Test.isRunning)
				WScript.Sleep 1000
		Wend

		'Show the Result
		WScript.StdOut.Write QTP.Test.Environment.Value("JenkinsTestCaseResult")
	End If
	WScript.Sleep 1000
Wend

'Close QTP
QTP.Test.Close
QTP.Quit

Changes to be done in QTP:

We will also need to do some changes in QTP’s test to set those Environment variables.

Whenever the test starts – set these variables.

JenkinsFlag is used to tell jenkins that a new testcase starts.

JenkinsTestCaseDescription is used for the test case description.

Environment.Value("JenkinsFlag") = "Y" 
Environment.Value("JenkinsTestCaseDescription") = "Verify Google search results"

Runner.vbs will look for these variables continuously. As soon as JenkinsFlag is set to Y, it will display the testcase description in the console which user can see in Jenkins.

At the end of the test, update the result as given below.

Environment.Value("JenkinsTestCaseResult") = "PASS"

I am just giving you an idea here – I do not update this for each & every testcase. It is all part of my framework.

Jenkins Console:

Our Runner.vbs is modified to show the results & as given below – I do not have to wait to see the results anymore – as and when it executes, i will get the results immediately by accessing Jenkins job’s console.
uft002

 

Summary:

This post gives only a very high level idea about the implementation.  Check the below article for more details and the scripts to download.

Github / SVN integration for UFT scripts

 

Happy Testing 🙂

 

Share This:

16 thoughts on “QTP/UFT – Display Test execution progress in Jenkins Console

  1. Hi, My QTP test uses a Keyword driven framework. There is a driver script which reads each test case from excel and executes it. Excel sheet will be updated for Pass or Fail. Should i set the below Environment variables in each and every functions in my QTP test?

    Environment.Value(“JenkinsFlag”)
    Environment.Value(“JenkinsTestCaseDescription”)

    1. I am using a Hybrid framework & I want to report each testcase result as my automation suites has hundreds of testcases. Each testcase will have many steps. I do not need every step result in the jenkins console.

      But in your Keyword driven framework, as you write your entire testcases in a spreadsheet & driver is responsible for reading & executing it, You need to just include those environment variables in the driver script. Once a step is executed, you update these variables in driver before reading the next step.

  2. Thanks.
    One comment : QTP.Test.Environment.Value(“JenkinsTestCaseResult”) has the value placed on execution meanwhile the QTP.Test.isRunning. Your last execution could not have the correct value

    1. True. It happens for the last test. I have added a ‘wait 1’ after setting the value for QTP.Test.Environment.Value(“JenkinsTestCaseResult”) to make it work.

  3. Very Good Article

    It is working fine with few issues

    I am using Hybrid Framework and i have configured all the Environment Variables as mention by you. But my First Test Case is not displaying the Pass/Fail Status and for rest all test cases i am getting it successfully. Could you please guide me. Here is my Out Put

    Building in workspace C:\Jenkins\workspace\ICL_Sanity_Test
    [ICL_Sanity_Test] $ cmd /c call C:\WINDOWS\TEMP\hudson3178053962083818972.bat

    C:\Jenkins\workspace\ICL_Sanity_Test>C:\Windows\SysWOW64\cscript.exe Sanity.vbs
    Microsoft (R) Windows Script Host Version 5.8
    Copyright (C) Microsoft Corporation. All rights reserved.

    TC01 : Create Contact in CLC : .
    TC02 : Update Contact in CLC : Passed.
    TC03 : Add Contact to Location Relationship : Passed.
    TC04 : Search Contacts in CLC : Passed.
    TC05 : Create Location in CLC : Passed.
    TC06 : Update Location in CLC : Passed.

    HERE IS MY SANITY.VBS

    ‘Create UFT object

    Set App = CreateObject(“QuickTest.Application”)
    App.Launch
    App.Visible = True
    App.WindowState = “Maximized”
    App.ActivateView “ExpertView”

    ‘Open UFT Test
    App.Open “C:\Users\AB82164\Desktop\ICLAutomation\SanityDriver”, True

    ‘Create Environment Variables to pass the results from QTP to runner.vbs
    App.Test.Environment.Value(“JenkinsFlag”) = “N”
    App.Test.Environment.Value(“JenkinsTestCaseDescription”) = “”
    App.Test.Environment.Value(“JenkinsTestCaseResult”) = “”

    ‘Set Result location
    Set qtpResultsOpt = CreateObject(“QuickTest.RunResultsOptions”)
    qtpResultsOpt.ResultsLocation = “Result path” ‘Set the results location

    ‘Run QTP test
    App.Test.Run qtpResultsOpt, FALSE

    ‘Write the result in the console
    While App.Test.isRunning
    If App.Test.Environment.Value(“JenkinsFlag”) = “Y” Then
    App.Test.Environment.Value(“JenkinsFlag”) = “N”

    ‘Show TC ID ,TC Name and Result
    WScript.StdOut.WriteLine App.Test.Environment.Value(“JenkinsTestCaseNumber”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseDescription”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseResult”) & “.”

    End If
    Wend

    App.Test.Close
    App.Quit
    Set App = Nothing

    1. How long each testcase takes to execute?
      Sanity.vbs & QTP are 2 different processes & they need to talk each other. You need to set enough ‘wait’.

    1. See… You need 2 while loops. One outer loop which keeps executing as long as QTP runs.
      inner loop is for updating the result. Wait for App.Test.Environment.Value(“JenkinsTestCaseResult”) to be not null

  4. Thanks bro

    I have update the script like below as per your comment

    While App.Test.isRunning
    If App.Test.Environment.Value(“JenkinsFlag”) = “Y” Then
    App.Test.Environment.Value(“JenkinsFlag”) = “N”

    ‘Wait till the test is executed & result is updated
    While (App.Test.Environment.Value(“JenkinsTestCaseResult”) = “” AND App.Test.isRunning)
    WScript.Sleep 1000
    Wend

    ‘Show TC ID ,TC Name and Result
    WScript.StdOut.WriteLine App.Test.Environment.Value(“JenkinsTestCaseNumber”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseDescription”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseResult”) & “.”

    End If
    Wend

    MY Result Comes like this. No First test case, only second one displays twice 🙁

    TC02 : Update Contact in CLC : Passed.
    TC02 : Update Contact in CLC : Passed.
    TC03 : Add Contact to Location Relationship : Passed.
    TC04 : Search Contacts in CLC : Passed.
    TC05 : Create Location in CLC : Passed.

  5. Is there any way to create defects in Jira automatically for failed scripts. We want to implement in such a defects are automatically created for failed scripts. when we have a new build then if failed scripts gets passed then defects are auto updated accordingly.
    Please suggest.

    Regards,
    Sanjay

    1. Jira has API exposed for most of the functionalities. So It is possible to create defects/update existing defects programmatically. You need to check with your Jira admin to access the API.

  6. Hı mate,

    I’m trying to initiate a similar scenario but in linux instead of windows. Could you please guide me? I need to execute several scripts and show the outputs of those scripts in console output or email report via email plugin and so on.. Biggest problem I am having at the moment is that, when I execute a shell script to check mount points lets say, build passes successfully but I do not see any results in console output. So this is basically my problem.. Any ideas would be appreciated very much!

    Thanks in advance

    1. Sorry that i do not understand your question. If possible, please give more information.
      I would advise you to raise your question in StackOverFlow.com to get the help immediately.

  7. I have my UFT on VM and I am trying to hookup with Teamcity server on cloud , I am running into 2 issues
    1. When I run above vb script(runner.vb) from command prompt , it does not displays output in command window but runs the test successfully and generate result.xml etc
    2. When i call this script from teamcity server it fails where another scripts works fine .

    Thanks in advance .

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.