We had seen the QTP/UFT integration & triggering QTP test scripts remotely using Jenkins in this article. (If you have not read that, I recommend you to read the article first).
In this article we will see how to display the QTP/UFT results in the Jenkins console.
I have more than 5000 automated test scripts which need to be executed very often remotely which takes more than 8 hours to execute. When I first used Jenkins to run my QTP test scripts in the remote slave machines, I had to wait for several hours to see the results. It was very annoying.
The setup was given below.
Solution:
- Runner.vbs uses QTP’s Automation Object Model (COM API) to launch, open the test & run..etc.
- Runner.vbs once launches QTP, QTP runs as a separate process -information is not exchanged between Runner.vbs & QTP.
- As Runner.vbs uses QTP’s COM API, it can access QTP’s Environment object using which the information can be exchanged.
Updated Runner.vbs:
We will modify the Runner.vbs slightly to display the results in the console.
'Create QTP object Set QTP = CreateObject("QuickTest.Application") QTP.Launch QTP.Visible = TRUE 'Open QTP Test QTP.Open "QTP Test Path", TRUE 'Set the QTP test path 'Create Environment Variables to pass the results from QTP to runner.vbs QTP.Test.Environment.Value("JenkinsFlag") = "N" QTP.Test.Environment.Value("JenkinsTestCaseDescription") = "" QTP.Test.Environment.Value("JenkinsTestCaseResult") = "" 'Set Result location Set qtpResultsOpt = CreateObject("QuickTest.RunResultsOptions") qtpResultsOpt.ResultsLocation = "Result path" 'Set the results location 'Run QTP test QTP.Test.Run qtpResultsOpt, FALSE 'Write the result in the console While QTP.Test.isRunning If QTP.Test.Environment.Value("JenkinsFlag") = "Y" Then QTP.Test.Environment.Value("JenkinsFlag") = "N" ' Show TC ID and Description WScript.StdOut.WriteLine QTP.Test.Environment.Value("JenkinsTestCaseNumber") & " : " & QTP.Test.Environment.Value("JenkinsTestCaseDescription") & " : " 'Wait till the test is executed & result is updated While (QTP.Test.Environment.Value("JenkinsTestCaseResult") = "" AND QTP.Test.isRunning) WScript.Sleep 1000 Wend 'Show the Result WScript.StdOut.Write QTP.Test.Environment.Value("JenkinsTestCaseResult") End If WScript.Sleep 1000 Wend 'Close QTP QTP.Test.Close QTP.Quit
Changes to be done in QTP:
We will also need to do some changes in QTP’s test to set those Environment variables.
Whenever the test starts – set these variables.
JenkinsFlag is used to tell jenkins that a new testcase starts.
JenkinsTestCaseDescription is used for the test case description.
Environment.Value("JenkinsFlag") = "Y" Environment.Value("JenkinsTestCaseDescription") = "Verify Google search results"
Runner.vbs will look for these variables continuously. As soon as JenkinsFlag is set to Y, it will display the testcase description in the console which user can see in Jenkins.
At the end of the test, update the result as given below.
Environment.Value("JenkinsTestCaseResult") = "PASS"
I am just giving you an idea here – I do not update this for each & every testcase. It is all part of my framework.
Jenkins Console:
Our Runner.vbs is modified to show the results & as given below – I do not have to wait to see the results anymore – as and when it executes, i will get the results immediately by accessing Jenkins job’s console.
Summary:
This post gives only a very high level idea about the implementation. Check the below article for more details and the scripts to download.
Github / SVN integration for UFT scripts
Happy Testing 🙂
Thanks
Hi, My QTP test uses a Keyword driven framework. There is a driver script which reads each test case from excel and executes it. Excel sheet will be updated for Pass or Fail. Should i set the below Environment variables in each and every functions in my QTP test?
Environment.Value(“JenkinsFlag”)
Environment.Value(“JenkinsTestCaseDescription”)
I am using a Hybrid framework & I want to report each testcase result as my automation suites has hundreds of testcases. Each testcase will have many steps. I do not need every step result in the jenkins console.
But in your Keyword driven framework, as you write your entire testcases in a spreadsheet & driver is responsible for reading & executing it, You need to just include those environment variables in the driver script. Once a step is executed, you update these variables in driver before reading the next step.
Thanks.
One comment : QTP.Test.Environment.Value(“JenkinsTestCaseResult”) has the value placed on execution meanwhile the QTP.Test.isRunning. Your last execution could not have the correct value
True. It happens for the last test. I have added a ‘wait 1’ after setting the value for QTP.Test.Environment.Value(“JenkinsTestCaseResult”) to make it work.
Very Good Article
It is working fine with few issues
I am using Hybrid Framework and i have configured all the Environment Variables as mention by you. But my First Test Case is not displaying the Pass/Fail Status and for rest all test cases i am getting it successfully. Could you please guide me. Here is my Out Put
Building in workspace C:\Jenkins\workspace\ICL_Sanity_Test
[ICL_Sanity_Test] $ cmd /c call C:\WINDOWS\TEMP\hudson3178053962083818972.bat
C:\Jenkins\workspace\ICL_Sanity_Test>C:\Windows\SysWOW64\cscript.exe Sanity.vbs
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.
TC01 : Create Contact in CLC : .
TC02 : Update Contact in CLC : Passed.
TC03 : Add Contact to Location Relationship : Passed.
TC04 : Search Contacts in CLC : Passed.
TC05 : Create Location in CLC : Passed.
TC06 : Update Location in CLC : Passed.
HERE IS MY SANITY.VBS
‘Create UFT object
Set App = CreateObject(“QuickTest.Application”)
App.Launch
App.Visible = True
App.WindowState = “Maximized”
App.ActivateView “ExpertView”
‘Open UFT Test
App.Open “C:\Users\AB82164\Desktop\ICLAutomation\SanityDriver”, True
‘Create Environment Variables to pass the results from QTP to runner.vbs
App.Test.Environment.Value(“JenkinsFlag”) = “N”
App.Test.Environment.Value(“JenkinsTestCaseDescription”) = “”
App.Test.Environment.Value(“JenkinsTestCaseResult”) = “”
‘Set Result location
Set qtpResultsOpt = CreateObject(“QuickTest.RunResultsOptions”)
qtpResultsOpt.ResultsLocation = “Result path” ‘Set the results location
‘Run QTP test
App.Test.Run qtpResultsOpt, FALSE
‘Write the result in the console
While App.Test.isRunning
If App.Test.Environment.Value(“JenkinsFlag”) = “Y” Then
App.Test.Environment.Value(“JenkinsFlag”) = “N”
‘Show TC ID ,TC Name and Result
WScript.StdOut.WriteLine App.Test.Environment.Value(“JenkinsTestCaseNumber”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseDescription”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseResult”) & “.”
End If
Wend
App.Test.Close
App.Quit
Set App = Nothing
How long each testcase takes to execute?
Sanity.vbs & QTP are 2 different processes & they need to talk each other. You need to set enough ‘wait’.
Thanks for your Reply bro.
Mine is a SOAP Services Framework and it takes less than 10 seconds for each test case
See… You need 2 while loops. One outer loop which keeps executing as long as QTP runs.
inner loop is for updating the result. Wait for App.Test.Environment.Value(“JenkinsTestCaseResult”) to be not null
Thanks bro
I have update the script like below as per your comment
While App.Test.isRunning
If App.Test.Environment.Value(“JenkinsFlag”) = “Y” Then
App.Test.Environment.Value(“JenkinsFlag”) = “N”
‘Wait till the test is executed & result is updated
While (App.Test.Environment.Value(“JenkinsTestCaseResult”) = “” AND App.Test.isRunning)
WScript.Sleep 1000
Wend
‘Show TC ID ,TC Name and Result
WScript.StdOut.WriteLine App.Test.Environment.Value(“JenkinsTestCaseNumber”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseDescription”) & ” : ” & App.Test.Environment.Value(“JenkinsTestCaseResult”) & “.”
End If
Wend
MY Result Comes like this. No First test case, only second one displays twice 🙁
TC02 : Update Contact in CLC : Passed.
TC02 : Update Contact in CLC : Passed.
TC03 : Add Contact to Location Relationship : Passed.
TC04 : Search Contacts in CLC : Passed.
TC05 : Create Location in CLC : Passed.
Is there any way to create defects in Jira automatically for failed scripts. We want to implement in such a defects are automatically created for failed scripts. when we have a new build then if failed scripts gets passed then defects are auto updated accordingly.
Please suggest.
Regards,
Sanjay
Jira has API exposed for most of the functionalities. So It is possible to create defects/update existing defects programmatically. You need to check with your Jira admin to access the API.
Hı mate,
I’m trying to initiate a similar scenario but in linux instead of windows. Could you please guide me? I need to execute several scripts and show the outputs of those scripts in console output or email report via email plugin and so on.. Biggest problem I am having at the moment is that, when I execute a shell script to check mount points lets say, build passes successfully but I do not see any results in console output. So this is basically my problem.. Any ideas would be appreciated very much!
Thanks in advance
Sorry that i do not understand your question. If possible, please give more information.
I would advise you to raise your question in StackOverFlow.com to get the help immediately.
I have my UFT on VM and I am trying to hookup with Teamcity server on cloud , I am running into 2 issues
1. When I run above vb script(runner.vb) from command prompt , it does not displays output in command window but runs the test successfully and generate result.xml etc
2. When i call this script from teamcity server it fails where another scripts works fine .
Thanks in advance .
Take a look at this [ http://www.testautomationguru.com/qtpuft-jenkins-github-svn-integration/ ] . I have shared the sample code in github to play around.
Ensure that you use CScript command. CScript runner.vbs