Hi, I have found a number of questions/answers on this forum and elsewhere regarding having to use command line to run multiple test suites when using Jenkins, because the Ranorex Jenkins plugin doesn't support multiple test suites.
The problem I am having is as follows: My Ranorex/Jenkins/Remote VM setup is working fine and successfully runs each test suite individually, but I can't workout/find the structure of the command line to get the 3 test suites to run one after another with a run configuration for each. Although it doesn't look like it, the text shown in the screen grab is one long line, not 4 individual lines.
What is happening, is that the test run runs, but it will only run the last test suite in the line (in the screen grab that is the Execute test suite), rather than running the 3 sequentially.
Any ideas or suggestions on how I should do this will be greatly accepted.
On a slightly different note, does anyone from the Ranorex support team know when or whether the Jenkins Ranorex plugin is going to be updated to let it work with multiple spreadsheets? In this thread about Jenkins https://www.ranorex.info/need-help-rano ... tml#p50021, Marcus from the Ranorex Support team mentions that he is working to update it, but that comment is from August 2018 and the plugin still hasn't been updated. In fact, looking in GIT, the last update to the plugin is 2019.
Cheers JimLin
Running multiple test suites with run configurations when using Jenkins and command line
Running multiple test suites with run configurations when using Jenkins and command line
You do not have the required permissions to view the files attached to this post.
Re: Running multiple test suites with run configurations when using Jenkins and command line
Hi JimLin,
as I see, you try to run your Tests.exe with multiple testsuite parameters.
Based on Ranorex documentation: https://www.ranorex.com/help/latest/ran ... execution/ this is currently not possible.
Tests.exe /testsuite...
Tests.exe /testsuite...
Tests.exe /testsuite...
This way the test suites are called sequentially one after another in different processes.
Greetings,
Gyuri
as I see, you try to run your Tests.exe with multiple testsuite parameters.
Based on Ranorex documentation: https://www.ranorex.com/help/latest/ran ... execution/ this is currently not possible.
What if you call your tests like this:You can specify only one test suite to execute with this argument. If you specify more than one, only the last one will be executed.
Tests.exe /testsuite...
Tests.exe /testsuite...
Tests.exe /testsuite...
This way the test suites are called sequentially one after another in different processes.
Greetings,
Gyuri
Re: Running multiple test suites with run configurations when using Jenkins and command line
Hi csaszi89,
Thank you for your reply. It does seem to fit with with the behaviour I have witnessed and it seems I had misunderstood what was possible.
This does seem to be a bit of a limitation to me. @Ranorex support, is this expected behaviour and is the Jenkins Ranorex plugin going to be updated to make this easier to setup?
Regards JimLin
Thank you for your reply. It does seem to fit with with the behaviour I have witnessed and it seems I had misunderstood what was possible.
This does seem to be a bit of a limitation to me. @Ranorex support, is this expected behaviour and is the Jenkins Ranorex plugin going to be updated to make this easier to setup?
Regards JimLin
Re: Running multiple test suites with run configurations when using Jenkins and command line
Gyuri, I have tried your suggestion and it does solve the problem and is a suitable way of doing it as far as I can work out. Thanks for your assistance.
Cheers JimLin
Re: Running multiple test suites with run configurations when using Jenkins and command line
besides the rx job (multibranchpipelinejob) in jenkins I have another job like this,
so i can run all predefined suites + runconfigs
please note I dont use the plugin, the rx job has to accept the parameters ( i my case this job builds rx & runs buildoutput with given parameters)
pipeline script:
so i can run all predefined suites + runconfigs
please note I dont use the plugin, the rx job has to accept the parameters ( i my case this job builds rx & runs buildoutput with given parameters)
pipeline script:
def dobuild(String jobname,String testenv,String testsuitename,String runconfigname) { try { ThisjobX=build( job: jobname,propagate: false,wait:true, parameters: [[$class: 'StringParameterValue', name: 'Testsuite', value: testsuitename], [$class: 'StringParameterValue', name: 'Runconfig', value: runconfigname], te rx branch [$class: 'StringParameterValue', name: 'Testenv', value: testenv]]) ; //println 'jobdone...'; if (ThisjobX.result=='SUCCESS' ) { } else { unstable(ThisjobX.result); } } catch (Exception err) { throw new Exception(err); } } import java.net.URLEncoder String downstreamjobname = "rx/" + URLEncoder.encode(yourbranchname); pipeline { agent none stages{ stage('Parallel run of tests') { steps { parallel ( T1 : { dobuild(downstreamjobname,testenv,'suite1.rxtst',runconfig1'); }, T2 : { dobuild(downstreamjobname,testenv,'suite1.rxtst',runconfig2'); }, T3 : { dobuild(downstreamjobname,testenv,'suite2.rxtst',runconfig1'); }, T4 : { dobuild(downstreamjobname,testenv,'suite2.rxtst',runconfig2'); }, T5 : { dobuild(downstreamjobname,testenv,'suite3.rxtst',runconfigx'); }