Hi Everyone,
we set up a CI environment with Jenkins: Linux server where Jenkins runs and a slave Windows VM with Ranorex Studio 9.0 installed where test should run.
Everything works properly if each job is launched manually (Build Now option), but if the same TS is launched by the schedule of the Job, it starts properly, but TC of TS fail, everytime for different specific reasons, but always because some elements are not found at all.
That sounds strange to me, because the build is correct and all the TS works properly if launched manually.
I tried to have a look on the web, but the only similar problem I found was related to GUI on RDP session and I think that is not my case.
Any suggestions?
Scheduled test run in Jenkins doesn't work properly
Re: Scheduled test run in Jenkins doesn't work properly
Hi,
At first, please make sure you are using latest 9.0.1. 9.0.0 was somewhat buggy
At next, if the test fails on "not found" element, could you please post an example xpath (stored in repo) and also Ranorex Snapshot (NOT screenshot) of the problematic element?
Additionally, does it happen always at the same place or it fails randomly at multiple places in test? If test fails with specific element, could you please add a "save snapshot" action, right before the failing place and save entire page, containing the failing element? This could help with narrowing down the reason of failing xpath.
At first, please make sure you are using latest 9.0.1. 9.0.0 was somewhat buggy
At next, if the test fails on "not found" element, could you please post an example xpath (stored in repo) and also Ranorex Snapshot (NOT screenshot) of the problematic element?
Additionally, does it happen always at the same place or it fails randomly at multiple places in test? If test fails with specific element, could you please add a "save snapshot" action, right before the failing place and save entire page, containing the failing element? This could help with narrowing down the reason of failing xpath.
Pavel Kudrys
Ranorex explorer at Descartes Systems
Please add these details to your questions:
Ranorex explorer at Descartes Systems
Please add these details to your questions:
- Ranorex Snapshot. Learn how to create one >here<
- Ranorex xPath of problematic element(s)
- Ranorex version
- OS version
- HW configuration
Re: Scheduled test run in Jenkins doesn't work properly
Thank you for the reply.
Do you really think the version update can help? I will try.
As far as the problem I am having, tests don't fail always on the same item, but usually stop at the first element they are looking for.
I have several TS and all of them cannot find the first element. I tried to change the test beginning sequence, but nothing changes.
I remind you that if I run each TS from Jenkins manually (not scheduled) everything works properly (elements are found and tests completed successfully). I cannot understand why the scheduled sessions give me this problem!
Element of the repository are there and the VM can find them if Jenkins jobs are not schedule but run with "Build Now" option.
An example of Jenkins log is:
[2019/05/09 11:10:35.746][Error ][Module]: Failed to wait for item to exist.<br/>Failed to find item 'Proj_Repository.Proj.LoginSplash'. Please check that your whitelist allows Ranorex to access the process.<br/>No element found for path '/dom[@domain='xxx.xxxxx.xx:8080']/body/div/div/div' within 30s.
[2019/05/09 11:10:35.848][Failure][Test]: Test Module 'LoginSetup' completed with status 'Failed'
....
[2019/05/09 11:11:09.022][Debug ][Logger]: Console logger stopping.
Build step 'Run a Ranorex test suite' changed build result to FAILURE
[JK_FX_TS_XXXXXX] $ cmd /c call C:\Users\aut\AppData\Local\Temp\jenkins3445835144195212420.bat
C:\JenkinsDockerWorkspace\workspace\JK_FX_TS_XXXXXX>cd "C:\JenkinsDockerWorkspace\workspace\JK_FX_TS_XXXXXX\CompressedReports"
C:\JenkinsDockerWorkspace\workspace\JK_FX_TS_XXXXXX\CompressedReports>exit 0
Sending e-mails to: [email protected]
Finished: FAILURE
Do you really think the version update can help? I will try.
As far as the problem I am having, tests don't fail always on the same item, but usually stop at the first element they are looking for.
I have several TS and all of them cannot find the first element. I tried to change the test beginning sequence, but nothing changes.
I remind you that if I run each TS from Jenkins manually (not scheduled) everything works properly (elements are found and tests completed successfully). I cannot understand why the scheduled sessions give me this problem!
Element of the repository are there and the VM can find them if Jenkins jobs are not schedule but run with "Build Now" option.
An example of Jenkins log is:
[2019/05/09 11:10:35.746][Error ][Module]: Failed to wait for item to exist.<br/>Failed to find item 'Proj_Repository.Proj.LoginSplash'. Please check that your whitelist allows Ranorex to access the process.<br/>No element found for path '/dom[@domain='xxx.xxxxx.xx:8080']/body/div/div/div' within 30s.
[2019/05/09 11:10:35.848][Failure][Test]: Test Module 'LoginSetup' completed with status 'Failed'
....
[2019/05/09 11:11:09.022][Debug ][Logger]: Console logger stopping.
Build step 'Run a Ranorex test suite' changed build result to FAILURE
[JK_FX_TS_XXXXXX] $ cmd /c call C:\Users\aut\AppData\Local\Temp\jenkins3445835144195212420.bat
C:\JenkinsDockerWorkspace\workspace\JK_FX_TS_XXXXXX>cd "C:\JenkinsDockerWorkspace\workspace\JK_FX_TS_XXXXXX\CompressedReports"
C:\JenkinsDockerWorkspace\workspace\JK_FX_TS_XXXXXX\CompressedReports>exit 0
Sending e-mails to: [email protected]
Finished: FAILURE
Re: Scheduled test run in Jenkins doesn't work properly
Hi Mafi,
It super-likely that you are either not starting Jenkins as a process or your GUI user session is not open when running the test from Jenkins:
1. Disable service/Start as process
2. Keep user session alive after closing remote desktop connection
Which one was it?
Regards,
McTurtle
It super-likely that you are either not starting Jenkins as a process or your GUI user session is not open when running the test from Jenkins:
1. Disable service/Start as process
2. Keep user session alive after closing remote desktop connection
Which one was it?
Regards,
McTurtle
Re: Scheduled test run in Jenkins doesn't work properly
Thank you.
Unfortunately I had already set both in the right way: I read them around in the net.
I think the problem is not Jenkins. I think the problem is Ranorex. Perhaps it is my knowledge of Rnx!
I noticed that main issues come from Edge: several objects, very often, are not recognized or clicked somewhere else. Sometimes, repository objects well recognized on physical machines are not well recognized in VM. And so on.
So, now, before deploy the test suites on CI system, I have to test it a lot in order to be sure everything is strong enough to be run in many different machines, but Edge is still a problem.
Unfortunately I had already set both in the right way: I read them around in the net.
I think the problem is not Jenkins. I think the problem is Ranorex. Perhaps it is my knowledge of Rnx!
I noticed that main issues come from Edge: several objects, very often, are not recognized or clicked somewhere else. Sometimes, repository objects well recognized on physical machines are not well recognized in VM. And so on.
So, now, before deploy the test suites on CI system, I have to test it a lot in order to be sure everything is strong enough to be run in many different machines, but Edge is still a problem.
Re: Scheduled test run in Jenkins doesn't work properly
Hi,
At first, Edge engine is going to be replaced with new chrome-based engine, so it's most probably not the best idea to concentrate on Edge tests development right now? I believe there is already available a beta of new Edge?
At next, if you are experiencing problems mainly on VM (with tests developed on physical machine), then you should most probably develop/optimize them directly on VM, where the tests will be running? I'm always developing all my tests directly on VM and it helps a lot.
Unfortunately, without seeing an example of failing xpath, exact error you are getting and ideally, also Ranorex snapshot (NOT screenshot) of the problematic element (taken both on VM and physical machine), there is not much else anyone here can do or suggest.
At first, Edge engine is going to be replaced with new chrome-based engine, so it's most probably not the best idea to concentrate on Edge tests development right now? I believe there is already available a beta of new Edge?
At next, if you are experiencing problems mainly on VM (with tests developed on physical machine), then you should most probably develop/optimize them directly on VM, where the tests will be running? I'm always developing all my tests directly on VM and it helps a lot.
Unfortunately, without seeing an example of failing xpath, exact error you are getting and ideally, also Ranorex snapshot (NOT screenshot) of the problematic element (taken both on VM and physical machine), there is not much else anyone here can do or suggest.
Pavel Kudrys
Ranorex explorer at Descartes Systems
Please add these details to your questions:
Ranorex explorer at Descartes Systems
Please add these details to your questions:
- Ranorex Snapshot. Learn how to create one >here<
- Ranorex xPath of problematic element(s)
- Ranorex version
- OS version
- HW configuration
Re: Scheduled test run in Jenkins doesn't work properly
Unfortunately it is not so easy to develop on VM directly due to logistics issue, but I will keep in mind your suggestion.
Thanks.
Thanks.