Remote execution of the same tests end in different results.

Ranorex Studio, Spy, Recorder, and Driver.
dmconsulting1
Posts: 169
Joined: Fri Nov 08, 2019 3:23 pm

Remote execution of the same tests end in different results.

Post by dmconsulting1 » Thu Feb 24, 2022 5:37 pm

I have a series of tests I develop locally, then test on the remote PC using the Remote tab in Studio, then I move the tests to DevOps and have them ran as part of the release pipeline. The issue I have is the results are different. These are the same test scripts running against the same remote agent PC using the same version of the software. How are my mouse clicks any different? How am I supposed to have faith in these results?
You do not have the required permissions to view the files attached to this post.

User avatar
odklizec
Ranorex Guru
Ranorex Guru
Posts: 7470
Joined: Mon Aug 13, 2012 9:54 am
Location: Zilina, Slovakia

Re: Remote execution of the same tests end in different results.

Post by odklizec » Fri Feb 25, 2022 10:56 am

Hi,

At first, I would suggest to get rid of all these pixel-based coordinate actions. Simply replace them with relative-based actions (like "Center").
RelativeCoord.png
Sadly, you will have to do this manually for all existing actions. For future, you can enable relative-based actions in Rx Settings, by selecting 'None' option. This is something I do immediately after installing Ranorex on a new machine.

The most common reason of "element is outside the visible desktop" error, is that the UI is either not fully loaded yet, or the element is scrolled outside the visible area of screen, typically, it happens with list/select boxes. You will have to ensure that the element you want to click is always visible. Sadly, there are elements, which resists Ensure Visible method, which is used by default for each repo element. So in case of certain elements, you will somehow have to scroll the UI to view.

Unfortunately, it;s hard to suggest something reliable without access to your environment. So I would suggest to do above changes, and then capture screenshots/snapshots here or there in report, to see what's the state of UI before performing problematic actions.
You do not have the required permissions to view the files attached to this post.
Pavel Kudrys
Ranorex explorer at Descartes Systems

Please add these details to your questions:
  • Ranorex Snapshot. Learn how to create one >here<
  • Ranorex xPath of problematic element(s)
  • Ranorex version
  • OS version
  • HW configuration

dmconsulting1
Posts: 169
Joined: Fri Nov 08, 2019 3:23 pm

Re: Remote execution of the same tests end in different results.

Post by dmconsulting1 » Fri Mar 11, 2022 7:43 pm

So why is it not finding this object remote then?

Remote:
cpmFailed.png
Local:
cpmPassed.png

Same test suite. same version of UAT
You do not have the required permissions to view the files attached to this post.

dmconsulting1
Posts: 169
Joined: Fri Nov 08, 2019 3:23 pm

Re: Remote execution of the same tests end in different results.

Post by dmconsulting1 » Fri Mar 11, 2022 7:46 pm

And if you look at the failed test screen shot, you can see the object.
cpmFound.png
You do not have the required permissions to view the files attached to this post.

dmconsulting1
Posts: 169
Joined: Fri Nov 08, 2019 3:23 pm

Re: Remote execution of the same tests end in different results.

Post by dmconsulting1 » Fri Mar 11, 2022 7:47 pm

This isn't the only object, just one example. There are a host of others. This test suite passes locally.

Fergal
Certified Professional
Certified Professional
Posts: 455
Joined: Tue Feb 18, 2014 2:14 pm
Location: Co Louth, Ireland
Contact:

Re: Remote execution of the same tests end in different results.

Post by Fergal » Tue Mar 15, 2022 4:52 pm

I've experienced similar issues in the past, with tests failing on a particular environment. Those issues were sometimes resolved by doing what odklizec suggested and changing the click location to "center".

Something else I noticed in your report is that the element was not found "...within 7s". That seems like a very short timeout, perhaps try increasing the search timeout for the problem elements and see if that helps. Tests on VMs and remote environments sometimes need a little longer to run successfully.

dmconsulting1
Posts: 169
Joined: Fri Nov 08, 2019 3:23 pm

Re: Remote execution of the same tests end in different results.

Post by dmconsulting1 » Thu Mar 17, 2022 3:28 pm

I increased the timeout t o 10s (12s total effective timeout) and it is still failing. I changed all the click actions to center and no longer get the warnings, but it's still failing to find the object CPM vs LEM. After the test run, the object shows as red in the repo, but if you hit the "Edit in spy" spy can locate the item just fine.