Ranorex Forum

Hi all,

Did you ever have that experience with Ranorex 5.4.1 on VM?
It impact greatly to the code since we use click everywhere.
It is on Windows 8. I have to replace it with btnElement.moveto() and btnElement.PerformClick()
It is very bizarre.

I don't see this with 5.4 on my regular machine.

Hi Betty,

Test automation on VM is always a bet in the lottery In general, virtual machines are often slower than physical machines and not all aspects works the same as on physical machines. So if your target is to test on VMs, you should probably also design your tests there.

What kind of VM do you use (VMWare, VirtualBox, any other)?
Do you have enabled HW virtualization on a physical machine, where you run VMs? If disabled, it may greatly impact on speed and stability of your VMs! This switch can be found in the physical machine BIOS, but not all processors support HW virtualization. So if you cannot find this switch in BIOS, your processor probably doesn't support it.

BTW, unless you really need to see the mouse pointer moving to a certain element, you don't have to use MoveTo() together with PerformClick(). PerformClick() alone should be just enough. On the other hand, there may be cases where moving the mouse prior the click is necessary

Hi Pavel,

The reason I am using moveto method is for users. So they can see what the program is actually doing.

I am very appreciated your feedback about running automation on VM. Just as you mentioned running automation on VM and Physical machine are very different. I am experiencing that now.
Test cases are running pretty well might not necessary good on VM. Vise Versa, test cases adjust to VM might sometimes get issue on physical machine.

For now, we have to decide whether we want to run it on Physical machine or VM since it will determine whether we need to develop test cases on physical machine or VM.

As for the VM configuration, I will pass your questions to our IT department since they have set for me.

Regards,
Betty

Pavel,

It seems that our VM is iffy. Today I encountered more weird behavior for both Firefox and Chrome on VM.
Click event is not fired, neither does PerformClick() for a specific test case. However I don't see the same behavior on my physical machine. Now I am using 5.4.1 on both physical and VM.

If we keep seeing some random behaviors like that, we might have to abandon the idea for running it on VM?

Hi Betty,

I think giving up on VM is not a very good idea. I think you just need to setup a reliable VM environment and run your VMs on a reliable HW with enough resources. I'm designing my tests (almost entirely web based, tested with IE) on my physical machine (i5 2.5GHz Win7/64 8GB RAM ) and run them via Jenkins on VirtualBox VM (Win7/64 3GB RAM). The only thing I had to do in my tests was to add some WaitForExists methods here or there. But I have no problem with recognizing web elements on VM. So my tests work the same both on physical and virtual machine. Surprisingly, they run even faster on VM, most probably due to the absence of antivirus on VM

c676228 wrote:If we keep seeing some random behaviors like that, we might have to abandon the idea for running it on VM?

Just my 2 cents: We run (almost) all of our automated tests (Ranorex tests Ranorex) on virtual machines and design/create the tests on physical machines - so just like you, I guess. Nevertheless, we don't experience issues like you describe.

Things we do to make automation work on any machine (VM or physical):

• We try to not use hard-coded timeouts and use relatively high timeouts for repository items (they are only reached in case of an error, anyway).
• Most of the problems we experienced were caused by how you remote into the virtual machine. E.g. leaving an RDP session open and having a bad connection could influence mouse events. Make sure to follow the guidelines on how to work remotely with Ranorex or see how other users handle that problem.

Besides, the virtual machines are hosted on a dedicated server, which is quite powerful. But still, when running tests, they run on about 9 to 20 virtual machines at the same time on one host.

Regards,
Alex
Ranorex Team

I would agree with Alex.

I use my laptop to design tests, and VMWare VMs to run the tests.

I would highly recommend moving to a VNC-based connection instead of RDP. I use TightVNC and it is much better than Remote Desktop at maintaining a stable connection and keeping the desktop active even when you disconnnect.

Hi Alex/krstcs,

Thanks for your input.

First of all, I am not dealing with RDP minimize issue yet. I put it aside for now. I might try later to use TightVNC if it is faster than RDP.

1) I seldom use hard delay now unless absolutely necessary, say, waitfor certain time for the tested application to send an email to a specific mailbox since the automation program is going to fetch it soon(of course, in the auto program, it will deal with it again when email is not arrived when it tries to fetch it).
2) If a click event is not fired frequently across almost all the test cases with Firefox specifically, I don't think it is an issue with bad connection. I believe something on VM is not right. I only see one test case that somehow both click and perform click not working for both Firefox and Chrome. While on my physical machine, I don't have this issue. That issue is duplicable.
3) Actually I have read guidelines on how to work remotely several times. I do have questions about that.
a) what does "same windows schema" mean? I can tell you that our VM and Physical machine are on the same domain.
b) The resolution for both our VM and physical machine is 1600X1200. but the display for physical machine is Dell U2412 M (Analog), while for VM, we have "Unknown Monitor on RDPUDD chained DD". Does that matter?
4) I somehow believe that something is not right. for example I experienced another weird behavior on
VM today in Ranorex Studio. here it is:
a) copy some text b) delete a line of code using shift+delete c) paste the text from a)
However, you won't see text from a) is pasted, instead you see text from b) is pasted. Isn't weird? This error can be duplicated and keeping doing this when I edit on VM. This doesn't happen when I use Ranroex on my physical machine.

Any idea?

Regards,
Betty

We are using Hyper-V.
My dev machine is: Intel Core i5-4690 3.5 GHz, RAM 8.0 GB. 232 GB hard drive.
VM is Intel Xeon E5-2620 v2 2.10GHz, 4 virtual processors, RAM 8.0 GB. 100 GB hard drive.
The host machine is Xeon E5-2620 v2 @2.10GHz(2 processors), RAM 64GB, Windows Server 2012 R2.
We have about 25 VMs hosted by this machine. On average we have about 10 running on a daily basis.

Do you see any differences or potential issues here?
Our IT guy says he probably cannot do anything about with it since we use SATA, not SSD etc.

With only 64GB of RAM on the HOST you are going to run out of RAM after 4 VMs are started, if they are each 8GB.

Windows reserves 1/2 of the RAM for system operations, which will leave only 32GB for other software (Hyper-V may be allowed more, but I'm not sure since I don't use it).

Even if the above isn't true, 10 VMs @ 8GB each is 80GB of RAM being used, which is more than your 64GB HOST can handle without paging, which will cause serious slow-downs.

If we assume your HOST's OS only uses 4GB of RAM to run (which it won't), you still only have 60GB available for your Hyper-V instance to use.

You probably should have at least 128GB or RAM for 10 hosts, or you should implement a dynamic host solution where you spin-up new VMs on different hosts after the first host is full.

The other option is to reduce the guest RAM to 4GB each, unless you need 8GB (again, Windows reserves 1/2 for system use).

Hi krstcs,

I have just checked. All other VMs are assigned about 3GB* 8 machines RAM + my VM-8GB -32GB.
It seems that we only have 64-32 GB=32GB, which is min for Windows operating system already?

The minimum total RAM depends on the version of Windows. Even the server versions, for the most part, can run on 8GB (except for the really big boys), depending on what you are doing on them.

But, like I said, the Windows OS reserves HALF of the total RAM for the system, in most cases. I'm not sure if Hyper-V changes that at all.

So, if you are only using about 3GB for most of the VMs, you should be OK with 64GB for the host, but only barely.

More memory is ALWAYS a good thing, if you can afford it, and if your system can handle it (the license for the server OS will dictate how much RAM is accessible by the system).

Are you only running one VM for your testing? If so, something to think about is what is going to happen when you start testing on multiple VMs at the same time? How much RAM will you need then? Are other VMs going to be competing for those resources at the same time?

If you can, I would suggesting getting your own test environment (VM Host, network, etc.) so you can more directly control the test environment. I, unfortunately, don't have that option myself, but I'm working on it.

Hope this helps you!

Hi krstcs,

When the VM is stable and not many VMs are running. it runs ok. Each test case takes about 3-4 min on average to finish depending on how complicated of the test case.
I am ok with it now. We might take your suggestion to set up our own test env to make sure no other VMs are competing for the resources.

As for Firefox not fire the click event, I have reported to Ranorex and the support noticed/saw that too during remote session. I assume the issue cannot be solved any time soon since the issue is on and off even though it occurs quite often.

Regards,
Betty