NUnit RemoteTestRunner slowness

Class library usage, coding and language questions.
acohen
Posts: 3
Joined: Tue May 21, 2013 3:36 pm

NUnit RemoteTestRunner slowness

Post by acohen » Tue May 21, 2013 3:43 pm

I have written a bunch on NUnit tests which use Ranorex to manipulate our Windows Forms app. The tests run fine in the NUnit consule or gui, but when I attempt to run them programatically using RemoteTestRunner, I get Ranorex Search timeouts. If I increase the search timeout in Ranorex, it does eventually find the elements and run the test, but it seems to be an order of magnitude slower than the same tests run from the console. Any help would be much appreciated.

acohen
Posts: 3
Joined: Tue May 21, 2013 3:36 pm

Re: NUnit RemoteTestRunner slowness

Post by acohen » Tue May 21, 2013 7:02 pm

Here is what I am trying to do. If I run this test from the GUI runner, it takes ~1 second. If I run it from RemoteTestRunner, it takes ~1 minute.

My Test:
[Test]
public void DebugTest()
{
DateTime start = DateTime.Now;
Ranorex.Form hf = Ranorex.Host.Local.FindSingle<Ranorex.Form("/form[@controlname=\'HomeForm\']");
DateTime done = DateTime.Now;
}

Running it:
public TestResult runTest()
{
NUnit.Core.CoreExtensions.Host.InitializeService();

String assembly = <my assembly path>;
TestPackage testPackage = new TestPackage(assembly);
RemoteTestRunner remoteTestRunner = new RemoteTestRunner();
remoteTestRunner.InitializeLifetimeService();
remoteTestRunner.Load(testPackage);
TestResult testResult = remoteTestRunner.Run(new NullListener(), new MyFilter1(), false, LoggingThreshold.Error);

return testResult;
}

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: NUnit RemoteTestRunner slowness

Post by Support Team » Wed May 22, 2013 3:52 pm

Hello,

Which NUnit and Ranorex version do you use?

We had a performance issue when using Ranorex with NUnit 2.6.2.
It was solved by replacing NUnit with version 2.5.2.

Could you please try to replace your NUnit version?

Thank you.

Regards,
Markus (T)

acohen
Posts: 3
Joined: Tue May 21, 2013 3:36 pm

Re: NUnit RemoteTestRunner slowness

Post by acohen » Fri May 24, 2013 4:11 am

We are running Ranorex 4.0.3 with NUnit 2.6.2. Unfortunately, NUnit 2.52 does not support .Net 4.0, so we are unable to use it. Is there a fix in the works? If so, any ETA?

Thanks

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: NUnit RemoteTestRunner slowness

Post by Support Team » Mon May 27, 2013 4:16 pm

Hello,

The mentioned issue was specific for one of our customers.
In general, we support NUnit 2.6.2 without any known issues.

Have you set the [STAThread] at the beginning of your Main method?
Is it possible to send us your project to [email protected] in order to analyze it?

Regards,
Markus (T)

Pavlo
Posts: 43
Joined: Fri Dec 30, 2011 10:55 am

Re: NUnit RemoteTestRunner slowness

Post by Pavlo » Tue Jun 18, 2013 8:57 am

Hi guys

Was there any continuation for this story?
I'm also using nUnit (currently use version 2.6.0) and execute all tests from nUnit GUI, so - can you please share results and recommendations how to properly use nUnit?

-re
Pavlo

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: NUnit RemoteTestRunner slowness

Post by Support Team » Wed Jun 19, 2013 1:23 pm

Hello,

Unfortunately, we haven't got a reply from acohen.
Do you face any (performance) issues with NUnit?

Have you seen our blog that describes Using NUnit for test execution?

Regards,
Markus (T)

Pavlo
Posts: 43
Joined: Fri Dec 30, 2011 10:55 am

Re: NUnit RemoteTestRunner slowness

Post by Pavlo » Fri Jun 21, 2013 11:06 am

There are minor performance issues when executing tests from nUnit, but those are not critical.
It takes bit more time to find elements, but did not block my tests.
I was looking for a ways how to improve that (if any).

-re
Pavlo

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: NUnit RemoteTestRunner slowness

Post by Support Team » Mon Jun 24, 2013 4:52 pm

Hello,

I assume that you have added '[STAThread]' in your test project.
Is it possible to narrow down this issue?
Do you see the same performance issue in other projects as well?
Is there a difference in the performance, if you try to run your test in Ranorex Studio?

Regards,
Markus (T)

brgibb
Posts: 39
Joined: Wed Jun 05, 2013 2:15 pm

Re: NUnit RemoteTestRunner slowness

Post by brgibb » Tue Aug 20, 2013 10:14 am

I have a similar issue I guess.

My Ranorex automation runs smoothly and swiftly in the Visual Studio IDE, but as soon as I run it via nUnit it takes noticeably longer. The mouse pointer stalls, flickers and takes much longer (sitting stationary) to find elements.

This behaviour is consistent across web-based interface I'm automating and an MSI installer that I'm also automating....but is more pronounced in the MSI installer....of course will use forms.

Both test cases (for now) are in the same C# Project.

The structure of all my nUnit tests are the following:

[TestFixture, RequiresSTA]

class LoginTest
{
[TestFixtureSetUp]
public static void FixtureSetUp()
{
TestReport.BeginTestCase("Login Tests");
//do something;
}

[TestFixtureTearDown]
public static void FixtureTearDown()
{
TestReport.EndTestCase();
//Something to do after all test in this class have run
}

[SetUp]
public static void TestSetUp()
{
Host.Local.OpenBrowser(WebURL, BrowserType);
}

[TearDown]
public static void TestTearDown()
{
Keyboard.Press("{ControlKey down}{F4}{ControlKey up}");
Ranorex.Delay.Seconds(1);
TestRun.SetEndOfTest();
}

[Test]
[Category("Login Test")]
public static void ValidAccount()
{
try
{
TestRun.SetStartOfTest();
ProjectName.doLogin(ValidTestUserName, ValidTestUserPassword);
ProjectName.doLogout();
Report.Success("Test Passed");
}

catch (Exception e)
{
TestReport.SetCurrentTestResult(TestResult.Failed);
Report.Failure("Test Failed : " + e.ToString());
Assert.Fail(e.ToString());
Report.Screenshot();
}
}



Notice I have the [RequiresSTA] set at the start. Do you think this has any relevance to the slowness I am seeing?

User avatar
Support Team
Site Admin
Site Admin
Posts: 12145
Joined: Fri Jul 07, 2006 4:30 pm
Location: Houston, Texas, USA
Contact:

Re: NUnit RemoteTestRunner slowness

Post by Support Team » Fri Aug 23, 2013 3:06 pm

Hello,

I don't think that the [RequiresSTA] is the problem, but it could be a problem in the NUnit settings.
Could you please check if each test is running in a separate process? If this is the case please change the setting and check if the performance is better.
Thank you!

Regards,
Bernhard