How to execute my test suite on my build server?

Experiences, small talk, and other automation gossip.
timbuethe
Posts: 15
Joined: Mon Aug 22, 2011 11:08 am

How to execute my test suite on my build server?

Post by timbuethe » Mon Aug 22, 2011 2:14 pm

Hi,

I've tested ranorex for some days now, created "MyTest", recorded some trivial tests and executed them successfully. I've then used the "MyTest.exe" to execute the tests at night on my Hudson build server. No problems so far.

Now the solution grew into multiple projects with a lot of modules and some user code and I'm facing some problems or have some questions: First of all, there are more than one exe file. I thought there would be one exe file for the whole solution / test suite and dlls for all the child projects, right? Why there are multiple exe files? Which exe should I use to run the tests on the build server? Can I use a single exe to execute the test suite including other projects or do I have to call multiple exe files?

thanks,
Tim

chfischer
Posts: 20
Joined: Fri Jan 14, 2011 10:44 am

Re: How to execute my test suite on my build server?

Post by chfischer » Mon Aug 22, 2011 6:01 pm

Hi

We have Ranorex running on Jenkins/Hudson. I would be happy to share some experience.

- We have separate Ranorex solutions which share one project (which holds the repositories, functions & code modules).

- We build the projects in a single Jenkins job using MSBUILD and deploy all the tests to a server as a zip file.

- We have individual jobs to execute the different tasks (although a multi project build could be used for this as well..)

- I use the publish html + xunit test plugin to visualize test results.

- I execute only one exe file for each individual test solution. What exe files to you get as a result?

timbuethe
Posts: 15
Joined: Mon Aug 22, 2011 11:08 am

Re: How to execute my test suite on my build server?

Post by timbuethe » Tue Aug 23, 2011 1:28 pm

Hi,

thanks for sharing your experience.
- We have separate Ranorex solutions which share one project (which holds the repositories, functions & code modules).
Do you mean, you have different .rxsln files and switch between them when using Ranorex Studio?
- We build the projects in a single Jenkins job using MSBUILD and deploy all the tests to a server as a zip file.
So you only checked the source files into your the VCS and produced executables inside the build? That's great and surly the way to go. However, since I have no experience with .Net, C# and MSBuild I just commit the .exe and .dll files too, right now.

Could you share your MSBuild file please?
- I use the publish html + xunit test plugin to visualize test results
How did you configure this? I guess you choose "Custom Tool", right? How does your "Pattern" look?

Thanks
Tim

BTW: I normally work with Java, so I know the concepts of build and test tools, I just don't have any experience in the Microsoft world so I have to ask alot of dumb questions, sorry.

chfischer
Posts: 20
Joined: Fri Jan 14, 2011 10:44 am

Re: How to execute my test suite on my build server?

Post by chfischer » Thu Sep 08, 2011 8:53 am

Do you mean, you have different .rxsln files and switch between them when using Ranorex Studio?
Yes. We have different test scenarios, which cover different areas of our application.
So you only checked the source files into your the VCS and produced executables inside the build? That's great and surly the way to go. However, since I have no experience with .Net, C# and MSBuild I just commit the .exe and .dll files too, right now.
Its actually quite easy using MSBUILD. I have a simple batch script (which I want to replace by a proper gradle solution) which compiles all the tests and deploys an archive to a server (thats one jenkins job) and a jenkins job per test scenario/Ranorex Solution to run them.

I've sent you an PM. Just send me an email and we can share this stuff and maybe put together some documentation on this. What do you think?

timbuethe
Posts: 15
Joined: Mon Aug 22, 2011 11:08 am

Re: How to execute my test suite on my build server?

Post by timbuethe » Thu Sep 08, 2011 9:08 am

Sorry I haven't updated my posting, I am some step further now.

My setup is the following now:
1. I only checkin sources into SVN ignoring binaries and reports (*.rxuser, *.jpg, *.rxlog, Reports, bin, obj). See Ranorex & VCS topic.
2. I've written a NAnt build file to compile the projects
3. I set up a Hudson/Jenkins job to compile and execute the tests