How To manually create testrail testruns for automated build testing in ranorex

Best practices, code snippets for common functionality, examples, and guidelines.
c4_mmartin
Posts: 7
Joined: Fri Feb 21, 2020 5:44 pm

How To manually create testrail testruns for automated build testing in ranorex

Post by c4_mmartin » Fri Feb 21, 2020 6:34 pm

Using testrail integration, I am trying to see if there is a easy way to simply create a new test run without running tests. Giving me the ability to record the test run id pre-automated tests. Then run tests against that test run in my CI (Jenkins)

Right now i am debugging a complicated job in which i attempt to either pull the test run id from a database. Or create a new test run and then append it to the database marked for this release cycle.

Is there a easy way to just simply create the test run in testrail that contains the tests needed for the run configuration?

Sorry if this is confusing, i just cant explain it easily.

HansSchl
Posts: 143
Joined: Wed Sep 19, 2018 10:05 am

Re: How To manually create testrail testruns for automated build testing in ranorex

Post by HansSchl » Thu Jan 14, 2021 8:32 am

@c4_mmartin, I am not sure what you want to achieve. Maybe what I have done can help you?
I created a system where, every time a new build pops out of the CI environment, a set of test suites is triggered, and the results are reported to TestRail. The steps are as follows:
  • The suites of the test project are compiled into *.exe files which are copied to the machine where automated tests should run.
  • The output of the CI machine goes to subversion into the "export" folder of the project that contains the source code of the AUT.
  • On the testing machine, a PowerShell script runs every 30 minutes and checks for new files in this "export" folder.
  • If new files are found, they are copied into the directory where the binaries of the AUT live, and the test are launched by running the *.exe files that had been created by compiling the test poject.
I separated the test machine from the CI machine for two reasons:
  • Tests take quite a while and would block the CI system if they run on the CI machine.
  • This whole thing is quite new, and the test machine still needs frequent configuration changes which may disrupt CI.
I hope this helps you a bit; if not, please post more details.
Hans

c4_mmartin
Posts: 7
Joined: Fri Feb 21, 2020 5:44 pm

Re: How To manually create testrail testruns for automated build testing in ranorex

Post by c4_mmartin » Thu Jan 14, 2021 4:52 pm

Thanks, i think we will eventually look to a different way of implementing testrail in ranorex.

Currently my setup is:

Jenkins hosts (Winders, linux)
Every test rack has one of each as an agent running
Using python i maintain a sql db of builds to automate testing and maintain racks status such as available or busy
Using jenkins i compile the ranorex project as it is maintained from git
Using jenkins i run a loop(cron) to check the DB for work
Using jenkins when work is triggered to run, i run a bunch of python scripts in jenkins jobs to setup the software and environment
Then fire off the jenkins jobs with parameters to run on which test rack with which testsuite and run config which may or may not report to testrail
Once the automated test is done, in python i find the job in testrail and then drop it in the right milestone(s) or create them

So on top of this my boss wants the testrail run to be configured differently but the only answer i circle back to is import a 3rd party lib for testrail or build my own api wrapper. One of the big problems is we have multiple testrail projects and he only wants to report with one. In ranorex importing the whole project required suites makes the ide unusable with all of the manual test cases (around 4200 or so) as it displays everything, not just ranorex marked test cases.

Maybe i am just over thinking it or this is too complex. thanks anyway.