Iterations counted as testcases in total tally

Ranorex Studio, Spy, Recorder, and Driver.
uhmdown
Posts: 54
Joined: Mon Apr 03, 2017 12:00 pm

Iterations counted as testcases in total tally

Post by uhmdown » Mon Apr 03, 2017 12:16 pm

When Data-Binding has been employed, each iteration appears in the Automation Report as a separate test case, and is counted in the total tally.

So if you have 3 testcases, and each run with 30 iterations, then the total count will be 3x30=90 instead of 3.
If one of them fails, then you will have 1 fail out of 90, instead of 1testcase out of 3.

This is pretty inconvenient for us, because we don't always consider an iteration as a separate testcase; for example, a test case with 30 iterations can really just be one testcase with 30 steps.
To have 3 testcases suddenly become 90 makes the reports unreliable for us, because it heavily skews the numbers.


Is there a way to not have the iterations not get counted in the total tally (other than manipulating the ranorex report xml file)?

User avatar
qwertzu
Posts: 284
Joined: Wed Jan 25, 2017 11:08 am

Re: Iterations counted as testcases in total tally

Post by qwertzu » Tue Apr 04, 2017 1:30 pm

Hi uhmdown,

With Ranorex 7.0, this can be done quite easily because iterations on smart folders are not counted.

So simply create a smart folder under a test case and bind your data on this smart folder. Independent on how many iterations there are, the report will always show if running this single test case was successful or not.

I created a short example with 3 test cases. Each of them have a smart folder as child on which some data is bound. In the 3rd smart folder I added a recording consisting of a report failure action.
After running, the report counts the 2 test cases as successful and 1 as failed.. (See screenshots)

I hope this helps.

regards,

qwertzu
You do not have the required permissions to view the files attached to this post.

uhmdown
Posts: 54
Joined: Mon Apr 03, 2017 12:00 pm

Re: Iterations counted as testcases in total tally

Post by uhmdown » Wed Apr 05, 2017 8:15 am

Yeah, I feel so silly. I saw the update almost immediately after I post, and it solved the iteration thing. Awesome : )