Page 1 of 1

Iterations counted as testcases in total tally

Posted: Mon Apr 03, 2017 12:16 pm
by uhmdown
When Data-Binding has been employed, each iteration appears in the Automation Report as a separate test case, and is counted in the total tally.

So if you have 3 testcases, and each run with 30 iterations, then the total count will be 3x30=90 instead of 3.
If one of them fails, then you will have 1 fail out of 90, instead of 1testcase out of 3.

This is pretty inconvenient for us, because we don't always consider an iteration as a separate testcase; for example, a test case with 30 iterations can really just be one testcase with 30 steps.
To have 3 testcases suddenly become 90 makes the reports unreliable for us, because it heavily skews the numbers.


Is there a way to not have the iterations not get counted in the total tally (other than manipulating the ranorex report xml file)?

Re: Iterations counted as testcases in total tally

Posted: Tue Apr 04, 2017 1:30 pm
by qwertzu
Hi uhmdown,

With Ranorex 7.0, this can be done quite easily because iterations on smart folders are not counted.

So simply create a smart folder under a test case and bind your data on this smart folder. Independent on how many iterations there are, the report will always show if running this single test case was successful or not.

I created a short example with 3 test cases. Each of them have a smart folder as child on which some data is bound. In the 3rd smart folder I added a recording consisting of a report failure action.
After running, the report counts the 2 test cases as successful and 1 as failed.. (See screenshots)

I hope this helps.

regards,

qwertzu

Re: Iterations counted as testcases in total tally

Posted: Wed Apr 05, 2017 8:15 am
by uhmdown
Yeah, I feel so silly. I saw the update almost immediately after I post, and it solved the iteration thing. Awesome : )