Repeating a test case upon failure
Repeating a test case upon failure
Hi,
I have been looking through the forums for a solution to repeat a test case after it has failed but was unable to come up with something concrete. After a test case has failed and it enters into it's teardown phase, I would like it to be able to restart the instance of the test case and run the test again. In order to do this, I figured there were two ways to accomplish this. Either simply make a call to the instance to run again or to setup a global parameter to control the amount of iterations that occur.
Any help is appreciated,
Steven
I have been looking through the forums for a solution to repeat a test case after it has failed but was unable to come up with something concrete. After a test case has failed and it enters into it's teardown phase, I would like it to be able to restart the instance of the test case and run the test again. In order to do this, I figured there were two ways to accomplish this. Either simply make a call to the instance to run again or to setup a global parameter to control the amount of iterations that occur.
Any help is appreciated,
Steven
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Repeating a test case upon failure
Hello Steven,
The easiest way to repeat the test case is to use Data-Driven Testing. You can set the error behavior setting to "Continue with iteration". The test case will repeat with the next iteration in case of an failure, but it will also repeat with the next iteration if no failure occurs. If you only want to start specific Recordings you can do so using the following code within the teardown region:
Bernhard
Ranorex Support Team
The easiest way to repeat the test case is to use Data-Driven Testing. You can set the error behavior setting to "Continue with iteration". The test case will repeat with the next iteration in case of an failure, but it will also repeat with the next iteration if no failure occurs. If you only want to start specific Recordings you can do so using the following code within the teardown region:
YourRecording.Start();Regards,
Bernhard
Ranorex Support Team
Re: Repeating a test case upon failure
Hi Bernhard,
I was looking more for a solution where I can repeat the current test case for all the test cases, which rules out the possibility of just calling a specific test case as I have a general teardown that is used for all the tests. If it was possible to iterate the same test case with the same set of variables, that would be great. Any more suggestions on this? Ranorex tests fail sometimes when trying to locate an element that was found before. Therefore, being able to repeat a test case from scratch will hopefully solve this problem.
Thanks,
Steven
I was looking more for a solution where I can repeat the current test case for all the test cases, which rules out the possibility of just calling a specific test case as I have a general teardown that is used for all the tests. If it was possible to iterate the same test case with the same set of variables, that would be great. Any more suggestions on this? Ranorex tests fail sometimes when trying to locate an element that was found before. Therefore, being able to repeat a test case from scratch will hopefully solve this problem.
Thanks,
Steven
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Repeating a test case upon failure
Hello Steven,
Does it help if you increase the search timeout to find the element instead of restarting the test case? Another thing you can try is to disable Folder Caching.
Regards,
Bernhard
Ranorex Support Team
Does it help if you increase the search timeout to find the element instead of restarting the test case? Another thing you can try is to disable Folder Caching.
Regards,
Bernhard
Ranorex Support Team
Re: Repeating a test case upon failure
Hi Bernhard,
Unfortunately, increasing the timeout length does not help. It is still preferred to repeat the test care upon failure. I noticed there are methods to run code modules and the test suite but I have not come across a method that will run a test case. I want to be able to run the same test case again so that in the report, it will appear as if it ran two iterations with the same parameters. Would this be possible to implement in the teardown?
Thanks,
Steven
Unfortunately, increasing the timeout length does not help. It is still preferred to repeat the test care upon failure. I noticed there are methods to run code modules and the test suite but I have not come across a method that will run a test case. I want to be able to run the same test case again so that in the report, it will appear as if it ran two iterations with the same parameters. Would this be possible to implement in the teardown?
Thanks,
Steven
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Repeating a test case upon failure
Hello Steven,
You could try to copy your test case and paste it after the original test case.
In the teardown region of your original test case you can check if the test case has failed. If this is the case you can activate the new test case as shown below. The specified test case will only be executed if the original test case has failed.
Regards,
Bernhard
Ranorex Support Team
You could try to copy your test case and paste it after the original test case.
In the teardown region of your original test case you can check if the test case has failed. If this is the case you can activate the new test case as shown below. The specified test case will only be executed if the original test case has failed.
if (Ranorex.Core.Testing.TestCase.Current.Status.Equals(Ranorex.Core.Reporting.ActivityStatus.Failed)) { TestSuite.Current.GetTestCase("TestCase1").Checked = true; } else { TestSuite.Current.GetTestCase("TestCase1").Checked = false; }I hope this workaround will help you!
Regards,
Bernhard
Ranorex Support Team
Re: Repeating a test case upon failure
Hi Bernhard,
I was wondering if instead of doing that, is there a way to call a test case in code?
Thanks again,
Steven
I was wondering if instead of doing that, is there a way to call a test case in code?
Thanks again,
Steven
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Repeating a test case upon failure
Hi Steven,
A test case is not designed to be executed from UserCode. There will be more possibilities in the test suite to work with conditions but meanwhile there is no other "supported" way.
Regards,
Markus
Ranorex Support Team
A test case is not designed to be executed from UserCode. There will be more possibilities in the test suite to work with conditions but meanwhile there is no other "supported" way.
Regards,
Markus
Ranorex Support Team
Re: Repeating a test case upon failure
Hi Markus,
That is unfortunate. Can you go more into detail of the possibilities to work with conditions in the test suite? I appreciate the suggestion from Bernhard but it does not seem like a very "clean" way that I would prefer at this moment.
Thanks,
Steven
That is unfortunate. Can you go more into detail of the possibilities to work with conditions in the test suite? I appreciate the suggestion from Bernhard but it does not seem like a very "clean" way that I would prefer at this moment.
Thanks,
Steven
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Repeating a test case upon failure
Hello Steven,
The possibility to work with conditions is planned for one of our future releases. But it is not certain which version will provide this feature.
Regards,
Bernhard
Ranorex Support Team
The possibility to work with conditions is planned for one of our future releases. But it is not certain which version will provide this feature.
Regards,
Bernhard
Ranorex Support Team
-
- Posts: 2
- Joined: Wed Jan 22, 2014 1:29 am
Re: Repeating a test case upon failure
Hi Ranorex Support,
To follow up on this thread, is there a way in Ranorex to re-try your test cases other than that what is mentioned above? Alternatively is there a way to add the test case again programmatically if the original test case has failed.
Also how will this work if it is a test suite setup you want to repeat instead of just the test case
Thanks,
Avinash
To follow up on this thread, is there a way in Ranorex to re-try your test cases other than that what is mentioned above? Alternatively is there a way to add the test case again programmatically if the original test case has failed.
Also how will this work if it is a test suite setup you want to repeat instead of just the test case
Thanks,
Avinash
Re: Repeating a test case upon failure
I would recommend that you look into using a Continuous Integration product (such as Jenkins, which is free) to do this.
Make your test cases very small, and create Jenkins jobs for each one. There is a plugin for Jenkins called "Naginator" that will allow you to specify how many times to rerun a job if it fails, and allows you to specify an interval to use in the repeats (so the second starts in 5 minutes after the first job, the third starts 10 minutes after the second, etc.).
This would probably be easier than trying to figure out how to do it in code. Plus, using a CI solution will allow you to do things in the future that will benefit you in the long run.
Make your test cases very small, and create Jenkins jobs for each one. There is a plugin for Jenkins called "Naginator" that will allow you to specify how many times to rerun a job if it fails, and allows you to specify an interval to use in the repeats (so the second starts in 5 minutes after the first job, the third starts 10 minutes after the second, etc.).
This would probably be easier than trying to figure out how to do it in code. Plus, using a CI solution will allow you to do things in the future that will benefit you in the long run.
Shortcuts usually aren't...
Re: Repeating a test case upon failure
Hi ...
I'm responding to an old thread - so this questions may have been addressed. As discussed in the previous thread - I also am looking for a solution that will allow me to instantiate and run methods in other classes.
My use case was recommended by Ranorex Support for a workaround to a Flash/Flex bug. When testing against Flex objects - when the flex item is visible - Ranorex will fail to find the item and throw an exception (see below).
The workaround suggested by Ranorex support is to close the Flex Object and reopen and try again. This seems to work. The problem is intermittent.
Failed to find item 'SummaryRegressionRepository.JSA.LibraryReference.PrintButton'.
No element found for path '/dom[@domain~'[a-z, A-Z]*.workbench.judicial.int.westgroup.com']/body/div/div/div[@id~'[0-9]*_judicialsummary.trialCourtOrderSummaryPage.section.widget.libraryReferenceWindow']/../..//button[@automationname='LibraryReferenceWidget.printButton']' within 45.1s.
It seems data from the Flex Object is stored in the browser (IE.
cache
We have created a WorkAround Test case that will execute before the suite of tests. The WorkAround Test will open all the flex objects prior to using the objects during the primary test.
Now the use case for access to Ranorex user classes ...
During the WorkAround Test a flex object will fail with a 'Failed to find item' exception. In these cases we would like to close the browser and begin again. We have several user classes for clearing data, closing a browser, opening a browser, setting data. It would be ideal if I could instantiate these classes and execute methods (.Run) from another class (i.e. WorkAround).
Below is some code.
The class below will clean data after a test process. It would be very cool if I could call this from another class in the Ranorex suite.
The code below is a example of how it would be nice to call the class above
from the user class below. They are in different Code Modules.
See the IF Clause below
I'm responding to an old thread - so this questions may have been addressed. As discussed in the previous thread - I also am looking for a solution that will allow me to instantiate and run methods in other classes.
My use case was recommended by Ranorex Support for a workaround to a Flash/Flex bug. When testing against Flex objects - when the flex item is visible - Ranorex will fail to find the item and throw an exception (see below).
The workaround suggested by Ranorex support is to close the Flex Object and reopen and try again. This seems to work. The problem is intermittent.
Failed to find item 'SummaryRegressionRepository.JSA.LibraryReference.PrintButton'.
No element found for path '/dom[@domain~'[a-z, A-Z]*.workbench.judicial.int.westgroup.com']/body/div/div/div[@id~'[0-9]*_judicialsummary.trialCourtOrderSummaryPage.section.widget.libraryReferenceWindow']/../..//button[@automationname='LibraryReferenceWidget.printButton']' within 45.1s.
It seems data from the Flex Object is stored in the browser (IE.

We have created a WorkAround Test case that will execute before the suite of tests. The WorkAround Test will open all the flex objects prior to using the objects during the primary test.
Now the use case for access to Ranorex user classes ...
During the WorkAround Test a flex object will fail with a 'Failed to find item' exception. In these cases we would like to close the browser and begin again. We have several user classes for clearing data, closing a browser, opening a browser, setting data. It would be ideal if I could instantiate these classes and execute methods (.Run) from another class (i.e. WorkAround).
Below is some code.
The class below will clean data after a test process. It would be very cool if I could call this from another class in the Ranorex suite.
Code: Select all
// Removes all summaries for current artifact being used in test
[TestModule("D8413E73-6B7F-4B63-9CCA-22C011D6402C", ModuleType.UserCode, 1)]
public class CleanupData : SummaryBase, ITestModule
{
public CleanupData()
{ }
void ITestModule.Run()
{
Report.Info("Removing all summaries in artifact with UUID = " + ArtifactUuid);
SummaryRestClient client = new SummaryRestClient();
client.deleteSummariesForArtifact(ArtifactUuid);
}
}
from the user class below. They are in different Code Modules.
See the IF Clause below
Code: Select all
/// <summary>
/// Description of WABAddNewHeadnote.
/// </summary>
[TestModule("CD0BBE37-4002-45EC-8B21-23A0FEDF4F3F", ModuleType.UserCode, 1)]
public class WABAddNewHeadnote : ITestModule
{
SummaryRegressionRepository repo = SummaryRegressionRepository.Instance;
/// <summary>
/// Constructs a new instance.
/// </summary>
public WABAddNewHeadnote()
{
// Do not delete - a parameterless constructor is required!
}
/// <summary>
/// This method will wait for the adapter to become visible. The Timer
/// object is created, configued and then started. We will wait until the
/// timer expires. If the timer will expire we will send a failure
/// mesage to the family.
/// </summary>
/// <remarks>You should not call this method directly, instead pass the module
/// instance to the <see cref="TestModuleRunner.Run(ITestModule)"/> method
/// that will in turn invoke this method.</remarks>
void ITestModule.Run()
{
Mouse.DefaultMoveTime = 300;
Keyboard.DefaultKeyPressTime = 100;
Delay.SpeedFactor = 1.0;
bool returnVal = true;
int retryCount = 0;
int retryLimit = 3;
while (retryCount < retryLimit)
{
Report.Info("Add new Headnote.");
SummaryRegressionRepository.Instance.JSA.Features.CreateNewDropdown.Click();
SummaryRegressionRepository.Instance.JSA.Features.AddNewHeadnote.Click();
SummaryRegressionRepository.Instance.Alert.OKButton.Click();
// Wait for the object to become visible
// waitForVisible() is not defined for this sample
if (!waitForVisible(SummaryRegressionRepository.Instance.JSA.Headnote.X))
{
// Headnote didn't open - will try again
Report.Info("Fail: Headnote didn't open. RetryCount: ");
// Here is where we need the ability to instantiate a user
// class and call methods.
// Something like below
CleanupData cleanupData = new CleanupData();
cleanupData.Run();
}
else
{
Report.Info("Headnote opened successfully.");
break;
}
}
}
}
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Repeating a test case upon failure
Hi,
I am afraid I didn't get the issue.
It is possible to start a code module from another module. You can even execute specific methods in the action table of a Recording, the Recording (in the Recording.UserCode.cs file) just needs to derive from the CodeModule where the method is defined in.
It is of course also possible to start specific code modules from code, you just need to use the TestModuleRunner.Run(ITestModule) method:
Regards,
Markus
I am afraid I didn't get the issue.
It is possible to start a code module from another module. You can even execute specific methods in the action table of a Recording, the Recording (in the Recording.UserCode.cs file) just needs to derive from the CodeModule where the method is defined in.
It is of course also possible to start specific code modules from code, you just need to use the TestModuleRunner.Run(ITestModule) method:
UserCodeModule1 userCodeModule = new UserCodeModule1(); // would be CleanupData in your case TestModuleRunner.Run(userCodeModule);Please let me know if I got this correctly.
Regards,
Markus