Possible to notify at point of failure?
Possible to notify at point of failure?
Hi,
We're evaluating Ranorex and it's looking pretty good. We're now at the stage of determining how we can handle recovery scenarios and notifications.
We are running a suite of tests, with each recording configured to try to continue its steps if there's a failure. Each case in the suite is then configured to continue to the next case if the recordings can't continue.
But rather than wait until the suite of tests has finished to get a notification and report (e.g. by email using the Send Mail module) is it possible to have an alert as soon as there's a failure in any of the recordings or cases?
Thanks,
Gav
We're evaluating Ranorex and it's looking pretty good. We're now at the stage of determining how we can handle recovery scenarios and notifications.
We are running a suite of tests, with each recording configured to try to continue its steps if there's a failure. Each case in the suite is then configured to continue to the next case if the recordings can't continue.
But rather than wait until the suite of tests has finished to get a notification and report (e.g. by email using the Send Mail module) is it possible to have an alert as soon as there's a failure in any of the recordings or cases?
Thanks,
Gav
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Possible to notify at point of failure?
Hi,
Regards,
Peter
Ranorex Team
If I can follow your description you want a notification if something fails, right? Or do you want a message box which pauses the automation?Gav wrote:But rather than wait until the suite of tests has finished to get a notification and report (e.g. by email using the Send Mail module) is it possible to have an alert as soon as there's a failure in any of the recordings or cases?
Regards,
Peter
Ranorex Team
Re: Possible to notify at point of failure?
Hi Peter,
That's right - we would like the tests to continue without pause, but to 'raise a flag' as soon as something fails so we can start working on it, rather than waiting for the report at the end.
Gav
That's right - we would like the tests to continue without pause, but to 'raise a flag' as soon as something fails so we can start working on it, rather than waiting for the report at the end.
Gav
Re: Possible to notify at point of failure?
I like this idea too 
If the ranorex assertions fire an even upon failure, we could add a handler for that event and ping out an email...

If the ranorex assertions fire an even upon failure, we could add a handler for that event and ping out an email...
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Possible to notify at point of failure?
Hi,
Would it be possible to post us a few samples, where the other two solutions aren't enough and you need an EventHandler?
Thank you.
Regards,
Peter
Ranorex Team
Currently it is possible to use the Ranorex Blog Customizing Ranorex Reports or you use the Tear Down part of the Test Case to handle this.Gav wrote:That's right - we would like the tests to continue without pause, but to 'raise a flag' as soon as something fails so we can start working on it, rather than waiting for the report at the end.
It would be possible to implement an EventHandler, but currently we don't know what this handler should do? Should this handler allow to execute "whatever" code or just logging code? Only on failure or even on success?sdaly wrote:If the ranorex assertions fire an even upon failure, we could add a handler for that event and ping out an email...
Would it be possible to post us a few samples, where the other two solutions aren't enough and you need an EventHandler?
Thank you.
Regards,
Peter
Ranorex Team
Re: Possible to notify at point of failure?
Hi Peter
The way I would imagine to use it would be like this -
The way I would imagine to use it would be like this -
Code: Select all
//register a handler for the validate 'failure' event
Validate.Failure += new EventHandler(Failure);
public static void Failure(object sender, RanorexEventArgs e){
sendEmail("Assertion failed - location is " + e.className + "::" + e.memberName + " - " + e.errorMessage);
}
Re: Possible to notify at point of failure?
...I should probably explain this a bit more!...
We use NUnit to control our tests - we determine failure by surrounding each test in a try/catch block. I would like to send out an email notification upon failure. We could add some code to our tear downs to send an email if the test has failed. We would have to remember to add this bit of logic to each of our teardowns though.
The advantages I see of having an event fired upon failure is -
>Say we have a validate.exists within a code module/helper method such as "Login()"...if a 'failure' event was fired here (with stacktrace details) which invokes a method (our handler) that sends an email, then we would get an email to say failure has occurred at 'Login' and not just that test x has failed.
>We could log failures to our DB for each validate, again with the class/member details...this would allow us to report on failing areas such as 'Login', instead of just being able to see failures at test level. We could quickly see 10 tests have failed but 10 'Login's have failed and therefore identify that there is a login issue.
>If we have a test which has several validates wrapped in a try/catch because we want it to try all steps, then we would get an email notification for each of the failure points, not just one big failure at the end of the test.
In general, it would just be a nice little feature which would open up some powerful opportunities...
I hope that makes sense
Sorry for hijacking this thread...all seems relevant though
We use NUnit to control our tests - we determine failure by surrounding each test in a try/catch block. I would like to send out an email notification upon failure. We could add some code to our tear downs to send an email if the test has failed. We would have to remember to add this bit of logic to each of our teardowns though.
The advantages I see of having an event fired upon failure is -
>Say we have a validate.exists within a code module/helper method such as "Login()"...if a 'failure' event was fired here (with stacktrace details) which invokes a method (our handler) that sends an email, then we would get an email to say failure has occurred at 'Login' and not just that test x has failed.
>We could log failures to our DB for each validate, again with the class/member details...this would allow us to report on failing areas such as 'Login', instead of just being able to see failures at test level. We could quickly see 10 tests have failed but 10 'Login's have failed and therefore identify that there is a login issue.
>If we have a test which has several validates wrapped in a try/catch because we want it to try all steps, then we would get an email notification for each of the failure points, not just one big failure at the end of the test.
In general, it would just be a nice little feature which would open up some powerful opportunities...
I hope that makes sense

Sorry for hijacking this thread...all seems relevant though

- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Possible to notify at point of failure?
Hey Scott,
We discussed this internally and we could implement an event handler.
But actually we don't know which arguments we should return.
We could return the first class/method which forces the validate? We could even return you the stacktrace, but for the most users the stacktrace is too complex and difficult to handle and thus not useful.
So what would you like to know when such an event handler is called?
Regards,
Peter
Ranorex Team
We discussed this internally and we could implement an event handler.
But actually we don't know which arguments we should return.
We could return the first class/method which forces the validate? We could even return you the stacktrace, but for the most users the stacktrace is too complex and difficult to handle and thus not useful.
So what would you like to know when such an event handler is called?
Regards,
Peter
Ranorex Team
Re: Possible to notify at point of failure?
Hi Peter
I guess the class/method which triggered the validate would be adequate and would keep it simple...so that, the validation message and the exception message?
Cheers
Scott
I guess the class/method which triggered the validate would be adequate and would keep it simple...so that, the validation message and the exception message?

Cheers
Scott
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Possible to notify at point of failure?
I added a feature request to our internal bug tracking system.
Regards,
Peter
Ranorex Team
You want the validation exception and the EventHandler exception inside the Report, right?sdaly wrote:the validation message and the exception message?
Regards,
Peter
Ranorex Team
Re: Possible to notify at point of failure?
Hi Peter
Those are the things I would like passed through the event args! So then in the custom event handler, you can do whatever!
Thanks
Scott
Those are the things I would like passed through the event args! So then in the custom event handler, you can do whatever!
Thanks
Scott
Re: Possible to notify at point of failure?
Please excuse my noobness, but if the feature request goes ahead, does it mean our developer guys will be able to get Ranorex to send notifications as soon as there is a failure in the test suite, while the test suite then attempts to carry on with other tests?
Gav
Gav
- Support Team
- Site Admin
- Posts: 12145
- Joined: Fri Jul 07, 2006 4:30 pm
- Location: Houston, Texas, USA
- Contact:
Re: Possible to notify at point of failure?
This feature request would just provide another way to react to validation failures using code, e.g. by sending an email.
http://www.ranorex.com/blog/customizing-ranorex-reports
http://www.ranorex.com/forum/send-mail- ... t2356.html
Regards,
Alex
Ranorex Team
That feature is already available. You just need to attach a custom logger that sends an email if a failure is reported. See following blog on how to customize Ranorex reports and the "Send Mail" automation module:Gav wrote:does it mean our developer guys will be able to get Ranorex to send notifications as soon as there is a failure in the test suite, while the test suite then attempts to carry on with other tests?
http://www.ranorex.com/blog/customizing-ranorex-reports
http://www.ranorex.com/forum/send-mail- ... t2356.html
Regards,
Alex
Ranorex Team
Re: Possible to notify at point of failure?
Hi Alex,
Apologies again, but I just need to be absolutely clear on this before I get our devs to have a look at it for me. Will the custom logger allow a message to be sent immediately that a step fails (regardless that the script will attempt to continue), without having to wait until the suite completes and the report containing failures and results is presented at the end?
Gav
Apologies again, but I just need to be absolutely clear on this before I get our devs to have a look at it for me. Will the custom logger allow a message to be sent immediately that a step fails (regardless that the script will attempt to continue), without having to wait until the suite completes and the report containing failures and results is presented at the end?
Gav
Re: Possible to notify at point of failure?
Gav, that is correct, you can do that. I implemented a solution using a custom logger last week (never even knew it existed!)... I'm going to write a blog post next week on it so I'll post you a link once I've done so...