Possible to notify at point of failure?

Gav · Post by **Gav** » Thu Oct 20, 2011 11:06 am

Hi,

We're evaluating Ranorex and it's looking pretty good. We're now at the stage of determining how we can handle recovery scenarios and notifications.

We are running a suite of tests, with each recording configured to try to continue its steps if there's a failure. Each case in the suite is then configured to continue to the next case if the recordings can't continue.

But rather than wait until the suite of tests has finished to get a notification and report (e.g. by email using the Send Mail module) is it possible to have an alert as soon as there's a failure in any of the recordings or cases?

Thanks,

Gav

Post by **Support Team** » Thu Oct 20, 2011 12:29 pm

Hi,

Gav wrote:But rather than wait until the suite of tests has finished to get a notification and report (e.g. by email using the Send Mail module) is it possible to have an alert as soon as there's a failure in any of the recordings or cases?

If I can follow your description you want a notification if something fails, right? Or do you want a message box which pauses the automation?

Regards,
Peter
Ranorex Team

Gav · Post by **Gav** » Thu Oct 20, 2011 1:59 pm

Hi Peter,

That's right - we would like the tests to continue without pause, but to 'raise a flag' as soon as something fails so we can start working on it, rather than waiting for the report at the end.

Gav

sdaly · Post by **sdaly** » Thu Oct 20, 2011 2:36 pm

I like this idea too

If the ranorex assertions fire an even upon failure, we could add a handler for that event and ping out an email...

Post by **Support Team** » Thu Oct 20, 2011 3:47 pm

Hi,

Gav wrote:That's right - we would like the tests to continue without pause, but to 'raise a flag' as soon as something fails so we can start working on it, rather than waiting for the report at the end.

Currently it is possible to use the Ranorex Blog Customizing Ranorex Reports or you use the Tear Down part of the Test Case to handle this.

sdaly wrote:If the ranorex assertions fire an even upon failure, we could add a handler for that event and ping out an email...

It would be possible to implement an EventHandler, but currently we don't know what this handler should do? Should this handler allow to execute "whatever" code or just logging code? Only on failure or even on success?

Would it be possible to post us a few samples, where the other two solutions aren't enough and you need an EventHandler?

Thank you.

Regards,
Peter
Ranorex Team

sdaly · Post by **sdaly** » Thu Oct 20, 2011 4:39 pm

Hi Peter

The way I would imagine to use it would be like this -

Code: Select all

//register a handler for the validate 'failure' event
Validate.Failure += new EventHandler(Failure);

public static void Failure(object sender, RanorexEventArgs e){
sendEmail("Assertion failed - location is " + e.className + "::"  + e.memberName + " - " + e.errorMessage);	
}

sdaly · Post by **sdaly** » Thu Oct 20, 2011 6:32 pm

...I should probably explain this a bit more!...

We use NUnit to control our tests - we determine failure by surrounding each test in a try/catch block. I would like to send out an email notification upon failure. We could add some code to our tear downs to send an email if the test has failed. We would have to remember to add this bit of logic to each of our teardowns though.

The advantages I see of having an event fired upon failure is -
>Say we have a validate.exists within a code module/helper method such as "Login()"...if a 'failure' event was fired here (with stacktrace details) which invokes a method (our handler) that sends an email, then we would get an email to say failure has occurred at 'Login' and not just that test x has failed.
>We could log failures to our DB for each validate, again with the class/member details...this would allow us to report on failing areas such as 'Login', instead of just being able to see failures at test level. We could quickly see 10 tests have failed but 10 'Login's have failed and therefore identify that there is a login issue.
>If we have a test which has several validates wrapped in a try/catch because we want it to try all steps, then we would get an email notification for each of the failure points, not just one big failure at the end of the test.

In general, it would just be a nice little feature which would open up some powerful opportunities...

I hope that makes sense

Sorry for hijacking this thread...all seems relevant though

Post by **Support Team** » Fri Oct 21, 2011 10:23 am

Hey Scott,

We discussed this internally and we could implement an event handler.
But actually we don't know which arguments we should return.
We could return the first class/method which forces the validate? We could even return you the stacktrace, but for the most users the stacktrace is too complex and difficult to handle and thus not useful.

So what would you like to know when such an event handler is called?

Regards,
Peter
Ranorex Team

sdaly · Post by **sdaly** » Fri Oct 21, 2011 11:04 am

Hi Peter

I guess the class/method which triggered the validate would be adequate and would keep it simple...so that, the validation message and the exception message?

Cheers
Scott

Post by **Support Team** » Fri Oct 21, 2011 12:12 pm

I added a feature request to our internal bug tracking system.

sdaly wrote:the validation message and the exception message?

You want the validation exception and the EventHandler exception inside the Report, right?

Regards,
Peter
Ranorex Team

sdaly · Post by **sdaly** » Fri Oct 21, 2011 1:06 pm

Hi Peter

Those are the things I would like passed through the event args! So then in the custom event handler, you can do whatever!

Thanks
Scott

Gav · Post by **Gav** » Fri Oct 21, 2011 2:43 pm

Please excuse my noobness, but if the feature request goes ahead, does it mean our developer guys will be able to get Ranorex to send notifications as soon as there is a failure in the test suite, while the test suite then attempts to carry on with other tests?

Gav

Post by **Support Team** » Mon Oct 24, 2011 10:24 am

This feature request would just provide another way to react to validation failures using code, e.g. by sending an email.

Gav wrote:does it mean our developer guys will be able to get Ranorex to send notifications as soon as there is a failure in the test suite, while the test suite then attempts to carry on with other tests?

That feature is already available. You just need to attach a custom logger that sends an email if a failure is reported. See following blog on how to customize Ranorex reports and the "Send Mail" automation module:
http://www.ranorex.com/blog/customizing-ranorex-reports
http://www.ranorex.com/forum/send-mail- ... t2356.html

Regards,
Alex
Ranorex Team

Gav · Post by **Gav** » Mon Oct 31, 2011 11:21 am

Hi Alex,

Apologies again, but I just need to be absolutely clear on this before I get our devs to have a look at it for me. Will the custom logger allow a message to be sent immediately that a step fails (regardless that the script will attempt to continue), without having to wait until the suite completes and the report containing failures and results is presented at the end?

Gav

sdaly · Post by **sdaly** » Mon Oct 31, 2011 12:11 pm

Gav, that is correct, you can do that. I implemented a solution using a custom logger last week (never even knew it existed!)... I'm going to write a blog post next week on it so I'll post you a link once I've done so...

Ranorex Forum

Possible to notify at point of failure?

Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?

Re: Possible to notify at point of failure?