Ranorex Forum

In Scotland many job descriptions ask for QTP experience.

Can someone who has used QTP give me a comparison of the two tools? I have only used Ranorex and T-Plan Robot, but cannot see how QTP can be any better, especially at its cost!

This is purely out of interest!

Thanks
Scott

Download the trial version of QTP
I think you will quickly hate it

I've had a play about with it. It seems similar to Ranorex but looks ancient and only allows you to script with VBscript and costs lots lots more. I can't get my head round it!

Many companies here in the US specifically ask for X tool or Y tool experience. A lot of this is because either A) someone in their past had used tool X to do some automation and it either became shelfware or the person left and no one knows what to do with it or B) they already have a team doing work in tool Y and need experienced people to fill in the team that know tool Y, and sometimes C) someone said automation needs to be done here and someone else had heard the name of the company and makes tool Z.

In any case a strong understanding of test automation and its principals with any automation tool will get you noticed. Many companies that have tried automation in the past or are ready to "give it a go" really don't know what they want or what can be done. They just think, well we need to do something to help. And in more cases than not they fail. I've gone into many companies as a consultant or an employee and worked to straighten them out. Many of them end up succeeding in some of their scope but many others still fail due to unwillingness to dedicate the proper resources and time or just figuring out that the initial steps of an automation project is too difficult and costly.

Regardless of my rant, If you are strong with automation (regardless of the tool or tools) you can make inroads into any company wanting an automation engineer. You can play up your experience and let them know that picking up a new tool is just a matter of semantics and getting used to that new tool. But you can also sell them on the benefits of your tool of choice (such as Ranorex) including power, ease of use, low cost of entry and support, support, support... These big companies really do lack in supporting their customers... Also, if you do interview with a company for automation (regardless of their tool) be sure to bring examples of how automation was a success in your past ALSO don't forget to talk about how it has failed and what was done to make it right. As a person who has interviewed many an aspiring automation engineer I know not everything succeeds 100% of the time so I want to hear about the failures, and what was done to fix them and in turn make you a better automation engineer.

Good luck!

In my observation, UI automation (or any automation) is similar to product development work and has all the characteristics of it in terms of resources, processes (requirements, design, dev,code, test, maintain...) etc. But management don't provide for these requirements in the same way they provide for Product dev.
Evidently one cannot expect a high level of investment because automation code is not a direct revenue generator.

But it's even a reasonable level of investment turns out to be an ROI challenge often. Then there is also the flakiness/brittleness associated with UI Automation (no matter what tool). An application can fail in a million ways and a programmed tool can only go so far before it gets too much to sustain that code. There seem to be a new paradigm to test below the UI (fixtures) in the application code etc... but reality: Lot of applications are legacy in some shape or form to support this and when it comes to release deadlines writing test fixtures is the last thing product managers/directors care about. So what does an automation engineer career hold?

More details comparing QTP and Ranorex:
http://www.ranorex.com/forum/comparison ... t1921.html

As a current QTP user, I'll give my own personal opinion.

QTP isn't better. It just has more customers.

It uses VBScript, which is better than proprietary automation languages, but worse than anything else. Also, its implementation of VBScript is frustrating. It has no support for public / global variables. Functions called in the Function Library or called Actions break this scope and have no access to these variables. The exception is if you perform a registry tweak to maintain access to a global dictionary, or if you use the proprietary 'Parameter("name") = value' to assign values to environmental variables, which are all stored as strings.

It isn't scalable. If you have a test with multiple actions that you wish to call from another test, you're shown a single, flat (not hierarchical) dropdown where you have to find (can't simply type) the name of the action you wish to add to your test. There are tons of poor interface design issues that I've found.

Exception Handling is not as strong as I desire. It exists (with options to skip to the next line of execution, run a separate function, or restart the computer and fail the test), but primarily due to the language used, there's no easy way to make nested exceptions (I do it, but I'm using silly work-arounds). VBScript's lovely (sarcasm) error handling involves only skipping to the next line or failing outright. This, combined with actions/the function library not having access to global variables (including the error object that is set when an error is detected) makes exception handling a challenge.

You can only have one test open at a time, which becomes a big pain when you're trying to modify actions called from other tests. You have to close out of your one test and open the other, a painful process.

The debugger leaves a lot to be desired. You have no way to view a call chain and see the line of execution in a script that called the currently executed action / function. You can't skip lines of execution. For some reason, "Step Over" doesn't even seem to work for me with actions (where called actions are executed and the test is paused after returning to the higher level action).

Support is lacking. The easiest way to resolve problems is to post/search in non-affiliated forums, as its main benefit is that it is the current industry leader (in terms of number of customers) and there are probably many other people who are dealing / have dealt with your issues. Their own support website is very messy (HP offers a ton of products for a ton of different market segments, and the site used doesn't seem to care that you only have a focus on one of their products. "support.openview.hp.com" and "irtc.hp.com" are 2 different sites that offer bits of the necessary support. I get patches from one location, and ask support questions from the other)

The Update Manager, which they may have finally fixed, was the only real support I cared about from HP. I wanted to get the latest features / patches. Frustratingly, this system was broken a significant amount of the time I used the product. HP eventually went to another upgrade system without notifying me, and I had to get a patch from their awful support website just to continue to get the upgrades I needed. This new upgrade system still didn't detect the latest version of QTP being released.

The product monetizes support for different GUI interfaces. Want Java detection? Buy an add-on, etc.

It isn't all bad. The actual object repository is pretty reliable and intuitive. There are tools to merge independent OR files. Support for Regex is nice. Offline Documentation is actually really good and fairly comprehensive. Fair VBScript support means additional functionality can be coded without it being an explicit feature of the test utility (I use VBscript commands to control other applications or perform DB queries). Keyword View seems like a reasonable way to allow non-coders to call actions created by coders without actually seeing any code.

You may notice that none of these features is something lacking in a modern automation tool like Ranorex, though. Honestly, I can't think of anything in QTP that can't be found in cheaper alternatives. Maybe that's because I'm not the customer HP was shooting for (someone reliant on the record/replay functionality with zero knowledge of scripting?), maybe my lack of experience with the tool (2 years) has hindered my ability to design around these issues, who knows. I just know that I can't stand these issues any more and am actively looking for alternatives.

Sorry for the long post / rant. I just thought I'd share my personal experience, and say that the main reason QTP is used by software companies and is requested as a job requirement has much less to do with the capability of the product and more to do with the fact that it's still, somehow, widely used in the industry.

mrusso wrote:... the main reason QTP is used by software companies and is requested as a job requirement has much less to do with the capability of the product and more to do with the fact that it's still, somehow, widely used in the industry.

IMHO, that's because of HP's aggressive sales and marketing teams, which will do anything to sale it...
In our company, we just gave them a call to inquire about QTP and few days later 3 HP reps showed up in our office to demonstrate their product! They even asked to talk to our CEO to convince him to buy it...
Well, at the end we bought Ranorex, but they (HP) called us few times to follow up...

Great post!
A good rant really helps

We we using TestPartner and it had many of the same problems that you mention for QTP. Its development language VBA was lacking lots of features that are so easy to do in C#/VB.net - it was very expensive, the IDE was not great, it had no source code control, support was not good etc. etc.

If we try to project into the future you will see that these "old" style automation tools in my opinion are going to die out, leaving tools like Ranorex, TestComplete etc. as the fore runners. Lets not also forget Microsoft is making inroads into GUI automation in Visual Studio 2010, and that will no doubt become popular also.

Ranorex is a cost effective, and feature rich automation tool, its choice of Dev language is perfect. However i still believe there is a few features missing when you want to scale your automation effort to run 1000's of tests per day. These are:

- Remote test distribution, execution
- Central database of test results, logs, output files
- Reporting tools for analysis of 1000's of results
- Integration with test case management, and defect tracking tools

All of the above are available with Microsoft Visual Studio 2010, Team Foundation Server etc.
But are costly!

Regrading the QTP customer base being large... QTP was years back mercury's product, introduced as a a replacement to Winrunner (which IMHO was a good but a coder's tool and not non-coder friendly). QTP started as Astra Quick test for web back 2003/04. HP acquired Mercury and hence the larger customer base.

yeah, the few times I have been to HP's website (printer drivers etc)... I simply hate the overwhelming content.

I know it's a while since this post was put up, but I thought I'd put in my 2 cents worth. I've used pretty much all the automation tools available over the past 15 years, including QTP for 10 years, Winrunner before that, Rational Robot, FTP and Ranorex for the past almost 4 years. (Still also using QTP).
Needless to say the success of your automation comes down to treating it as a development project (as others have said above). I've found ways over the years to sell that to clients and treat automation as such. This helps when you're able to provide previous project successes using the strategy your proposing, and having an existing automation framework that you can utilise for essentially any companies projects.
The secret for us was building a framework that is not tool dependant, hence wrapping all the tools specific code into your own core wrapper functions. Whilst this is a lot of work initially, in the long run it's only a little bit more than starting off with a new tool anyway, so we took on that challenge when we had to ditch QTP when our company changed UI's for the AUT, and QTP didn't support ExtJS at the time (kept crashing and support couldn't resolve the issue), hence our move to Ranorex.
The move to Ranorex however was pretty painless as we just replace all the QTP calls with the appropriate Ranorex calls within our framework. Admittedly there was the language issues of porting our existing framework from VBScript to C#, however we've now made our C# framework capable of calling scripted tool code too, hence we use our C# framework for running both our Ranorex and QTP scripts. As for the change to C#, VBscript is no comparision. The best bit being able to step forward and backward in C#, hell skip steps all together or go back to previous steps.
Now obviously vendors have a vested interest in trying to tie you into using their tools, so HP definitely don’t like you using QTP like this, hence don’t provide support for complex frameworks. They want you to be stuck into using their base functionality and hence make it difficult to swap tools once you've invested all that time and effort in their framework. My advice however is write your own, or at least wrap their functionality. Most tools are moving to C#/Java support these days, so with a framework like ours you can pretty much swap and change (if you really wanted to) however we’re more than happy with what Ranorex provides us, and at a reasonable price.
Whilst “free” tools are great, many companies don’t feel comfortable with the risks associated with using them. Either way, a framework like what I’ve suggested above can suitably slot in Selenium WebDriver or any other support simply enough. Mind you, from what I’ve heard from people who I’ve previously worked with on my current project that now use WebDriver, it’s still got some way to go before being close to a tool like Ranorex.

As a consultant I worked on a project to "sort out" a set of QTP tests and a Excel based QTP framework that had taken over two years to develop, and eventually wasn't being used at all. In the end I was able to completely rewrite and enhance their testing using Robot Framework and Selenium in Java in 4 months!

From that experience theres a few things I noticed about QTP:

- Generally large companies purchase the entire HP suite of tools as the senior managers with the purchasing power do not know of alternatives, and feel safer following the herd.

- QTP skills are often outsourced to India, Poland etc. where there are many resources available at low cost

- QTP users generally have limited programming experience. I met several QTP users who didnt know what a Class was or anything about OO programming, Junit, NUnit, CI, BDD etc.

- QTP users tend to write Excel frameworks which have to executed within QTP itself! Meaning your framework and tool are tied together.

All in all QTP is a reasonable tool if your forced to use it, but it hides a lot of bad practices, not only in the programming skill but also it doesnt give the automation engineer any real knowledge of automation technology. For example I quizzed a QTP user about MSAA and UI Automation in .net - he had no clue what I was talking about, all he know what "I point the Spy here, and capture and object"

After this project I moved onto another project that was in proof of concept phase. It was a web testing project where the client had no automation in place. We did a trial with QTP, and it recognised nothing on the web page, whereas selenium did. However the senior management of the consultancy company I worked for pitched the project using QTP telling the client they need to buy licenses up front. Of course they complained why they have to spend $1000's to know if QTP will work or not... whereas my pitch using Selenium would of got us the project. Needless to say we didnt get it...

jkrohn wrote: The move to Ranorex however was pretty painless as we just replace all the QTP calls with the appropriate Ranorex calls within our framework. Admittedly there was the language issues of porting our existing framework from VBScript to C#, however we've now made our C# framework capable of calling scripted tool code too, hence we use our C# framework for running both our Ranorex and QTP scripts. As for the change to C#, VBscript is no comparision. The best bit being able to step forward and backward in C#, hell skip steps all together or go back to previous steps.

I'm very curious to know how this was accomplished. Almost all of our new development in the last year or so has been in Ranorex, but we still maintain a legacy of QTP tests (years of development effort, and old regression we can't simply throw away).

We would love to just maintain and run everything in Ranorex, but I don't see how this is easily possible. How is you C# framework calling QTP scripts... and more importantly, how is it able to read the QTP repository without running QTP itself (the repository is a binary file, so it's not xpath)?

I don't see how this can be possible...