Page 1 of 1

TestSolution Lite Demo - A Ranorex Partner

Posted: Wed Sep 19, 2012 8:55 pm
by Dynamism Solutions
As A Proud Partner Of Ranorex. We here at Dynamism Solutions have built a Test Management Framework Based On The Power Of Ranorex.

Below is a video link to a lite demo of TestSolution - A Keyword driven Test Management Framework which allows your test teams to build both manual and automated tests through a simple GUI interface.
The concept is simple - all overloaded keywords and code are already built into the Application. Straight out of the box you can begin building automated Tests, The Automation expert can
empower their Manual Test team to speed up and increase the automated Test coverage through the use of our interface.
There are some other features we think are cool and would we love any feedback either on this forum or via direct contact.

Enjoy the demo.

http://dynamismsolutions.com.au/TestSol ... nDemo.html

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Wed Sep 19, 2012 10:18 pm
by Ciege
Honestly, I don't see what this provides much more than what is already built in to the Ranorex Studio with the recorder, although I think it still contains most of the limitations of a record and playback scenario.
Correct me if I am wrong, and I am quite frequently, but I don't see a benefit to using a tool like this as opposed to the power and benefits you get with writing real test automation code that is in a re-usable framework that can be shared across projects and teams.

That being said, I applaud your work and hope to see if you can overcome the limitations of true record and playback all while giving non programmer types a way to write powerful automation.

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Wed Sep 19, 2012 11:59 pm
by IanF
Ciege

I think you are missing the bigger picture here as do most automation experts. This methodology isn't about you as an automation expert.

One of the key failings of automation is the need to be able to understand and write code, there are many great testers out there who just don't want to or need to write code but they can add value to the automation effort with the right tool.

I know a little about this methodology a friend and I prototyped this a number of years ago. We developed an early script-less framework based on QTP over time it was given to non-automators to see if they could build automated tests. Within days we had people producing robust maintainable automation.

The other side of tools like this is the “Write Once Principle” The idea here is that you write the test once in the framework you can then choose what you want to do with this test. You can send it to the test management tool as an automated test or a manual test. You can send the test to the file system as a stand-alone automated test. You output manual test documentation. The key here is that what is produced by the framework is the central maintainable asset.

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Thu Sep 20, 2012 10:00 am
by artur_gadomski
- It seems like this tool ads no value to Ranorex. It only packages Ranorex into different box. Not technical people will not be able to use this tool.
- No save test, now way to organize tests, no way to add test data in a form that is not 'hardcoded' in a test? Or is it just not shown on the demo?
- You only show execute steps how do you validate? It's not a test if it just does a series of actions.
- In the Progress bar some actions seemed to have result as PASS and some as Complete without much pattern to them. Or maybe I just didn't look to closely.
- 'For Automation experts' part that shows how to add your own code is a joke. I can't imagine anybody being so masochistic to write actual code in this edit window. (Maybe letting people write 'plugins' or code actions in actual IDE and let people import them).

Generally this tool looks like hobby project. I hope it's Open Source or free because charging for it on top of charging for Ranorex is worse than a Nigerian scam. If you want to write tests that non technical people can read/write look into Keyword Framework especially RobotFramework. We managed to easily integrate Ranorex into it allowing us to write tests in plain text files and actual automation in dll's.

TestSolution is not a leader in test management. I don't know who is the leader but that is not it.

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Thu Sep 20, 2012 1:19 pm
by Dynamism Solutions
Hi All
We appreciate the feedback and views of all the members in the Ranorex and wider community.
The web demo is a lite version of the concepts of TestSolution.
The demo has pulled out some of the features to condense the demo to a single test run. Yes we could have shown more - in the end we showed the simplicity in building Tests.

We can say TestSolution contains a complete Test Plan, Test Lab, Data Forms, Test Versioning, Test Results, Test Reports and our own version of business process testing.
Ie. Tests can also become modular and be reused with different data of a testers own choosing or in built data in order to save reduce time on data creation - especially if having to create hundreds of data sets for an ERP system for example.
It also integrates into Test Management tools like QC for more robust reporting and currently we are working on serveral others.

TestSolution allows users to build automated tests quickly and reliably across different technologies at a GUI level, because the code is already written and exists in DLLs and is covered for many
types of objects and technologies - Thanks to the fantastic team at Ranorex!! For the times it isnt, the embedded IDE is there to use which we hope our Users will share with each other in a collaborative way.

TestSolution was developed from a team of Automation Testers and Software Developers with years of experience, we have seen the traps and pitfalls of trying to do Test Automation and achieve a decent ROI. The reality is Test Automation is difficult and finding the right people can be difficult as well.
We hope we are taking one step in the right direction. The Test community feedback is important to us. We want to ensure when TestSolution is in the wild that it can be a viable solution to any project, any where.
If you would like any more information you can contact us directly and we will be more than happy to answer any questions.

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Thu Sep 20, 2012 10:53 pm
by JimDonovan
I think we are all missing the point, this tool isn't meant for those of us that can write code...

In 15 years of Automated testing, the biggest issue I have found is that a project starts, the application starts being developed, the testers come along and start writing manual tests, the test manager or Project manager then looks for Automated test specialists - to convert those manual tests to automated - usually, there are `1, 2 maybe 3 Autmated Test specialists and up to 20 testers. Everyone sits back wondering why the automation team can't keep up with all the new tests being created.

Stop there... This so called '5th' generation test tool, along with others, is trying to address this problem. This will NEVER replce Automated Test Specialists, it is for THE TESTER, rather than have them write manual tests then us guys convert those tests (not getting into the argument of 'Should we be automating functional tests' - that is a project argument I have faced many times), these types of tools allow NON technical, i.e. those that can't, don't want to, don't have the time etc, to learn how to write code - to write a single pass test that outputs both manual and automated tests. I.e. they write it once using pretty easy steps and the result is a manual plain english step as well as an automated test.

From my experience, and obviously that of others too, this type of framework is quickly embraced by the test team, never by automated test specialists. I hate writing tests in these type of frameworks becuase as software engineers or Automated test specialists - we like to write code.

I'll give you a quick example of a project that works for my 'similar' framework:
team of 50 developers, 11 testers (including team lead), so far 12 year project. 1 Tester is an automated test specialist. The guy who was writing automated tests was VERY technical and VERY good at his job. the App kept changing and growing, but he could never get past 23% code coverage and his test coverage was falling further and further along.

New approach, have a small team come in for a few months, write a new framework that:
1. Allows manual testers to Create Auto and manual Tests in one pass - not two
2. Allows the tests to be saved and updated and executed from Test management tool or Run and reported on just from the framework
3. Code never changes, it is written once and is all just driven from the front end and stored in DB
4. TM tool upload creates a manual test and a Auto test that has no code - just call the driver and passes the script name (no updates needed to TM tool once uploaded)
5. Ability to update Actions and keywords straight from the 'very basic' but usable ide in a simple language
6. Reporting available form the framework or TM tool.
7. Framework allows modularisation - steps can be re-used in all tests - single place of change
8. Data availability that can shared and updated across all tests or per test

This is just the short of it, of course there is more... But the end result, the testers pick it up within a week, each day they write MANY complex tests that are both manual (really onlu just for documentation since it is also automated), and auto. Easy to execute, easy to report on, easy to update with changes as they relate straight back to the object map/repository within the framework. EASY! Still need automated test specialists to impart 'the art', update repositories, write good custom actions that won't work with keyword. So far I have used a very similar process to above for a number of different companies - it has worked everytime.

Jim

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Thu Sep 20, 2012 11:16 pm
by Ciege
Given everything you mentioned, and I commend you for writing so much, I just don't see what this (particular tool) gives you more than just record & playback...

Ranorex's record & playback (while still inherently lacking due to record & playback issues) is quite capable to reach the same benefit you mention... Manual testers can quickly generate repeatable automated tests that can be passed on to a more senior person to "fully/properly automate".

Further, in most companies I've worked in, there has been a dedicated test automation teams that has one or more people on each team. Usually at least 1 senior person writing the "hard stuff" (framework, API testing, unit testing, performance, etc...) and 1 or more more junior people doing the "easy stuff" (writing automated test cases based off of the framework implementation developed by the senior person, running and maintaining the tests, etc...).

I'm still a firm believer in automation *IS* development. Just like you wouldn't hire a "script kiddie" to write code for your company, why would you hire a manual tester to do automated testing? It doesn't make sense, it's prone to failure and your ROI is never going to materialize...

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Fri Sep 21, 2012 12:09 am
by IanF
Ciege

This methodology has nothing to do with record and playback at all. The process allows non-automators to write complete automation without the need to pass basic sic... recordings on. I have seen places that try this record and pass on process it simply dosen't work.

The other reason you fail to see value is you don't use an OR as you have stated in the past. I use to do this in QTP as well as I saw the OR file as an overhead I didn't need. However a good OR is vital to good automation process it is simply a lazy approach not to use an OR.

The Ranorex community is where the QTP community was 5 years ago and it seriously needs to get its head out of the sand. 5th generation frameworks are the future of automation. Business is getting vert tired of automation failures.

The other key advantage of this framework is standardisation. The process and methodology is locked in and it is very easy for new people to pick up the process. In the past traditional frameworks have been a liability if the orginal developer leaves. In most cases the automation has to be started again.

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Fri Sep 21, 2012 8:12 pm
by carsonw
I'm not sure if it's fair to characterize certain development choices as "lazy", I think we all make the best choices we can, given time, resource or even knowledge constraints.

That said, this has been an interesting read so far. I do agree that using an OR is most certainly a requirement - it would be unnecessarily challenging to update object properties that are used all over the code for a resource shared throughout the application, when you could just update it one place (the repository), that uses references throughout your code.

Unless I'm misunderstanding the above, we've implemented similar methods in our automation as well (and much to my surprise it was suggested we were limiting our approach to automation which I do not think is the case).

ALL tests are driven from the data for us (though we use Excel not SQL and my chief complaint is the limitation in data management as you'll see in other threads), and I truly believe this is the right way to go.

Although we don't have manual testers that create our "datasheets" that drive the tests, we very easily could and they would not require ANY knowledge of coding or automation at all. They would simply enter an action keyword on the datasheet and that would drive the test forward accordingly.

The various actions are handled through the framework, and a tester could write out whatever scenario he or she could dream up and it would go ahead and do it. There's no recording here (another topic, but recording is really not the way to go, it may be a quick win but it's a poor long term investment), it's all code.

I'm curious to know more specifics about how the Ranorex Community is 5 years behind the QTP one. Honestly, that's a pretty inflammatory statement and could be the springboard for some heated, and honestly, not useful discussion (aside from being an extremely sweeping generalization).

Rather, I'd like to know what knowledge/experience/advice a group of individuals five years in the future could share with those of us who are still in the present. I'm saying that tongue in cheek, but I really do mean it quite sincerely - if there are key lessons here that have been learned already and are being missed, please spell them out so we can all learn and move forward together.

Re: TestSolution Lite Demo - A Ranorex Partner

Posted: Sun Sep 23, 2012 9:56 pm
by IanF
Carsonw

My 5 year comment refers to how this Ranorex community interacts with each other the amount of sharing of common code snippets the active sharing of methodologies and framework designs.

One thing Ranorex has going for it is a pro-active company that wants to be engaged with the community.

This TestSolution framework is not the first of it kind I have seen. I was involved in a QTP framework that was very similar and it had the same acceptance issues from Automators but it was liked by manual testers.

It is very hard to get across in a few words on forum about what this methodology is all about but I can tell you its not record and playback it is not simply filling in data sheets to create a keyword driven process that requires code behind it.

Automators who like to write code all day hate this methodology. Automators who want to deliver maintainable robust automation quickly love this process.

I use to be the one who loved writing lines of automation code all day but now I like the methodology put forward by tools like TestSolution.