CA Service Virtualization Review

The web services protocol for SOAP requests and responses is the most valuable feature. The user guide should be updated to include more scenarios.

Valuable Features:

The web services protocol for SOAP requests and responses is the most valuable feature. It is easy to set up the test for them without much coding, thus it is easy to train new testers to work on it in a graphical development environment where one can see the flow of information visually like the TIBCO product. It is easier than SoapUI to follow.

Improvements to My Organization:

This product has introduced a simple-to-use middleware tool, which helped our team conduct web-services testing.

Room for Improvement:

The most important thing is to have integration with Excel. It has a way to read the Excel sheet but not to write onto it. If I need to generate customized reports in LISA in Excel sheets for management purposes (which can show me number of test cases pass/fail/not started, etc.), I can't do this since you can't ask Service Virtualization to put specific values in particular cells with a particular color, etc. We were able to achieve it only using Java, which meant a lot of scripting, error debugging and rework, and this defeats the purpose of it offering a codeless environment to testers. A simple groovy scripting in SoapUI can achieve all of the above.

The Service Virtualization section needs to be more intelligent in offering a good solution to customers. For example, the virtual service created via the request response pairs needs spoon feeding of responses before the service starts performing. And I am not sure if it serves the purpose of virtual environment till the actual system becomes available. I have yet to see a proper implementation and successful usage of a virtual service through it.

CA should provide a free trial version of the tool (was not available till version 7.0.10) so that clients can try it out before making a purchase.

Use of Solution:

I've used it for two years.

Deployment Issues:

There were no issues with the deployment.

Stability Issues:

The product is yet to become stable. I have seen instances when the tests have not been repeatable in LISA when scripting is done in Java. After removing and adding a character to the code, the test starts working again. Sometimes restarting it gets it working. Also, there are caching issues in LISA and it picks up the old values of the variables/parameters/properties in the tests even though the values have been updated and saved in the test before commencing the run. Thus, people prefer using more stable solutions like JMeter and SoapUI.

Scalability Issues:

There were no issues with the scalability.

Customer Service:

The customer support for LISA is average with the associates not being reachable all the time. Also, there were no specific communication protocols set up to interact with LISA support team and we were mostly relying on the Confluence forum to have our queries resolved. Also, the search results on Google do not provide answers or hints to the questions LISA testers face, unlike SoapUI and Ready API which have a lot of online support available. One can only rely on the user guide of LISA which does not cover all the scenarios.

Initial Setup:

The initial setup was straightforward, but the settings procedure, like providing the correct domain name, server name, username and password, did not work the first time they were provided from LISA.

Implementation Team:

I got it implemented through a vendor team. There are a couple of settings you have to play around with before it starts working.

Other Solutions Considered:

Choosing this product was a management decision and we would prefer other solutions because in order to achieve a customized solution one would have to do scripting. This tool needs to offer a lot more options before it can be marketed as a codeless environment that can really help an organization chive results with such a testing team that is not good at coding.

Other Advice:

QTP can achieve almost everything to do with scripting testing. Other tools like SoapUI/Jmeter being more popular become the obvious choice.

Add a Comment