On Introducing Automatic Test Case Generation in Practice: A Success
  Story and Lessons Learned
  The level and quality of automation dramatically affects software testing
activities, determines costs and effectiveness of the testing process, and
largely impacts on the quality of the final product. While costs and benefits
of automating many testing activities in industrial practice (including
managing the quality process, executing large test suites, and managing
regression test suites) are well understood and documented, the benefits and
obstacles of automatically generating system test suites in industrial practice
are not well reported yet, despite the recent progresses of automated test case
generation tools. Proprietary tools for automatically generating test cases are
becoming common practice in large software organisations, and commercial tools
are becoming available for some application domains and testing levels.
However, generating system test cases in small and medium-size software
companies is still largely a manual, inefficient and ad-hoc activity. This
paper reports our experience in introducing techniques for automatically
generating system test suites in a medium-size company. We describe the
technical and organisational obstacles that we faced when introducing automatic
test case generation in the development process of the company, and present the
solutions that we successfully experienced in that context. In particular, the
paper discusses the problems of automating the generation of test cases by
referring to a customised ERP application that the medium-size company
developed for a third party multinational company, and presents ABT2.0, the
test case generator that we developed by tailoring ABT, a research
state-of-the-art GUI test generator, to their industrial environment. This
paper presents the new features of ABT2.0, and discusses how these new features
address the issues that we faced.
