Effective Software Test Automation: Developing an Automated Software Testing Tool [Englisch] [Unbekannter Einband]

(1) developers or QA practitioners who need to quickly implement a testing tool that dramatically reduces the time it takes to execute test cycles. This book will service this audience as a technical manual for the tool, which can be downloaded in source and binary formats from the publisher's web site.

(2) developers who want a baseline tool that can be extended and modified to suit their specific needs, or to be integrated into (or augment) a suite of existing testing tools. The clear explanation of how this tool was designed and the code that makes up the tool will easily meet this audience segment's needs. More importantly, the book even shows how to develop assemblies with which to test the tool.

(3) developers who want to improve their own skills by examining an integrated application that has been engineered by experts. The tool upon which this book is based is one of the most elegantly designed and implemented examples of good programming practices and software engineering. Not only does it show how to harness some of the capabilities of the .NET framework and associated IDE, but also shows how to integrate into Microsoft Office applications and create a seamless enterprise application. The tool in this book is integrated into Microsoft Excel (version 2000), and the concepts, use of XML, and the way everything is tied into a coherent application that provides useful services exemplify how to develop business applications.

The tool itself is one of the most useful and clever test automation approaches I've seen in any environment. It auto-discovers what must be tested, and develops test scripts with virtually no intervention on the part of QA. It also dramatically reduces test time, and will significantly contribute to the deployment of defect-free applications, especially in a rapid development environment (such as those shops using extreme programming or agile methods). While the tool will not completely replace other testing tools, it will nicely augment them. I think the best use of this tool is in the development domain to be used for unit and integration testing. It also supports incremental regression testing, which can be effectively executed by the build manager before promoting to QA for final test and release. Of course, this tool will also fit nicely within the QA domain, especially with its ability to support both white- and black-box testing.

In combination this book and the associated tool are, in my opinion, important contributions to the QA profession, and to teams working with Microsoft technologies.

This book accomplishes two important goals. The first is the step by step creation of the test tool. I implemented the code to create the automated test tool to generate the data store and the test script up through chapter 7 without any problems. It worked as described, and the code, along with its description, for the remainder of the book was well presented. The second significant contribution the book makes is the tour it takes the reader through aspects of the .Net environment. It demonstrates the robust development qualities in .Net. An example of this is the hands-on discussion of the namespace system.Reflection. Further, it provides an insightful guide to referencing the MS Excel Object Library through example as the code dynamically creates a datastore in an Excel worksheet.

The authors also provide an overview on how to use the tool effectively and provide adequate notes on usage context. Towards the latter part of the book they are adamant that testers still need to analyze the requirements and understand the data used to test the application. For instance, testers still need to understand concepts such as boundary value analysis, equivalence partitions and other testing techniques which are more thoroughly addressed in other theoretical texts. It should be noted, that the tool presented by the authors does not guarantee code coverage (i.e. path analysis within a method). Readers will have a better appreciation for the tool if they study some of the classics in testing theory by Glenford Myers, Robert Poston and Paul C. Jorgensen.

The tool itself is certainly useful from the standpoint of regression testing. In other words, once an application is stable in its functionality, the test scripts generated can form a baseline of testing when changes are introduced. The authors do mention XP, but I have to comment as to the effective use of the tool in XP development. In XP, testing is based on test driven development (TDD). In TDD, the test script is written BEFORE the implementation code is ever produced. In TDD (agile development) the test harness is an up front statement of the design stating what the code should do - not what it already does because it hasn't been written yet. Their tool requires the implementation code (actually, the assembly) to be created before it creates the test script. Again, though, there are many development efforts where TDD is not used (the majority actually ) and the tool the authors present provides a good start for analysis of code completed by developers to generate an initial set of scripts when a maintenance environment receives transitioned code. Their observations of the spirit of XP are very accurate. I am only expressing a philosophical concern in the testing arena.

In essence, the tool can have benefits, but like any tool should be used in the proper context. Developers should not assume that any tool can replace human intuition and solid testing techniques.

Overall, I found the book to be well focused and clearly written. I only have two criticisms of the book itself, which is why I would rate it four out of five stars (which for me is a good rating). I think the title or cover should have noted something about .Net (e.g. "A Case Study Using .Net"), even though the authors claim the concepts are generic. The concepts are, but the implementation is not since legacy programming environments do not have support for integrating an XML framework in the code for comments. In my previous life as a UNIX and C developer and considering the cost, I would get very little benefit from the text since the "generic" part is limited after Chapter 2. Second, while the authors provided a high level discussion of the features the tool would provide in the later part of Chapter 2, it would have been helpful had there been illustrations with a brief description of how it would be used earlier. The tool creation was presented piece by piece, and while I was able to follow along with the authors, I found myself immediately skipping ahead to Chapter 7 to get an idea of what the end product would look like. After doing so, I went back to where I left off in Chapter 4 and the rest was fine. It's a small thing actually, but an early picture or two of where the code is taking the reader is helpful.