Not every software testing project can or should be automated. Before your department accepts a new test automation project, you should establish a process by which projects are reviewed and either accepted or rejected. This can be done with a simple Test Automation Acceptance Checklist.Repeatable Test Cases with Static Data
The true cost benefit of test automation is achieved only when same scripts are executed multiple times. The first execution is very expensive because it includes one-time cost of automation tools and 100% of Test Automation engineer’s time. When scripts are executed again, cost of test automation declines sharply. The tool has already been purchased and scripts have already been coded. If there have been changes in application, scripts may require maintenance before being executed. Maintenance on minor software updates should be minimal.
Because test automation is only successful when scripts can be executed multiple times, only application which require same test cases to be executed with same data are good candidates for automation. For example, a mortgage application that needs to be regression tested on a weekly basis could be a good candidate for test automation. Script maintenance is minimal and scripts can enter a mortgage application using same group of test data in a fraction of time it would take a manual tester to test same functionality.
On other hand, a mortgage origination system, which cannot use same test data for each iteration would not be a good automation candidate. Due to nature of mortgage systems, data could be staged in various states of approval or rejection, based on current data and departments who have already processed their part of mortgage application. If script cannot easily figure out what data to enter in software, it is not a good automation candidate.
Another problem with automating this type of complex system is that test environment often contains a sampling of production data that is refreshed on a periodic basis. Sometimes this can be overcome by rebuilding test data when test environment is refreshed. The feasibility of rebuilding test data on a regular basis depends on complexity of application. You will have to make that decision on a case-by-case basis.
Application or Environmental Stability
Environmental stability is crucial to a successfully automating a software testing project. Scripts cannot be coded in a timely manner if application environment is unavailable, experiences frequent down-times, or excessive defects and errors.
Little or No Application or Environment Downtime
It takes longer to write scripts than it does to manually test same functionality. Most automation tools are watered down version of C or Visual Basic, which means that writing automated scripts is essentially programming and takes adequate time and specialized skills. Unlike manual test cases, which can sometimes be written based off requirements and mock-ups, automated tools require actual application. When a test environment is unavailable, automation engineers cannot create scripts, which prolongs project and ends up costing more.