Not every software testing project can or should be automated. Before your department accepts a new test automation project, you should establish a process by which projects are reviewed and either accepted or rejected. This can be done with a simple Test Automation Acceptance Checklist.Repeatable Test Cases with Static Data
The true cost benefit of test automation is achieved only when
same scripts are executed multiple times. The first execution is very expensive because it includes
one-time cost of
automation tools and 100% of
Test Automation engineer’s time. When
scripts are executed again,
cost of test automation declines sharply. The tool has already been purchased and
scripts have already been coded. If there have been changes in
application,
scripts may require maintenance before being executed. Maintenance on minor software updates should be minimal.
Because test automation is only successful when
scripts can be executed multiple times, only application which require
same test cases to be executed with
same data are good candidates for automation. For example, a mortgage application that needs to be regression tested on a weekly basis could be a good candidate for test automation. Script maintenance is minimal and
scripts can enter a mortgage application using
same group of test data in a fraction of
time it would take a manual tester to test
same functionality.
On
other hand, a mortgage origination system, which cannot use
same test data for each iteration would not be a good automation candidate. Due to
nature of mortgage systems, data could be staged in various states of approval or rejection, based on
current data and
departments who have already processed their part of
mortgage application. If
script cannot easily figure out what data to enter in
software, it is not a good automation candidate.
Another problem with automating this type of complex system is that
test environment often contains a sampling of production data that is refreshed on a periodic basis. Sometimes this can be overcome by rebuilding
test data when
test environment is refreshed. The feasibility of rebuilding test data on a regular basis depends on
complexity of
application. You will have to make that decision on a case-by-case basis.
Application or Environmental Stability
Environmental stability is crucial to a successfully automating a software testing project. Scripts cannot be coded in a timely manner if
application environment is unavailable, experiences frequent down-times, or excessive defects and errors.
Little or No Application or Environment Downtime
It takes longer to write scripts than it does to manually test
same functionality. Most automation tools are watered down version of C or Visual Basic, which means that writing automated scripts is essentially programming and takes adequate time and specialized skills. Unlike manual test cases, which can sometimes be written based off requirements and mock-ups, automated tools require
actual application. When a test environment is unavailable, automation engineers cannot create scripts, which prolongs
project and ends up costing more.