<< Blog

When to do Manual Testing and when to do Automated Testing

December 20, 2011

The answer to the above question can be sought only after careful consideration of several factors that affect the testing. With automated testing tools on a rise in the market, several advanced test automation features are being introduced to make automated testing easy, affordable and offer more coverage.

As testers have begun to rely on the automated testing tool(s) to help them deliver defect free product(s), we have put across a series of questions to help you choose the best type of testing according to your needs and resources

How often do you plan to run a test suite?

If you need to run the test cases a lot number of times in a test cycle, automation testing is what you should be looking at. It can provide you a great return on your investment. Repeatability is not a time consuming activity when you consider automation.

How many test cases are you looking at testing?

Another factor to consider is the number of test cases that you have under the specific test suite. It is generally not a good investment to automate the testing of merely ten test cases, no matter how often they are going to be tested. However, if the number of test cases is high, and if these will be run more than once then automation testing can provide you with better results. The number of test cases should justify the cost of the automation tools.

Is it a high priority feature?

Well, some features have higher chances of failing than the others. Such high priority feature is better to be tested with an automation testing tool since with manual testing; there are chances that the test case or the feature might be missed during the testing cycle. The tests will be run every time in the same fashion with automated testing, thus eliminating any chances of human-induced errors.

Do you need to run the test cases in a predetermined order?

Some features have to be tested before the others. There are instances when a pre determined order has to be followed while testing the test cases. Achieving this can be cumbersome and time consuming in manual testing since the testers are either required to remember the order, or refer to an external source to know the order. But with automated testing, the scripts can be timed and ordered to execute as per our needs.

Does the test case need to be updated constantly?

If the test case will be frequently updated, then automating it may have to be given a second thought because changing the automated script of the test case is time consuming and defeats the purpose of automating it.

However with keyword driven or data driven framework, maintaining and changing of scripts is easier. So while automating such functionalities, choose a framework that eases maintainability. Also, most automated testing framework support reusability ensuring that there is no need to constantly rewrite test cases to test the same functionalities.

Are you planning on simultaneous running of test cases?

When your testing demands that you run the same set of test cases simultaneously on more than one machine, then you need to use automation testing. With manual testing, you cannot type the same test cases to run exactly at the same time on several machines. However, with automation testing, you can schedule the scripts so that the test cases are executed exactly at the same time on more than one system.

Do you need to test single functionality with multiple data sets?

Is there a need to run the same test case with multiple data? Then data-driven automation testing framework must be used to minimize effort and time spent in each test case. The data is fed from an external source (e.g. Excel) to each test case, thus ensuring that larger data coverage for a single test case is possible. When you opt for manual testing in this scenario, not only will your test team grow bored of testing the same functionality, but chances of missing one or two data sets is possible.

Is it a Regression/Smoke Test Suite?

Regression test suite consists of a selected number of test cases, which need to be tested after every defect fix cycle. Regression testing aims to ensure that no new defects have been introduced in the functionalities that were previously working. Sanity testing is run at the start of every test cycle and aims at ensuring that the basic functionalities of an application work as expected.

If it is a regression/sanity suite that you are looking to test, then automation testing is your safe bet. Reasons are simple:

• This regression/sanity test cases never/rarely change.
• The regression/sanity test suite consists of a fair number of test cases.
• These regression/sanity test cases have to be run repeatedly (after each round of new functionality testing).

Generation of Reports

With leading automation testing tools, the reports of testing activity are automatically generated once the testing cycle completes. This eases the job of identifying defects and empowers the test team to make a decision with respect to reliability of the product. Since the test log is readily available go-live decision can be taken quickly and efficiently.

Automation Testing Feasibility

Most automation testing tools are built in a way that enables the tester to test any kind of functionality right from the presence of a button to the absence of an image. However, some steps cannot be automated no matter which automation framework you choose. And in some cases, the automation framework that you have chosen to implement simply does not support the kind of testing you are expecting it to perform.

So, it is imperative that you do a feasibility study of the automation framework in conjunction with the test cases you are looking to automate.

Random Negative Testing

Also, with manual testing, random negative testing is a possibility but with automated testing it is not possible. So when it is merely testing of an interface, its user friendliness and navigation among web pages, manual testing can be chosen.

Test Team Expertise

Even if answers to all the above questions point at automation testing, the fact is you need the resources to implement it successfully. With manual testing, limited technical knowledge suffices, but when automation testing is considered, expert test team in the chosen framework and language is definitely needed. If you have a test team that is capable of handling the challenges of automating the test suite, then automation is a fair idea.

The simple fact that automated testing is quicker, more flexible and offers wider coverage with less effort definitely gives it a upper hand as compared to the straight forward manual testing. 

Despite the fact that the answers to above question may help decide on the kind of testing that must be chosen, one needs to take a decision based on experience, common sense and available expertise as well.

Check out how Automation Testing can help you in minimizing your manual testing efforts, Download the FREE trial version of TestingWhiz now!