You can buy the best golf clubs money can buy but it won’t magically lower your handicap unless you know how to use them. Software testing is no different; it won’t give you good results unless you know how, who, what and when to use it.
There are many ways you can approach software testing and we have recently written a best practice guide to QA Testing. However it’s easy to get confused by the sheer number of testing types and how they overlap, let alone what each of them does.
That’s why we’ve built for you The Ultimate Guide to Software Testing.
This is the how of software testing; how do you implement your testing strategy? We’ve split this section into two categories; Manual Testing and Automated Testing.
Manual testing is defined as testers manually executing test cases without the use of any automation tools. They play the role of the end-user and try to find as many bugs in the application as quickly as possible. The bugs are collated into a bug report, which is passed on to the developers to review and fix them.
An application cannot be tested using automation exclusively so manual testing plays a vital role in software testing. It requires a certain mindset; patience, creativity & open-minded amongst them.
We have put the below types of testing in the Manual Testing section:
- Exploratory Testing
- Manual Regression Testing
- Test Case Execution
What is it?
Exploratory testing relies on allowing the tester to have the freedom to interact with an application and react as they see fit. Good testers adapt and figure out what is needed rather than follow predefined test procedures. Nevertheless, some thought leaders in the software testing industry interpret exploratory testing as test design and test execution at the same time.
In order to maximise the results of exploratory testing, specific parameters must be given to the testers, e.g. what parts of an application to test, how long to test for, etc. Good exploratory testing is a planned activity, but not scripted.
|No need for long preparation||Difficult to get right|
|Fluid approach||Unstructured nature can lead to inefficiency|
|Discovery of more unique bugs/functionality problems||Requires a certain mindset|
|Helps in complex testing situations|
A major benefit is the preparation doesn't have to be exhaustive, although it is still needed. When executed right, exploratory testing is fluid without documentation or test cases. It makes it very effective in finding unique bugs and verifying functionality.
Exploratory testing is useful in complex testing situations when little is known about an application or more information is needed to write scripted tests.
A lack of planning before executing exploratory tests will lead to inefficient and unproductive results. Conversely, exploratory testing should not be scripted. It means getting the balance between the two is difficult.
Exploratory testing also relies heavily on the skill and mindset of the testers. A good exploratory tester requires many skills; lateral thinking, critical thinking, investigation skills, storytelling skills, good communication and technical skills.
Manual Regression Testing
What is it?
Manual regression testing is a method of verification, which is performed manually. It is used to confirm that a recent update, bug fix or code change has not adversely affected existing features. It utilises all or some of the already executed test cases which are re-executed to ensure existing functionality works correctly and no new bugs have been introduced.
Improves product quality
Time-consuming and inefficient
Detects any side effects of updates/bug fixes/code changes
May cause their own side effects
Regression testing is a necessity in all release cycles. When done correctly it can improve and maintain quality. Ideally, it should be performed after every single code commit. This ensures the need to only go back one commit to fix a problem, but this is not always practical.
When there are constant changes being implemented to an application, manual regression tests are very inefficient.
Test Case Execution
What is it?
Test cases help guide the tester through a sequence of steps to validate whether the application is working as intended. A good test case requires good writing skills, attention to detail and a good understanding of the application. Test case execution is the process of executing the code and comparing the expected and actual results. Test cases are assigned to testers to execute the tests, create the bug report and report the status of each one.
|Step-by-step process||Badly written test cases waste time|
|A good test case is reusable|
Testers like the step-by-step process of test cases, although it can be very repetitive. A good test case is reusable. This should be a consideration when writing them to save time in the long-term. Test cases provide comprehensive documentation for the area they are testing.
If test cases are badly written or are not clear, it will cause confusion or mistakes which will mean inaccurate results or the need to re-test.
Automated testing is a process in which an automation tool is used to execute pre-scripted test cases. The objective of automated testing is to simplify and increase efficiency in the testing process.
If a particular form of testing consumes a large percentage of QA, it could be a good candidate for automation. Repetitive tasks such as testing login processes or registration forms are good examples of when to use automated testing.
Using automated testing is undoubtedly quicker than manual testing. In terms of testing execution, it will increase productivity and reduce testing time for the majority of apps/websites. Even though set up costs are high, automated testing can save money in the long-term.
Repetitive tasks are inefficient when done manually, especially when they reoccur. There is also an increased chance of human error. Automated testing can eradicate this, depending on the quality and scope of the test cases.
However, there are certain test situations where automated testing won’t work such as user interface, documentation, installation, compatibility, and recovery. Even if you choose to automate, some form of manual testing will be needed.
Initial set up costs (automation tool purchase, training, maintenance of test scripts) are expensive. Also, if your app or website changes regularly, the cost and time associated with script maintenance will increase considerably.
We have put the below types of testing in the Automated Testing section:
- Unit Testing
- API Testing
- Automated Regression Testing
What is it?
Unit testing is testing individual units or components of an application. The aim is to ensure that each unit performs as designed. It is typically carried out by developers, not testers, as it requires a detailed knowledge of the internal program design and code.
|Finds bugs early||Hard to write|
|Facilitates change||Difficult to automate|
|Simplifies integration||Comprehensive version control needed|
When a failure occurs in a unit test, it’s either caused by a bug in the code or a problem with the actual unit test itself. Either way, it’s easy to pinpoint the problem and early enough in the development cycle to fix it. Unit testing ensures that the code functions properly as the code base grows. This streamlines the code to make it more readable and less complex. By verifying each unit, integration into an application is simpler.
Unit testing also provides guaranteed documentation for an application. This is helpful for other developers that need to find out what functionality is provided by a particular unit.
Good unit tests are complex to write. It can mean the test code is likely to be at least as buggy as the code it is testing. This scenario is the same for both manual and automated unit testing. It’s nearly impossible to evaluate every single execution path in all but the most basic applications. A comprehensive version control system is essential to record the changes in case anyone needs to refer back to previous versions.
What is it?
Application programming interfaces (API) testing means checking APIs directly. An API is a feature that allows one application to interact and communicate to other applications. It determines if the APIs meet expectations for functionality, reliability, performance, and security. This does not cover UI or UX testing. API testing involves sending calls to an API, receiving an output and recording a response.
|Verification of API functionality||Set up can be complex|
|Checks API integration with other applications||Coding knowledge required|
|Tests authentication credentials|
If an API is not tested properly, it may cause problems to not only the primary application but the other applications it integrates with. API testing provides a vital check to ensure this functionality works correctly.
Setting up a testing environment for API testing can be complex. Also, a good level of coding knowledge is necessary for API test cases.
Automated Regression Testing
By nature, regression testing requires constant repetition. It can be performed manually or using an automated method. The definition is the same as manual regression testing; it’s a method of verification but it is automated rather than performed manually.
|Improves product quality||Setting up automated regression tests is expensive|
|Detects any side effects of updates/bug fixes/code changes||High maintenance effort|
|May cause their own side effects|
Read more to learn who you use to test with. Whether it be In-house, Crowdsourced and/or Outsourced Testing, deciding who will execute your testing is a crucial part of your strategy.