6.1 Introduction to Automation Engineering
In prior units you were exposed to the traditional way of doing software testing. This involves generating test cases, mapping them to requirements and manually executing tests against a test system in a testing environment. You will have noticed that this is a time-consuming process that is difficult to do well. Many times these tasks are repetitive and cumbersome. As the need for more robust software testing grows, companies are discovering that making an investment in test automation pays real dividends. Today, manual software testing is giving way to more automated approaches, and companies are realizing the increase in software reliability and reduction in time to market as a result.
At its foundation, automation testing still has its roots in the testing pyramid as well as the manual testing techniques we discussed earlier. The astute tester still has to decide what needs to be tested in order to fulfill the acceptance criteria. The only key difference is that rather than doing these tasks by hand, we will set up scripts and programs to do them for us in an automated way.
The Benefits of Automation
There are several benefits we gain from automation. Here are several examples.
Efficiency Gains
Test automation significantly accelerates the testing process by executing test cases at a faster pace than manual testing. Instead of manually testing the same set of scenarios repeatedly, automation allows running these tests in a fraction of the time, promoting efficiency gains.
Repeatability and Consistency
Automated tests ensure that the same set of test cases is executed consistently, eliminating variations in test results. When testing a login functionality, automated scripts perform the same steps and verifications every time, ensuring a consistent testing approach.
Broad Test Coverage
Test automation allows for the execution of a large number of test cases across various scenarios, enhancing overall test coverage. Automated scripts can cover multiple user journeys, edge cases, and data variations, ensuring a comprehensive assessment of the software's functionality.
Reduced Human Error
Automation minimizes the risk of human-induced errors that may occur during repetitive or tedious manual testing. Automated tests follow pre-defined scripts, reducing the chances of overlooking test steps or making mistakes during the testing process.
Enhanced Regression Testing
Automation excels in executing regression tests to ensure that new code changes do not introduce defects into existing functionalities. After adding a new feature, automated regression tests quickly validate that previously working features remain unaffected by the recent changes.
Adaptability to Continuous Integration (CI) and Continuous Deployment (CD)
Automated tests seamlessly integrate into CI/CD pipelines, allowing for continuous testing and rapid feedback during the software development process. With each code commit, automated tests run automatically, providing immediate feedback to developers and preventing the integration of faulty code.
Challenges of Automation
Although the benefits of automation are great, there are significant drawbacks that need to be considered as well.
Initial Setup Time
Setting up the test automation environment, tools, and frameworks can be time-consuming, especially at the beginning of a project. Configuring automation tools, installing dependencies, and ensuring proper integration may require additional time and effort.
Maintenance Efforts
Automated test scripts need regular maintenance to adapt to changes in the application, making sure they remain effective over time. When a user interface element changes, automated scripts must be updated to reflect the new structure, requiring ongoing maintenance.
Selecting Suitable Test Cases
Identifying which test cases are suitable for automation can be challenging, as not all scenarios may yield significant benefits. Complex test cases with frequent changes may not be the best candidates for automation, as maintaining these scripts could outweigh the advantages.
Skillset Requirements
Test automation requires a certain skill set, and team members need to be trained to effectively use automation tools and frameworks. Team members may need to learn scripting languages or understand the intricacies of automation tools before effectively contributing to automated testing efforts.
Integration Challenges
Integrating automated testing into the entire development process, especially in legacy systems, can be challenging. Existing systems may not have clear interfaces for automation tools, requiring additional efforts to establish seamless integration.
False Sense of Security
Relying solely on automated tests may lead to a false sense of security, as they may not catch all types of defects. Automated tests may miss certain edge cases or scenarios that are crucial for thorough testing, potentially leading to unanticipated issues in production.
Deciding What to Automate
In order to be effective, choosing what and how much to automate is a non-trivial task. Let’s return again to our test pyramid.
- Unit tests ensure that individual components work correctly in isolation. These will be the easiest to automate and are often created at the same time as the code.
- Integration tests validate the interaction between these components. These are also relatively easy to create, but may be more complex than unit tests.
- End-to-end tests ensure the entire application functions as expected. This is the most difficult to automate, and the most likely to change frequently over time. These tests will often be the least desirable to automate.
- Prioritize creating the faster-running tests at the base of the pyramid (unit tests). They provide quick feedback to developers during the development cycle.
- Unit tests are also typically easier to maintain than end-to-end tests. By having a larger number of unit tests, maintenance costs are kept lower.
- Unit and integration tests are often more suitable for parallelization, enabling faster test execution in a continuous integration (CI) environment.
A very good treatment of this topic can be found here with practical examples in Java. https://martinfowler.com/articles/practical-test-pyramid.html
Useful Links: ←Unit 5.7 | Unit 6.2→ | Table of Contents | Canvas