Now that I have provided you with an overview of the Top 5 Common Challenges for Agile Testing Teams, I wanted to start diving into each of these challenges more thoroughly and offer some solutions to overcome them and improve your Agile testing results.
Agile development is a faster, more efficient and cost-effective method of delivering high-quality software. However, Agile presents testing challenges beyond those of waterfall development. That’s because Agile requirements are more lightweight, and Agile builds happen more frequently to sustain rapid sprints. Agile testing requires a flexible and streamlined approach that complements the speed of Agile.
The Challenge - Inadequate Test Coverage
Inadequate test coverage can cause big problems. It’s often the result of too few tests written for each user story and lacking visibility into code that was changed unexpectedly. Before you can have adequate test coverage you first must have a clear understanding of the features being delivered in the release. For each feature, you must understand how the feature is supposed to work, its constraints and validations, and its ancillary functions (such as logging and auditing). Agile developers build features based on a series of user stories (sometimes grouped by themes and epics). Creating your test scenarios at the user story level gives you the best chance of achieving optimal test coverage.
Once your QA and development teams agree on the features to be delivered, you can begin creating tests for each feature. Coding and test development should be done in parallel to ensure the team is ready to test each feature as soon as it’s published to QA.
Ensuring Comprehensive Test Results
Be sure to design a sufficient number of tests to ensure comprehensive results:
1. Positive Tests: Ensure that the feature is working as designed, with full functionality, is cosmetically correct, and has user-friendly error messages.
2. Negative Tests: Users often (okay, usually) start using software without first reading the manual. As a result, they may make mistakes or try things you never intended. For example, they may enter invalid dates, key-in characters such as dollar signs or commas into numeric fields, or enter too many characters (e.g., enter 100 characters onto a field designed for no more than 50). Users also may attempt to save records without completing all mandatory fields, delete records that have established relationships with other records (such as master/detail scenarios), or enter duplicate records. When you design tests, it’s important to understand the constraints and validations for each requirement and create enough negative tests to ensure that each constraint and validation is fully vetted. The goal is to make the code "dummy proof."
3. Performance Tests: As I will share later in this blog series, it’s a good idea to test new features under duress. There are ways to automate this but you should conduct some manual tests that perform timings with large datasets to ensure that performance doesn’t suffer too much when entering a large amount of data or when there are many concurrent users.
4. Ancillary Tests: Most well-designed systems write errors to log files, record changes to records via audits, and use referential integrity so whenever a master/child record is deleted, both records are deleted simultaneously. Imagine if your systems regularly run purge routines, you will need tests to ensure the purge is working - this is a good example of an ancillary test. Finally, most systems have security features that limit access to applications so specific people only have rights to specific functions. Ancillary tests ensure that log files and audits are written, referential integrity is preserved, security is embraced, and purge routines cleanly remove all related data as needed.
Securing Traceability
Applying a sufficient number of tests to each feature to fully cover all the scenarios above is called traceability. Securing traceability is as simple as making a list of features including the number and breadth of tests that cover positive, negative, performance, and ancillary test scenarios. Listing these by feature ensures that no feature is insufficiently tested or not tested at all.
Tracking Test Coverage in the Real World
The easiest way to track test coverage is by using a tool like QAComplete. The following resources will help you get on your way to improving and tracking test coverage in your own development shop.
How to track test coverage through traceability
How to try QAComplete for free
In my next blog post, I'll address the challenge of accidentally broken code due to frequent builds.
If you want to download the entire free white paper, get it here: Five Challenges for Agile Testing Testing Teams: Solutions to Improve Agile Testing Results.
Be sure to check out my next blog in the series: Agile Testing Challenges - Broken Builds.
See also:
[dfads params='groups=937&limit=1&orderby=random']