Test Management in Jira: Strategies for Success
As your organization is looking to further compress software delivery cycles, it’s important to explore the benefits of sophisticated test management solutions that support Jira workflows and keep pace with the fast-evolving testing landscape.
Dr. Milan Verma and Ashwini Sathe recently gave a presentation on the strategies for success in test management using Atlassian’s Jira product. The following are some of the key takeaways from the webinar.
The Importance of Test Management
The first reason companies need a test management tool is the pressure to deploy high quality software faster. A test management solution helps software teams address all the functions needed to do efficient testing, including planning, creation, execution, and reporting. It makes it easier for organizations to manage and gain insights from a wide variety of tests, including automation, manual, API, and unit tests.
A good test management tool also provides two-way traceability reports that will help a company assess the appropriate test coverage of its requirements. Traceability reports allow managers and QA Leads to track relevant quality metrics so that they can check on the release progress, identify bottlenecks in the process, and correct them early on.
This is especially important when you have multiple, globally distributed testing teams because it gives managers and executives visibility across the entire testing process to make sure your software teams are on track and aligned with the release goals. By allowing everyone from testers, developers, and managers to access all of your testing assets in a single centralized, shared space—which also provides other business stakeholders like project and product managers visibility into the release cycle--test management tools help foster a collaborative culture.
A real-time poll of the webinar audience found that 61% were using Jira, 26% spreadsheets, and 10% proprietary tools for a test management tool. These were almost the same exact percentages found in the SmartBear State of Software Testing, 2018 Industry Report, which was as based on more than 2,600 responses from a myriad of job roles across 16 major industries globally.
In the webinar, we discussed some of the challenges or limitations in using spreadsheets for test management:
- Spreadsheets are cumbersome to use to create and store test documents since, with every release, you have to maintain the increasing test case data and also ensure its quality
- Assessing test coverage can be a problem if you're using only spreadsheets for testing
- Spreadsheets make it hard for managers to gauge tester accountability
- Reporting and visualization of test data is limited, which impacts your ability to get the optimum visibility and traceability you need
- There is also lack of integration with test automation frameworks when teams use spreadsheets
Legacy Testing Tool Limitations
Similarly, legacy tools have their own set of limitations:
- Legacy solutions can put a strain on your wallet with a higher overall cost associated with onsite licensing versus the all-inclusive global licensing that’s available with a test management tool like Zephyr and others
- The siloed architecture of legacy tools can restrict the use of more modern tool sets such as automation and CI/CD tools
- The browser and operating system support are often limited
- The learning curve with legacy tools can be steep, which requires additional time and expense
- Since big companies are the ones that typically support these legacy solutions, it's hard to get information on the product roadmap or the future plans of these legacy offerings
In the second real-time poll of the webinar,i we discovered that most listeners were using Jira for defect tracking, followed closely by software development and test management.
Why Jira is So Popular
Jira is popular with software teams these days because it facilitates a very agile way of working since you can use it as a place to capture requirements, user stories, epics (collections of stories), new improvements, and other issue types. Jira also lets you set up boards (message spaces that display issues from one or more projects), giving you a flexible way of viewing, managing, and reporting on work in progress.
It also makes it easy to collaborate with other users by using Continuous Integration (CI) tools to bring data in, and then leverage other tools in the Atlassian stack such as Confluence to share data onto a wiki page.
Similarly, there are many benefits to using Jira for test management. You can build workflows within Jira configured to match your company’s business processes. Sharing Jira data internally within your organization will increase collaboration because you can pull in people from traditionally separated teams.
This means your business users will start talking to testers and developers about things like defects that testers, developers, or even users have found. Jira features like notifications, comments, and assignments will help get everyone engaged in your QA process.
There are some limitations to just using Jira by itself for test management, such as not being able to share test cases or work with external frameworks such as CI frameworks or automation frameworks. Users can take advantage of many of the strengths of Jira like its reports, dashboards, and charts, but you can’t access the data you need to create a traceability matrix.
Choosing a Test Management Solution that Works with Jira
There are two types of integrations you can choose to add testing functionality to Jira:
- Add-ons like Zephyr for Jira, which are internal integrations that live inside the Jira application and add testing functionality. Test issues can be created, executed, tracked, and reported on just like any other Jira issue in Zephyr for Jira, which has the same look and feel as Jira.
- An external integration with a dedicated test case management tool like Zephyr Standalone, a test management solution managed independently outside the Atlassian ecosystem that offers real-time Jira integration.
Both of these test management approaches offer real-time, bi-directional integration with Jira, which means your requirements, user stories, epics, sprints, new features, improvements, tasks and sub-tasks will be connected to your test cases, test case assignments, and execution information, as well as your defects.
Since Jira is configurable, the test management solution you choose needs to be customizable as well, so that you are able to create a workflow for test execution, test design, or test planning that is in alignment with your unique business processes.
While evaluating test management solutions, teams should also consider technical support offered by vendors, which includes supplying training, offering best practice sessions, and doing health checks to confirm if the tool meets your needs.
Pricing is another key consideration for picking the right solution. There are many different test management solutions available in the market and many different ways of doing pricing, such as site-oriented licenses, global licenses, concurrent licenses, or, if you're working with a plug-in inside of Jira, there are tier-oriented licenses.
Enhance Test Management with Zephyr for Jira
Zephyr for Jira offers a robust set of testing tools inside the Jira environment without users having to navigate elsewhere. They can get up-and-running with test management straight away because Zephyr for Jira provides an out-of-the-box Jira issue of type test. This Jira test issue can be used to capture all of your custom fields from the design stage to the execution stage.
As a plugin inside the Jira environment, Zephyr for Jira can be deployed in both the Jira cloud and Jira server. This means that when you create custom reports of, say, traceability reports, you can pick and choose your data points in Jira with JQL queries and filters, and you’ll be able to export this data out where it’s needed with comments or assignments. Since your business analysts are most likely using Jira, as are your developers, testing, and operations teams, this will help bring your teams closer, so they collaborate more.
As far as automation solutions go, Zephyr for Jira has a series of APIs called ZAPI (or Zephyr APIs) that can be used to view and upload data programmatically. You can use these APIs to seamlessly integrate with CI tools like Jenkins or Bamboo or even use the APIs by themselves to pull data out or push data in.
A look at Key Zephyr for Jira Features
After logging into Jira, you should see a top-level navigation bar with the following "Dashboard", "Projects", "Issues", and “Tests" tabs if you have the Zephyr for JIRA add-on installed:
It’s possible to use the generic Create button in Jira to create a test in the same way you can create any other issue type such as a Story, Epic, or Task. You can also leverage the issue navigator by selecting the issue type test to browse through all of the test cases you have.
There are several features associated with editing and executing a test including how a test case can be easily and quickly cloned, linked to any other Jira issue (Task, Improvement, Story, Epic etc.), or added to a specific test cycle of a version for test execution.
Once the test cases are executed, you can add comments, which can include execution information, actual test results, or any notes to the developer or management that you need to raise. You can also add supporting evidence by leveraging Zephyr’s image annotation tool Capture for Jira to take screenshots and video recordings and append them to your test case executions.
Advanced Reporting: Zephyr for Jira provides many ways to track and share the progress your tests. The test summary is a high-level view of your test cases, including how many have been executed, how many haven’t been executed, and how many haven't yet been assigned to testers.
Since Zephyr for Jira uses a standard Jira issue-type, there are a wide variety of out-of-the-box Jira gadgets that can be used to track testing in the Jira dashboards area.
Enhanced Traceability: Zephyr for Jira has very sophisticated traceability reports that show the links between the requirements, tests, test executions, and any issues that have been created to the record defects identified during testing.
These reports can ensure that all the requirements have been covered by test cases and that none have been overlooked. They also provide a clear Go / No Go indication of whether a release is meeting the requirements successfully and the software is ready to be shipped.
Traceability reports are useful during development as they highlight which specific defects are impacting the most requirements and, alternatively, which requirements have the most defects. This information can help teams prioritize bug-fixes or, in some cases, drive a scope change to drop problematic requirements for a particular release.
Watch the webinar here, "Test Management in Jira: Strategies for Success," to see how Agile teams can employ winning strategies to maximize the benefits of test management in Jira!