Test Management in Jira: Strategies for Success

  April 17, 2019

As your organization is looking to further compress software delivery cycles, it’s important to explore the benefits of sophisticated test management solutions that support Jira workflows and keep pace with the fast-evolving testing landscape.

SmartBear recently hosted a webinar on the strategies for success in test management using Atlassian’s Jira software. Here are the key takeaways from this webinar.

The Importance of Test Management

The increasing pressure to reduce the time to market without sacrificing quality, is one of the most significant pain points that leads companies to seek a test management tool.

A test management solution helps software teams by addressing all the necessary functions needed to achieve faster and efficient testing, including test planning, creation, execution, and reporting. It makes it easier for organizations to manage and gain insights on a wide variety of tests, ranging from automation, manual, API, to unit tests.

A good test management tool provides two-way traceability reports that helps teams assess the appropriate test coverage and allows them to track relevant quality metrics, so they can closely check on the release progress and identify bottlenecks early on. This is especially important when organizations have multiple, globally distributed testing teams, as it gives the managers and executive team visibility across the entire testing process and ensure that teams are well aligned on the release goals.

By allowing everyone from testers, developers, and managers to access all the testing assets in a single centralized, shared space—which also provides other business stakeholders like project and product managers visibility into the release cycle--test management tools help foster a collaborative culture.  

A real-time poll of the webinar audience found that the 61% were using Jira for a test management tool, 26% spreadsheets, and 10% proprietary tools. This was almost the same exact percentages found in the SmartBear State of Software Testing, 2018 Industry Report, which was based on more than 2,600 responses from a myriad of job roles across 16 major industries globally.

Spreadsheet Limitations

When teams count on spreadsheets (only) for test management, they can face many limitations, some of which we outlined below:

  • Hard to maintain the increasing test case data with every release
  • Difficult to assess appropriate test coverage as opposed to using agile tools like Jira
  • Challenging for Managers to efficiently track and gauge tester accountability
  • Reporting and visualization of test data is limited, which impact the team’s visibility
  • Lack of integration with test automation frameworks restricts automation efforts

Legacy Testing Tool Limitations

Similarly, legacy tools have their own set of limitations:

  • Higher overall cost associated with onsite licensing versus the all-inclusive global licensing that is available with a test management tool like Zephyr
  • The siloed architecture of legacy tools can restrict the use of more modern tool sets such as automation and CI/CD tools 
  • The browser and operating system support are often limited
  • The learning curve can be steep, which means added time and expense
  • Since big companies are the ones that typically support these legacy solutions, it's hard to get information on the product roadmap or the future plans of these legacy offerings

Why Jira is So Popular

Jira is becoming increasing popular with software teams because it facilitates a very agile way of working. You can use it as a place to capture requirements, user stories, epics (collections of stories), new improvements, and other issue types. Jira lets you set up boards (message spaces that display issues from one or more projects), giving you a flexible way of viewing, managing, and reporting on work in progress.

It also makes it easy to collaborate with other users by using Continuous Integration (CI) tools to bring data in, and then leverage other tools in the Atlassian stack such as Confluence to share data onto a wiki page.

Similarly, there are many benefits to using Jira for test management. You can build workflows within Jira configured to match your company’s business processes. Sharing Jira data internally within your organization will increase collaboration because you can pull in people from traditionally separated teams.

This means your business users will start talking to testers and developers about things like defects that testers, developers, or even users have found. Jira features like notifications, comments, and assignments will help get everyone engaged in your QA process.

There are some limitations to just using Jira by itself for test management, such as not being able to share test cases or work with external frameworks such as CI frameworks or automation frameworks. Users can take advantage of many of the strengths of Jira like its reports, dashboards, and charts, but you can’t access the data you need to create a traceability matrix. Therefore, it is important that you explore the right test management tool which can work seamlessly with Jira and empower your teams with critical test management capabilities.

Choosing a Test Management Solution that Works with Jira

There are two types of integrations you can choose to add testing functionality to Jira:

  1. Add-ons like Zephyr for Jira, which are internal integrations that live inside the Jira application and add testing functionality.  Test issues can be created, executed, tracked, and reported on just like any other Jira issue in Zephyr for Jira, which has the same look and feel as Jira.
  2. An external integration with a dedicated test case management tool like Zephyr Standalone, a test management solution managed independently outside the Atlassian ecosystem that offers real-time Jira integration.

Both of these test management approaches offer real-time, bi-directional integration with Jira, which means your requirements, user stories, epics, sprints, new features, improvements, tasks and sub-tasks will be connected to your test cases, test case assignments, and execution information, as well as your defects.

Since Jira is configurable, the test management solution you choose needs to be customizable as well, so that you are able to create a workflow for test execution, test design, or test planning that is in alignment with your unique business processes.

While evaluating test management solutions, teams should also consider technical support offered by vendors, which includes supplying training, offering best practice sessions, and doing health checks to confirm if the tool meets your needs.   

Pricing is another key consideration for picking the right solution. There are many different test management solutions available in the market and many different ways of doing pricing, such as site-oriented licenses, global licenses, concurrent licenses, or, if you're working with a plug-in inside of Jira, there are tier-oriented licenses.

Enhance Test Management with Zephyr for Jira

Zephyr for Jira offers a robust set of testing tools inside the Jira environment without users having to navigate elsewhere. They can get up-and-running with test management straight away because Zephyr for Jira provides an out-of-the-box Jira issue of type test.  This Jira test issue can be used to capture all of your custom fields from the design stage to the execution stage.

As a plugin inside the Jira environment, Zephyr for Jira can be deployed in both the Jira cloud and Jira server.  This means that when you create custom reports of, say, traceability reports, you can pick and choose your data points in Jira with JQL queries and filters, and you’ll be able to export this data out where it’s needed with comments or assignments. Since your business analysts are most likely using Jira, as are your developers, testing, and operations teams, this will help bring your teams closer, so they collaborate more.

As far as automation solutions go, Zephyr for Jira has a series of APIs called ZAPI (or Zephyr APIs) that can be used to view and upload data programmatically. You can use these APIs to seamlessly integrate with CI tools like Jenkins or Bamboo or even use the APIs by themselves to pull data out or push data in.

 A look at Key Zephyr for Jira Features

After logging into Jira, you should see a top-level navigation bar with the following "Dashboard", "Projects", "Issues", and “Tests" tabs if you have the Zephyr for JIRA add-on installed:

ASH1.png

It’s possible to use the generic Create button in Jira to create a test in the same way you can create any other issue type such as a Story, Epic, or Task. You can also leverage the issue navigator by selecting the issue type test to browse through all of the test cases you have.

There are several features associated with editing and executing a test including how a test case can be easily and quickly cloned, linked to any other Jira issue (Task, Improvement, Story, Epic etc.), or added to a specific test cycle of a version for test execution.

ASH2.png

Once the test cases are executed, you can add comments, which can include execution information, actual test results, or any notes to the developer or management that you need to raise. You can also add supporting evidence by leveraging Zephyr’s image annotation tool Capture for Jira to take screenshots and video recordings and append them to your test case executions.

Advanced Reporting: Zephyr for Jira provides many ways to track and share the progress your tests. The test summary is a high-level view of your test cases, including how many have been executed, how many haven’t been executed, and how many haven't yet been assigned to testers.

ASH3.pngSince Zephyr for Jira uses a standard Jira issue-type, there are a wide variety of out-of-the-box Jira gadgets that can be used to track testing in the Jira dashboards area. 

Enhanced Traceability: Zephyr for Jira has very sophisticated traceability reports that show the links between the requirements, tests, test executions, and any issues that have been created to the record defects identified during testing.

These reports can ensure that all the requirements have been covered by test cases and that none have been overlooked. They also provide a clear Go / No Go indication of whether a release is meeting the requirements successfully and the software is ready to be shipped.

Traceability reports are useful during development as they highlight which specific defects are impacting the most requirements and, alternatively, which requirements have the most defects. This information can help teams prioritize bug-fixes or, in some cases, drive a scope change to drop problematic requirements for a particular release. 

ASH4.pngWatch the webinar here, "Test Management in Jira: Strategies for Success," to see how Agile teams can employ winning strategies to maximize the benefits of test management in Jira!