When the development team refactors an API, it's purely an implementation matter, and therefore of no significance to testing, right?
Wrong, even triply wrong: testing expertise is crucial to a successful roll out, however sincerely the programmers believe its construction is "purely internal".
Check off these six hot spots to find problems before your customers do.
Make the most of testing ability
More precisely, refactoring without testers is a bad choice. A refactored API release is a reasonable goal: there's nothing wrong with a scheduled focus on paying down technical debt. A naive observer might conclude an unchanged contract means testers take a vacation: unchanged artifacts apply to an unchanged API.
That's a failure to think like a tester, though. The testing perspective recognizes every software encounter is an opportunity to uncover errors. Invariance of the API just gives more room to hunt for implicit elements, as opposed to the explicitly-documented interface. Here, then, are six ideas or attitudes about tests that apply to a refactoring project:
1. Don't skimp
Plan for a full roll-out test. That means, whatever your organization does for new versions with new features, schedule the testing crew for the same effort for the refactored release. The test plan and product documentation deserve thorough review and harmonization, even if it looks as though they haven't changed.
Unchanged documentation can acquire different meaning through time, as the larger software culture changes. A description of permissible values for one specific datatype for a 2010 customer base of twenty scientific organizations on internal networks might take on a whole new meaning when applied to two thousand commercial clients operating in hostile 2016 environments.
Testing pays off. Give your testers a full chance to spot problems, even though it's "only" a refactoring release. You'll be glad you did.
2. Design testability
One of the great contributions of testing experts is to help build in testability. Conventional organizations build products, then document what has been built, then test the correspondence of the documentation and product. Wise organizations, in contrast, design products, and build to those plans.
Testability can be part of the plan. The documented API published for customers might be partial: in many cases, a good product has a larger internal API which facilitates testing. Bring testers in at the beginning of a refactoring API to receive their help designing in testability. Keep testers in the conversation with product managers throughout a project to help make the most of the unique abilities testing brings.
3. Take combinatorics seriously
Imagine a refactoring release: the API is unchanged. All unit tests pass. Integration tests pass. Load tests pass. The previous release was well-received and robust. This one will be a success, right?
Not necessarily. I've lived through examples that fit this description, yet resulted in customers perceiving the release as a "failure" and "broken". A change in implementation that improves average performance, for instance, can deteriorate worst-case performance. Another true-life possibility: while unit tests pass, heavy load pushes an implementation to a point that a race condition violates the API contract.
Testing can never exhaustively exclude all errors, of course. Full engagement in a refactoring release, though, gives testers a chance not only to freshen individual tests, but also to design combinations of tests that effectively diagnose the release's health.
An important part of the testing mentality is the search for a "Goldilocks" result: not too hot, not too cold. It's wasteful to run lots of tests unlikely to turn up any problems, certainly; at the same time, too many errors slip through to customers that clever combination tests would have caught. Part of the value of a testing team is that they specialize in locating the happy ranges between those costly extremes.
4. APIs are special — maybe
Many product managers believe that APIs are special in regard to version compatibility. In this view, while consumer products can change interface — move buttons around, adjust color schemes, and so on — APIs have a particular obligation to backward compatibility. An extreme form of this principle puts a value on preservation of un-documented API features.
The consequence for testing? Before a refactoring project begins, the organization needs to clarify exactly which API it is preserving.
5. Business goals and technical goals
Earlier I wrote about the importance of testing even when technical specifications are unchanged. A variation of this theme occurs when development perceives technical specifications as constant, at the same time as larger business goals are shifting. An organization might change the branding or licensing or market position of an API with a fixed technical definition.
This situation again calls for plenty of testing help. Testers adopt a wider customer-centered perspective than developers often have; testers can take responsibility for ensuring that a refactored API fulfills updated business goals.
6. Sharpen your blades ahead of time
These tips have mostly emphasized that a refactoring release is like other releases, and needs all the usual involvement of testers. In a few aspects, a refactoring release is special. Testers need to be particularly well-prepared, for instance, to make sure that test plans, test data, on-call system engineers, test assignments, and other assets are all ready before the testing period begins.
As tempting as it can be to generate these items just-in-time, that is, during the testing interval itself, to do so is likely a mistake. The rhythm of a refactoring release might differ from more conventional releases. With adequate preparation ahead of time, testing can make sure that nothing blocks progress during testing execution.
Treat a refactoring release as a chance to "cut corners" or even eliminate testing, and unpleasant surprises are almost certain to erupt. Instead, let your starting point be that a refactoring release deserves just as much testing as any other release, then refine the release-specific test plan with attention to the half-dozen points above.
The right attitude and effort will pay off with a code base less in technical debt, a smooth release, and deeper understanding of how and ability to ensure testing supports delivery of high-quality products.
WSDL Refactoring with SoapUI NG Pro
The WSDL refactoring in SoapUI NG Pro easily updates all of your tests by providing a simple wizard that allows you to map old message elements to new ones instead of manually looking at, and replacing, project code. This feature also allows you to manually override or discard changes created by SoapUI NG during the process, ensuring that the correct result comes out in the end.
By using SoapUI NG Pro Refactoring you'll not only increase your test quality, but decrease the time invested while also minimizing the risks.
Learn more about this and other benefits of using SoapUI NG Pro.