Can Your Tool Test Itself? SoapUI 5.0 Can!
I've heard that Blocket.com, Sweden's largest second-hand sales website, (similar to Craigslist in the U.S.) has 45 developers and two testers. That sounds surprisingly low, considering they release new versions daily - or maybe just because of that fact.
On the other side of the scale, I sometimes tell the joke that, "I heard that Microsoft used 15,000 testers when testing Windows 7. That's meager; SoapUI has 500,000 testers!"
But, of course, SoapUI testers are not working full-time with testing SoapUI, rather they're using SoapUI to test their own APIs.
Still, I am amazed by Microsoft having a staggering 15,000 testers for a single product. Having massive numbers of testers is one way to ensure high quality. But automation has its place in running tedious or repetitive tasks, allowing humans to do more difficult, creative jobs such as exploratory testing.
That is, if you have tools that can properly automate your tests.
SoapUI is one such tool, automating the testing of APIs. Of course, SoapUI wasn't originally designed for testing itself. Still, I have always thought about how cool it would be if it could test itself, similar to how a C++ compiler can be used re-compile itself.
And now, thanks to the new REST Mocking feature added in SoapUI 5.0, we can automate even more of our own SoapUI testing than we could do before, by using SoapUI itself to, in part, test its own core functionality.
We just need to create test suites that call SoapUI's built-in REST Mocks that simulate APIs, executing all the positive and negative tests that you can use while calling APIs. Because the mocks can simulate both negative and positive responses we can test for the correct behavior. (And this of course works for SOAP Mocks too.)
Previously, we would have had to use APIs deployed outside SoapUI, so we couldn't say SoapUI tested itself. And, in practice, the problems of keeping the REST APIs up and running against testing clients made it too cumbersome to work with.
"But what about creating the mocks and test suites? Won't it take a lot of time?"
Actually, no. Using the new REST Discovery feature also added in SoapUI 5.0, we can now create the basic test cases by recording API interactions between a client and live API. And because the recordings also include responses, they can also be used to quickly create the REST Mocks we want to run the tests against.
All in all, this means that the core functionality of SoapUI can now be tested without being dependent on external resources. And that by using the product itself as the test tool! Which, I think, is kind of cool.
Even if we get 100% coverage of SoapUI testing using these types of tests, there's still plenty more to test - both on lower level and UX level. For the future, I can imagine how SoapUI could also test itself on the lower-level and UX-level.
Of course, none of this is actually necessary, since we use other automation tools for this. My point is simply that the tool should be able to test itself, and it is now able to do that. I leave it up to the reader imagining how it can be done.
By the way, SoapUI 5.0 supports OAuth 2.0, as well.
You can download the open source version on soapui.org or get a free trial of SoapUI Pro at SmartBear.com.