Sometimes things just click. You know, the feeling when you stumble upon something that makes perfect sense and just works. It feels pretty good, doesn’t it?
BDD, Specification-by-example and Gherkin
I am a developer going on my tenth professional year. During the last of them, software quality has become a passion of mine and I’m always looking for things to help me deliver better stuff. I started out with unit tests, moving on to TDD, and as of late I have been looking into behavior-driven development (BDD), specification-by-example and similar techniques. For me, it really makes sense to have readable requirements that can be executed and verified.
In order to write these kinds of requirements, we need some sort of tool, and there are a few. One example is Cucumber, which comes with the Gherkin language – known to the most of us as the “Given-When-Then” language.
Gherkin gives us three building blocks to work with:
The feature we are building, such as “Login”, “Withdrawal” or “PublishBlogPost”.
Examples of when this feature is used. Some will be happy, such as “SuccesfulLogin". Other examples are sad, such as “LoginFailed” or “NoConnectionWithBank”.
The steps describe what actions are needed to complete the scenario. This is where the three familiar words appear:
- Given – Provides preconditions, context and puts the system in a known state
- When – The key actions taken in interaction with the system
- Then – The expected outcome/result of the actions
Although not a Cucumber user, I find this way of structuring requirements and tests quite natural. It gives me a simple set of rules to follow and they let me focus on the right things at the right time.
At the company where I work we started using soapUI for our web services a few years ago. Going from manual to automated tests took some time, and we finally came to a point where there was a decent amount of tests. Having all of those tests was great, but introducing quantity both in tests and authors meant trouble –naming tests was one:
“Hey, dev! Test case 35 just failed!”
“Um… so what does test case 35 actually test?”
So, we were in desperate need of some structure. I started looking into what was available in soapUI: Test Suites, Test Cases and Test Steps. While messing around with these for a while it suddenly became apparent that soapUI was screaming for me to use one particular way of structuring stuff - a particular green way, a particular vegetable way.
Using Gherkin in soapUI
For me, Gherkin and soapUI seem like a match meant to be. Here’s how I see them mapping:
||Test SuiteFor each of the features, I’ll add one Test Suite. This doesn’t only give me a lot of test suites; it provides well needed boundaries for my tests. It also gives me easy naming for test suites – they will have the same name as the feature!
||Test CaseEach suite will contain several test cases for each feature. Some will be happy, some sad. I’ve found it useful to make this distinction very clear by prefixing the name of the test case with “Happy – “ or “Sad – “.
||Test stepEach of the test steps in my test case will now start with “GIVEN”, “WHEN” or “THEN”. I use this capital, screaming style to add to the sense of structure. Maybe syntax highlighting could be something for a future release of soapUI…(?)
Let’s assume that we have an authentication service in our architecture. Let’s also assume that it enables users to login and be authenticated. Our first feature is “Login”. Let's create a test suite for it:
And if you like, you can add the business value motivation part in the description of the test suite properties:
The next step is to identify a scenario and what gives more business value than "Successful login"? We'll create a test case for it and tag it as a “Happy” one:
Finally let’s create the test steps for this example:
There we are! Let's examine the steps in more detail.
We use the Given step to setup an existing user in our system. This might be a database operation, a set of service calls depending on the application environment. I always try to create as much of the context from scratch so that I am not relying too much on data still being in the database/system. What I found quite useful is to use the step type "Run test case". It allows reusing code between scenarios, many of them will have a similar setup. I also use properties of this test case to configure this particular test run. The reused test case can then provide me with the data it created, such as a user ID, to drive further testing.
In a service-oriented architecture, this will most certainly be a service call. We'll call the Authentication service using the Login method with some parameters, such as UserId = "OurNewlyCreatedUser".
Since the When is only meant to interact with the system, assertions should not be placed in these steps. SoapUI provides a simple way of adding asserts directly on the SOAP request step, but we need to ignore that possibility in order to keep the Given-When-Then-flow. Sometimes I keep “sanity-checks” as assertions here, such as “Not SOAP fault,” but never any assertions that represent business value.
This is where we want the assertions. We want the test case to fail in one of the Then clauses, or even better - not fail...
In previous versions of soapUI, the Then step required us to get down and dirty with Groovy. Not anymore. Since version 4.5 we have the assertion test step at our disposal which fits perfectly for our Gherkin purposes.
Once we have these three constructs in place it is easy to add more steps into the test, using And.
We can use this to add extra context in the Given, perform multiple actions in the When or assert many things in the Then. In my example I added an extra JDBC step to ensure that an audit of the login was stored in the database. Personally, I try to limit the number of When and Then steps, since it makes tests more fine-grained and easier to diagnose when they are failing – resulting in fewer “Test case 35? - Crap.”
Making this happen with soapUI really opened my eyes. I am a passionate TDD guy, but my unit tests are not supposed to be understood by the "non-developer" crowd. Since SoapUI is widely adopted where I work, I can now communicate requirements and the state of development better. It also helps me help others to understand the value of testing. Writing tests is not a separate activity, nor should it be performed by specific people – it is our common responsibility and a crucial part of developing quality software.
It seems that there are many tools that can be used to implement BDD, Specification by example or similar techniques. Even though soapUI is not explicitly marketed as a BDD tool, the fact that we had invested time with it made it a practical choice. What tool you use is not important, understanding the value of executable requirements in the shape of tests is.