Nick Olivo on Effective Automation Communication & Documentation
Here's a new article from Nick Olivo titled "Effective Automation Communication & Documentation":
This article provides a series of recommendations for successful communication and documentation regarding your automation efforts.
Anyone whos been working with QA automation for an extended period of time has heard the following questions at one point or another:
What does this subroutine do? How is it doing it?
How do I run the automated smoke test?
Is there an automated regression test for this bug?
Whats the syntax for that function you wrote?
How many TestComplete licenses do we have?
Is this test plan automated?
Will this script run on Windows 2000?
What language are you coding in?
Did the regression tests run successfully last night?
If youre lucky, people ask you questions like this just once and remember the answers that you gave. If youre unlucky, people assume they know the answer to something and are walking around with misinformation about what the automation youve painstakingly created can and cant do. There is a very basic solution to this - good communication and documentation.
These are both simple concepts, but its difficult for people to do them well. Sometimes this is because of time constraints youre under pressure to get bugs validated so you skip documenting the steps youre taking to perform those validations. Sometimes its because of misunderstandings someone misinterprets something youve said and now has an incorrect understanding of automation. And yet other times its because people dont know where to look for the information they need. In this article Ill present a few ways you can improve communication within your QA and development departments. The end result will provide people with the data that they need, and ultimately make your automation efforts more effective.
Item 1: The Automation Web Site
Many companies have a corporate intranet where departments can post information. This is a prime location to post your automation data. This can be as simple as a plain page with hyperlinks, or an elaborate page incorporating your corporate look and feel. Either way, you want a central location that anyone can go to at any time to get current information regarding your automation efforts. Many of the recommendations that I make in this article rely on having such a location in place. The types of documentation your website should include are:
- Function Documentation
- Test Results
- Test Cases
- Bug Validations
- Policies, Procedures & FAQs
- Other Reference Materials
Item 2: Comments
While not listed as an item intended for the automation web site, comments in your code are the first step in your communication effort. By documenting what your code does and how it works, you make it easier for other people to work with your code. I recommend adopting a commenting standard that includes the following:
- Standard script heading information
- Variable descriptions
- Function descriptions, parameters they accept and return values
Heres an example of the level of detail that should be included in your comments:
Item 3: Function Documentation
As you develop more and more scripts, youll most likely create several libraries of code that will be used across multiple projects. In this case, youll probably want documentation above and beyond code comments youll want help pages or a compiled help file. There are many utilities on the market that can be used for this purpose, some of which are free (jsdoc, for example) and some commercial utilities that are available for a small fee (TwinText, Doc-O-Matic). The beauty of these tools is that the documentation they generate is based off of your existing code comments, so there is very little setup, maintenance or overhead involved when using them. Once youve generated these pages, you can post them on your automation site, or just keep them in your script repository.
Heres what the documentation in Item 2 looks like after being run through TwinText:
Item 4: Test Results
Ive found one of the most frequently asked questions is Did the Such-and-Such script pass? You can deal with this issue in a couple of ways. The first is to have your script send an email (via TestCompletes BuiltIn.SendMAPIMail function) to a specified group of individuals once its completed. This is a good way to ensure that a core set of people are always informed of your scripts status. A second way is to have your scripts automatically copy their results files to a shared directory on your network (via the Log.SaveResultsAs command), and then include a pointer to that location on your automation website.
Item 5. Test Cases
Having test cases in a centralized location means everyone on your team always has access to the most up-to-date test scenarios. You can take this a step further by including automation information in each test case. This information simply states what is and isnt automated within a given test case, and what environments the script will run on. If youd like, you can also include notes on why something hasnt been automated. For example, if the test case isnt appropriate for automation because it verifies a file was printed successfully.
Item 6. Bugs
Many managers and test team leads often want to know how many bugs have automated validations in place. In order to provide this information quickly and efficiently, youll probably need to tweak your bug tracking system. Most bug tracking systems have an API that can be used to customize their functionality. Ask your system administrator to add an Automated check box to the bugs description page. Then, each time a bugs validation gets automated, check this box. This provides at a glance automation information to anyone viewing the bug.
Additionally, having this kind of information allows you to create automation specific queries against your bug tracking systems database. By running a query that searches for all records that have the aforementioned automation checkbox set to true, you can quickly create a list of all the bugs in your system that have been automated. You can then take that a step further and place the information in an html file and post it to your automation site or script repository.
Item 7. Policies, Procedures & FAQs
Its always a good idea to document your recommended scripting practices and keep them somewhere easily accessible to all members of your team. Be sure to include information such as naming conventions, the language youre coding in, commenting standards and how you want the code to flow. This is also the place where you can provide data on how many TestComplete licenses you have and who each of those licenses are allocated to.
Item 8. Other Reference Materials
I find it helpful to post a link to a help file for the language Im coding in on my network. TestComplete doesnt have built-in help for Jscript or VBScript, for example, so I have a help file I downloaded from Microsofts web site. You may also find it handy to post links to AQAs community page and message boards, or to other resources like QAForums.com.
Item 9. The Automation Newsletter
Placing information up on a website is great and provides a place for people to get information whenever they need it. Unfortunately, it also requires that people actively visit the site on a regular basis in order to stay up to date on your automation activities. My final suggestion is to send out a weekly automation newsletter. This is a document that gives a high level overview of all the ongoing automation activities and their statuses. It can also include other relevant pieces of information, such as where to find documentation on a new library you created or the acquisition of additional TestComplete licenses. This document should be kept short (1-2 pages maximum) so as to not overwhelm people with too much information. The newsletter allows you to push information highlights to your users, who can then go out to the automation site for more detailed information.
Automation provides a wealth of information. One of our primary challenges is to ensure that the right people have access to the right information at any given moment. By following these suggestions, youll be on your way to more effective automation communication. This will provide greater insight into automation throughout your organization, and thus strengthen all of your automation efforts.
About the Author
Nicholas Olivo has worked as a technical writer and automation specialist for over seven years. He enjoys automating laborious manual tasks so he and his co-workers can go out for coffee.
TwinText, Doc-O-Matic and TestComplete are all trademarks of their respective copyright holders.