Nick Olivo on Effective Automation Communication & Documentation
Test and Monitor | Posted November 29, 2005

Here's a new article from Nick Olivo titled "Effective Automation Communication & Documentation":


This article provides a series of recommendations for successful communication and documentation regarding your automation efforts.


Anyone who’s been working with QA automation for an extended period of time has heard the following questions at one point or another:


What does this subroutine do? How is it doing it?

How do I run the automated smoke test?

Is there an automated regression test for this bug?

What’s the syntax for that function you wrote?

How many TestComplete licenses do we have?
Is this test plan automated?
Will this script run on Windows 2000?

What language are you coding in?

Did the regression tests run successfully last night?


If you’re lucky, people ask you questions like this just once and remember the answers that you gave. If you’re unlucky, people assume they know the answer to something and are walking around with misinformation about what the automation you’ve painstakingly created can and can’t do. There is a very basic solution to this - good communication and documentation.

These are both simple concepts, but it’s difficult for people to do them well. Sometimes this is because of time constraints – you’re under pressure to get bugs validated so you skip documenting the steps you’re taking to perform those validations. Sometimes it’s because of misunderstandings – someone misinterprets something you’ve said and now has an incorrect understanding of automation. And yet other times it’s because people don’t know where to look for the information they need. In this article I’ll present a few ways you can improve communication within your QA and development departments. The end result will provide people with the data that they need, and ultimately make your automation efforts more effective.


Item 1: The Automation Web Site

Many companies have a corporate intranet where departments can post information. This is a prime location to post your automation data. This can be as simple as a plain page with hyperlinks, or an elaborate page incorporating your corporate look and feel. Either way, you want a central location that anyone can go to at any time to get current information regarding your automation efforts. Many of the recommendations that I make in this article rely on having such a location in place. The types of documentation your website should include are:

  • Function Documentation
  • Test Results
  • Test Cases
  • Bug Validations
  • Policies, Procedures & FAQs
  • Other Reference Materials


Item 2: Comments

While not listed as an item intended for the automation web site, comments in your code are the first step in your communication effort. By documenting what your code does and how it works, you make it easier for other people to work with your code. I recommend adopting a commenting standard that includes the following:

  • Standard script heading information
  • Variable descriptions
  • Function descriptions, parameters they accept and return values


Here’s an example of the level of detail that should be included in your comments:


Item 3: Function Documentation

As you develop more and more scripts, you’ll most likely create several libraries of code that will be used across multiple projects. In this case, you’ll probably want documentation above and beyond code comments – you’ll want help pages or a compiled help file. There are many utilities on the market that can be used for this purpose, some of which are free (jsdoc, for example) and some commercial utilities that are available for a small fee (TwinText, Doc-O-Matic). The beauty of these tools is that the documentation they generate is based off of your existing code comments, so there is very little setup, maintenance or overhead involved when using them. Once you’ve generated these pages, you can post them on your automation site, or just keep them in your script repository.


Here’s what the documentation in Item 2 looks like after being run through TwinText:


Item 4: Test Results

I’ve found one of the most frequently asked questions is “Did the Such-and-Such script pass?” You can deal with this issue in a couple of ways. The first is to have your script send an email (via TestComplete’s BuiltIn.SendMAPIMail function) to a specified group of individuals once it’s completed. This is a good way to ensure that a core set of people are always informed of your script’s status. A second way is to have your scripts automatically copy their results files to a shared directory on your network (via the Log.SaveResultsAs command), and then include a pointer to that location on your automation website.


Item 5. Test Cases

Having test cases in a centralized location means everyone on your team always has access to the most up-to-date test scenarios. You can take this a step further by including automation information in each test case. This information simply states what is and isn’t automated within a given test case, and what environments the script will run on. If you’d like, you can also include notes on why something hasn’t been automated. For example, if the test case isn’t appropriate for automation because it verifies a file was printed successfully.


Item 6. Bugs

Many managers and test team leads often want to know how many bugs have automated validations in place. In order to provide this information quickly and efficiently, you’ll probably need to tweak your bug tracking system. Most bug tracking systems have an API that can be used to customize their functionality. Ask your system administrator to add an “Automated” check box to the bug’s description page. Then, each time a bug’s validation gets automated, check this box. This provides at a glance automation information to anyone viewing the bug.

Additionally, having this kind of information allows you to create automation specific queries against your bug tracking system’s database. By running a query that searches for all records that have the aforementioned automation checkbox set to true, you can quickly create a list of all the bugs in your system that have been automated. You can then take that a step further and place the information in an html file and post it to your automation site or script repository.


Item 7. Policies, Procedures & FAQs

It’s always a good idea to document your recommended scripting practices and keep them somewhere easily accessible to all members of your team. Be sure to include information such as naming conventions, the language you’re coding in, commenting standards and how you want the code to flow. This is also the place where you can provide data on how many TestComplete licenses you have and who each of those licenses are allocated to.


Item 8. Other Reference Materials

I find it helpful to post a link to a help file for the language I’m coding in on my network. TestComplete doesn’t have built-in help for Jscript or VBScript, for example, so I have a help file I downloaded from Microsoft’s web site. You may also find it handy to post links to AQA’s community page and message boards, or to other resources like


Item 9. The Automation Newsletter

Placing information up on a website is great and provides a place for people to get information whenever they need it. Unfortunately, it also requires that people actively visit the site on a regular basis in order to stay up to date on your automation activities. My final suggestion is to send out a weekly “automation newsletter.” This is a document that gives a high level overview of all the ongoing automation activities and their statuses. It can also include other relevant pieces of information, such as where to find documentation on a new library you created or the acquisition of additional TestComplete licenses. This document should be kept short (1-2 pages maximum) so as to not overwhelm people with too much information. The newsletter allows you to push information highlights to your users, who can then go out to the automation site for more detailed information.

In Summary

Automation provides a wealth of information. One of our primary challenges is to ensure that the right people have access to the right information at any given moment. By following these suggestions, you’ll be on your way to more effective automation communication. This will provide greater insight into automation throughout your organization, and thus strengthen all of your automation efforts.

About the Author

Nicholas Olivo has worked as a technical writer and automation specialist for over seven years. He enjoys automating laborious manual tasks so he and his co-workers can go out for coffee.


TwinText, Doc-O-Matic and TestComplete are all trademarks of their respective copyright holders.


By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox