What Exactly are You Trying to Test?

How did we get way up here? Or, "record-and-playback technology made me do it!"


The record-and-playback capabilities of TestComplete (or of any test automation product) are quite often one of the first features to be touted when demonstrating the tool. This is, no doubt, in part because it is easy to show off; it has plenty of "wow-factor" and because potential customers connect with this feature much faster than they might with, say, object-driven testing or distributed testing subtleties. In spite of this, or perhaps as a result of this, a great many pundits in the world of quality assurance have taken to making their living by bashing record/playback testing, stating that this technique is a great unmet hype which is worse than unfulfilled; it's actually dangerous and detrimental to productive test plans. The reality, of course, is somewhere back behind that line in the dirt. Record/playback type testing tools are just that: tools. They are only as good as the professionals who use them. And while salespeople might make promises about what you can accomplish, the products themselves make no such offers. As with all tools, it is up to the user to learn how to get the most out of it and to avoid falling into a "silver bullet" mindset, thinking that the tool will solve all QA problems.



Why record-and-playback is not so terrible (or so wonderful):


Record/playback-style automation was born in the days of yore (15+ years ago); my first taste of it was with MS Visual Test. Let me take a moment to define this term: when I talk of record and playback, I mean the technique of recording a series of user interface interaction steps into a script (basically a big macro), playing those recordings back to emulate user interaction, and then performing some sort of validation to complete an automated test. Rather than thinking of this technique as either a godsend or a vile hype-fest, I recommend to think of recordings as a starting place.
This is true in any of three senses:
1 - The new user of TestComplete will find record and playback-style testing to be a great place to start getting to know the product and the general methodologies of automated testing.
2 - Automated functional and regression testing plans can be given a great send off by starting with recording. In most cases, you can have some working tests in place with a minimum of effort. These simple record and playback tests remain in place, getting the job done (and impressing management with the way the mouse goes flying about the screen).
3 - Not all recorded scripts have to be well thought out or even usable - they can be a way to get the names of many controls on a form at once. Quite often when starting work on a new form, I'll record a script that is simply me moving around the screen, clicking everything in sight - edit boxes, lists, combos, etc. For buttons, I'll usually click the mouse button down, drag the cursor off the button (so as to avoid actually activating it) and then release the mouse button. The end result of all this is a script that contains the names of all the controls I clicked, in TestComplete naming convention-style; a very nice resource to have.
Having said all that, there are definitely drawbacks to implementing a testing plan based on record-and-playback technology. For one thing, while replacing manual regression tests with recordings is a good start, if you've accepted all the defaults, your code can't stay there - it's rather inflexible.
For as much as I think of TestComplete (and the amazing upcoming TestComplete 4), it suffers from the same problem that Borland Delphi, MS Visual Studio and other "RAD" development tools do: that defaults are not necessarily "best practices".
When I was teaching myself object-oriented programming many years ago, I leaned heavily on the Delphi 1.0 IDE. The result? I learned to double-click a button control (to create a stub OnClick event handler) and to begin typing lots of implementation code there. That's bad. I spent some time unlearning that, I can tell you. The moral of this story is that what you can do with a development tool is not necessarily what you should do. The same is true with TestComplete. Just because you can record many scripts a certain way and play them back (unchanged) with success does not mean that this is the way to test automation bliss.
The other thing that accepting the default recorded scripts that TestComplete produces can do is to force you into testing a specific portion of your application repeatedly, when only once may suffice. Specifically, TestComplete's auto-generated script code forces testing of the Presentation Domain of your application, while you may prefer to be testing the Problem Domain.



Application Domains (or, not being part of the "problem")


Typically, an application under test will be made up of three domains (or areas of function). There's the presentation domain, the persistence domain and the alliteratively thoughtful problem domain. These are fairly straightforward and are important for you, as a QA professional, to be aware of:



  • Presentation: this is the user interaction model of the application. This is where the data, information and status of your application are presented to the user. Anything that the user is aware of should be coming through this domain. 

  • Persistence: this is the storage system of your application, or where data gets persisted. It might be a simple text file or a terabyte database, but it's the domain where the data your application manages is stored.

  • Problem: this is the "guts" of the application. This domain is responsible for getting data from the persistence domain, showing it to the user via the presentation domain, then taking user-entered data and processing it.

 


All three of these domains need testing. In fact, the presentation and problem domains have testing types specifically designed for them: functional and unit testing respectively. The important thing, though, is to be aware of what exactly you are trying to test and to manage your testing code accordingly. As I mentioned earlier, the script code generated by recording in TestComplete is presentation domain-centric. A familiar bit of code generated by TestComplete might be

[Jscript]
function Test2()
{
    var p, w
    p = Sys.OrdersProcess
    w = p.OrderForm
    w.Activate()
    w.Window("TEdit", "*", 10).Click(38, 8)
    Sys.Keys("Robert Leahey")
    w.Window("TEdit", "*", 6).Click(70, 10)
    Sys.Keys("123 My st.")
    w.Window("TEdit", "*", 5).Click(47, 0)
    Sys.Keys("Dallas")
    w.Window("TEdit", "*", 4).Click(102, 14)
    Sys.Keys("TX")
    w.Window("TButton", "OK").Click(56, 10)
}

Now, let's ignore the fact that there's been little name mapping done here, so that makes this a little less readable. Where I want to call your attention is the pairs of code lines that click an edit box, and then call Sys.Keys(). This is, in point of fact, exactly what I did when recording this example, so I can't fault TestComplete for that; I did click each edit box and then type in a new value. It is, however, at this point that we need to ask the penultimate question of this article: "are we testing how our UI handles data input (presentation domain), or are we testing how the data is dealt with by the application (problem domain)?" Or both, of course.
If we are testing how our application UI responds to user clicks and typing, etc. then it is right that we should accept the default model of emulating keystrokes and mouse events. If, however, we are strictly interested in how our application processes the data it is given (for instance, if we're checking validation logic), then we couldn't care less (at this point) how the UI manages the data; we just want to push the data in. In fact, emulating user interaction at this point is just slowing us down as Sys.Keys, for all its speed, still plays back rather slowly when you have hundreds of test cases in the queue.
If at this point in our testing, the UI serves simply as a way to get our data into the problem domain, then here we will need to do a little rewriting of the generated script code. This will serve to improve our process and will have the generous side effect of making our test code more flexible.



An alternative to Sys.Keys()


Open the DDT.mds project of your preferred language flavor in TestComplete. It's usually found in C:Program FilesAutomated QATestComplete 3SamplesScriptsDDT&NameMapping. If you launch the tested application (Orders.exe) and examine it in the Object Browser, you'll notice something important: most every edit control has a property called "wText". This is a read/write property and it give you scripting access to the text value that is in the edit control. This means that not only can you read the current value within an edit control, but also that you can set that value as though someone had typed it in (it's not entirely the same though, and we'll discuss that in our section on caveats later). The end result of all this is that instead of the default code generated by TestComplete during a recording…

function Test2()
{
    var p, w
    p = Sys.OrdersProcess
    w = p.OrderForm
    w.Activate()
    w.Window("TEdit", "*", 10).Click(38, 8)
    Sys.Keys("Robert Leahey")
}

We can skip the click and the keystrokes and tell TestComplete to set the value of the edit control directly:

function Test2()
{
    var p, w
    p = Sys.OrdersProcess
    w = p.OrderForm
    w.Activate()
    w.Window("TEdit", "*", 10).wText = "Robert Leahey"
}

The obvious difference is that, instead of clicking the control and then sending a stream of keystrokes at it, we're communicating with the control directly and telling it to set its text value. The less obvious advantage of this method is that you gain a measure of dynamic flexibility: it doesn't matter whether or not the control is visible, if it has focus, if it's overlapped, or much else. Whatever its condition, the text within the control is changed. Additionally, it's done so quite quickly. Rather than seeing the stream of keystrokes be sent to the edit control one-by-one, you'll see the value in the control change almost instantly. Let me tell you, when you're iterating through hundreds of different test cases in a data-driven testing scenario, this can make a huge difference. Finally, and more to the point, you don't put yourself in the position of validating the UI's functioning for every one of those test cases. Rather, you're evaluating the code in the problem domain, and perhaps having a different set of boundary cases for dealing with the UI.



Implementation and Caveats


So assuming that we're trying to test the problem domain, what does it look like to develop tests? Are we forbidden from recording scripts? Must we hand-write dozens of lines of obfuscated code to access the internal objects of our tested application?
Not so much.
We can still record scripts, we just need to tweak them as we proceed. It's good to know the UI controls that are being used in your tested application, as they will affect the script code that ultimately gets written, but most of the time it will be a simple matter of what we examined before: replacing the lines of code which click a control and send keys at it with a line of code which sets the value of the control directly.
So ultimately, instead of code like this that was originally recorded:

[Jscript]
function Test3()
{
    var p, w
    p = Sys.OrdersProcess
    w = p.Window("TOrderFrm", "New order")
    w.Activate()
    w.editCustName.Click(28, 14)
    Sys.Keys("Robert Leahey")
    w.edStreet.Click(101, 8)
    Sys.Keys("123 My Street")
    w.edCity.Click(56, 9)
    Sys.Keys("Denton")
    w.edState.Click(27, 15)
    Sys.Keys("TX")
    w.edZip.Click(65, 13)
    Sys.Keys("76201")
    w.btnOK.Click(40, 14)
}

Now we have this code, which is more readable:

function Test3()
{
    var p, w
    p = Sys.OrdersProcess
    w = p.Window("TOrderFrm", "New order")
    w.Activate()
    w.editCustName.wText = "Robert Leahey"
    w.edStreet.wText ="123 My Street"
    w.edCity.wText = "Denton"
    w.edState.wText = "TX"
    w.edZip.wText = "76201"
    w.btnOK.Click()
}

Not to stretch this too far, but the code we now have is just a step away from being converted into a reusable functional script. If we replace the hard-coded values above with parameters that are passed into the function, we can call the function from anywhere, passing in the values for a new customer:

function InsertCustomer(aCustName, aStreet, aCity, aState, aZip)
{
    var p, w
    p = Sys.OrdersProcess
    w = p.Window("TOrderFrm", "New order")
    w.Activate()
    w.editCustName.wText = aCustName
    w.edStreet.wText = aStreet
    w.edCity.wText = aCity
    w.edState.wText = aState
    w.edZip.wText = aZip
    w.btnOK.Click()
}

function CustomerInsertion()
{
    InsertCustomer("Robert Leahey", "123 My Street", "Denton", "TX", "76201")
}

I'm not going to stray too far off into that topic, because that's leading down the path to data-driven testing. Just know any time you have code that populates a set of fields, it's quite a simple matter to turn a recorded script into a flexible, reusable function.
I will, however, discuss some of the caveats involved with implementing some of the ideas in this article. "There's no such thing as a free lunch", and that's certainly true here.
One of the subtle issues to deal with here, and certainly a deal-breaker, is that of control change event handlers. The developers of your application may have code that is intended to execute when the value of your edit control changes. If that's the case, most likely the events tied to that control will only fire when text is manually typed into it (i.e., Sys.Keys()). If that's the case, you'll have to identify which controls maintain such events, and enter text using the Sys.Keys() method for those controls. To deal with this issue, I've often created a function (or an object-driven testing class) that knows how to enter text either way and will employ that function any time I'm entering text.
Thus, keeping to our previous examples, we might have:

function EnterText(aWindow, aControl, aManualEntry, aTextValue)
{
    aWindow.Activate();
    if (aManualEntry)
        {
            aControl.Click();
            Sys.Keys(aTextValue);
        }
    else
        {
            aControl.wText = aTextValue;
        }
}

function InsertCustomer_Opt(aCustName, aStreet, aCity, aState, aZip)
{
    var p, w
    p = Sys.OrdersProcess
    w = p.Window("TOrderFrm", "New order")
    w.Activate()
    EnterText(w, w.editCustName, false, aCustName);
    EnterText(w, w.edStreet, false, aStreet);
    EnterText(w, w.edCity, false, aCity);
    EnterText(w, w.edState, true, aCustName);
    EnterText(w, w.edZip, true, aZip);
    w.btnOK.Click()
}

 Thus, we can manually enter text or insert it, as needed, using a single method. If our needs should change, after developing our tests, we can simply change the aManualEntry flag.
 The other major caveat to consider before implementing this data entry approach is that of the possible existence of compound controls. Usually (at least with standard Windows controls) the text control being manipulated is the control that actually receives the text. i.e., if you click an edit box and start typing, it is the edit box object that receives and displays the text you enter. This is not always the case with third-party tools. Sometimes, in the interest of drawing a fancy box around a text control, or for other reasons, the text control that is visible on the window you're testing is not the control that will actually receive keystrokes when you start typing. Sometimes a control will have one object for display, and then it will dynamically create another object to receive input when the text control is focused. When that is the case, it's usually not possible to set the text of the control in question by using the wText property. However, the control will most likely have some other property that can be set ("Caption", "Text", "Value", etc.). If not, most likely the control will have a method of some sort that can be called to set the text value ("set_Text", "SetText", etc.). In a worst-case scenario, it may be that your test scripts will have to focus the text control (either with a call to Click() or to SetFocus()) and then work with the dynamically created edit control to enter the text. You can use the TestComplete Finder Tool to get information about the control when it is instantiated.


Conclusion


This article is not about advocating one method of text entry over another; rather it's more about stressing the importance of knowing what exactly you're trying to validate, or test. If you know, at each step of your test development, what your goals are and what you're trying to validate, your resulting test code will most likely be more flexible and more robust.


About The Author


Robert Leahey, one-time Director of Developer Relations for AutomatedQA, is now a developer and QA automation specialist for QuickCARE in Frisco, TX. You can reach Robert at rleahey@quickcare.com.


Close

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox

By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading