It's a typical scenario. You get in to work on a given morning, and during the team meeting someone discusses the fact that we need to "make sure that we are providing as much coverage as possible in our testing." Being ever diligent, we decide to re-double our efforts, and we reach for our tool of choice and strive to meet that goal… whatever that goal happens to be.
Do you recognize yourself in this scenario? If so, you're not alone. There's an old, often repeated adage that "when the only tool you have is a hammer, everything starts to look like a nail." We may think this is silly in today's technical and electronic world, but really, it's not. We have a tendency to want to get the most out of the tools that we use. That's normal, especially for tools we pay for. And that's even more common for people who buy tools, then try to find uses for them.
When I first bought the house I live in a number of years ago, I went out and bought a number of tools that I was absolutely certain I would need. Why? Because I'd read issues of "This Old House" for years, or had gone through reference books on DIY home repair, and they all made sense. Thus, armed with all of this knowledge, I went and bought a variety of tools. A table saw, a band saw, a scroll saw, a router and router table, a drill press (and assorted bits), a hand drill, an orbital sander, a planing sander, plus a number of other interesting and esoteric tools. I set up my garage with this amazing array of machinery… and there it all sat. In many cases, for years.
To my chagrin, many of the tools never got touched. If they did, it was because I tried to invent circumstances in which they would be useful. I knew I was in trouble the day I said to my wife, "Of course this mitre saw is needed. Think how great it will be to make picture frames for all of our pictures." To which she raised an eyebrow and said "Really? You're going to make picture frames?"
She wasn't being snarky. She was reminding me that, first, I'd never made a picture frame before, and second, there was no compelling need to justify investing in an expensive tool to do something that, honestly, I might do once or twice.
With that, I decided it was time to do a real world inventory on what tools I had, what ones I used, and why I used them - not why I wanted to have and use them, but where I really did put them to use for actual issues. Over the course of that experiment, I pared down the tools to a drill press, an electric rotary saw, a planing sander, a (manual) coping saw, and a good old-fashioned hand saw. Add a few screwdrivers of various sizes and lengths, plus a few wrenches, and I had the tools that would meet 90% of the real jobs I do, not the one I imagine myself doing some day. For those rare instances I actually need a more esoteric or unique tool, I can rent it for a day or a weekend. I consoled myself to the knowledge that those tools I didn't keep were donated to Habitat for Humanity, and that I was certain they would be put to far better use than I ever would or could.
Why Does This Matter?
I can see what you are thinking, "Isn't this a software blog? Why are you talking about shop tools?"
Well, it's because I see too many of us making this same mistake when it comes to picking out the software tools that we use. We have a grand idea of what we'd like to do, what we imagine we may do, without taking the time to consider what we actually do. This situation is also compounded by the fact that we have (often) already invested time and money selecting a tool that promises to do everything we ever imagined. We end up suffering because of a sunk cost fallacy: "We spent a lot of money on that tool, so you better figure out how to make good use of it!"
This is a dangerous situation to be in, because instead of being rational and considering if the tool you have purchased will actually meet your needs, you are instead trying our best to invent scenarios where the tool will work. Really, it's the software equivalent of my explaining how great it will be to make picture frames with my mitre saw, when I had no real need (or even interest) in making picture frames prior to this.
Instead of putting the tool first and then figuring out what we will use it for, we need to reverse this process. Before tools, before anything, take the time to really ask yourself "What is it that I want to do?" Step back and make a list to see what your goals are.
- Do you want to test to see if your HTTP responses are what you expect them to be?
- Do you want to find out if you can perform penetration testing on a server?
- Are you looking to run a sequence of commands over and over again?
- Do you want to set up an array of machines that will send requests to a server?
Stop and consider just those (deliberately) simple examples. If you do this exercise, you will notice that there are a variety of approaches we can use to accomplish these goals. In the example above with checking HTTP responses, we could use a flashy automated tool that allows us to control a web browser. On the other hand, using a command line tool like cURL might be a better fit. What if I want to do cross-browser testing of a particular feature? Would it make sense to maintain three separate unit tests that connect to different browsers? Would it be more advantageous and efficient to use a tool that allows me to loop through a test, but change the browser for subsequent iterations?
How about simulating load on a server? Does it make sense to invest in an expensive performance tool, or could the same process be performed with a series of shell scripts? If I wanted to expand this model, would it make sense to dedicate real world hardware to this process? If it's a relatively rare job, would it be more economical to set up a system that brings up an array of virtual machines (EC2 instances, etc.) once a month? Starting with "what's the goal?" often leads us to more (and better) solutions than to automatically say "How can I make Tool-X do that?"
Too often, we make a decision that we want to learn a particular tool, or the decision has been made for us before we even became a part of the process (many companies have already invested decades worth of engineering hours to develop automation scripts). While it might make sense to use the tools that are in place for current initiatives, don't fall into the trap of thinking that everything that follows on will match.
Instead of thinking about how to adapt the tool to a current challenge, try to block out the challenge you are facing in both general and specific terms. Try to keep the actual tool out of the conversation, if at all possible. Consider the goal, and then see what ways you can come up with to achieve the goal. In many cases, the commercial tool that is being used will be a good fit for the job. Often, the solution will be outside of the scope of the commercial tools.
And guess what: That's OK!
The purpose of tools is to make a job easier, or to perform a well-defined job more efficiently. Pounding a screw into a piece of drywall with a hammer doesn't make much sense. Let's stop thinking of our software tools in the same manner.