Rethinking Legacy Test Automation
I've been hearing this scenario a lot lately -
"We've got this legacy system that was designed back in the 90's using old C++/ Delphi/VB6, and we're going to be migrating it to .NET/ASP.NET/Some-other-technology within the next 6-8 months. Our plan is to automate the legacy application so we'll have time to manually test the new stuff."
At first glance, this might not seem like a bad plan, but it is. Let's break this scenario down a bit.
For starters, we've got a six to eight month time frame, before our product gets buried and is never seen again.
Second, you're going to spend at least a month coming up to speed on your test automation tool. This time may include training or it may just be your team tinkering with the tool, but no one's going to be an expert immediately after purchase. So after you learn, your product is alive only for another 5-7 months.
Now you spend the next 2 months building your tests out. You learn some tricks, apply what you've learned during test refactoring, and by the end of 3 months you've got a pretty healthy regression test suite set up. Your product only has 2-4 months left to live at this point.
Given that this is a legacy application that's being migrated onto a new platform, no new development is being done on the old platform. There may be a few minor bug fixes here and there, but development might as well be at a standstill. That means that in the remaining 2-4 months, you probably only need to run *one* test cycle, because only one new version with a couple of bug fixes is being put out.
As a result, your automation has just made both its maiden and final voyage in a single test cycle. And now you get to throw away everything you've built, because the old stuff won't run on the new system.
Here’s the thing folks: Regression automation is only worthwhile when it can be run over and over and over again. Creating automated tests for the scenario above is a huge waste of time. If you were going to end of life a product in 2 years, and there were still releases of the software being put out every six weeks, then automation will provide some benefits there. But in a case where the product's being put down and there's no new development being done, you're just wasting time and energy.
In the scenario above, I'd say keep manually testing the legacy app and focus on getting the groundwork laid for automating the new platform. You'll get a lot more mileage out of your automated tests in the long run.