As application development moves from the desktop to the Internet, developers and testers face new challenges in profiling and tooling Web applications. Here’s what you should expect in 2011.
The definition of application performance is undergoing a radical metamorphosis. Nearly every business application is networked to one degree or another, and applications that do so to a lesser degree (such as Microsoft Office) are increasingly being perceived as relics of an earlier era. Even if you began as a client/server developer, today you’re a Web developer. So if you’re serious about profiling — about giving yourself the clearest possible picture of how your application behaves throughout the entire system where your customers will likely use it — then accounting for factors such as network latency and server workloads becomes just as important as processor efficiency and memory leaks.
“Developers are coming into a wider world that is increasingly complex. And unfortunately, there are very complicated trade-offs that have to be made,” says Joshua Bixby, president of Web performance tools and services provider StrangeLoop. “This world is much more difficult than the one that they’re coming out of. I think the skills and the aptitude are similar, yet there’s a dogged perseverance that you need if you’re going to solve these problems. The permutations are such that there’s dramatically increasing complexity.”
For the last few decades, the performance of distributed, “three-tier,” client/server applications has been considered almost like the wave/particle nature of quantum physics. Each conclusion you draw from measurements made from the characteristics of one side of the system, reduces proportionally the reliability of any conclusion simultaneously drawn from the other side. The client application’s performance was measured using parameters under the client’s exclusive control: CPU cycles, memory consumption, hard drive latency. On the opposite side of the universe, tuning performance on the server or data center has been more macrocosmic in scale, dealing with balancing workloads, reducing processor latency, tightening clusters, rotating storage device priorities, and hunting down database deadlocks and bottlenecks.
It isn’t that these factors have stopped being important. But extremely rapid advances in Web application architectures have blurred and sometimes been obliterated the barriers between client and server in the system. The result — just as quantum physics may have predicted — is uncertainty.
One Universe, Many Facets
If you’re confused, you are not alone. One man credited with radical advances in Web apps in the last few years has delivered talks to development conferences this year titled simply, “Now What?”
The landscape for developers changed — for some, overnight. Browser speed became an issue in performance tuning. Heeding the call, browser makers like Mozilla, Google, Apple, and Opera Software accelerated their pace of development, on average boosting their products’ performance by 300% per year. Even Microsoft joined the fray, deploying a public beta of Internet Explorer 9 that was at least as capable as anyone else’s browser, if not more so. One of the jewels in Microsoft’s crown in 2009, Silverlight was back-burnered in 2010, with product managers scrambling to substantiate its relevance in manner that recalls to mind Zune, ActiveX, and Multiplan.
The incredible pace of development made plenty of room for Schiemann’s follow-up innovation: another toolkit for AJAX developers called Comet (if you’ve ever shopped the household goods aisle, you’ll understand the double-entendre) whose purpose is to completely close the gap between server and client. Now, stock traders can see real-time quotes in their browser without using expensive add-ons, and NASA can implement real-time telemetry tracking without resorting to expensive asynchronous tools or languages.
Moving the Lens
At StrangeLoop, Joshua Bixby leads a team whose job is to analyze network conditions the way meteorologists analyze the weather, and to adapt the apps and services they host to perform better under those conditions — making on-the-fly 2010-11 adaptations to 2008-09 applications. Last November, Bixby’s team ran a test on the extent to which deploying the Internet Explorer 9 beta could speed up Web apps. The results were without fanfare. In fact, IE9 might not speed up Web apps at all, at least not by itself.
Upon discovering Bixby’s report, much of the technology press (whose attention span of late mysteriously fails to extend past the 140th character) jumped to the erroneous conclusion that browser speed doesn’t matter. Some, in an extraordinary leap of logic, coupled that observation with the results of earlier studies showing faster servers did nothing to improve performance on slow and outdated browsers (such as IE6), to postulate that speed in and of itself is unattainable, and therefore not worth bothering with. Thus the entire performance tuning industry could pack up and shut down today with a whimper, not a bang.
What the press missed from Bixby’s report was this paragraph: “It’s crucial to bear in mind that a browser is just an application that renders a site to visitors. To take advantage of IE9’s potential to deliver faster performance, it’s up to site owners to optimize their sites so that IE9 can render it as quickly as possible.”
So does that mean speed and performance are the responsibility of the folks managing the servers? We asked Bixby, whose response indicated that delegating performance measurement to any one side of the system — server or client, browser or data center — is about as useful as an astronomer keeping only one side of his telescope polished.
“In the old world, developers would get on their desktop, in their office, with the data center in the basement, and have a relatively good sense for what performance would look like. From there, they started making educated decisions,” Bixby tells us. “The one main difference is that today, developers can’t get the context upon which to make decisions. From where they sit at their desk, without a great deal of effort, they can’t actually see the problem.”
Developers do lack some of the right tools, he continued, but more importantly, they lack a proper perspective. “Most developers that I speak to want to solve these problems, understand the challenges, and they have many of the tools to do it, but not all. There’s a definite tool gap, but the lens has to shift from what I see from my developer desktop in my office, to being far outside of the corporate environment, where the users are. And what that often means is, obviously a longer distance [and] a much worse network connection.
“I was on a call yesterday when a gentleman told me that his site came back in under one second, and it was 2 megs. He said, ‘I don’t have a performance problem.’ I said, ‘Great, let’s change the lens. Let me look at it from where I am.’ And it took 9.5 seconds for me. And I changed browsers; I said, ‘What browsers do your customers use?’ We looked across at a couple of browsers, and it started going up from 9 to 10 to 11. The lens is the key here. I tell customers and developers that the lens must include the browsers that their customers are using — most of the developers I know are on very modern browsers. That might not be the case with their users. What you see isn’t what happens.”
From Dylan Schiemann’s perspective, the “telescope” of overall performance is affected by multiple lenses, all of which provide varying levels of focus. The position and setting of those lenses change at different rates.
“With different rates of evolution, developers can update their code almost every day, [and] toolkits more regularly, whereas browsers and plugins are much slower,” Schiemann notes. “Of course, browsers and plugins can provide more revolutionary changes and improvements, whereas toolkits can only improve things to a certain level.” He adds that it’s possible to implement “best practices” solutions for certain toolsets, but only for a very limited period of time. “What's interesting is that best practices do change with various revolutions, because what was once a necessary hack might no longer be needed. Now, as far as measurable performance improvements are concerned, they’re really independent of the toolkit. Either the app is faster under your test scenario or it’s not, and many of the best practices are contradictory across browser implementations or usage scenarios.”
Up until 2010, Web developers’ best practice had been to build sites and apps around a “least common denominator,” to simply assume that everyone used the least capable browser and platform available: Internet Explorer 6 on a Windows Vista desktop. Any added performance that a more modern browser could produce, if that’s what the user has instead, may or may not be an extra bonus. And if the user’s on a smartphone rather than a desktop, maybe the developer can implement an alternate “m” site that looks like it emanated from an Edison stock ticker. (At least it would be fast.)
With that “LCD” in place, Bixby remarks, companies were free to pressure developers into building in more features rather than better ones — to be bigger rather than faster. If a feature set was too rich or demanding for little ol’ IE6 to manage, there was always Flash. As a result, if something went wrong, rather than blame yourself, “you could just blame Flash,” Bixby says. “There wasn’t a complexity there. It was something you just accepted, offloaded. There was no learning; that was just something [managers] expected [developers] to deal with. And if they didn’t, you were in the same boat as everyone else using Flash.”
“Today, when I have an app that can be accessed from IE6 or IE9 beta, or it can be on an iPhone,” Bixby continues, “the complexity of the problem is really almost increased by orders of magnitude. What we have found in our work is, I can actually solve a problem for IE7 that will hurt the performance of IE9, and I can solve a problem that helps caching in Firefox but that subsequently will hurt my iPhone. In the old world of a corporate app where you could mandate IE7, this problem of proliferation on the browser side was reduced. Today, when you take out the standard frameworks that you used to manage problems — Flash, Silverlight, whatever — you introduce a lot more developer control, and then you proliferate on the browser side. This problem becomes, for some companies, an insurmountable challenge.”
The Ten-Billion-Ton Elephant
The full extent of that challenge is realized once you take into account how the nature of faster clients (once customers finally install them) impacts the workload at the data center. “The largest companies in the world — Google, Yahoo, AOL — have been creating massive clusters of Hadoop servers to analyze, collect, and deal with these masses of data, and they’ve been doing it for years,” says Bixby. “For a normal organization, this big data problem is very, very difficult. So if you’re a mid-tier e-commerce player that collects 20 million hits per month, and you start having that data pour into your data center, how do you collect it? How do you store it? How do you analyze it? The real problem in this world is on that side.”
In 2011, Bixby predicts, browser makers will settle upon one standard methodology for their products to report precise timing and performance statistics back to servers, in real-time. For the first time ever, developers and admins will have one way to measure performance from the client’s perspective, to move the “lens” from the server site to the customer site. Typically when one releases floodgates, there’s a flood.
“Let’s say in a perfect world, everyone upgraded to browsers where I could collect 20 timing stamps on each of them, which is what we’re collecting on IE9. What do you do with that much data? This is a data problem that most organizations have no idea what to do with,” remarks Bixby. “It introduces a whole new science around big data and big data analysis, which today is thought to only be a problem for the Googles. But if you’re getting 20 million timings, times 20 items, and you’re doing this for a year, you’re now in a world where your traditional database infrastructure doesn’t stand up. There’s no way you can use SQL Server for this. How many servers do you need? What do you do about storage? If you actually got the metrics, what do you do with them? And there’s no standard way; this is a very difficult problem, and this is going to lag far behind the browser’s ability to collect the data.”
A great many organizations, says Bixby, don’t even know they have this problem. One effective presentation he’s given his clients, he says, is a simple video, in real-time, of their Web sites’ performance from the end user’s perspective, in dead silence. “What’s really important is that emotional transition to believing that what you’re seeing on your desktop is wrong. That’s a difficult transition,” he notes, “but once it’s made, I find that in organizations, a light bulb goes off, and the world changes.”