Avoiding the Grinch this Holiday Season: Benchmarking
Develop | Posted October 30, 2012

While some things about website speed are absolutes - for instance, if it’s taking 10 seconds before your page appears to a user, you’re toast – understanding relative performance is critical. Benchmarking is the process of evaluating a website’s operational attributes by comparison to the competition or the landscape.

 

A best practice for benchmarking is to pay attention to four major fields: technical performance, user experience, consistency, and competitive benchmarking.

When I say technical performance, I’m talking about the traditional measures of web performance that talk to web page load time. This is the time it takes to download each and every asset of web page. Here’s a sample of what that report looks like for Google’s website:

action-name-google

It’s no longer sufficient to view website performance through the technical lens because there are many optimization techniques that allow us to spend 4 or more seconds downloading all the page resources and still present an excellent user experience in just over a second. While this particular sample took 4.2 seconds to load the entire page from a user experience perspective, the page was visible in just over a second - 1.17 to be exact. And the page was fully rendered in the visible portion of the screen by 2.4s:

 

 First Paint:

amazon_-_first_paint-resized-600

 

Above the Fold:

amazon_-_aft-resized-600

Understanding and managing Web performance, both perceived and technical, is incredibly important. The averages, however, can sometimes hide important information making it very important to also look at consistency. Here are two reporting items that can really help understand variability.

Min, Max, Avg, and Standard Deviation data from the detail report:

 

deviation-data-web-monitoring

 

And here’s a look at how performance varies by hour of day for last week:

web-monitoring-amazon-data

Ok, so Amazon.com doesn’t have a lot of variability. It’s Amazon.

Competitive benchmarking is last on the list, but certainly not least. I worked closely this year with a customer to complete a really great benchmarking analysis of their website performance characteristics for Home Page, Product Search, Product Detail Page, Add to Cart, and Checkout vs. key competitors.

This is what that analysis looked like:

web-monitoring-analysis

And, of course, we offer business benchmarks for quite a few industries websites. But benchmarking is just a small part of preparing yourself for the looming holiday rush. To really ensure that you keep the Grinch away this Friday, check back for next week's installment on load testing.

See also:

Close

By submitting this form, you agree to our
Terms of Use and Privacy Policy

Thanks for Subscribing

Keep an eye on your inbox for more great content.

Continue Reading

Add a little SmartBear to your life

Stay on top of your Software game with the latest developer tips, best practices and news, delivered straight to your inbox