Christmas eCommerce planning – Checking It Twice or Losing it!

Date: 15th August 2011
Author: Deri Jones

It’s that confusing time of year when the sun is shining outside the window, but the big projects on the go are Christmas ones!

It’s a no-brainer ROI wise, that eCommerce retail sites need to prepare for the Christmas rush, there is simply such a high value of sales at stake for most players.

Getting the IT eCommerce technology ready is therefore a common activity – but sometimes we see Load Test planning activity that is missing the boat – especially if the business teams have rather left the IT teams to it, and to have not provided helpful targets as to what is required.

First Tip: All too often, eRetail web site Load testing is done at too late to be properly prepared for Christmas.

You need to allow enough time after the testing is completed, to work out if any remedial work is needed, and fit that work in before the Christmas traffic starts to ramp up in November.

Hence why August and September are our busy months for pre-Christmas load testing projects.

So maybe your organisation is well organised, and does run website load testing before November.

Are you suffering under a false sense of security –  due to the fact that the testing is done a very basic level, such that it does not represent real usage at all?

So our second tip, is to ensure that when you check your site early, you are genuinely checking what the users do – otherwise you’re missing out the first check, and the second check will be your real users finding it hard to part with their money on your site in the real Christmas sales!

The most common form of, let’s call it ‘bare minimum’ Christmas load testing we see is where the IT team have been given no specification instructions at all, so have understandly written a test spec from their perspective, not the end users.

So they will have defined 3 or four different kinds of pages:

  • product page
  • category list page
  • home page
  • etc

For each page type they’ll list a few URLs – maybe for product pages, they’ll make a list of URls for hundreds of products.

And then load tested by generating traffic that hits those pages:  probably in the percentage split between them, that is based on some web analytics data of the real world split between the different page types.

The IT team may even have done testing using free software tools, as the Business didn’t quite allocate a clear budget – credit to the IT team for the doing the best they could with the resources available to them

The problem of this approach is two fold:  firstly that it misses out key blocks of functionality on the site, such as Search, and the vital CheckOut functions of Address lookUp and etc.

But more importantly still, no real user just hits a few URls at random, they follow a User Journey route through the site, to achieve their goal, and click on links from each page, to get to the next.

So your load testing also needs to have virtual User Journeys that click through the site, in the same way.

That requires your load test technology to look into the page contents at every step, work out the links available, and follow on as defined in your test specification.  E.g the most obvious example of dynamic choices is that after a product search, the virtual user should choose a product at random from the list offered, and not choose hit a pre-determined URL  for a product that may or may not even show in the search!

To kick off discussions between Business teams and IT teams:   these 3 Journeys are a good place to start agreeing a specification:

  • Find a product and Add to Basket – (using the navigation menus – and choosing random categories and products each time)
  • Search for a product and Add to Basket – ( not using navigation this time: the search phrases to be randomly chosen from say 100 real world examples)
  • CheckOut – exercising Address Lookup (so using a set of random port codes and etc).

In conclusion, if well before Christmas you check your site, confidence across your company in the ability of the technology to support your sales targets will grow.