Monitoring Shared-eCommerce Platform MultiChannel User Experience

Date: 25th October 2011
Author: Deri Jones

We have been involved in some interesting User Journey monitoring projects recently, for a couple of retailers trying to cover the full multi-channel user experience.

To achieve full multi-channel understanding measurement of dynamic User Journeys on the public website have been complemented with Journeys on the internal  in-store  shopping Kiosks and ePos systems as well as with Mobile User Journeys on the m-web mobile website.

But things are more tricky for clients on shared eCommerce platforms – the likes of Fresca,  Venda, IBM WebSphere and so on – that make it tricky to be sure your website will be ready for big traffic peaks.

Hosted eCommerce solutions are not suitable for every retailer –  in theory they may be less flexible than a roll-your-own web platform, but the benefit is removing the maintenence and mangement software and infrastructure  from the retailer. Ideally this is supported with a software base that is already proven, and an infrastructure that has near perfect downtime, and no outages.

That kind of reliability promise sounds like a good strategy to avoid the damaging social media sentiment effect that errors and slow downs can cause throughout your multi-channel enviroment as a result of a negative user experience on any part of it, whether it is the main website, the m-web or the in-store Kiosks that underperform.

There are many hosted eCommerce solution names to choose from, such as Venda, BT Fresca, Web Sphere, Hybris, Actinic, Advanysy, Screen Pages, Snow Valley and TradeIT (redTechnology), all of which have their own specialities and focus areas.

However, there is one area of brand damage that can result from using a hosted eCommerce solution over which you have no control – and that is the issue of  Capacity or handling traffic peaks.

It’s not easy to measure Capacity of a shared platform.

A shared platform could be your organisation’s use of Cloud hosting; or a hosted eCommerce solution – but there are some fundamental challenges that are caused in their use.

Compare the situation of a regular eCommerce multi-channel site – with one on a shared platform.

With a regular eCommerce multi-channel site, you own (or rent) and are in control of a bunch of hardware – the web servers in your web farm.

If you want to do some Load Testing – to get hard evidence for your Business teams as to whether your systems can handle the upcoming traffic peaks,  Xmas or a sales season: then you can appoint a load testing team to provide the work.

And typically, so as to avoid upsetting real users on your site, you’d schedule any testing to be done out of hours – or at least at your quietest, least profitable periods – choosing a time when there as few genuine shoppers on your site as possilbe, so that the slow down during load testing has the least impact.

So far so good:  because you’re testing out of hours, you know there is essentially no one on the site, except your load test virtual traffic.  When the load testing finds the throughput capacity limits of your site on your own hardware then you know, that is the limit, there’s no doubt or uncertainty.  The Load testing will  show  the capacity of the site as a % of the ‘Last Sale Traffic’  from the last real peak period:  if you’ve designed your User Journeys well to reproduce the traffic cross section of Browsing, Searching, Adding to Basket and Checking-Out. It will also show the max throughout Journey by Journey for each.

Now compare the same Load Test, when your site is hosted on a shared eCommerce solution.

Testing out of hours – means that not only is there virtually no load from real users on your site – but there is also no load from visitors to the other retailers hosted at the same vendor.

Your provider surely has enough capacity to handle all their clients’ traffic in parallel: at least for normal, non-peak traffic levels!

So when testing out of hours, all that infrastructure which is enough to support their 10 or 20 or 50 clients; is available to your load testing alone.

So it should be very easy for your load testing to show the hosting can handle just your traffic peaks! Because you’re not competing with users from other clients of your vendor.

But… that doesn’t prove that there is sufficient capacity for your future real-user traffic peaks, when the other clients at your solution provider will also have their own traffic peaks too.

Another issue may be that another client of your provider decides to carry out their load testing at a time that is typically quiet for them, slowing down the site and taking up capacity, at a time that is normally very busy and valuable for you.

So there’s added complexity here when you put in trust in a supplier due to looser brand control, you are exposed to having  a nasty user experience in your most important sales seasons, with the brand damage, impact on sales and hurtful social media sentiment effect that can follow.

Don’t Panic

In the inimitiable words from the Hitch Hikers Guide to the Galaxy:  Don’t panic.

If you’re using a hosted eCommerce solution like Venda, Snow Valley or Fresca etc – there’s still a lot of value in load testing, to uncover technology blips and bottlenecks in advance of the start of your big shopping seasons.

A project we handled this month for one of the biggest UK Retailers, using a half-way house managed eCommerce solution proved of genuine value  when we showed hard evidence that one of their 3 major brand sites could only handle 1/4 the traffic of the other two on some dynamic User Journeys for vital shopping tasks.

The hoster, knowing that expected peak sales traffic was to be hightest for that brand was then able to work with client to plan for this and moved it to it’s own, better specification, web farm infrastructure. However, it’s dangerous to assume that just because you have discovered one issue that it will always be plain sailing because some small set-up errorswith the new set up meant that the resulting performance was actually lower in practice than on the older, lower spec system! Always test – even your newest kit!

Not discovering that until the actual Christmas sales rush would have allowed a nasty impact on sales across the multi-channel offering.

Your Multichannel systems never stand alone

Of course as multi-channel offerings provide the various interfaces for mobile users, PC users, in-store users  there is a trend that more and more of your page content is actually served by seperate 3rd parties to your main content.

So, for example, even with a hosted eCommerce provider, it is very common that your product images are being served from a seperate web farm managed by Scene7 or similar.

Or your social media pages.

Or even your m-web entirely: as some retailers have  sub-contracted their m-web out entirely to specialist m-web providers.

So all those reason – the User Experience that your technology provides when shoppers reach one of your multi-channel sites depends on several web farms:  so capacity planning  and load testing needs to cover those bases too: not just your main eCommerce solution.