Monitoring video streaming – the BBC set a benchmark

Date: 24th June 2010
Author: Deri Jones

I picked up the discussion at about the quality (or otherwise) of the two main UK  TV channels video streaming online:  BBC versus ITV.

As someone in London noted:

“…However, the worst thing about the online coverage is the quality. The small screen shown above isn’t quite big enough to watch the game properly, but if you enlarge the coverage to full screen the resolution is not good enough and a blurry mess ensues”.

“Worse still, the coverage is jerky and needs to reload / buffer every few minutes…

Video streaming monitoring is something we’ve been doing more of lately: and it’s interesting the variety of different ways in which people do it.

Some run streaming servers in-house, others use specialist streaming 3rd-parties: or roll their content round a CDN network.  Some wrap their streams in encrypted channels, or DRM protect them; some use industry standard protocols and some wrap their own bespoke things round complex home-grown streaming server software on non-standard  TCP ports.

Whichever way, if there’s streaming on your site, confidence that users are getting a fast and smooth experience is not something that can just be assumed.  But of course, given the widely varying and potentially complex technology being used to deliver what the management may think is ‘just a video stream’, it’s often impossible for organisations to measure and monitor the user experience of their own streams, with the tools available to them

An example of the effort needed, for the video streaming monitoring we set up for the BBC over the world-cup, we had to create multi-step User Journeys that handled the first few steps of a user’s journey, navigating from the homepage through to choose the game to watch – and then monitored the streaming.

Fortunately, our existing web monitoring service is designed for handling the most rich multi-page User Journey, with log-ins, dynamic random selections in real time and so on:  so the pre-steps in the journey before the actual streaming starts were straightforward.

As with all project plans it was a last minute dash as more in depth monitoring was required than anticipated: but they were successfully set up in advance of the first game, and were able to provide the BBC team with the real time picture of performance.

Our journey script was also able to handle the fact that there are not live matches being streamed 24/7: between matches, the Journey looks for a live stream and doesn’t find it of course: so to avoid false alarms we simply coded it so that this didn’t throw an error .

And on Wednesday’s England game in the afternoon, at 1:0 at half time the graphs from our monitoring showed the streaming was as good as the mood of the English fans!