Unearthing hidden functionality flaws

Can we solely rely on server uptime or URL availability to monitor the performance of web systems? Today’s web applications no longer simply serve static pages, but generate dynamic content and offer complex functionality. All too often simple page uptime/downtime methods will fail to reflect a user’s experience when navigating the site.

They may overlook hidden flaws. It is not uncommon to discover that although the home page is fine, credit card orders online or user registrations are not working, sometimes for many hours or even days; and the simple server or page-based checking systems in place have not raised an Alert.

Even making minor changes to a website can adversely affect site visitors. The impact can be immediate or over time, but may never be spotted by simple monitoring. For example, design flaws may cause occasional error rates of up to, say, 2 or 3% - but only when certain forms are used in certain ways. These can frustrate real users, but are lost in the noise of the site web logs. In any case they are not easily reproducible at the console and thus are hard to follow-up, so tend to be ignored by hard-pressed Web Admin staff.

So, how best to unearth such hidden functionality flaws? To address the matter, SciVisum has developed a functional monitoring approach. It’s at the heart of the company’s recently-launched SV-Monitor Service – a service to track and respond to the smallest of flaws across all the functionality elements of even the most complex dynamically-driven web applications.

Functionality testing goes beyond page availability. It considers user behaviour, whether typical or random; the actual way a user might register, browse and make on-line purchases. It can be applied equally to Internets, Extranets and internal Intranet applications.

The performance, error-rate and consistency of website functionality is measured by systematically modelling user behaviour. Test experts script all the key functional elements of a web site in multi-page User Journeys; registration processes, searching, browsing for products or making online purchases. The test engine then runs 24/7, visiting the site and reproducing the same user activity. It inserts random steps – such as choosing a product at random before proceeding. Analysis, graphical views of each step taken and the overall User Journey performance are displayed on a flexible Portal.

Subscribers are alerted when a problem arises and can easily track medium or long-term changes to their site, over time lines of their choice.

Where a site features dynamic content and offers complex functionality, it’s well worth considering the functional monitoring approach.

The SV-Monitor service follows a functional monitoring approach and can provide information beyond page availability testing alone. Is the user experience getting better over time? What is the effect of new features on the performance of existing functionality, how does today’s performance under today’s load compare with earlier periods when there were fewer users? Which individual functional elements have changed the most?

So what are the specific benefits of the functional monitoring approach as instanced by the SciVisum SV-Monitor service?

• Real time analysis: SMS/emails alerts and live data of current and recent performance – to speed problem-response analysis
• Powerful portal: performance graphs, error tables and data analysis can be viewed across any time line, current or historic
• Comprehensive reports: monthly, in-depth statistics, a quick managerial overview with performance comparisons of current versus previous time periods
• User-configurable parameters: subscribers can configure their own ‘Quality cut-off time‘ individually for each User Journey
• Adaptive test frequency: default test interval of 5 minutes decreases when a User Journey is in error; to more accurately track the duration of errors, without generating excessive test load under normal operation.

Underpinning the whole functional monitoring approach at SciVisum is the flexible, semi-intelligent test engine developed in-house. This combined with expert staff takes testing beyond trivial home page performance to tracking the user experience of today’s fully featured dynamic web applications.