Managing bandwidth for increased traffic

Broadband has expanded three-fold in just two years. More than 60% of UK Internet connections, including a third of all households, are now on broadband. And, to top it all, broadband is getting faster - 2Mbps is standard and 8Mbps common.

Some providers now offer up to 24Mbps. With users accessing at such speeds it's important to ensure that web systems can respond fast too. So how best can you identify when and for which functionality you need to better manage bandwidth? And how best can you manage the available bandwidth to offset increases in traffic?

Caching

Once a user has received static content, they shouldn't have to ask your web server for it again. Caching can both increase the speed of your site and reduce the required bandwidth. But, sadly, the majority of sites don't use caching as well as they might. Servers often have to re-deliver the same content.

Check your caching headers such as 'Expires' and 'Last-modified' - these define how pages are handled by caches, what is to be stored and when/if revalidation is required. Ideally cache headers should be set individually for each type of page. Start with elements that are large and popular, such as home page images, and with documents that won't be changed - datasheets, annual reports and the like.

Products such as the freeware mod_expires for Apache, or the CacheRight for IIS can help.

Compression

Where HTML content is large, compression can bring major savings in bandwidth and file size. 30% to 60% reductions in download times are typical. The delivery time saved on static HTML pages can far outweigh the additional CPU time at the server.

There are fewer benefits with dynamic web pages. Data produced uniquely on the fly needs to be compressed in real time. However, http compression can be added to most web servers at little cost. For Apache web servers, mod_gzip is widely used, also DynaGZip.

Microsoft's IIS web server in practice offers only limited support for HTTP/1.1 compression. However, you can add third-party flexible compression handling to IIS using jetNexus, Redline, TurboIIS orPipeBoost.

Content distribution

Separating content from one central location to multiple locations, some closer to your end-users, should be considered. Pictures, sound or video clips - which don't change - can be separated from complex web application functions such as dynamic HTML page building. A separate, slimmed down, low flexibility server tuned for speed will give better performance on less hardware. Equally bulky content, such as multimedia, dynamic content or https encrypted traffic, can be separated onto a specialised server better able to perform those tasks.

By placing your image server somewhere physically different to your main servers, you can keep the image traffic off the connection that your HTML pages are delivered over. If you are hosting on your own premises, consider a low-cost hoster to house the server for your images. This is cheaper than increasing bandwidth. If you have a significant number of users overseas, keep page delivery down by reducing long-distance traffic with an extra image server closer to them.

Clever content management

Areas of the site produced dynamically, which do not contain unique customer data, are ideal for redesign. Can they be turned into static pages and prebuilt (or precompressed) ready for delivery? Pages produced dynamically that remain unchanged for short periods, eg news pages, can be converted into static pages.

Page Optimisation

One of the most important, cheapest and often overlooked ways of improving page delivery is simple optimisation.

• Reduce the number of objects per page
• Use as few backend database queries as you can
• Optimise graphics and image files - keep graphics small.
• Reduce white space from stored HTML, CSS and JavaScript, so that only useful HTML is delivered to the user
• Use style sheets to define page layout, reducing HTML page bulk
• Avoid using image files for text headings, use pure text

These tweaks as a side effect will also often improve page accessibility, in the WAI sense.

Web accelerators

Web accelerator products, sometimes called 'reverse proxy' servers, reduce page delivery times using a variety of techniques by caching, compression, pre-fetching pages that users are likely to follow next, filtering out unwanted objects such as ads and maintaining persistent TCP connections. However, prices vary widely - from the free open source Squid, to ones selling for six figure sums.

Load testing/capacity planning

Load and stress testing is needed to ensure in advance that your web systems can support expected peak traffic levels. But it doesn't just tell you what the maximum capacity will be or if and when more hardware is needed to handle the desired traffic levels. Often load testing can identify below par areas of the web application's architecture or coding which when fixed can provide several-fold capacity gains.