Pipelining and Task Management

Pipelining is something that all the major browsers support. It means that parallel requests will be sent to a given web server to load any given page. IE pipelines up to two requests at a time per server, whereas firefox by default will pipeline 4 requests (one reason that firefox is perceived to be quicker than IE). I believe Opera makes 4 requests by default as well, but I don't know for sure.

You can increase the pipeline values to help your own browsing experience, but as a webmaster you probably want to at least view your own site in the same way that most other people do. We can, however, take advantage of this standard to tweak our own sites in order to maximize the perceived performance experience by our website visitors, and actually reduce the overall workload on our servers at the same time - thereby making the overall performance of our sites better.

I recently did a little bit of very basic load testing on my own server and came to discover that certain heavily scripted pages (like vBulletin pages) decrease the capacity of the web servers Transactions Per Second number significantly (by two orders of magnitude). Other static content was handled significantly faster, and faster still through the use of a high performance web server package geared towards static content delivery.

The idea, when thinking about Pipelining and Task Management where web site performance is concerned, is to set up separate processes to handle http requests in the most efficient way possible. Use a high performance static content server to handle static requests. Use a high performance dynamic content server to handle dynamic logic. And where possible, offload web serving tasks from logic intensive applications.

The following diagram depicts the idea visually (click the image to see the full sized diagram):

In the above diagram, there are two static content servers set up (lighttpd), an apache server as the main server, and a Tomcat server proxied through apache. Once the initial page request is responded to, the end users computer can then request up to 12 consecutive objects simultaneously to complete the page load. (3 server addresses * 4 pipelined requests). You certainly could use apache's virtual host directives to serve static content directly from within a single application, but lighttpd's caching of compressed objects is better than what apache offers, and has a significantly smaller memory and CPU footprint per request. Using the pre-fork MPM with apache, depending on which modules you have loaded, you can see a per-request memory footprint of up to 20 Megabytes (more commonly about 2 Mb). Offloading delivery of static content to lighttpd can reduce memory usage and speed delivery, while at the same time reducing the CPU expense of individual requests increasing the capability of your server to handle a high number of simultaneous requests.

Proxying java applications through apache allows for another nicety - it reduces the need for Tomcat to serve up static content and allows the engine to concentrate on it's primary purpose - processing Java. You also get the side benefit of being able to serve java in a subsection of a site rather than run an entire site with java, or having to push java functionality out to a subdomain. Under high load which is more likely when serving static content, Tomcat can be reduced to very poor performance as the most important task would be thread creation to deal with the load rather than processing data. The fact that the java requests keep piling up inherently increases the amount of time that it takes for Tomcat to clear the queue and return to a fast response mode. It is a nasty vicious circle.

Consider these rough Transactions Per Second numbers that are based on my own extremely basic load performance analysis.

Apache static content delivery is somewhere in the neighborhood of 3000 transactions per second.
Apache with php and database intensive processing is reduced to the neighborhood of 5-20 transactions per second, but can be increased significantly with less intensive application interaction.
Lighttpd static content delivery is somewhere in the neighborhood of 6000 transactions per second.
Tomcat delivers static content somewhere in the neighborhood of 2300 transactions per second.
Tomcat delivers java intensive pages somewhere in the neighborhood of 350 transactions per second.

Now, if either apache or tomcat get bogged down with static requests, their capability for maximizing throughput on non-static requests are reduced. If those duties are offloaded, the real world capabilities of these applications are closer to their load tested maximum numbers.

Since there is such a significant difference between static and dynamic content delivery, static servers should never reach maximum throughput capability and therefore should always be able to manage your traffic - even at 8 times what your apache server is getting. It doesn't mean there is a free for all for using large images, javascript, or css files but at the same time the inclusion of these scripts will affect page load times less dramatically - at least for broadband users. Dial-up users will most likely not be able to take advantage of pipelining to the degree that broadband users can - however by spreading content across physical server addresses you wll ensure that their bandwidth usage is maximized.