F5 Tech Focus

Using Stateful Test Tools to Verify WebAccelerator Performance Gains

While stateful test tools are perfect for providing the background stateful traffic that help test products like the F5 WebAccelerator under load, they dramatically understate the response time improvement and bandwidth reduction WebAccelerator brings to web applications.
by Peter Murray Technical Marketing Manager

Tech-Focus Paper Using Stateful Test Tools to Verify WebAccelerator Performance Gains

Contents
Introduction WebBrowserOperation WebAcceleratorOperation StatefulTestToolOperation TestToolCachingLimitations OtherTestToolShortcuts EffectiveWebAcceleratorTesting Conclusion AssociatedReading 3 3 5 6 6 7 8 9 10

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

Introduction
WebAccelerator is widely known for its remarkable ability to improve web application performance by speeding web page loading and for its ability to reduce network bandwidth. However, testing WebAccelerator performance is not as simple as connecting a stateful test tool and checking results. The reason for this is that the only meaningful way to understand WebAccelerator performance improvements is to measure response time from a fully functioning web browser. Because stateful test tools do not adequately emulate client behavior, they cannot provide accurate results and are therefore inadequate for testing WebAccelerator performance alone. A better approach is to test with a stateful testing tool in combination with a performance-testing tool like Symphonic TrueView/TrueSpeed or an HTTP monitoring plug-in tool such as HTTPWatch or HTTPFox, which operate in tandem with a web browser. Testing using both tools together can demonstrate the dramatic performance gains available with WebAccelerator. Stateful test tools provide a powerful way to test the effects of network load on stateful devices such as servers, firewalls, Application Delivery Controllers and others. Although load testing tools such as HP LoadRunner, Ixia IxLoad, Spirent TestCenter, BreakingPoint and others offer high quality results useful for measuring the effects of stateful application traffic on a network, they lack the ability to measure how applying web acceleration technology to applications improves user experience. While stateful test tools are perfect for providing the background stateful traffic that help test products like the F5 WebAccelerator under load, they dramatically understate the response time improvement and bandwidth reduction WebAccelerator brings to web applications.

Web Browser Operation

Before looking at stateful test tool behavior, it is important to understand how a web browser operates. For example, suppose John wants to access www.example.com. He types http://www.example.com into the browser address field to access the default page for the example.com web server. The browser issues an HTTP GET command for the site (for example GET 102.0.2.100/). The server that answers the request

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

sends back an HTTP response that typically contains a list of files (content) the browser requires to build the web page. The browser downloads the files and uses them to build and display the web page for www.example.com. When a browser downloads the objects for a web page, it makes the decision whether to cache each object based on HTTP response headers contained in the server response. The first time the browser downloads a page, none of the content is present locally. If a piece of content is marked as cacheable, the browser saves it in a local disk cache along with information about how long the content is valid. If the content is marked as non-cacheable, the browser displays it without saving to disk cache. On subsequent visits, when the browser encounters a reference to an object in an HTML page that it has previously stored in its local disk cache, it checks for the contents validity. If it is still valid, the browser loads the content from disk rather than requesting it from the server. If the content has expired, the browser validates the content with the server. If an E-Tag, exists (which uniquely identifies an object), the browser sends a GET request with an If-None-Match header containing the E-Tag value for the content. If the object has an associated modification date, the browser sends a GET request with an If-Modified-Since header. If both an E-Tag and a modification date are present, the browser sends both. If the content is still valid, the server sends a 304 (Not-Modified) HTTP response code, which directs the browser to load the object from cache. This ensures that the browser downloads new content only if the content on the server is newer than the one that is available in its cache. By loading static content from cache, the browser dramatically reduces access time and improves page loading and therefore the user experience. During web page loading, the web browser may also process one or more JavaScripts. Most web browsers stop all other processing while they process each JavaScript. If a browser downloads multiple JavaScripts, script processing is sequential. Figure 1 shows an example trace where a series of JavaScripts execute serially during a page download.

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

Figure 1. JavaScript processing

WebAccelerator Operation
WebAccelerator resides on a network between origin servers and clients. It proxies all client requests to an origin server and speeds web applications by reducing the number and size of objects that need to be retrieved from a server. Examples include: 1. 2. Compressing objects to reduce object size whenever possible Setting long object expiration times to increase the use of cached objects vs. downloading objects from a server 3. Modifying server responses to enable the browser to open more simultaneous server connections 4. Serving objects from WebAccelerator rather than from an origin server

First, WebAccelerator examines HTTP response headers and objects. When it encounters an object that is cacheable per the configured acceleration policy, it stores the object and forwards it on to the requesting browser. WebAccelerator subsequently serves the object until it expires rather than requesting it from the origin server. When the object expires, WebAccelerator performs a conditional GET request to determine whether it is still valid, as described above. Another important WebAccelerator function is to examine content and take action based on policies. A policy controls how WebAccelerator interacts with an application to improve performance. For example, a policy may mark an object as cacheable and set an expiration date even though the application marks it as non-cacheable. In this case, WebAccelerator also caches the object and serves it to other requesting clients directly until it expires. The result is a dramatic decrease both in the number of requests that the origin server has to process and network utilization.

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

Stateful Test Tool Operation

Stateful test tools emulate realistic network traffic in order to test network devices and in some cases, to stress-test servers. The test-tool user typically programs the device with a series of actions that emulate client behavior, for example, web page downloads, DNS requests, email retrieval and other applications. The tool typically repeats one or a few sets of actions to simulate large numbers of users. During the test, the tool records statistics about how the application or Device Under Test (DUT) performs under varying load. Stateful test tools cannot completely emulate browser behavior because they lack the processing, memory and disk resources to emulate client processing for thousands of web browsers. Instead, they take necessary short cuts to emulate user traffic. Unlike browsers, most test tools require the tester to create an explicit request for every object because they are unable to process HTTP response information or scripts, and then use the information to download a complete web page. This can make the process of creating a realistic test more difficult and much more time-consuming.

Test Tool Caching Limitations

Tools that do not read HTTP response headers additionally do not save downloaded content, nor do they process caching instructions in HTTP responses, so they do not emulate browser cache operation. Some other test tools can use the information in an HTTP response to download a complete web page, which somewhat simplifies configuration. However, running tests using a tool that more completely emulates browser behavior are very slow and do not scale. Moreover, even these tools cannot process and use caching instructions in HTTP response headers. Instead, some can be programmed to skip downloads for objects the tester configures as cacheable. In this case, the test tool downloads an object only the first time it encounters it during a test session. This is known as a session cache. The session cache is in fact a list of objects previously downloaded by an individual user. During a test session, the tester can configure the test tool to download these objects only once.

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

Other Test Tool Shortcuts

Test tools take other shortcuts to test using large numbers of emulated clients. These shortcuts can distort web page response times by shortening or lengthening transaction times. Some examples are: Some test tools suspend all action while waiting for DNS resolution, unnecessarily lengthening web page response time. Real web browsers process DNS requests in the background and therefore are able to continue processing and issuing requests while waiting for DNS resolution Test tools do not process java scripts. Instead, scripts are downloaded, but ignored. This understates response time by incorrectly eliminating scriptprocessing time and therefore understates the WebAccelerator benefit. Real browsers process java scripts sequentially and stop all other processing while it processes a script. Some tools do not open the appropriate number of connections when requesting web content, distorting response time by either lengthening response time by using too few concurrent open connections to a web site or unrealistically shortening response time by opening too many. Some tools either do not support sufficient username/password combinations to enable per-user authentication, or sufficient user certificates to emulate large numbers of users. Even when able to perform these actions, these parameters can be cumbersome to configure a realistic number of users. When too few username/password combinations are available, tests are run with the same, small subset of users. Support for too few user certificates has a similar effect, which can make it difficult to test with HTTPS. Some test tools operate through a socket connection rather than emulating a true TCP stack, therefore misstating TCP response time characteristics. The test tool may emulate too many TCP retries, or user requests may sit in a queue waiting for processing after a response is received from a server. The result is response time is overstated and page loading takes longer than it would with a real web browser.

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

The bottom line is that test tools cannot truly emulate HTTP browser behavior. Often, these test tools report little performance gain and sometimes, a performance decline when testing products such as WebAccelerator. The best that can be said for these tools is that they can provide an insight into the first visit to a web site. They can also be useful to show limited performance gains when many users are accessing a page at peak traffic times because the WebAccelerator will respond more quickly to object requests than a server after a first request where WebAccelerator caches an object. However, a better solution exists: test with a combination of stateful test tools and either an HTTP performance tool or a browser with an HTTP performance plug-in tools to collect, view and save actual HTTP browser-based response time information.

Effective WebAccelerator Testing

Fortunately, simultaneous testing with a stateful test tool and either an HTTP performance testing tool or a real web browser and an HTTP performance plug-in can clearly demonstrate how WebAccelerator shortens user response time to benefit applications. First, configure the test tool to generate the required amount of background traffic, using a realistic mix of traffic types and load. Then, while the stateful test tool is running a test, run a simultaneous test using a performance-testing tool like Symphonic TrueView/TrueSpeed or a standard web browser in combination with an HTTP measurement plug-in such as HttpWatch. During the test, take actions that emulate a real session and use the associated application to measure results. The following steps are a rough outline for running tests: 1. 2. 3. 4. 5. Configure a test that reflects typical application traffic on the stateful test tool. Install an HTTP measurement tool plug-in for your browser. Start the test tool without the WebAccelerator in the test bed. Open the browser and clear the cache. Restart the web browser. Run a baseline test by making a first-time request to each web page to be tested. 6. Repeat the baseline test to make a repeat visit to each web page to be tested.
8

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

7. 8. 9.

Stop the test, and then save the monitoring application results. Start another test with WebAccelerator in the test bed. Open the browser and clear the cache. Restart the web browser. Run the same baseline test to make a first-time visit with WebAccelerator installed and configured to accelerate your web application(s).Repeat the baseline test to make a repeat visit to your application(s).

10. Stop the test, and then save the monitoring application results. 11. Compare the results from the baseline test without WebAccelerator. Note performance improvements (if any) with repeat visits. Next note performance improvements for WebAccelerator first and repeat visits. Compare the results to demonstrate how WebAccelerator can help improve both first time and repeat visits.

Conclusion
WebAccelerator can dramatically improve web application response times and user satisfaction, especially for remote users over a WAN. When tested properly, the combination of stateful test tools and a performance-testing tool like Symphonic TrueView/TrueSpeed or an HTTP monitoring plug-in tool such as HttpWatch will show the true benefit WebAccelerator can deliver to your users.

F5 Tech Focus Using Stateful Test Tools to Verify WebAccelerator Performance Gains

Associated Reading
Browser Behavior https://rate.f5.com/files/folders/acceleration_architects_archive/entry9008.aspx

HTTP Fundamentals https://www.gosavo.com/PROD/Document/Document.aspx?id=109597&view=

How to Test with TrueSpeed https://rate.f5.com/files/folders/tools/entry6356.aspx

Showing WebAccelerator Value HttpWatch https://rate.f5.com/forums/post/6062.aspx

F5 Networks, Inc. Corporate Headquarters

401 Elliott Avenue West Seattle, WA 98119 +1-206-272-5555 Voice (888) 88BIGIP Toll-free +1-206-272-5556 Fax www.f5.com

F5 Networks Asia-Pacific
+65-6533-6103 Voice +65-6533-6106 Fax info.asia@f5.com

F5 Networks Ltd. Europe/Middle-East/Africa

+44 (0) 1932 582 000 Voice +44 (0) 1932 582 001 Fax emeainfo@f5.com

F5 Networks Japan K.K.

+81-3-5114-3200 Voice +81-3-5114-3201 Fax info@f5networks.co.jp

10
TF-Stateful tools WA.pdf/08 2008 F5 Networks, Inc. All rights reserved. F5, F5 Networks, the F5 logo, BIG-IP, FirePass and iControl are trademarks or registered trademarks of F5 Networks, Inc. in the U.S. and in certain other countries.