Verktyg

Case: SEMA - Swedish Emergency Management Agency

How to verify, test and improve performance of a web site ahead of an upcoming, expected traffic peak load

The Swedish Emergency Management Agency – SEMA (Krisberedskapsmyndigheten (KBM), (http://www.krisberedskapsmyndigheten.se/) is a governmental body with the task to coordinate crisis preparedness in the Swedish society.

During the Spring of 2008, SEMA carried out a vast communications exercise called SAMÖ2008. The purpose of the exercise was to try the preparedness of financial systems against a possible IT-based attack. The exercise drew appox. 5000 participants from the Swedish Government, other central governmental bodies, the counties, organizations and companies as well as a large society network.

The subject of the exercise was organized IT-attacks which would threaten or bring down the financial systems and which could lead to a crises of confidence in the society; in a case like this, the traditional payment systems normal functions are affected by very severe distortions.

The exercise was a simulated crises scenario, which played out with the participants in the exercise and was headed up by SEMA, where approx. 100 participants were active in the so called counter scenario which brought the exercise game forward.

"Large communications exercises is an important means to strengthen the capacity to handle crises and to drill coordination between different sectors which work with crises preparedness in the society", said Helena Lindberg, leader of the exercise SAMÖ2008 and General Director of SEMA in a press release.

Technical challenge

The exercise had a central web site - www.samö2008.se – which delivered continuously updated information during the exercise for all participants. The performance of this site was of ultimate importance as the thousands of participants in the exercise periodically all logged in during a few minutes to update themselves on their exercise status.

The issues at hand for SEMA was, amongst others, “can the site handle 5000 logged on users during the one and same minute?” and “how should we best scale up our web application and server environment to be able to avoid bottlenecks?”.

The technical supervisor with SEMA, Per Söderström, designed together with Apica a load test of the production environment to verify the maximum performance of the site. The test was designed to simulate a scenario where 1,000 up to 10,000 users logs on to the site during the one and same minute and gets information from a number of sub-pages. Test results were aggregated regarding:

- No. of active users

- Data from server-CPU and web server

- Response times for the scenario as well as URL’s

Test results and actions

The result of the first test was that the site without any kind of trimming could cope with approx. 4000 concurrent logged in users. The response times were however to long.

This result did not certify that the site could live up to the expected size of traffic. One of the actions taken which had an impact was to make the size of the landing page smaller. But this was not sufficient.

The solution was to put a separate front-end cache before SAMÖ:s web site. This meant that the ordinary web site was offloaded all static traffic. For larger web-sites this is usually called a CDN (Content Delivery Network).

But why cannot the ordinary cache on a typical web server handle this?

The answer is that it depends on which kind of cache is implemented in the web application. A separate front end cache typically often delivers much better performance than the cache built in an ordinary web server. It is also very important that the no of inquiries per second to the web server goes down drastically, since the cache treats all static content. The CPU load is thus drastically lowered.

It is however worth mentioning that even if the content is flagged as cached in a web server, the actual no of hits per second is a separate problem when you reach high volumes of inquiries to the site.

It is impossible to have a general opinion on how a web cluster/web server will handle high load. The only way to be certain is to load test the production environment of the system and analyze how the separate components of the site reacts.

A front-end cache based on Varnish gives a much better throughput for static content than an equivalent web server. The most simple way of explaining it is that design and structure in the code is optimized specifically for delivery of images and not to generate complex web pages or all the rest of the functionality that comes with a modern web server.

On a source code level, Varnish is optimized to deliver maximum data per instruction. That type of optimization is impossible to achieve on a conventional web server.

Samö				Without Varnish 4000 users
Test	AV Time	<= 90 %	AV Size	URL
[1]	1'888 ms	5'161 ms	15'126 bytes	GET http://www.samo2008.se:80/
[2]	637 ms	1'547 ms	852 bytes	GET http://www.samo2008.se:80/styles/samo.css
[3]	641 ms	1'624 ms	1'739 bytes	GET http://www.samo2008.se:80/styles/reset.css
[4]	654 ms	1'629 ms	2'635 bytes	GET http://www.samo2008.se:80/styles/specialpages.css
[5]	1'910 ms	4'811 ms	15'145 bytes	GET http://www.samo2008.se:80/styles/global.css
[6]	2'681 ms	6'725 ms	23'615 bytes	GET http://www.samo2008.se:80/scripts/DOMAssistant.js
[7]	661 ms	1'675 ms	1'735 bytes	GET http://www.samo2008.se:80/scripts/global.js
[8]	2'992 ms	7'476 ms	21'622 bytes	GET http://www.samo2008.se:80/WebResource.axd?d=1jbvyOWjrSv4Om
[9]	2'965 ms	7'572 ms	22'186 bytes	GET http://www.samo2008.se:80/WebResource.axd?d=eUGjkbaL2CxyowU
[10]	4'516 ms	10'915 ms	34'821 bytes	GET http://www.samo2008.se:80/util/javascript/episerverscriptmanager.js
[11]	694 ms	1'904 ms	1'954 bytes	GET http://www.samo2008.se:80/images/samo_logo.gif
[12]	625 ms	1'674 ms	873 bytes	GET http://www.samo2008.se:80/images/bg_mainnavigation_separator.png
[13]	642 ms	1'812 ms	789 bytes	GET http://www.samo2008.se:80/images/bg_menutopspacer.png
[14]	4'707 ms	11'381 ms	38'411 bytes	GET http://www.samo2008.se:80/images/top.png
[15]	590 ms	1'662 ms	824 bytes	GET http://www.samo2008.se:80/images/bg_mainnavigation.png
[16]	603 ms	1'669 ms	1'158 bytes	GET http://www.samo2008.se:80/images/bg_contentwrapper.png
[17]	607 ms	1'676 ms	776 bytes	GET http://www.samo2008.se:80/images/bg_footer.png
Total	28'013 ms	61'430 ms	184'261 bytes	17 URLs
Samö				With Varnish 4000 users
[1]	12 ms	9 ms	15'126 bytes	GET http://www.samo2008.se:80/
[2]	1 ms	2 ms	852 bytes	GET http://www.samo2008.se:80/styles/samo.css
[3]	1 ms	2 ms	1'739 bytes	GET http://www.samo2008.se:80/styles/reset.css
[4]	1 ms	2 ms	2'635 bytes	GET http://www.samo2008.se:80/styles/specialpages.css
[5]	3 ms	3 ms	15'145 bytes	GET http://www.samo2008.se:80/styles/global.css
[6]	3 ms	3 ms	23'615 bytes	GET http://www.samo2008.se:80/scripts/DOMAssistant.js
[7]	1 ms	2 ms	1'735 bytes	GET http://www.samo2008.se:80/scripts/global.js
[8]	5 ms	7 ms	21'622 bytes	GET http://www.samo2008.se:80/WebResource.axd?d=1jbvyOWjrSv4Om
[9]	5 ms	7 ms	22'186 bytes	GET http://www.samo2008.se:80/WebResource.axd?d=eUGjkbaL2CxyowU
[10]	3 ms	3 ms	34'821 bytes	GET http://www.samo2008.se:80/util/javascript/episerverscriptmanager.js
[11]	1 ms	2 ms	1'954 bytes	GET http://www.samo2008.se:80/images/samo_logo.gif
[12]	1 ms	2 ms	873 bytes	GET http://www.samo2008.se:80/images/bg_mainnavigation_separator.png
[13]	1 ms	2 ms	789 bytes	GET http://www.samo2008.se:80/images/bg_menutopspacer.png
[14]	3 ms	3 ms	38'411 bytes	GET http://www.samo2008.se:80/images/top.png
[15]	1 ms	2 ms	824 bytes	GET http://www.samo2008.se:80/images/bg_mainnavigation.png
[16]	1 ms	2 ms	1'158 bytes	GET http://www.samo2008.se:80/images/bg_contentwrapper.png
[17]	1 ms	2 ms	776 bytes	GET http://www.samo2008.se:80/images/bg_footer.png
Total	44 ms	47 ms	184'261 bytes	17 URLs

For more information on the case above, please contact:

Sven Hammar

CEO, Apica AB

Phone: 08 – 555 830 30

Mobile: 0735 - 05 90 68

E mail: sven.hammar@apicasystem.com

and

Per Söderström

IT-manager, KBM

Phone: 08 - 593 71 000

E-mail: Per.Soderstrom@kbm-sema.se

Cache and acceleration of web traffic

So called HTTP-acceleration efficiently quickens up your web traffic and in a cost efficient way. Apica offers a packaged solution based on Varnish (Apica Acceleration) which is the market leading HTTP-acceleration solution.

No difficult integration is necessary as Apica delivers a pre-configured solution on optimized hardware with tested Varnish parameters.

Apicas http acceleration is a complete solution. The hardware is delivered on a rental basis. Hardware guarantee as well as continuous upgrades of Varnish and operating systems is included. Apica verifies the installation with a load test and are able to guarantee a performance enhancement. Availability and 24/7monitoring of the server is included in the as an optional service, Apica also offers response time measurement with Apica Availability.

Front-End Cache

All web pages normally consist of dynamic and static content. Static content is defined as data which does not change or is made unique for each visitor. Images and style sheets (i e formats css, gif and jpg) are typical examples of static content. Static content is excellent for caching on a separate server, a so called http-accelerator

Of the total bandwidth a web page needs to be loaded by one single user, the static content typically uses well over 50 % of the bandwidth and CPU-capacity of the web server. The point of a front-end cache is to take load of “unnecessary” tasks of the web servers behind (that is, to serve static content) and thereby improve the performance – often increasing the performance by factor 5-10 compared to a standard installation.

Varnish

Varnish is a leading software for http-cache based on open source, with absolute top class performance. Varnish is today used by several leading newspapers’ web sites and other large sites, and has a rapidly growing community. Apica possesses expert competence in Varnish configuration.