SMP and lighty

Overview

lighty is a single-threaded, single-process, event-based, non-blocking-IO web server. While this is perfect for the performance it doesn't scale very well over CPUs. In most cases you don't have to care about it as lighty will usually not be able to utilize the full CPU to fill the network-pipe. All the heavy jobs are moved to external applications (see mod_fastcgi).

But there are several cases where it makes sense to have several processes running and since 1.4.x we have support to spawn several processes to listen on a single socket:

server.max-worker = 8

CPU-Bound

It has been shown that lighty scales mostly linearly on multicore, multi-processor boxes if you set the number of workers equal to two times the number of cores if you are CPU bound (heavy rewrites, heavy proxying). Usually you don't have to care about CPU-bound processing.

IO-Bound

If you have to serve a lot of large files with a high concurrency, you will notice that you will be IO-bound to you hard-drive. lighty will spend most of the time waiting for the hard-drive to seek to the right sector to send out the file.

As lighty will wait for the disk, all connections in the process will have to wait. In order to process more requests concurrently, you should create 4 to 16 work processes. This will allow you to both send and wait for data at the same time.

See http://blog.lighttpd.net/articles/2005/11/11/optimizing-lighty-for-high-concurrent-large-file-downloads for more information.

NFS

Using NFS is basically the same as being IO-bound: most of the time is spent waiting on the remote system to deliver the content or to stat()/open() a file. In those cases you want to include the number of workers to spread the probability over several processes:

server.max-worker = 8

A value between 8 and 16 should handle most setups.

Problems

By running 2 or more processes on the same socket you will have a better concurrency, but will have a few drawbacks that you have to be aware of:

• mod_accesslog might create broken access logs, as the same file is opened twice and is NOT synchronized.
• mod_status will have <n> separate counters, one set for each process.
• mod_rrdtool will fail as it receives the same timestamp twice.