This has been observed with versions 1.4.5, 1.4.8 and the latest SVN as of 8. Jan 2006
Kernel 2.6, debian sarge, the machine has 2GB RAM and CPU load is usually around 60%
Lighttpd runs with the usual small memory footprint for an apparently random amount of time. Then the lighttpd process grows to a massive size - most often up to 1.4 GB. This growth starts suddenly, and the process often reaches 1.4GB within a few seconds. It's not something that happens slowly.
At this point, it may or may not keep serving pages for some time. After a while though, lighttpd just "stops" serving pages. Sometimes, after another dozen minutes or so, lighttpd TERMINATES without any log output. The php processes spawned by fastcgi always survive. When lighttpd has stopped serving pages but not terminated itself yet, a kill -9 is required to get rid of the process.
CPU usage remains nominal except for the ensuing swapping, of course.
Request logs shows no anomalies. Error log shows no strange messages at all, except for a few send idle timeouts. This growth can happen without any error log messages at all though. Debug logs show no anomalies. I tried disabling logging, tried all sorts of schedulers, tried changing fcgi settings, nothing helped.
Sometimes, lighttpd grows to 10MB, 450MB or 800MB in memory within a few seconds and then stops growing for several hours, and keeps running normally for some time. However, lighttpd never runs for longer than 12 hours before it eventually continues to grow to 1.4GB, stops serving pages, and then dies.
My workaround consists of a cronjob that tries to access a php page on that same machine, and restarts lighttpd if the page doesn't come up within 10 seconds. This job
top - 21:33:07 up 6:01, 3 users, load average: 19.68, 9.56, 4.12
Tasks: 146 total, 3 running, 143 sleeping, 0 stopped, 0 zombie
Cpu(s): 55.6% us, 35.5% sy, 0.0% ni, 0.0% id, 7.2% wa, 0.3% hi,
1.3% si
Mem: 2054544k total, 2047504k used, 7040k free, 4460k buffers
Swap: 9839804k total, 2816312k used, 7023492k free, 202712k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
25414 www-data 16 0 3871m 1.3g 1856 D 4.0 67.8 0:25.83 lighttpd
2939 mysql 16 0 262m 33m 6552 S 0.0 1.7 0:01.56 mysqld
2296 root 18 0 27904 17m 4816 S 0.0 0.8 0:03.11 spamd
25444 www-data 15 0 13620 10m 4180 S 1.1 0.5 0:08.17 php
25427 www-data 15 0 12804 9.9m 4180 S 0.0 0.5 0:08.97 php
25457 www-data 15 0 12768 9.9m 4180 S 1.1 0.5 0:09.79 php
