Ticket #758 (new defect)

Opened 2 years ago

Last modified 1 week ago

memory leak

Reported by: pts@inf.bme.hu Assigned to: jan
Priority: normal Milestone:
Component: core Version: 1.4.19
Severity: normal Keywords:
Cc: Blocking:
Need Feedback: 1

Description

lighttpd-1.4.11 (Jul 12 2006 20:07:41) - a light and fast webserver has memory leak -- about 60 megabytes accumulating each day. 

server.modules = (
 "mod_rewrite",
 "mod_redirect",
 "mod_alias",
 "mod_access",
 "mod_cml",
 "mod_auth",
 "mod_evasive",
 "mod_status",
 "mod_setenv",
 "mod_fastcgi",
 "mod_proxy",
 "mod_simple_vhost",
 "mod_evhost",
 "mod_userdir",
 "mod_cgi",
 "mod_scgi",
 "mod_compress",
 "mod_ssi",
 "mod_flv_streaming",
 "mod_webdav",
 "mod_usertrack",
 "mod_expire",
 "mod_secdownload",
 "mod_rrdtool",
 "mod_accesslog" )
rrdtool.binary="/usr/bin/rrdtool"
rrdtool.db-name="/opt/lighttpd/lighttpd.rrd"
server.document-root        = "/opt/lighttpd/root/"
allow-x-send-file="enable"
webdav.activate="disable"
webdav.is-readonly="enable"
index-file.names            = ( "index.php", "index.html",
                                "index.htm", "default.htm" )
mimetype.assign             = (
  ".pdf"          =>      "application/pdf",
  ".sig"          =>      "application/pgp-signature",
  ".spl"          =>      "application/futuresplash",
  ".class"        =>      "application/octet-stream",
  ".ps"           =>      "application/postscript",
  ".torrent"      =>      "application/x-bittorrent",
  ".dvi"          =>      "application/x-dvi",
  ".gz"           =>      "application/x-gzip",
  ".pac"          =>      "application/x-ns-proxy-autoconfig",
  ".swf"          =>      "application/x-shockwave-flash",
  ".tar.gz"       =>      "application/x-tgz",
  ".tgz"          =>      "application/x-tgz",
  ".tar"          =>      "application/x-tar",
  ".zip"          =>      "application/zip",
  ".mp3"          =>      "audio/mpeg",
  ".m3u"          =>      "audio/x-mpegurl",
  ".wma"          =>      "audio/x-ms-wma",
  ".wax"          =>      "audio/x-ms-wax",
  ".ogg"          =>      "application/ogg",
  ".wav"          =>      "audio/x-wav",
  ".gif"          =>      "image/gif",
  ".jpg"          =>      "image/jpeg",
  ".jpeg"         =>      "image/jpeg",
  ".png"          =>      "image/png",
  ".xbm"          =>      "image/x-xbitmap",
  ".xpm"          =>      "image/x-xpixmap",
  ".xwd"          =>      "image/x-xwindowdump",
  ".css"          =>      "text/css",
  ".html"         =>      "text/html",
  ".htm"          =>      "text/html",
  ".js"           =>      "text/javascript",
  ".asc"          =>      "text/plain",
  ".c"            =>      "text/plain",
  ".cpp"          =>      "text/plain",
  ".log"          =>      "text/plain",
  ".conf"         =>      "text/plain",
  ".text"         =>      "text/plain",
  ".txt"          =>      "text/plain",
  ".dtd"          =>      "text/xml",
  ".xml"          =>      "text/xml",
  ".mpeg"         =>      "video/mpeg",
  ".mpg"          =>      "video/mpeg",
  ".mov"          =>      "video/quicktime",
  ".qt"           =>      "video/quicktime",
  ".avi"          =>      "video/x-msvideo",
  ".asf"          =>      "video/x-ms-asf",
  ".asx"          =>      "video/x-ms-asf",
  ".wmv"          =>      "video/x-ms-wmv",
  ".bz2"          =>      "application/x-bzip",
  ".tbz"          =>      "application/x-bzip-compressed-tar",
  ".tar.bz2"      =>      "application/x-bzip-compressed-tar"
 )
server.tag                 = "httpd"
accesslog.filename          = "/opt/lighttpd/access.log"
server.errorlog             = "/opt/lighttpd/error.log"
url.access-deny             = ( "~", ".inc" )
$HTTP["url"] =~ "\.pdf$" {
  server.range-requests = "disable"
}
$HTTP["remoteip"] != "1.2.3.4" {
  url.access-deny = ( "" )
}
static-file.exclude-extensions = ( ".php", ".pl", ".fcgi" )
server.port                = 1234
dir-listing.activate       = "enable"
fastcgi.server = ( ".php" => (( 
                     "bin-path" => "/opt/lighttpd/php5-cgi",
                     "socket" => "/opt/lighttpd/php5-fastcgi.socket",
                     # vvv Dat: multiply max-procs and PHP_FCGI_CHILDREN
                     "max-procs" => 1,
                     "bin-environment" => ( 
                       "PHP_FCGI_CHILDREN" => "4",
                       "PHP_FCGI_MAX_REQUESTS" => "10000"
                     ),
                     "bin-copy-environment" => (
                       "PATH", "SHELL", "USER"
                     ),
                     "broken-scriptfilename" => "enable",
                     "idle-timeout" => 20,
                 )))

Attachments

Change History

08/05/2006 01:16:43 AM changed by bob@redivi.com

I can confirm that this is the case, using the FreeBSD port. In my test, it grew faster than that (60M in a couple hours); it seemed to grow linearly with traffic.

This bug in particular was a deal breaker for me. I was trying lighttpd out as an alternative to pound. Performance is good, and memory usage is good when it starts, but it's simply too broken for me to actually use. Maybe I'll check back in a few months.

(follow-up: ↓ 7 ) 08/25/2006 09:21:12 AM changed by anonymous

Lighty memory usage seems to grow linearly with traffic on my Linux boxes as well. After the typical high traffic evening the process has grown from 11 MB to 450 MB. Kill+restart fixes the problem :-/

(in reply to: ↑ 4 ) 09/02/2006 07:32:23 PM changed by msolo

Replying to anonymous:

Lighty memory usage seems to grow linearly with traffic on my Linux boxes as well. After the typical high traffic evening the process has grown from 11 MB to 450 MB. Kill+restart fixes the problem :-/

This is most likely not a true leak - this is probably a symptom of the internal buffer reuse. If you change BUFFER_MAX_REUSE_SIZE to zero in settings.h, all memory will be free'd.

03/27/2007 02:27:37 PM changed by alex

Can anybody confirm the fix by setting BUFFER_MAX_REUSE_SIZE = 0? Is this ticket abandoned?

09/14/2007 12:24:14 PM changed by Dmitryus

  • blocking changed.
  • pending changed.

BUFFER_MAX_REUSE_SIZE=0 not fix problem. I'm using lighttpd 1.4.17 mod_status + mod_proxy + mod_access + mod_secure_download - for high load - download server - memory leaks 500mb every day.

10/11/2007 02:48:01 PM changed by frostyplanet@gmail.com

  • version changed from 1.4.11 to 1.4.18.

I use lighttpd(1.4.18) + mod_proxy with squid in front of another web server, memory leak of lighttpd process cause my 1G ram and 1G swap space consumed in a couple of hours.

(follow-up: ↓ 12 ) 10/30/2007 06:52:24 PM changed by mark@meebo.com

People experiencing the memory growth: Do you use https?

(in reply to: ↑ 11 ; follow-up: ↓ 13 ) 11/15/2007 07:12:19 PM changed by anonymous

Replying to mark@meebo.com:

People experiencing the memory growth: Do you use https?

Same problem. Not using https.

(in reply to: ↑ 12 ) 11/19/2007 08:35:55 AM changed by anonymous

  • priority changed from normal to high.

Replying to anonymous:

Replying to mark@meebo.com:

People experiencing the memory growth: Do you use https?

Same problem. Not using https.

Here too - no https.

11/19/2007 04:38:56 PM changed by Linux

  • priority changed from high to normal.

[Here too - no https.] << My comment ( forgot change name )

Settin

" BUFFER_MAX_REUSE_SIZE= 0 server.max-request-size = 300000 server.max-keep-alive-requests = 10 server.max-keep-alive-idle = 5 " In the server config helps me a lot ;)

11/25/2007 12:04:57 AM changed by logikcoder

Experiencing the same here. I have to restart it every night or it will get over 100mb.

Currently using: mod_rewrite, mod_redirect, mod_access, mod_fastcgi, mod_simple_vhost, mod_cgi, mod_accesslog, mod_geoip.

server.max-keep-alive-requests = 128, server.max-keep-alive-idle = 30, server.max-read-idle = 60, server.max-write-idle = 240,

12/08/2007 01:03:35 PM changed by Linux

  • priority changed from normal to high.

Constantly i got the same problem. I've changed server and nothing.

lighttpd 22494 0.0 17.4 97508 84352 ? S Dec04 0:05 /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf (512RAM)

Help please.

12/10/2007 02:04:00 AM changed by frostyplanet@gmail.com

  • component changed from core to build_autotools.

Sorry, my previous description may not be accury. I run lighttpd 1.4.18, fastcgi and php5, with Website providing download service using php's readfile() method. Memory usage of process lighttpd growed rapidly beyond 1G in half an hour and later used out swap. If put another server lighttpd + mod_proxy + squid to cache access of my website, both merchine suffered memory growth. Latter I found removing readfile() method in the download service php page can solve the problem :P

12/10/2007 02:27:35 AM changed by frostyplanet@gmail.com

  • component changed from build_autotools to core.

Sorry again for my poor English and careless.In my previous post "merchine" should be "machines","accury" should be "accurate". And my servers are gentoo linux, kernel 2.6.22, php 5.2.14-p20070914-r2, lighttpd 1.4.18 only with mod_fastcgi and mod_accesslog. I have tried Apache to make sure php library and webpage are OK.

(follow-up: ↓ 26 ) 12/10/2007 11:55:34 AM changed by darix

if lighttpd has access to the files you want to send to the client, use X-LIGHTTPD-send-file. see the mod_fastcgi documentation.

12/10/2007 12:23:30 PM changed by Linux

My config is:

Lighttpd 1.4.18 (installed from .deb package) PHP 5 (Static, in use (exec/read/system/fopen etc/..) Mysql (Only installed, not used)

Should i try compile lighttpd?

12/10/2007 12:37:58 PM changed by darix

no. it wouldnt change anything.

12/10/2007 07:43:29 PM changed by Linux

Oops.. Right now 60% Memory usage by the lighttpd process, load going up. Swap getting fulled, damn.

01/03/2008 08:46:43 PM changed by István

Hi,

I have similar problem. I have a lighttpd server, which used for proxy task. All requests to port 80 served by a background apache server.

Internet -> lighttpd -proxy-> apache

If several downloads (some CD ISO images etc.) started and running, the lighttpd eats all the memory and swap space and stops working.

I use this instance of lighttpd only for proxy. Should I use squid instead? :/

01/29/2008 05:44:07 PM changed by marcello100

Are you going to repair that memory leak in 1.5.0 ? is this leak also in 1.5beta ? Lighty is great,but this leak sucks :(

(in reply to: ↑ 20 ) 02/01/2008 04:43:46 PM changed by frostyplanet@gmail.com

Replying to darix:

if lighttpd has access to the files you want to send to the client, use X-LIGHTTPD-send-file. see the mod_fastcgi documentation.

I tried. It doesnt't help. By adding this option the lighttpd leaks severly, while its mem usage is normal without this option.

02/05/2008 12:30:35 PM changed by Olaf van der Spek

I'm running into the same issue on one Debian Etch server.

Uptime 2 days 6 hours 47 min 44 s Requests 2 Mreq Traffic 50.69 Gbyte

lighttpd 1.4.13 Server-Features RegEx? Conditionals enabled Network Engine fd-Event-Handler poll Config-File-Settings Loaded Modules indexfile access alias accesslog rewrite redirect status fastcgi simple_vhost dirlisting staticfile

02/05/2008 12:31:36 PM changed by Olaf van der Spek

Forgot to add the actual memory usage: from top: VIRT: 935m, RES = 828m.

02/06/2008 08:41:42 PM changed by Olaf van der Spek

At the moment we're at 1612m virt, 1.1g res.

02/16/2008 03:33:00 PM changed by stbuehler

  • pending set to 1.

Please use valgrind to see whether and where memory leaks.
See http://trac.lighttpd.net/trac/wiki/HowToReportABug

02/28/2008 07:05:31 AM changed by masryalex@gmail.com

I have the same problem but php-cgi process are the reason for memory leak

03/07/2008 08:52:34 AM changed by anonymous

Haven't seen increasing memory usage on FreeBSD 6.2-RELEASE, via the lighttpd port.

Running a 1.4 million page view / month site, serving static media and thumbnailing via cgi (Django application)

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND

94485 www 1 4 0 16404K 3820K kqread 2 62:12 0.00% lighttpd

That's 62 hours of CPU time on a box with 136 days uptime.

Also haven't seen this issue on the same platform serving 900'000 page views / month via php-fastcgi backend, lighttpd only.

03/24/2008 12:08:27 PM changed by anonymous

Wood Stair Railings: The beauty of this material is that they can be carved into any shape,painted or stained any color, andwill compliment a wood or composite deck beautifully. Metal Stair Railings: There are many different types of metalused to make balusters and handrails.

05/02/2008 04:47:33 AM changed by anonymous

  • version changed from 1.4.18 to 1.4.19.

05/02/2008 06:10:32 AM changed by stbuehler

  • priority changed from high to normal.
  • severity changed from major to normal.

It has been now more than two months that i asked for a valgrind log, and no one even responded to that => dropping Priority/Severity to normal.

And I think most "leaks" reported here are just high memory usages due to big files sent from a backend (fastcgi/proxy); it is known that lighty caches the complete response in memory from the backend as fast as possible, and that memory is not freed but reused later.

05/07/2008 07:26:43 AM changed by anonymous

Dunno, might want to take a look at this, and contact the original poster: http://trac.lighttpd.net/trac/ticket/1642

Add/Change #758 (memory leak)




Change Properties