Request: mod_rewrite check if ile exists (like mod_rewrite in apache)

http://pixel.global-banlist.de/

see the mod_magnet articles there, especially http://pixel.global-banlist.de/2006/10/6/dr-magneto-vs-mr-404-handler .

this bug is a wont fix

I looked at the mod_magnet solution but that seemed to have a high performance hit. So I modified a copy of mod_rewrite to support a new pair of options: url.rewrite-repeat-NF and url.rewrite-once-NF. These only apply the rewrite if the file corresponding to the URI does NOT exist in the Filestore (NF = Not Filestore).

See here for more information: http://lmmrtech.com/lighttpd_mod_rewrite_with_file_exists_check

so your site is doing more than 4000-5000r/s?

thats the value i reached with a running a mod_magnet script.

i forgot to mention. the normal speed of the box is not much higher.

Not even remotely close to that! However I do hope that the load will grow dramatically and I want to be prepared. The mod_magnet+lua approach is clearly very flexible but it does involve a high performance hit according to the article mentioned above (http://pixel.global-banlist.de/2006/10/6/dr-magneto-vs-mr-404-handler). Since I don't need the extra flexibility and want to maximize the performance I think my patch the way to go.

Please remove this Spam! And by the way, is there something new for this topic?

i still dont think we need that patch. mod_magnet can easily handle that for you.

mod_magnet cannot handle it easily, maybe it can handle it, but i don't know how :( The problem is, that i can check if file exists with mod_magnet, but lighttpd cannot parse the lighttpd rewrite rules AFTER mod_magnet. So i check if file exists. If it exists, I return a 200 and provide the static page. But if it does not exists i dont know what to do, that lighttpd uses then the rewrite rules and send the php file to the fastcgi backend. There is no possibility to use lua between rewrite rules and fastcgi backend.

there is. load mod_rewrite before mod_magnet.

but I want only use the mod_rewrite IF the file does not exist :(

-> mod_magnet before mod_rewrite, and in the lua file two conditions 1. file exists -> return 200 2. file does not exist -> rewrite and parse php

use mod_magnet. We will provide more examples later.

is there a reason why this can't be put into lighttpd? http://lmmrtech.com/lighttpd_mod_rewrite_with_file_exists_check

seems like he did all the heavy lifting already. i know he posted it above, but it seems like this could be a great fix without requiring magnet.

why should we maintain 2 solutions if mod_magnet can do that and even more?

so is mod_rewrite being deprecated then? why keep it at all?

mod_magnet is a much more robust solution. it's like why have simple-vhost and evhost then? one is quick and dirty, the other is much more feature-rich.

i don't really want to learn lua or try to debug/figure out if it is meeting my needs 100% with a basic -f / -d apache emulation. there has been a lot of demand for this, and it seems like a couple lua scripts made but specific for people's installs. seems like if a few lines of code can be thrown in to mod_rewrite that get triggered only if the file does not exist, why not? it won't add much overhead *unless* that condition is true...

I agree, I don't want to take the performance hit of dropping into mod_magnet. I'd rather just use re-write rules directly. I know mod magnet is feature rich, and flexible but why force us to take a performance hit just to use one of its features?

seriously: please prove that mod_magnet is a performance problem for _you_.

I recommend the patch made by lmmrtech to be merged. It shouldn't make lighttpd too much bulkier.

It is a better option than embedding another interpreted language. I think 90% of the CMS's out there that need clean urls only need to check for file or directory existance, and do not need the extra fancy overhead of LUA.

http://lmmrtech.com/lighttpd_mod_rewrite_with_file_exists_check

Yes please integrate this in mod_rewrite - preferably also in the 1.4.x branch. May be, that LUA is a great language and mod_magnet is also really cool. But I (and I'm sure many others out there) don't need either to fulfill the (and my client's) demands.

Integration of this would make transition from Apache much easier.

mod_rewrite is voodoo, but anybody switching has this voodoo already worked out. It's annoying to have 13 lines of boilerplate lua code to get effect of a simple built-in flag!

And what exactly stops you for using rewrite-once line like this:

"^{/files.*\.(jpg|jpeg|gif|png|swf|avi|mpg|mpeg|mp3|flv|ico|css)$" => "$0"}

with the file extensions you need? It means rewrite the file to itself, exit rewrite.

Works just fine under medium load (single cpu box, joomla, wordpress, simple machines forums, ~80req/s) - no mentionable performance issues.

that doesn't check if the file exists or not.

that just does a rewrite based on the incoming URI. nothing more.

this is for a filesystem-level check if a file exists or not... two totally different things.

Replying to mike503:

that doesn't check if the file exists or not. that just does a rewrite based on the incoming URI. nothing more. this is for a filesystem-level check if a file exists or not... two totally different things.

That's right, but in 99% of cases 'if file exists' check is used to exclude some parts of the website from rewrite rule. The remaining 1% in most cases could be handled by setting 404 file.

I do know that neither of that solutions really provides feature that this issue was opened for, but it really narrows down the list of people who actually need it, thus priority trivial.

No, a mod_rewrite file check would provide the ideal solution.

Your solution is a poor one, and not future-proof. It requires you to define all the assets ahead of time that you want parsed normally. How do you know of all the file extensions that will be there?

Sure, you've taken care of the basics and most common, but if you allow user uploads or have anything else not included in that line, it will not process it correctly.

Simply put, it's not the solution to this problem and is the common misconception with all rewrite rules I keep seeing for lighty, nginx, and other webservers. People think that they need to call out the jpg/jpeg/gif/css/blah/blah separately and then they're covered, and most of the time it's a "solution" that does not truly work - your solution again does not check the filesystem at all; you can still get 404 errors on a .jpg file, because the rewrite rule you've made doesn't care if the file exists. It might "work fine" for you but it's not the solution that people are asking for, and some day if you happen to upload a file that is not matched in your regexp, you'll have that issue too.

This ticket is for a check if the file exists, the webserver needs to check the filesystem for it first. Period. Please stop changing the status.

This seems like a pretty basic feature which could be implemented (and has) with almost no performance hit in C, or done with mod_magnet as another commenter mentioned could handle mod_rewrite's job anyway. There are reasons for mod_rewrite's existance.

I can't believe this is such a debate for such a simple feature. Could we meet in the middle and at least add the patch as a compile-time option with configure?