Bug #1717

Google's urlfetch from appEngine and lighttpd HTTP 400 response

Added by Anonymous 180 days ago. Updated 73 days ago.

Status:New Start:
Priority:Normal Due date:
Assigned to:jan % Done:

0%
Category:core
Target version:1.5.0
Pending:

Yes

 Resolution:
Patch available:

Description

Hi,
I started using urlfetch from google's appEngine (http://code.google.com/appengine/docs/urlfetch/overview.html) and I noticed that it doesn't work for some sites powered by lighttpd. I tried the example given in the link above but replaced the URL with lighttpd.net:


from google.appengine.api import urlfetch

url = "http://www.lighttpd.net/" 
result = urlfetch.fetch(url)
if result.status_code == 200:
  self.response.out.write(result.content)
else:
  self.response.out.write(result.status_code)

I always get back a 400 (Bad Request) response in return. I tried with youtube.com with the same result. I am rather new with Python so it maybe is something I don't do right.

I also tried this on a local lighttpd 1.5 instance with the same result. Here's the debug trace I see:


configfile-glue.c.500: (trace) === start of 1 condition block ===
configfile-glue.c.258: (trace) is condition [3] (global/HTTPhost=~(^|\.)local.com[:0-9]*$) already valid ? nej
configfile-glue.c.521: (trace) [1] result: unknown
2008-07-11 01:55:01: (response.c.137) Response-Header:
HTTP/1.1 400 Bad Request
Content-Type: text/html
Content-Length: 349
Connection: close
Date: Fri, 11 Jul 2008 01:55:01 GMT
Server: lighttpd

Please help!
florin

-- florin

History

07/30/2008 09:05 PM - stbuehler

The request would be the interesting thing.

08/02/2008 11:54 AM - stbuehler

The framework sends a "Content-Length: 0" header in a GET request:

rfc 2616, 4.3:


   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers. A message-body MUST NOT be included in
   a request if the specification of the request method (section 5.1.1)
   does not allow sending an entity-body in requests. A server SHOULD
   read and forward a message-body on any request; if the request method
   does not include defined semantics for an entity-body, then the
   message-body SHOULD be ignored when handling the request.

So i guess google shouldn't send the content-length header in that case.

10/26/2008 10:46 PM - ralf

stbuehler wrote:

The framework sends a "Content-Length: 0" header in a GET request:

rfc 2616, 4.3:

[...]

So i guess google shouldn't send the content-length header in that case.

hehe, the words of a developer.

i think lighttpd should read 0 bytes (as lighttpd do if there is no content-length header).

Every (ok, the mostly used) webserver - Apache - works with the request.

Also available in: Atom PDF