[Mongrel] Transfer-Encoding: Chunked

Eden Li eden at mojiti.com
Mon Feb 18 19:36:43 EST 2008


Mind posting a patch?

As an aside, the j2me people seem to think CE requests are part the  
http 1.1 spec (rfc2616):

   http://developers.sun.com/mobility/midp/questions/chunking/

and the minimal justification they give is that "interactive  
applications may not know ahead of time how much data they're going to  
send" (paraphrase).

On Feb 19, 2008, at 4:44 AM, Todd Fisher wrote:

> I implemented a chunked transfer encoding to handle ESI in pages...  
> Each fragment is sent as a chunk... Since, I can't determine ahead  
> of time what the content-length would be as the individual ESI  
> fragments are of unknown length until the page is assembled...
>
>  I can see where a mobile device might want to do this because of  
> memory constraints, however.... if the device has what's it's  
> sending stored somewhere one would hope it can measure that  
> something's length, ahead of time - writing the Content-length  
> first... Then connect the write end of the socket to the read end of  
> the file...   and away it goes never storing more then the OS  
> decides to keep in memory...
>
> This page might be helpful... http://www.jmarshall.com/easy/http/#http1.1c2
>
>
>
> On Feb 18, 2008 7:39 AM, Zachary Roetemeyer <zprime at gmail.com> wrote:
> The upload is coming from a mobile device with J2ME, and it does this
> by default with anything over 2K, and apparently there's no way to
> turn it off.
>
> On Feb 17, 2008, at 8:18 PM, Zed A. Shaw wrote:
>
> > On Fri, 15 Feb 2008 14:35:04 -0600
> > Zachary Roetemeyer <zprime at gmail.com> wrote:
> >
> >> I'm working with someone who is having issues with a Transfer-
> >> Encoding: Chunked upload. With out a content-length set in the
> >> request, everything is treating the content length as 0 and the  
> file
> >> is more or less getting sent to the bit bucket.  Does anyone have  
> any
> >> experience with this kind of issue?  We started proxying with
> >> nginx(returned a HTTP Error 411), then we tried to directly  
> expose a
> >> mongrel and ended up getting strange results, and errors in the
> >> mongrel logs.  The client is uploading with a mobile phone using
> >> J2ME.
> >
> > Uh, you aren't allowed to set a Content-Length AND use CE in the  
> same
> > request.  That would let the two contradict so it's an RFC  
> violation.
> > Actually, WebLogic makes this mistake and we had to school them  
> about
> > it until they told us to turn it off.  (WebLogic is such a piece of
> > shit).
> >
> > My RFC knowledge on CE is rusty since I banished that shitty part of
> > HTTP from my mind when I implemented it in RFuzz, but my thoughts  
> are:
> >
> > 1) CE wasn't designed for clients to upload, since no browser does  
> it.
> > 2) It's originally for a server to send periodic parts of a request
> > without having to know it's total final size.
> > 3) Anyone doing a CE as a client is seriously screwing with the RFC
> > and
> > really exercising the corners of it for no apparent reason since CE
> > isn't needed by a client (the client knows the size, or better).
> > 4) This puts a burden on the server to process the CE as it comes
> > over, which has the same problems as mime boundaries but with a more
> > horribly designed part of the RFC.
> > 5) My recommendation is that you just don't do this.  The value add
> > for
> > a client (or even modern servers really) is quite minimal.
> >
> > Now, how you go about implementing it:
> >
> > A) Read up on how to write a Mongrel handler.
> > B) Register it at the front of your web app or create a separate
> > server
> > that runs just mongrel (probably your best if you want any kind of
> > performance).
> > C) Your handler would detect the CE via the normal headers and such,
> > and then process the final request body as it comes in for each  
> chunk.
> > D) This might involve subclassing the Http processing parts that
> > handle
> > the body and doing a bit of other hacking, and it's seriously  
> fucking
> > dumb because after that you've gotta process mime boundaries and  
> other
> > stuff, which will punish your server heavily.
> >
> > Enjoy!
> >
> > --
> > Zed A. Shaw
> > - Hate: http://savingtheinternetwithhate.com/
> > - Good: http://www.zedshaw.com/
> > - Evil: http://yearofevil.com/
> > _______________________________________________
> > Mongrel-users mailing list
> > Mongrel-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/mongrel-users
>
> Zachary Roetemeyer
> zprime at gmail.com
>
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users



More information about the Mongrel-users mailing list