[Mongrel] [WARNING] don't use ruby-sendfile

Zed Shaw zedshaw at zedshaw.com
Thu Aug 3 11:30:57 EDT 2006


On Thu, 2006-08-03 at 09:40 -0500, James Ludlow wrote:
> On 8/3/06, Zed Shaw <zedshaw at zedshaw.com> wrote:
<snip>
> Setting the performance issue aside for a moment, if I have a long
> download (i.e. streaming video), I'm going to tie up a mongrel process
> for one user for a relatively long time.  This isn't a crisis, because
> I can just spawn more, and I'm more constrained by the bandwidth at
> the switch than by memory or CPU time.
> 
I think you're misunderstanding the purpose of install ruby-sendfile.
Even *without* sendfile installed Mongrel won't block on sending a file.
It'll still chug away on multiple concurrent files without stopping
anything else.  There's a limit of course, but it's not locked like with
Rails.

What sendfile does is this:

http://www.die.net/doc/linux/man/man2/sendfile.2.html

It gives ruby the sendfile API.  Problem with the sendfile API is that's
it's implemented wildly differently on all the OS and even different
versions of the OS.  On some it blocks.  On others it can throw
EINTERRUPT.  On still others it doesn't even do zero-copy.

This is the main reason it causes instability in Mongrel.

> My use of the word "performance" was pretty vague and imprecise.  I'll
> try it without sendfile, but with more mongrel processes and see what
> happens.
> 
You shouldn't need more Mongrel processes.

<snip>
> I appreciate the warning.  With what seems like a sudden influx of
> stability problems and now this announcement about sendfile, I'm
> wondering if something else might be going on though.  I'm not saying
> that ruby-sendfile isn't broken, but maybe something else changed
> recently that exposed this bug.

No, what actually happened is Coda Hale put sendfile in a list of gems
to install.  Previously most folks weren't installing it.  Now that
they're following those docs they were just installing it arbitrarily
without understanding what it did.

With an increase in different people using sendfile for different apps I
started getting complaints about stability.  Telling them to remove
sendfile fixed all their problems.

Also, in every OS that I've tried it on the errors are related to
differences in how sendfile is implemented.  For example, FreeBSD was
aborting sendfile with EINTERRUPT and returning the amount sent
requiring that you send the remainder (which isn't very helpful).  It
was also doing this on exactly 32k boundaries, but only when the server
is overloaded.

Anyway, let me know how it works.


-- 
Zed A. Shaw
http://www.zedshaw.com/
http://mongrel.rubyforge.org/
http://www.railsmachine.com/ -- Need Mongrel support?



More information about the Mongrel-users mailing list