[Mongrel] mongrel memory usage ballooning and process stomping
Zed A. Shaw
zedshaw at zedshaw.com
Thu Jan 18 19:07:27 EST 2007
On Thu, 18 Jan 2007 14:28:50 -0600
"Joey Geiger" <jgeiger at gmail.com> wrote:
> Is mongrel not releasing the memory used for these operations? I need
> to use send_file to stream some static images (4k max) (approved,
> declined, etc) which rails then turns into new cached images. This
> allows a group of people to get the current status of an item via html
> email with images. It's a neat hack, but I'd like to make it better.
> >But mongrel will just buffer these chunks into a StringIO
> >until rails is completely finished and then mongrel will send the
> >entire thing to the client. This means that every time you stream
> >data like this, the entire image has to be loaded into ram in mongrel
> >before it will send it to the client.
No, first off, doing it this way is incredibly bad simply because you're holding up rails while you send the files. No matter how big the size this is a waste. Look at using X-Sendfile headers to get the web server to do this for you.
Second, "mongrel" seems to be said in place of "whatever ruby stuff i'm currently running". I think this comes from people who are used to saying "Apache" when they mean "all my php stuff" or "tomcat" when they mean "all my java stuff". In those environments the whole stack is also mostly the web server and the web servers are gigantic usually in comparison to whatever you wrote. In the case of Mongrel, the "web server" is just a very tiny little piece of the whole stack. It's literally like %1 of the total code in the system. To really start thinking clearly about what you're using you have to start thinking in terms of "mongrel+rails+[my included libs]" to find your problems. This is also true in the PHP and Java worlds, but it's not as important.
Third, Remember, *LOTS* of people run mongrel without memory leaks. I'd put out that there's easily thousands of mongrels running without leaks. If *YOU* are seeing leaks *YOU* are doing something wrong. Sorry to be harsh, but first thing people jump at is Mongrel and then we find out they're doing 10-20 really dumb things they should check on first. Always assume that what you've got sucks, then after you've ruled out all possible potential problems with what you've written then start looking at the "m+r+[mil]" stack as potential causes.
Finally, if you're doing the image generation and sending via a consistent URL, it wouldn't be hard to write a Mongrel handler that dealt with this work without touching rails at all. Take a look at lib/mongrel/handlers.rb for tons of examples.
Hope that helps.
Zed A. Shaw, MUDCRAP-CE Master Black Belt Sifu
http://www.awprofessional.com/title/0321483502 -- The Mongrel Book
http://www.lingr.com/room/3yXhqKbfPy8 -- Come get help.
More information about the Mongrel-users