[Mongrel] Memory usage issues

Jason Burks jburks725 at gmail.com
Thu Jun 29 08:14:00 EDT 2006


Hello all,

I'm new to Mongrel and am very impressed with what I see so far.  This
thing beats FastCGI hands down, and is even simpler to use than SCGI.
Great job Zed!  Thanks for all the hard work.

I am a bit surprised at the memory usage patterns though.  I'm not
sure if I'm doing something wrong (most likely I am), or if this is a
problem with the app I'm running (Typo trunk), but after the initial
mongrel_rails startup, each server uses about 22MB of memory.
However, after serving pages for a day or so, that memory usage will
have increased from 22MB to ~40MB.  If I leave the server running, it
just keeps climbing.  My memory usage with FastCGI was pretty stable
at about 22-24MB per dispatcher, regardless of how long they'd been
running.  I'm far from a high-traffic site, so this is a bit
unexpected.

I've tried this both in a single-server setup and using
mongrel_cluster, and the results are similar.  I'm running on Debian
testing behind LiteSpeed web server.  I keep up to date with Typo
trunk, and am using the latest gems for mongrel, mongrel_cluster, and
sendfile.  I tried removing sendfile, and that slowed things down, but
didn't improve the memory consumption, so I put it back in.

I've used both httperf and ab to test the app server, and they both
show the memory consumption problem.  I've done the testing both
through LiteSpeed as the proxy and by connecting directly to the
port(s) mongrel is running on.  When starting mongrel_rails, I use -e
production, and the cluster setup also specifies production for the
environment.  Just to be doubly sure, I also set RAILS_ENV to
production on the command line and force it to production in
config/environment.rb as well.  None of these things seems to help.

Some system information:

burks at cracked:~$ uname -a
Linux www.jasonburks.com 2.6.16.13-xenU #1 SMP Sat May 13 14:18:58 EDT
2006 i686 GNU/Linux

burks at cracked:~$ ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

burks at cracked:~$ gem list

*** LOCAL GEMS ***

actionmailer (1.2.1, 1.2.0, 1.1.5, 1.1.3)
    Service layer for easy email delivery and testing.

actionpack (1.12.1, 1.12.0, 1.11.2, 1.11.0)
    Web-flow and rendering framework putting the VC in MVC.

actionwebservice (1.1.2, 1.1.0, 1.0.0, 0.9.3)
    Web service support for Action Pack.

activerecord (1.14.2, 1.14.0, 1.13.2, 1.13.0)
    Implements the ActiveRecord pattern for ORM.

activesupport (1.3.1, 1.3.0, 1.2.5, 1.2.3)
    Support and utility classes used by the Rails framework.

cmdparse (2.0.2, 2.0.0)
    Advanced command line parser supporting commands

daemons (0.4.4)
    A toolkit to create and control daemons in different ways

fcgi (0.8.6.1)
    FastCGI ruby binding.

gem_plugin (0.2.1)
    A plugin system based only on rubygems that uses dependencies only

highline (1.2.1, 1.2.0, 1.0.1)
    HighLine is a high-level command-line IO library.

mongrel (0.3.13.2, 0.3.13.1, 0.3.13)
    A small fast HTTP library and server that runs Rails, Camping, and
    Nitro apps.

mongrel_cluster (0.2.0)
    Mongrel plugin that provides commands and Capistrano tasks for
    managing multiple Mongrel processes.

rails (1.1.2, 1.1.0, 1.0.0, 0.14.3)
    Web-application framework with template engine, control-flow layer,
    and ORM.

rake (0.7.1, 0.6.2)
    Ruby based make-like utility.

sendfile (0.9.2)
    Ruby interface to sendfile(2) system call

sources (0.0.1)
    This package provides download sources for remote gem installation

Now, if I'm not mistaken, Typo trunk uses it's own version of Rails as
an external resource.  I checked, and I'm at svn 4511 of Rails in
vendor/rails in my Typo install.  My cluster config looks like this:

burks at cracked:/var/www/typo/htdocs$ cat config/mongrel_cluster.yml
---
user: nobody
port: "8000"
environment: production
group: www-edit
address: 127.0.0.1
pid_file: log/mongrel.pid
servers: 2

I'm really not sure what else I should be looking at to try to solve
the memory issues.  I can certainly set up a cron job to restart the
service every day, but I shouldn't think this is necessary.  The
following message (with different action names) shows up a few times
in log/production.log:

ActionController::UnknownAction (No action responded to 2005):
    /vendor/rails/actionpack/lib/action_controller/filters.rb:368:in
`perform_action_without_benchmark'
    /vendor/rails/actionpack/lib/action_controller/benchmarking.rb:69:in
`perform_action_without_rescue'
    /usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
    /vendor/rails/actionpack/lib/action_controller/benchmarking.rb:69:in
`perform_action_without_rescue'
    /vendor/rails/actionpack/lib/action_controller/rescue.rb:82:in
`perform_action'
    /vendor/rails/actionpack/lib/action_controller/base.rb:381:in
`process_without_filters'
    /vendor/rails/actionpack/lib/action_controller/filters.rb:377:in
`process_without_session_management_support'
    /vendor/rails/actionpack/lib/action_controller/session_management.rb:117:in
`process'
    /vendor/rails/railties/lib/dispatcher.rb:38:in `dispatch'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel/rails.rb:85:in
`process'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:563:in
`process_client'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:562:in
`process_client'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:648:in `run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:648:in `run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:637:in `run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:969:in `run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel.rb:968:in `run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/bin/mongrel_rails:119:in `run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/lib/mongrel/command.rb:211:in
`run'
    /usr/lib/ruby/gems/1.8/gems/mongrel-0.3.13.2/bin/mongrel_rails:227
    /usr/bin/mongrel_rails:18

log/mongrel.log shows the following a whole bunch (94,000+ times in
the last week):

Thu Jun 29 02:53:07 EDT 2006: BAD CLIENT (127.0.0.1): Invalid HTTP
format, parsing fails.

I haven't turned on debug mode yet because I wasn't sure what sort of
info might be helpful for troubleshooting this.  Google didn't seem to
yield much in the way of similar problems, and I'm far from a Ruby or
Rails expert, so I'm not really sure how to proceed.  Any guidance or
suggestions or requests for additional debugging information to assist
me in stabilizing my memory usage would be greatly appreciated.

Again, Zed and others, great work!  I'm really impressed with Mongrel
so far.  Please help me to keep my doggie lean and happy...

Regards,
Jason Burks


More information about the Mongrel-users mailing list