[Mongrel] Re-map Portion of Ruby Heap Holding Optree to Save Child Server Memory?

Randy Parker randy.j.parker at gmail.com
Wed Feb 6 10:30:59 EST 2008


I wonder if it is possible to reduce the memory footprint of a pack of 6
mongrels by about 70 meg.

In reading Hongli's blog about his revisions to Ruby GC (last Monday Matz if
he could use Hongli's work), I was wondering at the size (in RSS) of all the
mongrels in a cluster.  It has always seemed just too big for my C-binary
intuition.  Why is so much of the memory of the parent not shared by the
children?  In other words, why are all the children so bloated?  Obviously,
something prevents Copy-On-Right from being applied to far more of the
memory than I expect.  (I expect the stack to be private, and a bit of
process-malloc'ed heap).
The pmap of a running mongrel says that by far the bulk of the memory
immediately following launch is mapped anon / private for
libruby.so,  23M
libnkf.so      6M for kanji
libsyck.so   7M for yaml

I guess that is because Ruby is a dynamic language, and always allows for a
running proc to modify the optree itself ('monkey patching').  So, in
contrast to a compiled binary where the program code is all in the text seg,
Ruby keeps the 'partly compiled bytecode' optree in private heap, where each
proc can mess with it at any time.

My question is, how important is that privacy?  I know lots of modules
monkey patch as they load, but I can't think of any cases where they modify
the optree at runtime.  I'm sure there are some, but do they happen in 99%
of real-world rails apps?  I must be overlooking some common cases...?

But if runtime optree modification is not always required, would it be
reasonable to have a flag that would re-mmap() the optree portions of the
heap to 'read-only' prior to the forking of child servers?  The payoff would
be that the read-only parts (which may be a couple of dozen meg) could be
treated as COW pages, saving them from being copied into each mongrel in a
cluster.  It just annoys me to think that all these mongrels are burdening
the system by lugging around useless private copies of the exact same
optree!

One of the arguments for threading Ruby and Rails is that server procs are
so big. DHH has argued that proc-granularity is vastly simpler and safer
(bug-wise), and I agree.  But I would like to reduce the seeming bloat of a
pack of mongrels.

Another unrelated thought is that I've never written a rails app that used
Kanji, so why can't I compile a version of mongrel without NKF?  Why don't
most Westerners run app servers without Kanji?  The trend in Rails is
towards plug-ins for everything except the genuine core, why not apply the
same philosophy to app servers?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080206/adac6421/attachment.html 


More information about the Mongrel-users mailing list