[Mongrel] garbage collection patch

Evan Weaver evan at cloudbur.st
Wed Feb 6 18:13:30 EST 2008


Some VSZ/RSS sizes for the front page of a large application:

Fresh caches, minimal AR usage, startup:
GC.enabled: 120200 66732
GC.disabled: 293492 239900

Fresh caches, one request.
GC.enabled: 127744 73360
GC.disabled: 345296 289616

All stale caches, one request:
GC.enabled: 528536 472840
GC.disabled: 1203112 1147276

This is actually not nearly as bad for GC.disabled as I expected.

Evan

On Feb 6, 2008 10:46 AM, Roger Pack <lists at ruby-forum.com> wrote:
> > Disabling GC around the requests would guarantee that the size of the
> > Mongrel process will balloon to the size of all objects combined in
> > the most heavyweight request.
>
> Definitely a trade-off between RAM and CPU.  I would say that those who
> are CPU not RAM bound would be most interested in it
>
>
> >  Remember that the Ruby heap never
> > returns space to the OS.
> (well it does if you're extremely lucky and a heap chunk happens to be
> entirely freed of its ruby objects, but, since the heap chunks are
> allocated in larger and larger blocks, this is unlikely and, as you
> noted, probably close to 'never')
> > As soon as you re-enable the GC, the entire
> > Ruby heap (now 4-5x bigger than it normally would be) will get paged
> > back in to physical RAM.
> Right.  No savings in terms of RAM being swapped out (except using the
> method you suggested below).  I believe the main advantage to GC'ing
> 'only every so often' or 'once per request'  would be not that you use
> less RAM, but that you use keep all that (bloated) RAM in memory and
> traverse it far less frequently.  So it "might" save CPU at the expense
> of RAM.  The overhead of mongrel +rails setup is (for me) around 40MB.
> So basically ruby, every 8MB of allocated memory, is traversing the 48MB
> of memory, which sets it down to 40MB.  So for it to create 50MB of
> memory (however many requests that is) it will traverse ~5*50MB memory =
> 250MB.  If you leave it to only GC after ~40MB have been allocated,  it
> traverses 100MB once (and sets it back down to 40).  As you noted it
> does use more RAM, and none of that RAM can healthfully reside in swap.
> Significant, at least for those with lots of RAM?   I don't know.
>
> > As I understand it, the point of disabling the GC is to allow part of
> > the heap to swap out. This has no benefit if you enable the GC
> > after--you have to disable the GC, Kernel.fork, run the request, let
> > the request thread die, and then re-enable the heap in the parent.
> That might be a quite useful RAM-wise--its like 'ignoring' the garbage
> generated with a request and deserves consideration.  Nice.
>
> > I'm doubtful that there is much benefit but it deserves some testing.
> >
> > Incidentally the Ruby heap (and the GC) should have nothing to do with
> > Imagemagick. Extensions that use malloc() are a totally different
> > scenario.
> IIRC when garbage collection begins it also requests the extensions to
> clean themselves up, too, though I'll admit I never saw or understood
> how this is accomplished within gc.c.  It's possible that it just called
> 'cleanup' on extensions' (now old) ruby allocated objects, which are
> linked to their own malloc'ed objects and know how to clean them up.
>
> > Evan
>
> Thanks Evan.
> -Roger
> --
>
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC


More information about the Mongrel-users mailing list