How to manage growing memory with Rainbows!

Eric Wong normalperson at yhbt.net
Thu Feb 14 07:15:12 UTC 2013


Claudio Poli <claudio at audiobox.fm> wrote:
> Il giorno 12/feb/2013, alle ore 06:00, Eric Wong <normalperson at yhbt.net> ha scritto:
> 
> Using LD_PRELOAD with libtcmalloc

Last I checked, tcmalloc never releases memory to the OS, so that
could be a problem.  (Giving memory back to the kernel and then
getting it back soon afterwards is slow because the kernel must
clear that memory, first, so most malloc implementations (including
glibc) will try to keep memory for the process)

> > You can try MALLOC_ARENA_MAX_=1 to limit the number of arenas if you
> > want.  That might reduce fragmentation since the GVL in MRI means
> > it's unlikely to hit malloc lock contention (glibc uses multiple
> > malloc arenas to avoid contention by default).
> 
> I didn't knew about this setting, might be worth a try, thanks.

You might also want to try MALLOC_MMAP_THRESHOLD_=131072 to disable the
sliding sbrk/mmap window (you can try larger/smaller values).  The
latest Linux man-pages releases have better mallopt(3) documentation
(but the arenas stuff was only documented on Dreppers blog, AFAIK (and
glibc.git/malloc/*.[ch] comments)

> > However, you can safely send SIGQUIT to any Rainbows! worker (bypassing
> > master) whenever you feel memory usage is high, master will restart it.
> 
> Will Rainbows! wait after the last request before restarting?

Yes, SIGQUIT => graceful shutdown will always work for normal apps.
The only exception is the new, optional rack.hijack (in rainbows.git).

> What Rainbows! strategy would you run on 1.9.3 given that some API call might take 800ms/1200ms (uncached) and the number of requests is fairly high?

It depends on your app.  For standard Ruby API (Net::*),
ThreadSpawn/Pool should be the most compatible and work well enough.
These will work great if you're using Celluloid, too.

EventMachine can scale very well but it requires your network-capable
gems/libraries to use EM to benefit.  Same issue with cool.io.

I also like XEpollThread{Pool/Spawn}, but they're Linux-only and
less-used.


More information about the rainbows-talk mailing list