Unicorn is killing our rainbows workers

[Eric Wong]

Samuel Kadolph <samuel.kadolph at shopify.com> wrote:
> On Thu, Jul 26, 2012 at 8:11 PM, Eric Wong <normalperson at yhbt.net> wrote:
> >> Our ops guys have been busy so I don't have the output from lsof but
> >> it didn't look like it was spawning any extra threads or opening any
> >> unexplainable connections. But I think we should have been checking
> >> the worker processes and not the master, right?
> >
> > Definitely check the master, too.  It's the master that seems to
> > believe it's suspended, so that makes me believe something is wrong
> > with the master (and this is likely due to preload_app).
> >
> >> Haven't tried disabling preload_app yet but we have tried
> 
> I've got the output of lsof and ls at https://gist.github.com/3190171.

Thanks, that's the output for the master?  I don't see anything
obviously wrong.

I seem to recall the Ruby library responsible for the following log file
also spawns its own background thread, but your "ls" only shows 2 tasks
(instead of 3):

> ruby    26564 root    9w   REG              202,1     51221   529742 APP_PATH/shared/log/newrelic_agent.log

> $ ls /proc/26564/task/
> 26564  27052

(While the Ruby code for the module responsible for that log file is
 technically "open", it's not Free, so I'm not comfortable looking at
 that code).