Strange quit behavior
ajsharp at gmail.com
Tue Aug 23 12:49:23 EDT 2011
On Tuesday, August 23, 2011 at 12:12 AM, Eric Wong wrote:
> Did you send any signals to 18862 while you were making this strace?
Not while, no. I had already sent a USR2 signal to this process (the old master), and then I attached strace to it.
I'll try sending another USR2 signal next time while strace is attached.
> > I went ahead and ran strace with the same flags on the *new* master,
> > and saw a bunch of output that looked bundler-related:
> > https://gist.github.com/138344b5b19ec6ba1a4c
> > Even more bizarre, eventually the process started successfully :-/ Is
> > it possible this had something to do with strace de-taching?
> That looks like pretty normal "require" behavior. strace would slow
> down requires, a lot.
> So this was with preload_app=true? While you're debugging problems,
> I suggest keeping preload_app=false and worker_problems=1 to minimize
> the variables.
Ok, I'll change those and report back. I'm guessing you meant worker_processes (not problems)?
> > You can see this in the unicorn.stderr.log file I included in the
> > gist. Check out these two lines in particular, which occur 25 minutes
> > apart:
> > I, [2011-08-23T02:15:08.396868 #22169] INFO -- : Refreshing Gem list
> > I, [2011-08-23T02:40:16.621210 #22925] INFO -- : worker=1 spawned pid=22925
> Wow, it takes 25 minutes to load your application? strace makes the
> application /much/ slower, so I can actually believe it takes that long.
No, my mistake. Loading the application only takes about 10 seconds, and I only had strace attached to this process for a few seconds (less than 10). My point here was to show that the new master just spun for a good 25 minutes (presumably trying to load files over and over again), and then, seemingly out of nowhere, the new master came up and spawned the new workers.
Next time I'll try to get attached with strace earlier and record more output.
> Loading large applications is very slow under Ruby 1.9.2, there's some
> pathological load performance issues fixed in 1.9.3.
Yep, I've read about those, and I've seen Xavier's patch, but I don't think that's the issue here (though, it appears that's why the files attempting to be loaded in the strace output do not exist). Under normal circumstances, loading the app takes about 10 seconds and doesn't peg the cpu while doing it.
> So you're saying /without/ strace, CPU usage _stays_ at 100% and _never_
> goes down?
> Generally, can you reproduce this behavior on a plain (empty) Rails
> application with no extra gems?
Good idea, I'll try that next.
More information about the mongrel-unicorn