pid file deleted briefly when doing hot restart
normalperson at yhbt.net
Tue Nov 27 21:51:46 UTC 2012
Petteri Räty <betelgeuse at gentoo.org> wrote:
> On 27.11.2012 4.02, Eric Wong wrote:
> > Petteri Räty <betelgeuse at gentoo.org> wrote:
> >> On 27.11.2012 2.35, Eric Wong wrote:
> >>>> nginx does not explicitly unlink the old pid file before it renames it
> >>>> out of the way so yes matching nginx in that regard changes the behavior
> >>>> exactly how I originally asked but you were against that. Maybe the
> >>>> point is moot though.
> >>> Ah, I thought you wanted the pid file to be replaced without
> >>> a window where the file is non-existent (or empty).
> >> That would be ideal but I meant this thread to be explicitly about the
> >> unlink and resulting couple seconds window. Now that I have spend some
> >> thinking about the issue here's an approach using hard links that can be
> >> used to replace the pid without window of non-existance:
> > Yes, this is ideal, but unfortunately, the window of non-existence is
> > necessary for compatibility with some existing (nginx-based) scripts.
> Can you point out what those scripts are?
(Revisiting old archives ...)
OK, I misremembered the situation...
TL;DR: there's too much historical baggage, as unicorn inherits traits
from both mongrel and nginx. A change to fix one pid file monitoring
solution will likely break something else. Of course PID files
are broken _anyways_, so relying on them for monitoring is a
terrible idea, *especially* when it's something like nginx/unicorn
where a master process gets replaced.
1) Before unicorn, there was Mongrel: it was important for somebody to
create the PID file early in the process, before binding the listen
socket. I seem to recall this being done to make some other health
2) nginx prevents PID file clobbering by attempting to bind the listen
socket before writing the pid file. This is the best way to prevent
PID clobbering for a fresh process.
Unfortunately, unicorn needed to inherit trait (1) from Mongrel to
ease migration for Mongrel users.
However, I also decided PID file clobber prevention in nginx was useful,
so unicorn will not overwrite a PID file of a valid process.
Unfortunately, the unicorn implementation of PID file clobber prevention
is racy as it cannot copy nginx behavior here while preserving Mongrel
Thus, unicorn unlinks the PID file before writing a new one.
unicorn could rename the PID file out of the way before writing
a new one (as nginx does) and not suffer bad consequences, but
the file still disappears momentarily.
There could be a minor tweak to allow clobbering of an existing
PID file iff it matches an expected value (of the original process),
but I don't think its worth the effort to implement at this point;
as it will still be racy.
Really, PID files are horrible, and MORE horrible than usual
because we support process upgrade/backout via fork+exec().
More information about the mongrel-unicorn