PID files not being removed on shutdown
normalperson at yhbt.net
Tue Jun 18 16:48:32 UTC 2013
Jon Wood <jon at hubbub.co.uk> wrote:
> I'm quite consistently seeing an issue where Unicorn leaves its PID
> files in place after a clean shut down, which is causing some issues
> when deploying releases that update Unicorn config.
> We're doing hot restarts with USR2+QUIT, and using Upstart for process
> supervision based on the presence of a PID file (this admittedly might
> be a mistake, but I've not seen any better solutions).
It's probably better to test the listening port unicorn uses
(perhaps setup a generic health check endpoint) via socat/curl
to detect if unicorn is down.
> This works fine in the case of a hot restart since Unicorn correctly
> updates the PID file, however when we change the configuration for
> Unicorn shutting down the master process doesn't remove the PID file,
> so the supervising Upstart script doesn't detect that the master has
> been shut down and bring up a new one.
> I've created a gist of our Upstart and Unicorn configuration files.
> As I say, we might just be doing something stupid here, but Google
> doesn't seem to have shed any light on the issue.
>  https://gist.github.com/jellybob/3789c3f3a00989b63d74
Do you have anything logging to stderr_path? That might give you a
better idea of what's wrong. Unfortunately, PID files are always bound
to be racy so perhaps supervisor scripts should test and endpoint of an
app using socat or curl.
More information about the mongrel-unicorn