[PATCH] avoid potential Logger deadlock in SIGQUIT and SIGUSR1

Eric Wong normalperson at yhbt.net
Tue Aug 30 19:32:32 EDT 2011

If any combination of SIGQUIT and SIGUSR1 are sent to a
Rainbows! worker in a /very/ short period of time, the Mutex
used by the default Logger implementation may deadlock since
Mutex synchronization is not reentrant-safe.

Users of alternative logger implementations (or monkey-patched
ones) are possibly not affected.  Users of the logger_mp_safe.rb
monkey-patch distributed[1] with unicorn are not affected.

[1] http://unicorn.bogomips.org/examples/logger_mp_safe.rb
 I've pushed to rainbows.git   I haven't seen Rainbows! hit it,
 but I've encountered this elsewhere.

 I don't have anything else for a bit, so maybe I'll just
 release 4.3.1 tonight.  Anybody have any other outstanding issues?

 lib/rainbows/base.rb |    7 +++++--
 1 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/lib/rainbows/base.rb b/lib/rainbows/base.rb
index d1c5c08..54f1d2d 100644
--- a/lib/rainbows/base.rb
+++ b/lib/rainbows/base.rb
@@ -18,8 +18,11 @@ module Rainbows::Base
     # we're don't use the self-pipe mechanism in the Rainbows! worker
     # since we don't defer reopening logs
     Rainbows::HttpServer::SELF_PIPE.each { |x| x.close }.clear
-    trap(:USR1) { reopen_worker_logs(worker.nr) }
-    trap(:QUIT) { Rainbows.quit! }
+    # spawn Threads since Logger takes a mutex by default and
+    # we can't safely lock a mutex in a signal handler
+    trap(:USR1) { Thread.new { reopen_worker_logs(worker.nr) } }
+    trap(:QUIT) { Thread.new { Rainbows.quit! } }
     [:TERM, :INT].each { |sig| trap(sig) { exit!(0) } } # instant shutdown
     Rainbows::ProcessClient.const_set(:APP, Rainbows.server.app)
     logger.info "Rainbows! #@use worker_connections=#@worker_connections"
Eric Wong

More information about the rainbows-talk mailing list