From normalperson at yhbt.net Sun Nov 1 15:57:32 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 1 Nov 2009 20:57:32 +0000 Subject: unicorn.git news for 2009.11.01 Message-ID: <20091101205732.GA6336@dcvr.yhbt.net> I've pushed out a handful of portability fixes to allow all the tests to pass under FreeBSD 7.2 (1.8.7-p174 and 1.9.1-p243). Hopefully these fixes extend to other *BSDs out there, too. Please let us know if there are any other issues I've missed and we can work on them together. All of the issues were bugs with the test code rather than Unicorn itself, so folks running 0.93.5 in production on FreeBSD 7.2 should be fine already. There are also some small cleanups and one new feature. Eric Wong (9): cleanup: avoid redundant error checks for fstat test_helper: connect(2) may fail with EINVAL GNUmakefile: fix non-portable tar(1) usage tests: provide a pure Ruby setsid(8) equivalent more portable symlink awareness for START_CTX[:cwd] test_signals: avoid portability issues with fchmod(2) cleanup error handling and make it less noisy Do not override Dir.chdir in config files configurator: add "working_directory" directive * git://git.bogomips.org/unicorn.git * http://git.bogomips.org/cgit/unicorn.git * http://unicorn.bogomips.org/HACKING.html -- Eric Wong From epaulin at gmail.com Tue Nov 3 04:44:18 2009 From: epaulin at gmail.com (HaiMing Yin) Date: Tue, 3 Nov 2009 17:44:18 +0800 Subject: workers does not seems to exit when served one client Message-ID: <43f51f830911030144o62cdce0agb593cc25d35edf@mail.gmail.com> quote from http://unicorn.bogomips.org/: {{{ workers all run within their own isolated address space and only serve one client at a time for maximum robustness. }}} top shows: {{{ 25398 www-data 20 0 268m 137m 3364 S 0 1.7 27:49.41 unicorn_rails 25400 www-data 20 0 266m 135m 3364 S 0 1.7 25:24.31 unicorn_rails 25397 www-data 20 0 265m 134m 3364 S 0 1.7 31:50.72 unicorn_rails }}} pid are the same since unicorn_rails worker started, and the RES and TIME+ column clearly shows that unicorn_rails did not exit after served one client. What I'm doing wrong? -- Regards, Josh.Yin/??? Friendfeed: http://friendfeed.com/yinhm From hbartels at i-neda.com Tue Nov 3 07:19:07 2009 From: hbartels at i-neda.com (huet bartels) Date: Tue, 03 Nov 2009 12:19:07 +0000 Subject: Sysadmin Setup question for Unicorn Message-ID: <1257250747.10247.135.camel@hbartels-laptop> Dear List, I am very new to ruby and this list. I am trying to setup unicorn to run on ubuntu 8.10. When I try to run unicorn I am getting the following error. with the following command I get the following error. Is this because I dont have a ruby rails app installed yet? unicorn at install01:/app/unicorn/config$ unicorn_rails -c config.ru I, [2009-11-03T13:05:47.609036 #30824] INFO -- : unlinking existing socket=/app/unicorn/tmp/sockets/unicorn.sock I, [2009-11-03T13:05:47.609458 #30824] INFO -- : listening on addr=/app/unicorn/tmp/sockets/unicorn.sock fd=3 I, [2009-11-03T13:05:47.609541 #30824] INFO -- : Refreshing Gem list /app/ruby/bin/unicorn_rails must be run inside RAILS_ROOT: # The other error i get is when I run unicorn with the command line. unicorn at install01:/app/unicorn/config$ unicorn -c config.ru {:daemonize=>false, :unicorn_options=> {:listeners=>[], :config_file=>"/app/unicorn/config/config.ru"}, :app=> #} I, [2009-11-03T13:03:53.555204 #30820] INFO -- : unlinking existing socket=/app/unicorn/tmp/sockets/unicorn.sock I, [2009-11-03T13:03:53.555538 #30820] INFO -- : listening on addr=/app/unicorn/tmp/sockets/unicorn.sock fd=3 I, [2009-11-03T13:03:53.555596 #30820] INFO -- : Refreshing Gem list Exception `NoMethodError' at config.ru:6 - undefined method `worker_processes' for # Exception `NoMethodError' at /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:130 - undefined method `worker_processes' for # config.ru:6: undefined method `worker_processes' for # (NoMethodError) from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/builder.rb:29:in `instance_eval' from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/builder.rb:29:in `initialize' from config.ru:1:in `new' from config.ru:1 from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:697:in `eval' from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:130 from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:697:in `call' from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:697:in `build_app!' from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:170:in `start' from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:20:in `run' from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:165 from /app/ruby/bin/unicorn:19:in `load' from /app/ruby/bin/unicorn:19 unicorn at install01:/app/unicorn/config$ Now is this error generanted because it requires a rails envoiroment. I am happy to be pointed to a website that will help me learn what is required if such a thing exists. thank you for your time regards Huet Bartels From normalperson at yhbt.net Tue Nov 3 11:56:27 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 3 Nov 2009 16:56:27 +0000 Subject: workers does not seems to exit when served one client In-Reply-To: <43f51f830911030144o62cdce0agb593cc25d35edf@mail.gmail.com> References: <43f51f830911030144o62cdce0agb593cc25d35edf@mail.gmail.com> Message-ID: <20091103165627.GA28983@dcvr.yhbt.net> HaiMing Yin wrote: > quote from http://unicorn.bogomips.org/: > > {{{ > workers all run within their own isolated address space and only serve > one client at a time for maximum robustness. > }}} > > top shows: > > {{{ > 25398 www-data 20 0 268m 137m 3364 S 0 1.7 27:49.41 > unicorn_rails > 25400 www-data 20 0 266m 135m 3364 S 0 1.7 25:24.31 > unicorn_rails > 25397 www-data 20 0 265m 134m 3364 S 0 1.7 31:50.72 unicorn_rails > }}} > > pid are the same since unicorn_rails worker started, and the RES and > TIME+ column clearly shows that unicorn_rails did not exit after > served one client. > > What I'm doing wrong? Hi HaiMing, You're doing nothing wrong except misunderstanding that phrase. "one client at a time" means it's not possible to be servicing more clients than there are worker_processes (the kernel will buffer them). Basically your workers <=> clients mapping will look like this with Unicorn: unicorn master \_ unicorn worker[0] | \_ client[0] \_ unicorn worker[1] | \_ client[1] \_ unicorn worker[2] | \_ client[2] ... \_ unicorn worker[M] \_ client[M] Where in other servers (such as Rainbows!) it'll look like this with multiple clients running under one worker: rainbows master \_ rainbows worker[0] | \_ client[0,0] | \_ client[0,1] | \_ client[0,2] | ... | \_ client[0,N] ... \_ rainbows worker[M] \_ client[M,0] \_ client[M,1] \_ client[M,2] ... \_ client[M,N] -- Eric Wong From normalperson at yhbt.net Tue Nov 3 12:07:35 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 3 Nov 2009 17:07:35 +0000 Subject: Sysadmin Setup question for Unicorn In-Reply-To: <1257250747.10247.135.camel@hbartels-laptop> References: <1257250747.10247.135.camel@hbartels-laptop> Message-ID: <20091103170735.GB28983@dcvr.yhbt.net> huet bartels wrote: > Dear List, > > I am very new to ruby and this list. > > I am trying to setup unicorn to run on ubuntu 8.10. When I try to run > unicorn I am getting the following error. > > with the following command I get the following error. Is this because I > dont have a ruby rails app installed yet? Hi Huet, Yes, you need a Ruby on Rails app to run `unicorn_rails' and you should be able to run any Rack application with `unicorn'. > unicorn at install01:/app/unicorn/config$ unicorn_rails -c config.ru > I, [2009-11-03T13:05:47.609036 #30824] INFO -- : unlinking existing > socket=/app/unicorn/tmp/sockets/unicorn.sock > I, [2009-11-03T13:05:47.609458 #30824] INFO -- : listening on > addr=/app/unicorn/tmp/sockets/unicorn.sock fd=3 > I, [2009-11-03T13:05:47.609541 #30824] INFO -- : Refreshing Gem list > /app/ruby/bin/unicorn_rails must be run inside RAILS_ROOT: # no such file to load -- config/boot> Also, your config.ru is the rackup config file, not the Unicorn config file which you would specify with the "-c" option. You only need a config.ru if you're using a non-Rails Rack application (but you can use one with Rails, too). You don't need to pass any switches for the config.ru, either, if it's in ./config.ru it'll automatically be detected and otherwise you can just pass it as the first non-option argument: unicorn production.ru Here, you're specifying Unicorn config directives in your config.ru: > I, [2009-11-03T13:03:53.555596 #30820] INFO -- : Refreshing Gem list > Exception `NoMethodError' at config.ru:6 - undefined method > `worker_processes' for # > Exception `NoMethodError' > at /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:130 - undefined method `worker_processes' for # > config.ru:6: undefined method `worker_processes' for > # (NoMethodError) config.ru is meant to be used with all Rack-able servers (Mongrel, Thin, Passenger, Unicorn) whereas the Unicorn config file is only meant for Unicorn. > I am happy to be pointed to a website that will help me learn what is > required if such a thing exists. http://rack.rubyforge.org/ has some good links on how to setup config.ru -- Eric Wong From chris at ozmm.org Tue Nov 3 12:13:13 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Tue, 3 Nov 2009 09:13:13 -0800 Subject: Sysadmin Setup question for Unicorn In-Reply-To: <1257250747.10247.135.camel@hbartels-laptop> References: <1257250747.10247.135.camel@hbartels-laptop> Message-ID: <8b73aaca0911030913g5cdad024obdcf2864f00cd52e@mail.gmail.com> On Tue, Nov 3, 2009 at 4:19 AM, huet bartels wrote: > I am trying to setup unicorn to run on ubuntu 8.10. ?When I try to run > unicorn I am getting the following error. > > with the following command I get the following error. ?Is this because I > dont have a ruby rails app installed yet? Yes. unicorn_rails must be run from RAILS_ROOT. See http://unicorn.bogomips.org/unicorn_rails_1.html > The other error i get is when I run unicorn with the command line. > > unicorn at install01:/app/unicorn/config$ unicorn -c config.ru > {:daemonize=>false, > ?:unicorn_options=> > ?{:listeners=>[], :config_file=>"/app/unicorn/config/config.ru"}, > ?:app=> > > #} > I, [2009-11-03T13:03:53.555204 #30820] ?INFO -- : unlinking existing > socket=/app/unicorn/tmp/sockets/unicorn.sock > I, [2009-11-03T13:03:53.555538 #30820] ?INFO -- : listening on > addr=/app/unicorn/tmp/sockets/unicorn.sock fd=3 > I, [2009-11-03T13:03:53.555596 #30820] ?INFO -- : Refreshing Gem list > Exception `NoMethodError' at config.ru:6 - undefined method > `worker_processes' for # > Exception `NoMethodError' > at /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:130 - undefined method `worker_processes' for # > config.ru:6: undefined method `worker_processes' for > # (NoMethodError) > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/builder.rb:29:in `instance_eval' > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/rack-1.0.1/lib/rack/builder.rb:29:in `initialize' > ? ? ? ?from config.ru:1:in `new' > ? ? ? ?from config.ru:1 > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:697:in `eval' > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:130 > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:697:in `call' > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:697:in `build_app!' > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:170:in `start' > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/lib/unicorn.rb:20:in `run' > > from /app/ruby.1.8.7-p174/lib/ruby/gems/1.8/gems/unicorn-0.93.4/bin/unicorn:165 > ? ? ? ?from /app/ruby/bin/unicorn:19:in `load' > ? ? ? ?from /app/ruby/bin/unicorn:19 > unicorn at install01:/app/unicorn/config$ > > Now is this error generanted because it requires a rails envoiroment. You need a separate unicorn.rb, distinct from your config.ru when using Unicorn. It appears you are using the proper config option (worker_processes) but putting it in the wrong file (config.ru). config.ru defines your app. Unicorn runs your app. The two are interchangeable, and this is a good thing. Trying moving your unicorn specific logic into a unicorn.rb and launching it with `unicorn -c unicorn.rb config.ru` -- Chris Wanstrath http://github.com/defunkt From wayne at larsen.st Tue Nov 3 20:24:16 2009 From: wayne at larsen.st (Wayne Larsen) Date: Tue, 3 Nov 2009 19:24:16 -0600 Subject: [PATCH] set RACK_ENV on startup Message-ID: Most stupid patch ever, but is there a reason not to set RACK_ENV on startup with `unicorn`, as RAILS_ENV is set with `unicorn_rails`? Thanks Wayne --- bin/unicorn | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/bin/unicorn b/bin/unicorn index 0fed11e..9967f87 100755 --- a/bin/unicorn +++ b/bin/unicorn @@ -58,6 +58,7 @@ opts = OptionParser.new("", 24, ' ') do |opts| opts.on("-E", "--env ENVIRONMENT", "use ENVIRONMENT for defaults (default: development)") do | e| + ENV['RACK_ENV'] = e env = e end -- 1.6.0.1 From normalperson at yhbt.net Tue Nov 3 22:06:35 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 3 Nov 2009 19:06:35 -0800 Subject: [PATCH] set RACK_ENV on startup In-Reply-To: References: Message-ID: <20091104030635.GA5282@dcvr.yhbt.net> Wayne Larsen wrote: > Most stupid patch ever, but is there a reason not to set RACK_ENV on > startup with `unicorn`, as RAILS_ENV is set with `unicorn_rails`? Hi Wayne, Does anything use/depend on it? `unicorn' is modeled after `rackup' and I don't think it's a good idea to expose things if nothing uses it (rackup does not set it, either). RAILS_ENV is an accepted standard for Rails applications and there are plenty of things that already depend on it. -- Eric Wong From wayne at larsen.st Tue Nov 3 22:51:12 2009 From: wayne at larsen.st (Wayne Larsen) Date: Tue, 3 Nov 2009 21:51:12 -0600 Subject: [PATCH] set RACK_ENV on startup In-Reply-To: <20091104030635.GA5282@dcvr.yhbt.net> References: <20091104030635.GA5282@dcvr.yhbt.net> Message-ID: On 2009-11-03, at 9:06 PM, Eric Wong wrote: > Wayne Larsen wrote: >> Most stupid patch ever, but is there a reason not to set RACK_ENV on >> startup with `unicorn`, as RAILS_ENV is set with `unicorn_rails`? > > Hi Wayne, > > Does anything use/depend on it? `unicorn' is modeled after `rackup' > and > I don't think it's a good idea to expose things if nothing uses it > (rackup does not set it, either). > Passenger passes the RACK_ENV value to apps: http://www.modrails.com/documentation/Users%20guide%20Nginx.html#RackEnv As does thin: http://github.com/macournoyer/thin/blob/master/lib/thin/controllers/controller.rb#L169 Sinatra uses it to set its environment: http://github.com/sinatra/sinatra/blob/master/lib/sinatra/base.rb#L1013 As does Merb: http://github.com/merb/merb/blob/master/merb-gen/lib/generators/templates/application/merb_stack/config.ru > RAILS_ENV is an accepted standard for Rails applications and there are > plenty of things that already depend on it. > It seems to me that RACK_ENV is a semi-standard. From a search of the rack group, this has come up before (unanswered): http://groups.google.com/group/rack-devel/browse_frm/thread/109241d9246f91ab/4a0d2f61a7851a6c?lnk=gst&q=RACK_ENV#4a0d2f61a7851a6c or http://tinyurl.com/yl9re66 And this discussion: http://groups.google.com/group/rack-devel/browse_frm/thread/822ce0551fbefa27/d52ea85fcb4e5a51?lnk=gst&q=RACK_ENV#d52ea85fcb4e5a51 also at http://tinyurl.com/yhssua2 where some suggested that it should be available in the rack environment instead of as a global ENV variable. It seems like the pragmatic answer would be to set it. Wayne From normalperson at yhbt.net Wed Nov 4 00:29:08 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 3 Nov 2009 21:29:08 -0800 Subject: [PATCH] set RACK_ENV on startup In-Reply-To: References: <20091104030635.GA5282@dcvr.yhbt.net> Message-ID: <20091104052908.GA10096@dcvr.yhbt.net> Wayne Larsen wrote: > On 2009-11-03, at 9:06 PM, Eric Wong wrote: >> Wayne Larsen wrote: >> >> Does anything use/depend on it? `unicorn' is modeled after `rackup' >> and >> I don't think it's a good idea to expose things if nothing uses it >> (rackup does not set it, either). >> > Passenger passes the RACK_ENV value to apps: > http://www.modrails.com/documentation/Users%20guide%20Nginx.html#RackEnv > > As does thin: > http://github.com/macournoyer/thin/blob/master/lib/thin/controllers/controller.rb#L169 > > Sinatra uses it to set its environment: > http://github.com/sinatra/sinatra/blob/master/lib/sinatra/base.rb#L1013 > > As does Merb: > http://github.com/merb/merb/blob/master/merb-gen/lib/generators/templates/application/merb_stack/config.ru > It seems like the pragmatic answer would be to set it. Agreed. Thanks for the research and links! I've pushed out the following change: >From c7f2242a53ceec6892bd72f0df771266d5193004 Mon Sep 17 00:00:00 2001 From: Wayne Larsen Date: Tue, 3 Nov 2009 21:12:47 -0800 Subject: [PATCH] bin/unicorn: set ENV["RACK_ENV"] on startup Although not currently part of the Rack specification, ENV["RACK_ENV"] is at least a de facto standard. Some of the popular Rack servers (Thin, Passenger) and frameworks (Merb, Sinatra) already set or use it. ML-Ref: Acked-by: Eric Wong [ew: setenv always, not just on CLI + commit message] --- bin/unicorn | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/bin/unicorn b/bin/unicorn index 0fed11e..1916098 100755 --- a/bin/unicorn +++ b/bin/unicorn @@ -118,6 +118,8 @@ if config =~ /\.ru$/ end end +ENV['RACK_ENV'] = env + require 'pp' if $DEBUG app = lambda do || -- Eric Wong From epaulin at gmail.com Wed Nov 4 00:33:38 2009 From: epaulin at gmail.com (HaiMing Yin) Date: Wed, 4 Nov 2009 13:33:38 +0800 Subject: workers does not seems to exit when served one client In-Reply-To: <20091103165627.GA28983@dcvr.yhbt.net> References: <43f51f830911030144o62cdce0agb593cc25d35edf@mail.gmail.com> <20091103165627.GA28983@dcvr.yhbt.net> Message-ID: <43f51f830911032133w2a6b89a1x7611f56aeedf902e@mail.gmail.com> On Wed, Nov 4, 2009 at 12:56 AM, Eric Wong wrote: > > You're doing nothing wrong except misunderstanding that phrase. > > "one client at a time" means it's not possible to be servicing more > clients than there are worker_processes (the kernel will buffer them). Thanks for your explanation and sorry for my misunderstanding. I'm running unicorn at production about one week, it works great, tnx again for your work! -- Regards, Josh.Yin/??? Friendfeed: http://friendfeed.com/yinhm From godfat at godfat.org Wed Nov 4 07:37:16 2009 From: godfat at godfat.org (=?UTF-8?B?TGluIEplbi1TaGluIChha2EgZ29kZmF0IOecn+W4uCk=?=) Date: Wed, 4 Nov 2009 20:37:16 +0800 Subject: About Unicorn Rack handler Message-ID: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> Hi, A couple days ago, I was trying to run Unicorn for Ramaze, and found that `Unicorn.run' didn't share the same interface with other Rack handlers, i.e. `options[:Host]' and `options[:Port]' Because of this, I can't just use: Rack::Handler.register('unicorn', 'Unicorn') And invoke this: Ramaze.start(:adapter => 'unicorn', :port => 8080) To address this, I added a simple wrapper in Innate (which is the core of Ramaze): http://github.com/godfat/innate/commit/9d607f41fdeeca366a9a07155e685ae2605c7025 In short, simply hack the config to: {:listeners => ["#{config[:Host]}:#{config[:Port]}"]} Should we adapt to this interface in Unicorn::Configurator, or provide an additional Rack handle to adapt to this, or maintain this Rack handler in Rack repository, just like the others? http://github.com/rack/rack/tree/master/lib/rack/handler I'm looking forward to your opinion, many thanks! cheers, From normalperson at yhbt.net Wed Nov 4 12:07:15 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 4 Nov 2009 09:07:15 -0800 Subject: About Unicorn Rack handler In-Reply-To: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> Message-ID: <20091104170714.GC5282@dcvr.yhbt.net> "Lin Jen-Shin (aka godfat ??)" wrote: > Hi, > > A couple days ago, I was trying to run Unicorn for Ramaze, > and found that `Unicorn.run' didn't share the same interface > with other Rack handlers, i.e. `options[:Host]' and `options[:Port]' > > Because of this, I can't just use: > > Rack::Handler.register('unicorn', 'Unicorn') > > And invoke this: > > Ramaze.start(:adapter => 'unicorn', :port => 8080) > > To address this, I added a simple wrapper in > Innate (which is the core of Ramaze): > > http://github.com/godfat/innate/commit/9d607f41fdeeca366a9a07155e685ae2605c7025 > > In short, simply hack the config to: > > {:listeners => ["#{config[:Host]}:#{config[:Port]}"]} > > Should we adapt to this interface in Unicorn::Configurator, > or provide an additional Rack handle to adapt to this, > or maintain this Rack handler in Rack repository, just like the others? > http://github.com/rack/rack/tree/master/lib/rack/handler I think making Unicorn.run add to :listeners if :Host or :Port are set will work. I'm not sure if launching Unicorn directly from `rackup' can ever work right without being too intrusive to the other servers, so having a Unicorn Rack handler in distributed with Rack would't make sense. Unicorn needs to capture ARGV (before option parsing) and parse its own config file, neither of which is doable out-of-the-box with rackup. Does something like this work? (not tested). diff --git a/lib/unicorn.rb b/lib/unicorn.rb index 0f2b597..d4a00e0 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -17,6 +17,14 @@ module Unicorn class << self def run(app, options = {}) + # compatibility with other interfaces (Ramaze) + host = options.delete(:Host) + port = options.delete(:Port) + if host || port + port ||= Const::DEFAULT_PORT + host ||= Const::DEFAULT_HOST + (options[:listeners] ||= []) << "#{host}:#{port}" + end HttpServer.new(app, options).start.join end end On the other hand, does Innate make it possible to do transparent upgrades (since rackup does not)? I'll look into it a bit more later... -- Eric Wong From normalperson at yhbt.net Wed Nov 4 12:13:55 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 4 Nov 2009 17:13:55 +0000 Subject: [PATCH] set RACK_ENV on startup In-Reply-To: <20091104052908.GA10096@dcvr.yhbt.net> References: <20091104030635.GA5282@dcvr.yhbt.net> <20091104052908.GA10096@dcvr.yhbt.net> Message-ID: <20091104171355.GC28983@dcvr.yhbt.net> Eric Wong wrote: > I've pushed out the following change: Actually, I think I'll go with this: >From ae2afbcc7dbac0af3256aa8b46afebf6e309bac0 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Wed, 4 Nov 2009 17:09:26 +0000 Subject: [PATCH] bin/unicorn: allow RACK_ENV to be passed from parent This makes our RACK_ENV handling like our RAILS_ENV handling for unicorn_rails, removing the redundant local variable. --- bin/unicorn | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) diff --git a/bin/unicorn b/bin/unicorn index 1916098..225e819 100755 --- a/bin/unicorn +++ b/bin/unicorn @@ -3,7 +3,7 @@ require 'unicorn/launcher' require 'optparse' -env = "development" +ENV["RACK_ENV"] ||= "development" daemonize = false listeners = [] options = { :listeners => listeners } @@ -58,7 +58,7 @@ opts = OptionParser.new("", 24, ' ') do |opts| opts.on("-E", "--env ENVIRONMENT", "use ENVIRONMENT for defaults (default: development)") do |e| - env = e + ENV["RACK_ENV"] = e end opts.on("-D", "--daemonize", "run daemonized in the background") do |d| @@ -118,8 +118,6 @@ if config =~ /\.ru$/ end end -ENV['RACK_ENV'] = env - require 'pp' if $DEBUG app = lambda do || @@ -135,7 +133,7 @@ app = lambda do || Object.const_get(File.basename(config, '.rb').capitalize) end pp({ :inner_app => inner_app }) if $DEBUG - case env + case ENV["RACK_ENV"] when "development" Rack::Builder.new do use Rack::CommonLogger, $stderr -- -- Eric Wong From godfat at godfat.org Wed Nov 4 23:47:04 2009 From: godfat at godfat.org (=?UTF-8?B?TGluIEplbi1TaGluIChha2EgZ29kZmF0IOecn+W4uCk=?=) Date: Thu, 5 Nov 2009 12:47:04 +0800 Subject: About Unicorn Rack handler In-Reply-To: <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> <20091104170714.GC5282@dcvr.yhbt.net> <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> Message-ID: <15d206ce0911042047i54ab72a2wd579efb523642026@mail.gmail.com> 2009/11/5 Lin Jen-Shin (aka godfat ??) : > On Thu, Nov 5, 2009 at 1:07 AM, Eric Wong wrote: >> On the other hand, does Innate make it possible to do transparent >> upgrades (since rackup does not)? ?I'll look into it a bit more later... > > I am not sure what do you mean transparent here, Oops, accidentally sent incomplete mail, sorry. I am not sure what do you mean transparent here, but I would guess Innate work better than rackup. :p cheers, From godfat at godfat.org Wed Nov 4 23:43:33 2009 From: godfat at godfat.org (=?UTF-8?B?TGluIEplbi1TaGluIChha2EgZ29kZmF0IOecn+W4uCk=?=) Date: Thu, 5 Nov 2009 12:43:33 +0800 Subject: About Unicorn Rack handler In-Reply-To: <20091104170714.GC5282@dcvr.yhbt.net> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> <20091104170714.GC5282@dcvr.yhbt.net> Message-ID: <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> On Thu, Nov 5, 2009 at 1:07 AM, Eric Wong wrote: > I think making Unicorn.run add to :listeners if :Host or :Port are set > will work. ?I'm not sure if launching Unicorn directly from `rackup' can > ever work right without being too intrusive to the other servers, so > having a Unicorn Rack handler in distributed with Rack would't make > sense. ? Unicorn needs to capture ARGV (before option parsing) and parse > its own config file, neither of which is doable out-of-the-box with > rackup. I see. On the other hand, it would be great if `rackup' could work with Unicorn directly. Perhaps something like: rackup -r config/unicorn.rb -s unicorn -p 12345 config.ru This would require unicorn config to change to something like: Unicorn::Configurator.instance.instance_eval{ # original config content } Then `Unicorn.run' should check if Configurator singleton has been configured, and use it instead. I would want this because I have a rackup cluster script. If this would work, then I don't have to change the existing script to adapt to Unicorn with a -c option, along with changing `rackup' to `unicorn'. Here's the script: http://github.com/godfat/app-deploy/blob/master/lib/app-deploy/rack_cluster.rb Here's an example config for `rack_cluster': http://github.com/godfat/app-deploy/blob/master/example/rack_cluster.yaml I know I won't need rack "cluster" once I switch to Unicorn. Perhaps what I should do is just drop rack cluster support, then many things could be simplified. > Does something like this work? (not tested). > > diff --git a/lib/unicorn.rb b/lib/unicorn.rb > index 0f2b597..d4a00e0 100644 > --- a/lib/unicorn.rb > +++ b/lib/unicorn.rb > @@ -17,6 +17,14 @@ module Unicorn > > ? class << self > ? ? def run(app, options = {}) > + ? ? ?# compatibility with other interfaces (Ramaze) > + ? ? ?host = options.delete(:Host) > + ? ? ?port = options.delete(:Port) > + ? ? ?if host || port > + ? ? ? ?port ||= Const::DEFAULT_PORT > + ? ? ? ?host ||= Const::DEFAULT_HOST > + ? ? ? ?(options[:listeners] ||= []) << "#{host}:#{port}" > + ? ? ?end > ? ? ? HttpServer.new(app, options).start.join > ? ? end > ? end Yes, work fine at first try. Many thanks for your work! The working `start.rb' for Ramaze is: require 'unicorn' Rack::Handler.register('unicorn', 'Unicorn') Ramaze.start(:adapter => 'unicorn', :port => 7000) > On the other hand, does Innate make it possible to do transparent > upgrades (since rackup does not)? ?I'll look into it a bit more later... I am not sure what do you mean transparent here, From normalperson at yhbt.net Thu Nov 5 02:28:45 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 4 Nov 2009 23:28:45 -0800 Subject: About Unicorn Rack handler In-Reply-To: <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> <20091104170714.GC5282@dcvr.yhbt.net> <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> Message-ID: <20091105072845.GA23256@dcvr.yhbt.net> "Lin Jen-Shin (aka godfat ??)" wrote: > On Thu, Nov 5, 2009 at 1:07 AM, Eric Wong wrote: > > I think making Unicorn.run add to :listeners if :Host or :Port are set > > will work. ?I'm not sure if launching Unicorn directly from `rackup' can > > ever work right without being too intrusive to the other servers, so > > having a Unicorn Rack handler in distributed with Rack would't make > > sense. ? Unicorn needs to capture ARGV (before option parsing) and parse > > its own config file, neither of which is doable out-of-the-box with > > rackup. > > I see. On the other hand, it would be great if `rackup' could work with > Unicorn directly. Perhaps something like: > > rackup -r config/unicorn.rb -s unicorn -p 12345 config.ru As I explained in the other email, this unfortunately can't ever give you all the features that Unicorn has. I've made an effort to keep everything else as compatible and the migration paths as easy as possible. > I would want this because I have a rackup cluster script. > If this would work, then I don't have to change the existing > script to adapt to Unicorn with a -c option, along with > changing `rackup' to `unicorn'. > > Here's the script: > http://github.com/godfat/app-deploy/blob/master/lib/app-deploy/rack_cluster.rb > > Here's an example config for `rack_cluster': > http://github.com/godfat/app-deploy/blob/master/example/rack_cluster.yaml > > I know I won't need rack "cluster" once I switch to Unicorn. > Perhaps what I should do is just drop rack cluster support, > then many things could be simplified. Yeah, it's probably simpler to just use "unicorn" than to wrap it. It's really hard use the process management functionality in a libified form since we rely on things like $0 and ARGV. On the other hand, since Unicorn is designed to work with nginx, you can use the same scripts/signals for managing both. > > Does something like this work? (not tested). > > > > diff --git a/lib/unicorn.rb b/lib/unicorn.rb > > index 0f2b597..d4a00e0 100644 > > --- a/lib/unicorn.rb > > +++ b/lib/unicorn.rb > > @@ -17,6 +17,14 @@ module Unicorn > > > > ? class << self > > ? ? def run(app, options = {}) > > + ? ? ?# compatibility with other interfaces (Ramaze) > > + ? ? ?host = options.delete(:Host) > > + ? ? ?port = options.delete(:Port) > > + ? ? ?if host || port > > + ? ? ? ?port ||= Const::DEFAULT_PORT > > + ? ? ? ?host ||= Const::DEFAULT_HOST > > + ? ? ? ?(options[:listeners] ||= []) << "#{host}:#{port}" > > + ? ? ?end > > ? ? ? HttpServer.new(app, options).start.join > > ? ? end > > ? end > > Yes, work fine at first try. Many thanks for your work! > The working `start.rb' for Ramaze is: > > require 'unicorn' > Rack::Handler.register('unicorn', 'Unicorn') > Ramaze.start(:adapter => 'unicorn', :port => 7000) > > > On the other hand, does Innate make it possible to do transparent > > upgrades (since rackup does not)? ?I'll look into it a bit more later... > > I am not sure what do you mean transparent here, Can you try and see if the USR2 handling of Unicorn allows a transparent upgrade here? If it doesn't then I don't think it's worth supporting an interface that's crippled compared to using the plain `unicorn' command. -- Eric Wong From normalperson at yhbt.net Thu Nov 5 02:29:00 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 4 Nov 2009 23:29:00 -0800 Subject: About Unicorn Rack handler In-Reply-To: <15d206ce0911042047i54ab72a2wd579efb523642026@mail.gmail.com> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> <20091104170714.GC5282@dcvr.yhbt.net> <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> <15d206ce0911042047i54ab72a2wd579efb523642026@mail.gmail.com> Message-ID: <20091105072900.GA11050@dcvr.yhbt.net> "Lin Jen-Shin (aka godfat ??)" wrote: > 2009/11/5 Lin Jen-Shin (aka godfat ??) : > > On Thu, Nov 5, 2009 at 1:07 AM, Eric Wong wrote: > >> On the other hand, does Innate make it possible to do transparent > >> upgrades (since rackup does not)? ?I'll look into it a bit more later... > > > > I am not sure what do you mean transparent here, > > Oops, accidentally sent incomplete mail, sorry. > > I am not sure what do you mean transparent here, > but I would guess Innate work better than rackup. :p Transparent, zero-downtime upgrades (USR2 + QUIT) that Unicorn can do like nginx[1]. The issue here with rackup is that it changes ARGV with the option parser before Unicorn can get to it. So when Unicorn receives the USR2 signal and respawns itself, it won't be able to spawn a child with the same command-line options as its parent. Using the "unicorn" script will allow it to always save its ARGV before OptionParser has a chance to mangle it. [1] - http://unicorn.bogomips.org/SIGNALS.html -- Eric Wong From normalperson at yhbt.net Thu Nov 5 05:06:37 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 5 Nov 2009 02:06:37 -0800 Subject: [ANN] unicorn 0.94.0 - small fixes and new features Message-ID: <20091105100637.GA9992@dcvr.yhbt.net> Unicorn is a HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: The HTTP parser is fix for oddly-aligned reads of trailers (this technically affects headers, too, but is highly unlikely due to our non-support of slow clients). This allows our HTTP parser to better support very slow clients when used by other servers (like Rainbows!). Fortunately this bug does not appear to lead to any invalid memory accesses (and potential arbitrary code execution). FreeBSD (and possibly other *BSDs) support is improved and and all the test cases pass under FreeBSD 7.2. Various flavors of GNU/Linux remains our primary platform for development and production. New features added include the "working_directory" directive in the configurator . Even without specifying a "working_directory", symlink-aware detection of the current path no longer depends on /bin/sh so it should work out-of-the-box on FreeBSD and Solaris and not just systems where /bin/sh is dash, ksh93 or bash. User-switching support is finally supported but only intended for use in the after_fork hook of worker processes. Putting it in the after_fork hook allows allows users to set things like CPU affinity[1] on a per-worker basis before dropping privileges. The master process retains all privileges it started with. The ENV["RACK_ENV"] (process-wide) environment variable is now both read and set for `unicorn' in the same way RAILS_ENV is used by `unicorn_rails'. This allows the Merb launcher to read ENV["RACK_ENV"] in config.ru. Other web servers already set this and there may be applications or libraries that already rely on this de facto standard. Eric Wong (26): cleanup: avoid redundant error checks for fstat test_helper: connect(2) may fail with EINVAL GNUmakefile: fix non-portable tar(1) usage tests: provide a pure Ruby setsid(8) equivalent more portable symlink awareness for START_CTX[:cwd] test_signals: avoid portability issues with fchmod(2) cleanup error handling and make it less noisy Do not override Dir.chdir in config files configurator: add "working_directory" directive configurator: working_directory is expanded configurator: set ENV["PWD"] with working_directory, too configurator: working_directory affects pid, std{err,out}_paths configurator: update documentation for working_directory TODO: remove working_directory bit, done Util.reopen_logs: remove needless Range worker: user/group switching for after_fork hooks Fix autoload of Etc in Worker for Ruby 1.9 bin/unicorn: allow RACK_ENV to be passed from parent tests for RACK_ENV preservation http: allow headers/trailers to be written byte-wise http: extra test for bytewise chunked bodies tee_input: do not clobber trailer buffer on partial uploads test_exec: ensure master is killed after test Util::tmpio returns a TmpIO that responds to #size TODO: remove user-switching bit, done unicorn 0.94.0 Wayne Larsen (1): bin/unicorn: set ENV["RACK_ENV"] on startup [1] - Unicorn does not support CPU affinity directly, but it is possible to load code that allows it inside after_fork hooks, or even just call sched_tool(8). -- Eric Wong From godfat at godfat.org Thu Nov 5 10:13:46 2009 From: godfat at godfat.org (=?UTF-8?B?TGluIEplbi1TaGluIChha2EgZ29kZmF0IOecn+W4uCk=?=) Date: Thu, 5 Nov 2009 23:13:46 +0800 Subject: About Unicorn Rack handler In-Reply-To: <20091105072845.GA23256@dcvr.yhbt.net> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> <20091104170714.GC5282@dcvr.yhbt.net> <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> <20091105072845.GA23256@dcvr.yhbt.net> Message-ID: <56e58d960911050713h48790d01w120bec84c7ca0f3e@mail.gmail.com> 2009/11/5 Eric Wong : > As I explained in the other email, this unfortunately can't ever give > you all the features that Unicorn has. > > I've made an effort to keep everything else as compatible and the > migration paths as easy as possible. Many, many thanks for your effort on Unicorn. It's awesome, in all ways. I would switch all services at my working place to Unicorn at some point. > Can you try and see if the USR2 handling of Unicorn allows > a transparent upgrade here? ?If it doesn't then I don't think > it's worth supporting an interface that's crippled compared > to using the plain `unicorn' command. I think it does work. This is start.rb: ### #!/usr/bin/env ruby require File.expand_path('app', File.dirname(__FILE__)) require 'unicorn' Rack::Handler.register('unicorn', 'Unicorn') Ramaze.start(:adapter => 'unicorn', :port => 7000) ### It won't work if I invoke start.rb like this: $ ruby start.rb (perhaps it's impossible to get the `ruby' ?) but it does work if I invoke this way: $ ./start.rb (tested with different debug output in lib/unicorn.rb) On the other hand, `ramaze' command won't work, and I guess it's because it uses `rackup', which you've explained why it won't. I am wondering.. Could `rackup' be fixed for this? For instance, use the same way as `unicorn' in `rackup'? If so, I would be glad to try to fix that in rackup. I would try it later in my free time. cheers, From normalperson at yhbt.net Thu Nov 5 18:19:24 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 5 Nov 2009 15:19:24 -0800 Subject: About Unicorn Rack handler In-Reply-To: <56e58d960911050713h48790d01w120bec84c7ca0f3e@mail.gmail.com> References: <56e58d960911040437g12cb6291rb19c2fe21cae834@mail.gmail.com> <20091104170714.GC5282@dcvr.yhbt.net> <15d206ce0911042043x34d29e8fp28b982c359252b4e@mail.gmail.com> <20091105072845.GA23256@dcvr.yhbt.net> <56e58d960911050713h48790d01w120bec84c7ca0f3e@mail.gmail.com> Message-ID: <20091105231924.GB7131@dcvr.yhbt.net> "Lin Jen-Shin (aka godfat ??)" wrote: > 2009/11/5 Eric Wong : > > As I explained in the other email, this unfortunately can't ever give > > you all the features that Unicorn has. > > > > I've made an effort to keep everything else as compatible and the > > migration paths as easy as possible. > > Many, many thanks for your effort on Unicorn. > It's awesome, in all ways. I would switch all services > at my working place to Unicorn at some point. Cool, good to know :) > > Can you try and see if the USR2 handling of Unicorn allows > > a transparent upgrade here? ?If it doesn't then I don't think > > it's worth supporting an interface that's crippled compared > > to using the plain `unicorn' command. > > I think it does work. This is start.rb: > > ### > #!/usr/bin/env ruby > require File.expand_path('app', File.dirname(__FILE__)) > require 'unicorn' > Rack::Handler.register('unicorn', 'Unicorn') > Ramaze.start(:adapter => 'unicorn', :port => 7000) > ### > > It won't work if I invoke start.rb like this: > $ ruby start.rb > (perhaps it's impossible to get the `ruby' ?) > but it does work if I invoke this way: > $ ./start.rb > (tested with different debug output in lib/unicorn.rb) Yeah, running "ruby start.rb" throws off $0 unless start.rb is in $PATH. "ruby ./start.rb" should work, too, but it's ugly. I think there are/were cases where "#!/usr/bin/env ruby" won't even work as a shebang, too. > On the other hand, `ramaze' command won't work, > and I guess it's because it uses `rackup', which > you've explained why it won't. > > I am wondering.. Could `rackup' be fixed for this? > For instance, use the same way as `unicorn' in `rackup'? > If so, I would be glad to try to fix that in rackup. > I would try it later in my free time. It would require `rackup' to do a "require 'unicorn'" before it does any option parsing. That would be unnecessarily intrusive if one has Unicorn installed but wants to test with a different server. -- Eric Wong From dstamat at elctech.com Thu Nov 5 18:18:54 2009 From: dstamat at elctech.com (Dylan Stamat) Date: Thu, 5 Nov 2009 15:18:54 -0800 Subject: Logging Message-ID: <149C8E09-664C-45D8-8CB9-AA970C298770@elctech.com> Is there a standard approach for logging when using unicorn_rails? Is it just a matter of specifying std_err/std_out, or setting the logger in a unicorn.rb? Thanks! == Dylan From normalperson at yhbt.net Thu Nov 5 18:49:45 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 5 Nov 2009 15:49:45 -0800 Subject: Logging In-Reply-To: <149C8E09-664C-45D8-8CB9-AA970C298770@elctech.com> References: <149C8E09-664C-45D8-8CB9-AA970C298770@elctech.com> Message-ID: <20091105234945.GC7131@dcvr.yhbt.net> Dylan Stamat wrote: > Is there a standard approach for logging when using unicorn_rails? > Is it just a matter of specifying std_err/std_out, or setting the logger > in a unicorn.rb? Both work, I think most folks just set their stderr/stdout_paths and leave logger as-is unless they're logging to syslog or a network device. Unicorn itself doesn't log a lot, but you can use Rack::CommonLogger or Clogger into your Rails/Rack middleware stack to do request logging. Any File objects with sync=true and opened with the append flag will get reopened with the USR1 signal including the log files Rails sets up by default. -- Eric Wong From leebankewitz at gmail.com Fri Nov 6 16:19:33 2009 From: leebankewitz at gmail.com (Lee Bankewitz) Date: Fri, 6 Nov 2009 16:19:33 -0500 Subject: Log line formatting Message-ID: Howdy. I'm running Unicorn 0.94 in production (its working wonderfully). ?One question about the log line formats in the unicorn log. In my unicorn config file, I have: logger Logger.new("log/unicorn.log") I see timestamps on the the initial log lines when unicorn first starts, i.e.: "I, [2009-11-06T12:25:13.812367 #2437] ?INFO -- : listening on addr=0.0.0.0:3000 fd=5" But the format of the log line changes soon thereafter: "master process ready" "worker=0 ready" any ideas why this is happening? I'd prefer to have the initial format for every line. thx, -lee From normalperson at yhbt.net Fri Nov 6 16:40:05 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 6 Nov 2009 21:40:05 +0000 Subject: Log line formatting In-Reply-To: References: Message-ID: <20091106214005.GA17969@dcvr.yhbt.net> Lee Bankewitz wrote: > Howdy. > > I'm running Unicorn 0.94 in production (its working wonderfully). ?One > question about the log line formats in the unicorn log. > > In my unicorn config file, I have: > logger Logger.new("log/unicorn.log") > > I see timestamps on the the initial log lines when unicorn first starts, i.e.: > "I, [2009-11-06T12:25:13.812367 #2437] ?INFO -- : listening on > addr=0.0.0.0:3000 fd=5" > > But the format of the log line changes soon thereafter: > "master process ready" > "worker=0 ready" > > any ideas why this is happening? I'd prefer to have the initial format > for every line. Hi Lee, This with Rails? Unfortunately, Rails seems to monkey patch the core Logger format methods. If you find a clean solution to this without bad side effects, please let the rest of us know, thanks. -- Eric Wong From leebankewitz at gmail.com Fri Nov 6 17:09:30 2009 From: leebankewitz at gmail.com (Lee Bankewitz) Date: Fri, 6 Nov 2009 17:09:30 -0500 Subject: Log line formatting In-Reply-To: <20091106214005.GA17969@dcvr.yhbt.net> References: <20091106214005.GA17969@dcvr.yhbt.net> Message-ID: On Fri, Nov 6, 2009 at 4:40 PM, Eric Wong wrote: > Lee Bankewitz wrote: >> Howdy. >> >> I'm running Unicorn 0.94 in production (its working wonderfully). ?One >> question about the log line formats in the unicorn log. >> >> In my unicorn config file, I have: >> logger Logger.new("log/unicorn.log") >> >> I see timestamps on the the initial log lines when unicorn first starts, i.e.: >> "I, [2009-11-06T12:25:13.812367 #2437] ?INFO -- : listening on >> addr=0.0.0.0:3000 fd=5" >> >> But the format of the log line changes soon thereafter: >> "master process ready" >> "worker=0 ready" >> >> any ideas why this is happening? I'd prefer to have the initial format >> for every line. > > Hi Lee, > > This with Rails? ?Unfortunately, Rails seems to monkey patch the core > Logger format methods. ?If you find a clean solution to this without > bad side effects, please let the rest of us know, thanks. > > -- > Eric Wong > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > Thanks Eric, Rails does indeed override the default formatting of Logger. I got around it for now by setting the formatter explicitly to the ruby default formatter, as follows: # config/unicorn.rb unicorn_logger = Logger.new("unicorn.log") unicorn_logger.formatter = Logger::Formatter.new logger unicorn_logger I think its bad behavior by rails though, so I'll see if there's any interest in a Rails patch to address this. -lee From normalperson at yhbt.net Fri Nov 6 17:56:07 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 6 Nov 2009 22:56:07 +0000 Subject: Log line formatting In-Reply-To: References: <20091106214005.GA17969@dcvr.yhbt.net> Message-ID: <20091106225607.GA17951@dcvr.yhbt.net> Lee Bankewitz wrote: > On Fri, Nov 6, 2009 at 4:40 PM, Eric Wong wrote: > > Lee Bankewitz wrote: > >> Howdy. > >> > >> I'm running Unicorn 0.94 in production (its working wonderfully). ?One > >> question about the log line formats in the unicorn log. > >> > >> In my unicorn config file, I have: > >> logger Logger.new("log/unicorn.log") > >> > >> I see timestamps on the the initial log lines when unicorn first starts, i.e.: > >> "I, [2009-11-06T12:25:13.812367 #2437] ?INFO -- : listening on > >> addr=0.0.0.0:3000 fd=5" > >> > >> But the format of the log line changes soon thereafter: > >> "master process ready" > >> "worker=0 ready" > >> > >> any ideas why this is happening? I'd prefer to have the initial format > >> for every line. > > > > Hi Lee, > > > > This with Rails? ?Unfortunately, Rails seems to monkey patch the core > > Logger format methods. ?If you find a clean solution to this without > > bad side effects, please let the rest of us know, thanks. > > Thanks Eric, > > Rails does indeed override the default formatting of Logger. I got > around it for now by setting the formatter explicitly to the ruby > default formatter, as follows: > > # config/unicorn.rb > unicorn_logger = Logger.new("unicorn.log") > unicorn_logger.formatter = Logger::Formatter.new > logger unicorn_logger Thanks, I've been meaning to start an FAQ RDoc for stuff like this. I wonder if: Configurator::DEFAULTS[:logger].formatter = Logger::Formatter.new is enough. I still try to avoid relative paths in config files, especially since Logger.new("unicorn.log") sets the path immediately and can't be affected by the new "working_directory" directive in 0.94.0 > I think its bad behavior by rails though, so I'll see if there's any > interest in a Rails patch to address this. Completely agreed that this is bad behavior by Rails. I'm very much against the idea of propagating/encouraging things like this and fixing the problem at the source is the way to go. -- Eric Wong From normalperson at yhbt.net Sat Nov 7 00:24:52 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 6 Nov 2009 21:24:52 -0800 Subject: Fwd: [PATCH (geoip)] use IO.pread from the io-extra lib if possible Message-ID: <20091107052452.GA31584@dcvr.yhbt.net> I sent this to the author of geoip a few days ago and haven't heard back, yet. Since he may be on vacation or busy, somebody may also hit this problem in the mean time, I figured I'd post the patch + git repo I sent him here. Unicorn users: The pread(2)/pwrite(2) syscalls are very useful for making libraries like this one (that rely on an on-disk database) compatible with preload_app. TokyoCabinet is a great example of a library that's already compatible with Unicorn+preload_app out-of-the-box. ----- Forwarded message from Eric Wong ----- From: Eric Wong To: Clifford Heath Subject: [PATCH (geoip)] use IO.pread from the io-extra lib if possible Hi Clifford, One user of Unicorn[1] with the "preload_app" feature ran into problems with the open file descriptor being shared across multiple processes. I've only lightly tested this, but it seems to work. Of course I'll send you updates in case I notice anything broken. I've also pushed the below patch up to git://yhbt.net/geoip.git in case you'd rather "git pull" than "git am". Full disclosure: I'm also a contributor to io-extra, but I've kept the dependency optional in case folks can't install or use io-extra because of an unsupported OS. PS: btw, might want to update the http://geoip.rubyforge.org site to point to the up-to-date repository :) [1] - http://unicorn.bogomips.org/ >From ec75c0fc83e22a4c8c90a896b51291ef01773d79 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Tue, 3 Nov 2009 13:06:06 -0800 Subject: [PATCH] use IO.pread from the io-extra lib if possible This allows the objects created in one process to be used concurrently with forked child processes in addition to multiple threads, as the pread(2) system calls are designed for concurrent operations across the board on POSIX systems). This is useful in cases when a master process forks off several child processes to serve queries. In case io-extra is not available or not installable, then we'll fall back on using a Mutex and at least still get thread-safety. --- lib/geoip.rb | 51 +++++++++++++++++++++++++++++++-------------------- 1 files changed, 31 insertions(+), 20 deletions(-) diff --git a/lib/geoip.rb b/lib/geoip.rb index 4a9b4c2..6fce6dd 100644 --- a/lib/geoip.rb +++ b/lib/geoip.rb @@ -43,6 +43,11 @@ $:.unshift File.dirname(__FILE__) #=end require 'thread' # Needed for Mutex require 'socket' +begin + require 'io/extra' # for IO.pread +rescue LoadError + # oh well, hope they're not forking after initializing +end class GeoIP VERSION = "0.8.4" @@ -429,7 +434,7 @@ class GeoIP # +filename+ is a String holding the path to the GeoIP.dat file # +options+ is an integer holding caching flags (unimplemented) def initialize(filename, flags = 0) - @mutex = Mutex.new + @mutex = IO.respond_to?(:pread) ? false : Mutex.new @flags = flags @databaseType = GEOIP_COUNTRY_EDITION @record_length = STANDARD_RECORD_LENGTH @@ -530,11 +535,9 @@ class GeoIP private def read_city(pos, hostname = '', ip = '') - record = "" - @mutex.synchronize { - @file.seek(pos + (2*@record_length-1) * @databaseSegments[0]) - return nil unless record = @file.read(FULL_RECORD_LENGTH) - } + off = pos + (2*@record_length-1) * @databaseSegments[0] + record = atomic_read(FULL_RECORD_LENGTH, off) + return nil unless record && record.size == FULL_RECORD_LENGTH # The country code is the first byte: code = record[0] @@ -655,11 +658,8 @@ class GeoIP throw "Invalid GeoIP database type, can't look up Organization/ISP by IP" end pos = seek_record(ipnum); - record = "" - @mutex.synchronize { - @file.seek(pos + (2*@record_length-1) * @databaseSegments[0]) - record = @file.read(MAX_ORG_RECORD_LENGTH) - } + off = pos + (2*@record_length-1) * @databaseSegments[0] + record = atomic_read(MAX_ORG_RECORD_LENGTH, off) record = record.sub(/\000.*/n, '') record end @@ -686,11 +686,8 @@ class GeoIP throw "Invalid GeoIP database type, can't look up ASN by IP" end pos = seek_record(ipnum); - record = "" - @mutex.synchronize { - @file.seek(pos + (2*@record_length-1) * @databaseSegments[0]) - record = @file.read(MAX_ASN_RECORD_LENGTH) - } + off = pos + (2*@record_length-1) * @databaseSegments[0] + record = atomic_read(MAX_ASN_RECORD_LENGTH, off) record = record.sub(/\000.*/n, '') if record =~ /^(AS\d+)\s(.*)$/ @@ -739,10 +736,8 @@ class GeoIP offset = 0 mask = 0x80000000 31.downto(0) { |depth| - buf = @mutex.synchronize { - @file.seek(@record_length * 2 * offset); - @file.read(@record_length * 2); - } + off = @record_length * 2 * offset + buf = atomic_read(@record_length * 2, off) buf.slice!(0... at record_length) if ((ipnum & mask) != 0) offset = le_to_ui(buf[0... at record_length].unpack("C*")) return offset if (offset >= @databaseSegments[0]) @@ -761,6 +756,22 @@ class GeoIP def le_to_ui(s) be_to_ui(s.reverse) end + + # reads +length+ bytes from +offset+ as atomically as possible + # if IO.pread is available, it'll use that (making it both multithread + # and multiprocess-safe). Otherwise we'll use a mutex to synchronize + # access (only providing protection against multiple threads, but not + # file descriptors shared across multiple processes). + def atomic_read(length, offset) + if @mutex + @mutex.synchronize { + @file.seek(offset) + @file.read(length) + } + else + IO.pread(@file.fileno, length, offset) + end + end end if $0 == __FILE__ -- Eric Wong ----- End forwarded message ----- From hukl at berlin.ccc.de Mon Nov 9 11:11:07 2009 From: hukl at berlin.ccc.de (John-Paul Bader) Date: Mon, 9 Nov 2009 17:11:07 +0100 Subject: Running the tests - on FreeBSD Message-ID: Hi, I just read that the FreeBSD support was improved which is great news for me as I am deploying almost exclusively on unicorn and made already good experiences with it on 7.2 Now I'd love to run the tests by myself but somehow fail to do so. I cloned the git repo and adjusted my local.mk, then i ran make test in the unicorn directory but I'm just getting a "`test' is up to date." Is there something obvious missing or is the process different on FreeBSD. Kind regards, John From normalperson at yhbt.net Mon Nov 9 13:27:04 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 9 Nov 2009 10:27:04 -0800 Subject: Running the tests - on FreeBSD In-Reply-To: References: Message-ID: <20091109182704.GB816@dcvr.yhbt.net> John-Paul Bader wrote: > Hi, > > I just read that the FreeBSD support was improved which is great news > for me as I am deploying almost exclusively on unicorn and made already > good experiences with it on 7.2 > > Now I'd love to run the tests by myself but somehow fail to do so. > > I cloned the git repo and adjusted my local.mk, then i ran make test in > the unicorn directory but I'm just getting a "`test' is up to date." > > Is there something obvious missing or is the process different on > FreeBSD. Hi John, You need GNU make, which should be gmake on FreeBSD systems. I just pushed this out: >From d5908cae3da3b2fac66407ed1b34fb8e3f6551bb Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Mon, 9 Nov 2009 10:20:25 -0800 Subject: [PATCH] HACKING: update with "gmake" in examples Most GNU users already know their "make" is GNU make but it may not be obvious to non-GNU users. --- HACKING | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-) diff --git a/HACKING b/HACKING index 08aa76d..119b6b7 100644 --- a/HACKING +++ b/HACKING @@ -16,6 +16,9 @@ Tests are good, but slow tests make development slow, so we make tests faster (in parallel) with GNU make (instead of Rake) and avoiding Rubygems. +Users of GNU-based systems (such as GNU/Linux) usually have GNU make installed +as "make" instead of "gmake". + Since we don't load RubyGems by default, loading Rack properly requires setting up RUBYLIB to point to where Rack is located. Not loading Rubygems drastically lowers the time to run the full test suite. You @@ -25,15 +28,15 @@ file is provided for reference. Running the entire test suite with 4 tests in parallel: - make -j4 test + gmake -j4 test Running just one unit test: - make test/unit/test_http_parser.rb + gmake test/unit/test_http_parser.rb Running just one test case in a unit test: - make test/unit/test_http_parser.rb--test_parse_simple.n + gmake test/unit/test_http_parser.rb--test_parse_simple.n === HttpServer @@ -103,11 +106,11 @@ It is easy to install the contents of your git working directory: Via RubyGems (RubyGems 1.3.5+ recommended for prerelease versions): - make install-gem + gmake install-gem Without RubyGems (via setup.rb): - make install + gmake install It is not at all recommended to mix a RubyGems installation with an installation done without RubyGems, however. -- Eric Wong From normalperson at yhbt.net Thu Nov 12 05:04:49 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 12 Nov 2009 02:04:49 -0800 Subject: dealing with client disconnects with TeeInput Message-ID: <20091112100449.GA1929@dcvr.yhbt.net> Foreword: this probably doesn't affect nginx+Unicorn users, which is the recommended configuration for the vast majority of sites. It probably affects Rainbows! users using Thread* or Revactor the most, and probably some Unicorn users serving Intranet clients directly. When clients are uploading large files, there's always a good possibility of them disconnecting before the upload ends. For other web app servers it's not much of a problem: they read the entire upload before attempting to process things; so the app never sees a prematurely disconnected client. However Rainbows! and Unicorn have the TeeInput class which allows real-time processing of uploads as they occur. Now, we _want_ the exception to be thrown and application to stop processing the dead client request immediately. I've made changes in unicorn.git and rainbows.git to ensure no EOFError exceptions from the socket are silenced, not just ones from reading trailers. However, this means (many more) socket errors will be seen within the application and any global exception trappers they use will see them as well. For Rails (and possibly other frameworks), this can mean very messy log files with large backtraces. So, would making a Unicorn::Disconnect < EOFError exception class and raising it with a short/empty backtrace on EOFErrors be the best way to go? That way those global exception trappers can distinguish between EOFError exceptions raised by Unicorn/Rainbows! itself and other code that Unicorn/Rainbows does not care about, and log appropriately... The other option we have is catch/throw. We can avoid worrying about the stack trace entirely, and middlewares that opt-in can still capture and log the disconnect if they want to. More maintenance overhead for Rainbows! with all its concurrency models, but this is a situation where I think catch/throw is appropriate for given the current middleware/application stacks these days. Thanks for reading. -- Eric Wong From clifford.heath at gmail.com Thu Nov 12 15:50:05 2009 From: clifford.heath at gmail.com (Clifford Heath) Date: Thu, 12 Nov 2009 20:50:05 +0000 Subject: [PATCH (geoip)] use IO.pread from the io-extra lib if possible In-Reply-To: <20091107052452.GA31584@dcvr.yhbt.net> References: <20091107052452.GA31584@dcvr.yhbt.net> Message-ID: <78ECBB6C-EB6E-46EF-AF70-7C5F72470BBD@gmail.com> Patch applied and pushed as 0.8.6, available on gemcutter.org Clifford Heath. On 07/11/2009, at 5:24 AM, Eric Wong wrote: > I sent this to the author of geoip a few days ago and haven't heard > back, yet. Since he may be on vacation or busy, somebody may also hit > this problem in the mean time, I figured I'd post the patch + git > repo I sent him here. > > Unicorn users: > > The pread(2)/pwrite(2) syscalls are very useful for making libraries > like this one (that rely on an on-disk database) compatible with > preload_app. TokyoCabinet is a great example of a library that's > already compatible with Unicorn+preload_app out-of-the-box. > > ----- Forwarded message from Eric Wong ----- > > From: Eric Wong > To: Clifford Heath > Subject: [PATCH (geoip)] use IO.pread from the io-extra lib if > possible > > Hi Clifford, > > One user of Unicorn[1] with the "preload_app" feature ran into > problems > with the open file descriptor being shared across multiple processes. > > I've only lightly tested this, but it seems to work. Of course > I'll send you updates in case I notice anything broken. > > I've also pushed the below patch up to git://yhbt.net/geoip.git > in case you'd rather "git pull" than "git am". > > > Full disclosure: I'm also a contributor to io-extra, but I've kept > the dependency optional in case folks can't install or use io-extra > because of an unsupported OS. > > > PS: btw, might want to update the http://geoip.rubyforge.org > site to point to the up-to-date repository :) > > [1] - http://unicorn.bogomips.org/ > > From ec75c0fc83e22a4c8c90a896b51291ef01773d79 Mon Sep 17 00:00:00 2001 > From: Eric Wong > Date: Tue, 3 Nov 2009 13:06:06 -0800 > Subject: [PATCH] use IO.pread from the io-extra lib if possible > > This allows the objects created in one process to be used > concurrently with forked child processes in addition to multiple > threads, as the pread(2) system calls are designed for > concurrent operations across the board on POSIX systems). This > is useful in cases when a master process forks off several child > processes to serve queries. In case io-extra is not available > or not installable, then we'll fall back on using a Mutex and at > least still get thread-safety. > --- > lib/geoip.rb | 51 ++++++++++++++++++++++++++++++ > +-------------------- > 1 files changed, 31 insertions(+), 20 deletions(-) > > diff --git a/lib/geoip.rb b/lib/geoip.rb > index 4a9b4c2..6fce6dd 100644 > --- a/lib/geoip.rb > +++ b/lib/geoip.rb > @@ -43,6 +43,11 @@ $:.unshift File.dirname(__FILE__) > #=end > require 'thread' # Needed for Mutex > require 'socket' > +begin > + require 'io/extra' # for IO.pread > +rescue LoadError > + # oh well, hope they're not forking after initializing > +end > > class GeoIP > VERSION = "0.8.4" > @@ -429,7 +434,7 @@ class GeoIP > # +filename+ is a String holding the path to the GeoIP.dat file > # +options+ is an integer holding caching flags (unimplemented) > def initialize(filename, flags = 0) > - @mutex = Mutex.new > + @mutex = IO.respond_to?(:pread) ? false : Mutex.new > @flags = flags > @databaseType = GEOIP_COUNTRY_EDITION > @record_length = STANDARD_RECORD_LENGTH > @@ -530,11 +535,9 @@ class GeoIP > private > > def read_city(pos, hostname = '', ip = '') > - record = "" > - @mutex.synchronize { > - @file.seek(pos + (2*@record_length-1) * > @databaseSegments[0]) > - return nil unless record = @file.read(FULL_RECORD_LENGTH) > - } > + off = pos + (2*@record_length-1) * @databaseSegments[0] > + record = atomic_read(FULL_RECORD_LENGTH, off) > + return nil unless record && record.size == FULL_RECORD_LENGTH > > # The country code is the first byte: > code = record[0] > @@ -655,11 +658,8 @@ class GeoIP > throw "Invalid GeoIP database type, can't look up > Organization/ISP by IP" > end > pos = seek_record(ipnum); > - record = "" > - @mutex.synchronize { > - @file.seek(pos + (2*@record_length-1) * > @databaseSegments[0]) > - record = @file.read(MAX_ORG_RECORD_LENGTH) > - } > + off = pos + (2*@record_length-1) * @databaseSegments[0] > + record = atomic_read(MAX_ORG_RECORD_LENGTH, off) > record = record.sub(/\000.*/n, '') > record > end > @@ -686,11 +686,8 @@ class GeoIP > throw "Invalid GeoIP database type, can't look up ASN by > IP" > end > pos = seek_record(ipnum); > - record = "" > - @mutex.synchronize { > - @file.seek(pos + (2*@record_length-1) * > @databaseSegments[0]) > - record = @file.read(MAX_ASN_RECORD_LENGTH) > - } > + off = pos + (2*@record_length-1) * @databaseSegments[0] > + record = atomic_read(MAX_ASN_RECORD_LENGTH, off) > record = record.sub(/\000.*/n, '') > > if record =~ /^(AS\d+)\s(.*)$/ > @@ -739,10 +736,8 @@ class GeoIP > offset = 0 > mask = 0x80000000 > 31.downto(0) { |depth| > - buf = @mutex.synchronize { > - @file.seek(@record_length * 2 * offset); > - @file.read(@record_length * 2); > - } > + off = @record_length * 2 * offset > + buf = atomic_read(@record_length * 2, off) > buf.slice!(0... at record_length) if ((ipnum & mask) != 0) > offset = le_to_ui(buf[0... at record_length].unpack("C*")) > return offset if (offset >= @databaseSegments[0]) > @@ -761,6 +756,22 @@ class GeoIP > def le_to_ui(s) > be_to_ui(s.reverse) > end > + > + # reads +length+ bytes from +offset+ as atomically as possible > + # if IO.pread is available, it'll use that (making it both > multithread > + # and multiprocess-safe). Otherwise we'll use a mutex to > synchronize > + # access (only providing protection against multiple threads, > but not > + # file descriptors shared across multiple processes). > + def atomic_read(length, offset) > + if @mutex > + @mutex.synchronize { > + @file.seek(offset) > + @file.read(length) > + } > + else > + IO.pread(@file.fileno, length, offset) > + end > + end > end > > if $0 == __FILE__ > -- > Eric Wong > > ----- End forwarded message ----- From sunaku at gmail.com Thu Nov 12 18:36:37 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Thu, 12 Nov 2009 15:36:37 -0800 Subject: where to chmod socket file? Message-ID: Hello, I set the socket for my app to reside in /tmp/ because my app's Capistrano deploy directory is NFS-mounted: listen '/tmp/my_app.sock' That socket file is being created with mode 0777 + sticky bit. I don't want others to accidentally delete or write to this socket file, so I added the following line to my before_fork() block: before_fork do |server, worker| File.chmod 0600, '/tmp/my_app.sock' # ... end Is there a better place to put this chmod? Or maybe tell unicorn to create the socket with mode 0600? Thanks for your consideration. From normalperson at yhbt.net Thu Nov 12 21:03:52 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 12 Nov 2009 18:03:52 -0800 Subject: where to chmod socket file? In-Reply-To: References: Message-ID: <20091113020351.GA5577@dcvr.yhbt.net> Suraj Kurapati wrote: > Hello, > > I set the socket for my app to reside in /tmp/ because my app's > Capistrano deploy directory is NFS-mounted: > > listen '/tmp/my_app.sock' > > That socket file is being created with mode 0777 + sticky bit. I > don't want others to accidentally delete or write to this socket file, > so I added the following line to my before_fork() block: > > before_fork do |server, worker| > File.chmod 0600, '/tmp/my_app.sock' > # ... > end > > Is there a better place to put this chmod? Or maybe tell unicorn to > create the socket with mode 0600? Hi Suraj, That's probably the best place to put chmod for now... I could be persuaded to add a :umask option for listen. E.g.: listen '/tmp/my_app.sock', :umask => 0077 On the other hand, I don't think it's even possible for others to accidentally delete the socket if it's in /tmp (the directory itself should be sticky, not the socket file). I don't think world-read/writability is a problem for deployed apps. Making sockets world-read/writable fits the model of localhost-bound TCP sockets better: it's one step easier for people to port/change existing testing/monitoring tools from the TCP ones. Also, in my experience with FastCGI deployments, a less permissive umask was often a source of breakage/confusion for FastCGI apps. TCP sockets don't have this problem, and I've seen people prefer it for that reason alone. -- Eric Wong From sunaku at gmail.com Fri Nov 13 17:54:05 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Fri, 13 Nov 2009 14:54:05 -0800 Subject: zero downtime deploys Message-ID: Hello, I tried using preload_app with the "zero downtime deploys" trick of sending SIGQUIT to the old master in before_fork(), as described in: http://unicorn.bogomips.org/Unicorn/Configurator.html http://github.com/blog/517-unicorn This significantly reduced, but did not eliminate, the transition time when the new workers take over (step #2 below): 1. old master (and its workers) is killed in before_fork() 2. workers re-establish DB connections in after_fork() 3. workers are ready to work, at the bottom of after_fork() Why do we kill the old master in before_fork() when the new workers are really ready to work much later? Shouldn't we kill the old master at the *bottom* of after_fork() --- when the new workers are really ready to work? Thanks for your consideration. From normalperson at yhbt.net Fri Nov 13 18:12:31 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 13 Nov 2009 23:12:31 +0000 Subject: zero downtime deploys In-Reply-To: References: Message-ID: <20091113231231.GB27461@dcvr.yhbt.net> Suraj Kurapati wrote: > Hello, > > I tried using preload_app with the "zero downtime deploys" trick of > sending SIGQUIT to the old master in before_fork(), as described in: > > http://unicorn.bogomips.org/Unicorn/Configurator.html > http://github.com/blog/517-unicorn > > This significantly reduced, but did not eliminate, the transition time > when the new workers take over (step #2 below): > > 1. old master (and its workers) is killed in before_fork() > 2. workers re-establish DB connections in after_fork() > 3. workers are ready to work, at the bottom of after_fork() > > Why do we kill the old master in before_fork() when the new workers > are really ready to work much later? Shouldn't we kill the old master > at the *bottom* of after_fork() --- when the new workers are really > ready to work? Hi Suraj, It depends on your setup, mainly memory constraints. I know some deployments run very close to the memory limit of the box and those examples are geared for that. We should clarify that in the documentation. I also know some setups that always startup with worker_processes=1 and then SIGTTIN through an external script gradually. Obviously I would always try to avoid maintenance during peak traffic because performance inevitably suffers somewhat (and human error is always possible :) -- Eric Wong From sunaku at gmail.com Fri Nov 13 18:25:15 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Fri, 13 Nov 2009 15:25:15 -0800 Subject: zero downtime deploys In-Reply-To: <20091113231231.GB27461@dcvr.yhbt.net> References: <20091113231231.GB27461@dcvr.yhbt.net> Message-ID: On Fri, Nov 13, 2009 at 3:12 PM, Eric Wong wrote: > Suraj Kurapati wrote: >> Why do we kill the old master in before_fork() when the new workers >> are really ready to work much later? > > It depends on your setup, mainly memory constraints. ?I know some > deployments run very close to the memory limit of the box and those > examples are geared for that. Ah, that makes more sense. Thanks. From normalperson at yhbt.net Fri Nov 13 20:16:58 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 13 Nov 2009 17:16:58 -0800 Subject: dealing with client disconnects with TeeInput In-Reply-To: <20091112100449.GA1929@dcvr.yhbt.net> References: <20091112100449.GA1929@dcvr.yhbt.net> Message-ID: <20091114011658.GA18151@dcvr.yhbt.net> Eric Wong wrote: > So, would making a Unicorn::Disconnect < EOFError exception class and > raising it with a short/empty backtrace on EOFErrors be the best way to > go? That way those global exception trappers can distinguish between > EOFError exceptions raised by Unicorn/Rainbows! itself and other code > that Unicorn/Rainbows does not care about, and log appropriately... I actually named it Unicorn::ClientShutdown since I figured the name would be more descriptive. Here's what I've pushed out to unicorn.git: >From e4256da292f9626d7dfca60e08f65651a0a9139a Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Sat, 14 Nov 2009 00:23:19 +0000 Subject: [PATCH] raise Unicorn::ClientShutdown if client aborts in TeeInput Leaving the EOFError exception as-is bad because most applications/frameworks run an application-wide exception handler to pretty-print and/or log the exception with a huge backtrace. Since there's absolutely nothing we can do in the server-side app to deal with clients prematurely shutting down, having a backtrace does not make sense. Having a backtrace can even be harmful since it creates unnecessary noise for application engineers monitoring or tracking down real bugs. --- lib/unicorn.rb | 6 ++++ lib/unicorn/tee_input.rb | 11 ++++++++ test/unit/test_server.rb | 61 ++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 76 insertions(+), 2 deletions(-) diff --git a/lib/unicorn.rb b/lib/unicorn.rb index a696402..c6c311e 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -8,6 +8,12 @@ autoload :Rack, 'rack' # a Unicorn web server. It contains a minimalist HTTP server with just enough # functionality to service web application requests fast as possible. module Unicorn + + # raise this inside TeeInput when a client disconnects inside the + # application dispatch + class ClientShutdown < EOFError + end + autoload :Const, 'unicorn/const' autoload :HttpRequest, 'unicorn/http_request' autoload :HttpResponse, 'unicorn/http_response' diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb index 69397c0..50ddb5b 100644 --- a/lib/unicorn/tee_input.rb +++ b/lib/unicorn/tee_input.rb @@ -135,10 +135,21 @@ module Unicorn end end finalize_input + rescue EOFError + # in case client only did a premature shutdown(SHUT_WR) + # we do support clients that shutdown(SHUT_WR) after the + # _entire_ request has been sent, and those will not have + # raised EOFError on us. + socket.close if socket + raise ClientShutdown, "bytes_read=#{@tmp.size}", [] end def finalize_input while parser.trailers(req, buf).nil? + # Don't worry about throw-ing :http_499 here on EOFError, tee() + # will catch EOFError when app is processing it, otherwise in + # initialize we never get any chance to enter the app so the + # EOFError will just get trapped by Unicorn and not the Rack app buf << socket.readpartial(Const::CHUNK_SIZE) end self.socket = nil diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb index bbb06da..a7f6a35 100644 --- a/test/unit/test_server.rb +++ b/test/unit/test_server.rb @@ -12,11 +12,12 @@ include Unicorn class TestHandler - def call(env) - # response.socket.write("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\n\r\nhello!\n") + def call(env) while env['rack.input'].read(4096) end [200, { 'Content-Type' => 'text/plain' }, ['hello!\n']] + rescue Unicorn::ClientShutdown => e + $stderr.syswrite("#{e.class}: #{e.message} #{e.backtrace.empty?}\n") end end @@ -103,6 +104,62 @@ class WebServerTest < Test::Unit::TestCase assert_equal 'hello!\n', results[0], "Handler didn't really run" end + def test_client_shutdown_writes + sock = nil + buf = nil + bs = 15609315 * rand + assert_nothing_raised do + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("PUT /hello HTTP/1.1\r\n") + sock.syswrite("Host: example.com\r\n") + sock.syswrite("Transfer-Encoding: chunked\r\n") + sock.syswrite("Trailer: X-Foo\r\n") + sock.syswrite("\r\n") + sock.syswrite("%x\r\n" % [ bs ]) + sock.syswrite("F" * bs) + sock.syswrite("\r\n0\r\nX-") + "Foo: bar\r\n\r\n".each_byte do |x| + sock.syswrite x.chr + sleep 0.05 + end + # we wrote the entire request before shutting down, server should + # continue to process our request and never hit EOFError on our sock + sock.shutdown(Socket::SHUT_WR) + buf = sock.read + end + assert_equal 'hello!\n', buf.split(/\r\n\r\n/).last + lines = File.readlines("test_stderr.#$$.log") + assert lines.grep(/^Unicorn::ClientShutdown: /).empty? + assert_nothing_raised { sock.close } + end + + def test_client_shutdown_write_truncates + sock = nil + buf = nil + bs = 15609315 * rand + assert_nothing_raised do + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("PUT /hello HTTP/1.1\r\n") + sock.syswrite("Host: example.com\r\n") + sock.syswrite("Transfer-Encoding: chunked\r\n") + sock.syswrite("Trailer: X-Foo\r\n") + sock.syswrite("\r\n") + sock.syswrite("%x\r\n" % [ bs ]) + sock.syswrite("F" * (bs / 2.0)) + + # shutdown prematurely, this will force the server to abort + # processing on us even during app dispatch + sock.shutdown(Socket::SHUT_WR) + IO.select([sock], nil, nil, 60) or raise "Timed out" + buf = sock.read + end + assert_equal "", buf + lines = File.readlines("test_stderr.#$$.log") + lines = lines.grep(/^Unicorn::ClientShutdown: bytes_read=\d+/) + assert_equal 1, lines.size + assert_match %r{\AUnicorn::ClientShutdown: bytes_read=\d+ true$}, lines[0] + assert_nothing_raised { sock.close } + end def do_test(string, chunk, close_after=nil, shutdown_delay=0) # Do not use instance variables here, because it needs to be thread safe -- Eric Wong From normalperson at yhbt.net Sat Nov 14 19:24:34 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 14 Nov 2009 16:24:34 -0800 Subject: where to chmod socket file? In-Reply-To: <20091113020351.GA5577@dcvr.yhbt.net> References: <20091113020351.GA5577@dcvr.yhbt.net> Message-ID: <20091115002433.GA29378@dcvr.yhbt.net> Eric Wong wrote: > Suraj Kurapati wrote: > > Hello, > > > > I set the socket for my app to reside in /tmp/ because my app's > > Capistrano deploy directory is NFS-mounted: > > > > listen '/tmp/my_app.sock' > > > > That socket file is being created with mode 0777 + sticky bit. I > > don't want others to accidentally delete or write to this socket file, > > so I added the following line to my before_fork() block: > > > > before_fork do |server, worker| > > File.chmod 0600, '/tmp/my_app.sock' > > # ... > > end > > > > Is there a better place to put this chmod? Or maybe tell unicorn to > > create the socket with mode 0600? > > Hi Suraj, > > That's probably the best place to put chmod for now... I could be > persuaded to add a :umask option for listen. E.g.: > > listen '/tmp/my_app.sock', :umask => 0077 Hi Suraj, just pushed this out: >From 07767ea2733ed5276ec638fa50102dccb0b2991e Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Sat, 14 Nov 2009 15:28:37 -0800 Subject: [PATCH] configurator: listen :umask parameter for UNIX sockets Typically UNIX domain sockets are created with more liberal file permissions than the rest of the application. By default, we create UNIX domain sockets to be readable and writable by all local users to give them the same accessibility as locally-bound TCP listeners. This only has an effect on UNIX domain sockets. This was inspired by Suraj Kurapati in cfbcd2f00911121536rd0582b8u961f7f2a8c6e546a at mail.gmail.com --- lib/unicorn/configurator.rb | 14 +++++++++++++- lib/unicorn/socket_helper.rb | 2 +- test/unit/test_socket_helper.rb | 14 ++++++++++++++ 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb index d68897b..2d92aa3 100644 --- a/lib/unicorn/configurator.rb +++ b/lib/unicorn/configurator.rb @@ -291,10 +291,22 @@ module Unicorn # +:delay+: seconds to wait between successive +tries+ # # Default: 0.5 seconds + # + # +:umask+: sets the file mode creation mask for UNIX sockets + # + # Typically UNIX domain sockets are created with more liberal + # file permissions than the rest of the application. By default, + # we create UNIX domain sockets to be readable and writable by + # all local users to give them the same accessibility as + # locally-bound TCP listeners. + # + # This has no effect on TCP listeners. + # + # Default: 0 (world read/writable) def listen(address, opt = {}) address = expand_addr(address) if String === address - [ :backlog, :sndbuf, :rcvbuf, :tries ].each do |key| + [ :umask, :backlog, :sndbuf, :rcvbuf, :tries ].each do |key| value = opt[key] or next Integer === value or raise ArgumentError, "not an integer: #{key}=#{value.inspect}" diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb index f792562..1c56be2 100644 --- a/lib/unicorn/socket_helper.rb +++ b/lib/unicorn/socket_helper.rb @@ -88,7 +88,7 @@ module Unicorn "socket=#{address} specified but it is not a socket!" end end - old_umask = File.umask(0) + old_umask = File.umask(opt[:umask] || 0) begin UNIXServer.new(address) ensure diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb index dbca69b..c35b0c2 100644 --- a/test/unit/test_socket_helper.rb +++ b/test/unit/test_socket_helper.rb @@ -63,6 +63,20 @@ class TestSocketHelper < Test::Unit::TestCase File.umask(old_umask) end + def test_bind_listen_unix_umask + old_umask = File.umask(0777) + tmp = Tempfile.new 'unix.sock' + @unix_listener_path = tmp.path + File.unlink(@unix_listener_path) + @unix_listener = bind_listen(@unix_listener_path, :umask => 077) + assert UNIXServer === @unix_listener + assert_equal @unix_listener_path, sock_name(@unix_listener) + assert_equal 0140700, File.stat(@unix_listener_path).mode + assert_equal 0777, File.umask + ensure + File.umask(old_umask) + end + def test_bind_listen_unix_idempotent test_bind_listen_unix a = bind_listen(@unix_listener) -- Eric Wong From sunaku at gmail.com Sat Nov 14 20:52:36 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Sat, 14 Nov 2009 17:52:36 -0800 Subject: where to chmod socket file? In-Reply-To: <20091115002433.GA29378@dcvr.yhbt.net> References: <20091113020351.GA5577@dcvr.yhbt.net> <20091115002433.GA29378@dcvr.yhbt.net> Message-ID: Eric Wong wrote: > [PATCH] configurator: listen :umask parameter for UNIX sockets Awesome! Thanks Eric. From normalperson at yhbt.net Sat Nov 14 21:06:10 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 14 Nov 2009 18:06:10 -0800 Subject: Unicorn FAQ section added Message-ID: <20091115020610.GA29681@dcvr.yhbt.net> Hi all, If you haven't noticed, I've added a FAQ section to the site at http://unicorn.bogomips.org/FAQ.html Let us know if we've missed anything and send patches here if you can. -- Eric Wong From normalperson at yhbt.net Sun Nov 15 17:34:04 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 15 Nov 2009 14:34:04 -0800 Subject: [ANN] unicorn 0.95.0 - we <3 Rainbows! and ponies too Message-ID: <20091115223404.GA26852@dcvr.yhbt.net> Unicorn is a HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: Mostly internal cleanups and documentation updates. Irrelevant stacktraces from client disconnects/errors while reading "rack.input" are now cleared to avoid unnecessary noise. If user switching in workers is used, ownership of logs is now preserved when reopening worker logs (send USR1 only to the the master in this case). The timeout config no longer affects long after_fork hooks or application startups. New features include the addition of the :umask option for the "listen" config directive and error reporting for non-portable socket options. No ponies have ever been harmed in our development. Eric Wong (28): unicorn.1: document RACK_ENV changes in 0.94.0 HACKING: update with "gmake" in examples don't nuke children for long after_fork and app loads local.mk.sample: steal some updates from Rainbows! Load Unicorn constants when building app tee_input: fix RDoc argument definition for tee Add FAQ FAQ: fix links to Configurator docs tee_input: better premature disconnect handling tee_input: don't shadow struct members raise Unicorn::ClientShutdown if client aborts in TeeInput tee_input: fix comment from an intermediate commit FAQ: additional notes on getting HTTPS redirects right configurator: update RDoc and comments in examples bump version to 0.95.0pre configurator: listen :umask parameter for UNIX sockets preserve user/group ownership when reopening logs old_rails/static: avoid freezing strings old_rails: autoload Static const: no need to freeze HTTP_EXPECT test_server: ensure stderr is written to before reading tee_input: expand client error handling replace "rescue => e" with "rescue Object => e" socket_helper: do not hide errors when setting socket options socket_helper: RDoc for constants ClientShutdown: RDoc Rakefile: add raa_update task tee_input: client_error always raises -- Eric Wong From james at lovedthanlost.net Sun Nov 15 18:21:34 2009 From: james at lovedthanlost.net (James Turnbull) Date: Mon, 16 Nov 2009 10:21:34 +1100 Subject: Exception `Errno::EAGAIN' Message-ID: <4B008CFE.6020206@lovedthanlost.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I get this repeated error - about once a second: Exception `Errno::EAGAIN' at /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 - Resource temporarily unavailable - accept(2) Exception `Errno::EAGAIN' at /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 - Resource temporarily unavailable - accept(2) Exception `Errno::EAGAIN' at /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 - Resource temporarily unavailable - accept(2) Which seems to be related to: ready.each do |sock| begin process_client(sock.accept_nonblock) nr += 1 alive.chmod(m = 0 == m ? 1 : 0) rescue Errno::EAGAIN, Errno::ECONNABORTED end break if nr < 0 end My unicorn.conf is: worker_processes 4 working_directory "/etc/puppet" listen '/tmp/puppet.sock', :backlog => 1 listen 8140, :tcp_nopush => true timeout 10 pid "/tmp/puppet.pid" preload_app true GC.respond_to?(:copy_on_write_friendly=) and GC.copy_on_write_friendly = true before_fork do |server, worker| # the following is recomended for Rails + "preload_app true" # as there's no need for the master process to hold a connection # defined?(ActiveRecord::Base) and # ActiveRecord::Base.connection.disconnect! # the following allows a new master process to incrementally # phase out the old master process with SIGTTOU to avoid a # thundering herd (especially in the "preload_app false" case) # when doing a transparent upgrade. The last worker spawned # will then kill off the old master process with a SIGQUIT. old_pid = "#{server.config[:pid]}.oldbin" if old_pid != server.pid begin sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU Process.kill(sig, File.read(old_pid).to_i) rescue Errno::ENOENT, Errno::ESRCH end end # optionally throttle the master from forking too quickly by sleeping sleep 1 end The platform is Fedora 10, Ruby 1.8.6. Oddly, if I comment out the preload I get: Exception `Errno::EPERM' at /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639 - Operation not permitted - /tmp/0.520680132392046 Unhandled listen loop exception #. /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639:in `chmod' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639:in `worker_loop' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:534:in `spawn_missing_workers' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:534:in `fork' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:534:in `spawn_missing_workers' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:530:in `each' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:530:in `spawn_missing_workers' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:540:in `maintain_worker_count' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:215:in `start' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:28:in `run' /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/bin/unicorn:165 /usr/bin/unicorn:19:in `load' /usr/bin/unicorn:19 Repeatedly. Regards James Turnbull - -- Author of: * Pro Linux System Administration (http://tinyurl.com/linuxadmin) * Pulling Strings with Puppet (http://tinyurl.com/pupbook) * Pro Nagios 2.0 (http://tinyurl.com/pronagios) * Hardening Linux (http://tinyurl.com/hardeninglinux) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEVAwUBSwCM/iFa/lDkFHAyAQIiSggA5r3rFPGiY5AeratLgxPvUDE1OjnrhRGn HxoRAcM901BNvXT5i7fX5fMTayW7vPXiKd0IazottCDgd40TYlFTQZNpU9hUDx+g LRAKQ5M5gHTkFX6nqxFlys8WIA3xseDBBUBNxGnZ4YGbqUMg7tQ258b41hiU+4pW fxQgzYN7JjRQ7/u1Px+86aEEdSGHFYNAOSwG15b+iE/7hQbiWrlFIL7p7XcZqT8M UxDm6eTkhP1kry7zJ4wTxVsreXgCi75AaCJBfIIoKs0NMParJ/rBWB+42ZA+1t5/ aEaKXBOi47Kw3CmbsSh6Bi4zbTyuavCX6/tfOMNXx6jXygvykUzpgw== =pJbE -----END PGP SIGNATURE----- From normalperson at yhbt.net Sun Nov 15 18:47:39 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 15 Nov 2009 15:47:39 -0800 Subject: Exception `Errno::EAGAIN' In-Reply-To: <4B008CFE.6020206@lovedthanlost.net> References: <4B008CFE.6020206@lovedthanlost.net> Message-ID: <20091115234739.GA26275@dcvr.yhbt.net> James Turnbull wrote: > I get this repeated error - about once a second: > > Exception `Errno::EAGAIN' at > /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 - > Resource temporarily unavailable - accept(2) > Exception `Errno::EAGAIN' at > /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 - > Resource temporarily unavailable - accept(2) > Exception `Errno::EAGAIN' at > /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 - > Resource temporarily unavailable - accept(2) > > Which seems to be related to: > > ready.each do |sock| > begin > process_client(sock.accept_nonblock) > nr += 1 > alive.chmod(m = 0 == m ? 1 : 0) > rescue Errno::EAGAIN, Errno::ECONNABORTED > end > break if nr < 0 > end Hi James, This is expected. The kernel wakes up all the workers when there's _one_ connection and they all race to accept() one client connection. One wins and accepts the connection, 3 lose and go back to sleep. With lots of worker processes, this can be a thundering herd problem but that's why we encourage Unicorn as a backend server, not as a frontend server. If we only have one listener, we could do a blocking accept() call to avoid thundering herds under Linux, but testing on a 16-core box, I don't remember being able to measure a performance improvement. But otherwise we have to use select() with multiple listeners. nginx uses a variety of non-portable locking mechanisms to implement its accept mutex, but that's too much work under Ruby for little/no benefit. In the nginx case, sometimes even disabling the accept mutex entirely gives better performance. > My unicorn.conf is: > > worker_processes 4 > working_directory "/etc/puppet" > listen '/tmp/puppet.sock', :backlog => 1 > listen 8140, :tcp_nopush => true > timeout 10 > pid "/tmp/puppet.pid" > > preload_app true > GC.respond_to?(:copy_on_write_friendly=) and > GC.copy_on_write_friendly = true > > before_fork do |server, worker| > # the following is recomended for Rails + "preload_app true" > # as there's no need for the master process to hold a connection > # defined?(ActiveRecord::Base) and > # ActiveRecord::Base.connection.disconnect! > > # the following allows a new master process to incrementally > # phase out the old master process with SIGTTOU to avoid a > # thundering herd (especially in the "preload_app false" case) > # when doing a transparent upgrade. The last worker spawned > # will then kill off the old master process with a SIGQUIT. > old_pid = "#{server.config[:pid]}.oldbin" > if old_pid != server.pid > begin > sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU > Process.kill(sig, File.read(old_pid).to_i) > rescue Errno::ENOENT, Errno::ESRCH > end > end > > # optionally throttle the master from forking too quickly by > sleeping > sleep 1 > end > > The platform is Fedora 10, Ruby 1.8.6. > > Oddly, if I comment out the preload I get: > > Exception `Errno::EPERM' at > /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639 - > Operation not permitted - /tmp/0.520680132392046 Do you by any chance do user switching in your application or config.ru? Instead, since 0.94.0, I would do this: after_fork do |server, worker| worker.user('user', 'group') if Process.euid == 0 end That way only the worker process drops permissions. The master really doesn't do anything interesting. Of course *I* would just start Unicorn as a regular user and forget the complexity of user switching entirely... -- Eric Wong From james at lovedthanlost.net Sun Nov 15 19:08:56 2009 From: james at lovedthanlost.net (James Turnbull) Date: Mon, 16 Nov 2009 11:08:56 +1100 Subject: Exception `Errno::EAGAIN' In-Reply-To: <20091115234739.GA26275@dcvr.yhbt.net> References: <4B008CFE.6020206@lovedthanlost.net> <20091115234739.GA26275@dcvr.yhbt.net> Message-ID: <4B009818.1020102@lovedthanlost.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Eric Wong wrote: > This is expected. The kernel wakes up all the workers when there's > _one_ connection and they all race to accept() one client connection. > One wins and accepts the connection, 3 lose and go back to sleep. With > lots of worker processes, this can be a thundering herd problem but > that's why we encourage Unicorn as a backend server, not as a frontend > server. Ah okay - makes sense. > Do you by any chance do user switching in your application > or config.ru? In the application we do. > Instead, since 0.94.0, I would do this: > > after_fork do |server, worker| > worker.user('user', 'group') if Process.euid == 0 > end > > That way only the worker process drops permissions. The master really > doesn't do anything interesting. Of course *I* would just start > Unicorn as a regular user and forget the complexity of user switching > entirely... Okay thanks - will give that a shot. Regards James Turnbull - -- Author of: * Pro Linux System Administration (http://tinyurl.com/linuxadmin) * Pulling Strings with Puppet (http://tinyurl.com/pupbook) * Pro Nagios 2.0 (http://tinyurl.com/pronagios) * Hardening Linux (http://tinyurl.com/hardeninglinux) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQEVAwUBSwCYGCFa/lDkFHAyAQIyPggA5XPzSOYgf4ekdBGoRcs4wp885eC8C/wY BIjzdxilCEhA8mqBw4Hrjo5sOebglOukACFla7QaDqyk2t1velQEpxj8sR2StNEe HHTbng/Jq3ryt4BxNucQWdKvdPTI3WJ3I9APqtGxNyh72FpY08uvAlUv02S8b/Aj mtFqbqi2gu+tkbOoqW1AeJEabjJHd7HyPhf2YlHowBaHG61bzrFRnuhvqN8R6SAq j3LI1kRpnTYdwKYEy3sLfR0OUSVmUuDkWjRkYt2tR2fAwCKkq2oupmcDUrvaT2Y2 fbQ8YJzoGjwVVvD1On/u6iNiTHZcOReYiHRsAhmtbaS+eHldSukHQw== =EyRA -----END PGP SIGNATURE----- From dstamat at elctech.com Mon Nov 16 14:30:44 2009 From: dstamat at elctech.com (Dylan Stamat) Date: Mon, 16 Nov 2009 11:30:44 -0800 Subject: rainbows for sleepy/lethargic apps Message-ID: <76461C1B-E671-4DA3-BED0-12F9E571125A@elctech.com> I'm working on a project that has very bad application performance, specifically, tons of long running requests due to view and (specifically) database contention. While I haven't tested our Unicorn setup under load yet, tweak backlog counts, etc... I'm wondering if I'd be better off giving Rainbows! a shot? In the Rainbows! AppPool diagram, I'm having a hard time understanding the N:P relationship. When the P threshold is met, what happens to N (the client)? Also, with both Unicorn/Rainbows!, is there a explicit timeout number set on the clients (and how does that differ between Unicorn/Rainbows!)? For instance, in our Nginx config, I have proxy_buffering turned on, with proxy_read_timeout and proxy_send_timeout's, set pretty high. Lastly... ?You rock Eric! == Dylan From normalperson at yhbt.net Mon Nov 16 16:26:41 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 16 Nov 2009 13:26:41 -0800 Subject: rainbows for sleepy/lethargic apps In-Reply-To: <76461C1B-E671-4DA3-BED0-12F9E571125A@elctech.com> References: <76461C1B-E671-4DA3-BED0-12F9E571125A@elctech.com> Message-ID: <20091116212641.GA7093@dcvr.yhbt.net> Dylan Stamat wrote: > I'm working on a project that has very bad application performance, > specifically, tons of long running requests due to view and > (specifically) database contention. > > While I haven't tested our Unicorn setup under load yet, tweak backlog > counts, etc... I'm wondering if I'd be better off giving Rainbows! a > shot? In the Rainbows! AppPool diagram, I'm having a hard time > understanding the N:P relationship. When the P threshold is met, what > happens to N (the client)? Hi Dylan, With AppPool, request headers are fully parsed and end up getting queued up in userspace. No request bodies are read (since AppPool is only compatible with the Thread-based concurrency models at the moment). Rainbows is designed for apps that are intentionally/unavoidably sleepy. Since I imagine the database is within your control, I'd optimize that first and fall back to using Unicorn with a higher :backlog. Since you're hitting database contention, throwing more concurrency at the database with Rainbows! is probably the wrong thing to do... If your database is just on a high latency network for whatever reason, then maybe Rainbows! with Neverblock drivers can help. Otherwise if your database is bogged down because of memory/CPU/disk, then you probably want fewer things hitting it at once. Can you shard, cache or otherwise offload DB requests? With slow views, Unicorn is pretty effective at using all available cores already with multiple worker processes. Even with a Ruby VM with non-crippled native threads (Rubinius maybe, not MRI 1.9.1) I doubt it'll help with CPU/memory-bound rendering performance. > Also, with both Unicorn/Rainbows!, is there a explicit timeout number > set on the clients (and how does that differ between > Unicorn/Rainbows!)? For instance, in our Nginx config, I have > proxy_buffering turned on, with proxy_read_timeout and > proxy_send_timeout's, set pretty high. Based on your nginx config, the easiest thing would probably be to set a high :backlog for now and then work on tuning/optimizing/avoiding your database ASAP. Views are harder to optimize, Unicorn itself always uses as much resources as the OS can give it to render. A lot of slow views I've seen hit a lot of memory allocation, so check out tcmalloc (included with REE) or use 1.9 (possibly with tcmalloc, too). 1.9 has some good optimizations for allocating shorter strings (should be more effective on 64-bit) and small hashes/arrays, too. My personal style is also to use destructive methods as much as possible (concat/<< vs +/+=, map! vs map, gsub! vs gsub, tr! vs tr, etc...). Avoid creating lots of OpenStruct, too, excessive use of define_method/metadef allocates a lot of short-lived objects and thrashes memory. The timeout in Unicorn affects the entire request (including all I/O). The timeout in Rainbows only kicks in when the process is completely deadlocked. Since apparently lots of "normal" HTTP clients have ridiculous keepalive times, Rainbows! will be getting a separate keepalive_timeout directive soon that only affects reading HTTP headers. I don't plan enforcing a timeout when processing request bodies or app responses (those can probably be done on a more controlled manner in the app/middleware). > Lastly... ?You rock Eric! Thanks, but much of the credit goes to random folks all over who have helped with this (and projects leading up to this), too :> -- Eric Wong From mguterl at gmail.com Tue Nov 17 12:50:16 2009 From: mguterl at gmail.com (Michael Guterl) Date: Tue, 17 Nov 2009 12:50:16 -0500 Subject: unicorn, rails, and symlinked deployment Message-ID: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> First let me say thanks for Unicorn, it has helped us fill a gap that Passenger could not fill. Like many using Rails, we use capistrano for deployment. At the end of each deployment we use the standard capistrano deploy:cleanup task to remove old releases. Everything is fine until we cleanup the release directory from which unicorn_rails was originally launched. When this happens we get an error in our unicorn error log. reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb error reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb: Errno::ENOENT No such file or directory - /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb I'm sure I'm not the only who has experienced this. Does anyone have any recommendations for handling this situation? Best regards, Michael Guterl From normalperson at yhbt.net Tue Nov 17 17:20:24 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 17 Nov 2009 22:20:24 +0000 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> Message-ID: <20091117222023.GA24006@dcvr.yhbt.net> Michael Guterl wrote: > First let me say thanks for Unicorn, it has helped us fill a gap that > Passenger could not fill. > > Like many using Rails, we use capistrano for deployment. At the end > of each deployment we use the standard capistrano deploy:cleanup task > to remove old releases. Everything is fine until we cleanup the > release directory from which unicorn_rails was originally launched. > When this happens we get an error in our unicorn error log. > > reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb > error reloading > config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb: > Errno::ENOENT No such file or directory - > /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb > > I'm sure I'm not the only who has experienced this. Does anyone have > any recommendations for handling this situation? Hi Michael, Unicorn was definitely implemented with Cap in mind. Which version of Unicorn are you running? 0.94.0 got better symlink detection for non-GNU/Linux systems, so if you start in /home/deploy/public_html/rm/current, then it should detect it and use /home/deploy/public_html/rm/current/config/unicorn.rb. If you don't start in your "current" symlink dir, 0.94.0 also got support for "working_directory" in your config file. -- Eric Wong From mguterl at gmail.com Tue Nov 17 17:08:46 2009 From: mguterl at gmail.com (Michael Guterl) Date: Tue, 17 Nov 2009 17:08:46 -0500 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> Message-ID: <944a03770911171408h10e85449r9e5ae9d93793d44e@mail.gmail.com> On Tue, Nov 17, 2009 at 12:50 PM, Michael Guterl wrote: > First let me say thanks for Unicorn, it has helped us fill a gap that > Passenger could not fill. > > Like many using Rails, we use capistrano for deployment. ?At the end > of each deployment we use the standard capistrano deploy:cleanup task > to remove old releases. ?Everything is fine until we cleanup the > release directory from which unicorn_rails was originally launched. > When this happens we get an error in our unicorn error log. > > reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb > error reloading > config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb: > Errno::ENOENT No such file or directory - > /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb > > I'm sure I'm not the only who has experienced this. ?Does anyone have > any recommendations for handling this situation? > I should also point out that in my unicorn logs when I start the app it references the path in it's symbolic link form: I, [2009-11-17T17:06:10.215485 #30857] INFO -- : unlinking existing socket=/home/deploy/public_html/rm/current/tmp/sockets/unicorn.sock I, [2009-11-17T17:06:10.227485 #30857] INFO -- : listening on addr=/home/deploy/public_html/rm/current/tmp/sockets/unicorn.sock fd=3 I, [2009-11-17T17:06:10.227485 #30857] INFO -- : Refreshing Gem list But when I send HUP to restart, it references the actual path, not the symlink. reloading config_file=/home/deploy/public_html/rm/releases/20091117215841/config/unicorn.rb Refreshing Gem list done reloading config_file=/home/deploy/public_html/rm/releases/20091117215841/config/unicorn.rb Best regards, Michael Guterl From hbartels at i-neda.com Wed Nov 18 03:58:20 2009 From: hbartels at i-neda.com (huet bartels) Date: Wed, 18 Nov 2009 08:58:20 +0000 Subject: Nginx Conf Message-ID: <1258534700.7580.8.camel@hbartels-laptop> Dear all, I know members of the list use nginx to forward to unicorn. Would some be so kind as to share a code snippet on how they have configured nginx to use unicorn. thanks in advance Huet Bartels From hukl at berlin.ccc.de Wed Nov 18 04:35:46 2009 From: hukl at berlin.ccc.de (John-Paul Bader) Date: Wed, 18 Nov 2009 10:35:46 +0100 Subject: Nginx Conf In-Reply-To: <1258534700.7580.8.camel@hbartels-laptop> References: <1258534700.7580.8.camel@hbartels-laptop> Message-ID: <489AA732-2AD2-4CF9-B330-4E4C37196EA4@berlin.ccc.de> On a FreeBSD 7.2 varnish:80 -> nginx:8080 -> unicorn and for ssl nginx:443 -> varnish:80 -> nginx 8080 -> unicorn Config: http://pastie.textmate.org/703919 Kind regards, John On 18.11.2009, at 09:58, huet bartels wrote: > Dear all, > > I know members of the list use nginx to forward to unicorn. Would some > be so kind as to share a code snippet on how they have configured nginx > to use unicorn. > > thanks in advance > > Huet Bartels > > > > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From mguterl at gmail.com Wed Nov 18 10:34:38 2009 From: mguterl at gmail.com (Michael Guterl) Date: Wed, 18 Nov 2009 10:34:38 -0500 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <20091117222023.GA24006@dcvr.yhbt.net> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> Message-ID: <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> On Tue, Nov 17, 2009 at 5:20 PM, Eric Wong wrote: > Michael Guterl wrote: >> First let me say thanks for Unicorn, it has helped us fill a gap that >> Passenger could not fill. >> >> Like many using Rails, we use capistrano for deployment. ?At the end >> of each deployment we use the standard capistrano deploy:cleanup task >> to remove old releases. ?Everything is fine until we cleanup the >> release directory from which unicorn_rails was originally launched. >> When this happens we get an error in our unicorn error log. >> >> reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb >> error reloading >> config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb: >> Errno::ENOENT No such file or directory - >> /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb >> >> I'm sure I'm not the only who has experienced this. ?Does anyone have >> any recommendations for handling this situation? > > Hi Michael, > > Unicorn was definitely implemented with Cap in mind. ?Which version of > Unicorn are you running? ?0.94.0 got better symlink detection for > non-GNU/Linux systems, so if you start in > /home/deploy/public_html/rm/current, then it should detect it and use > /home/deploy/public_html/rm/current/config/unicorn.rb. > I am using unicorn v0.94.0 on Ubuntu 8.04. > If you don't start in your "current" symlink dir, 0.94.0 also got > support for "working_directory" in your config file. > I set this option in my config, however, it did not help. After some experimentation I realized the problem is in the script I use for managing unicorn via /etc/init.d -- http://gist.github.com/237953 Essentially start does this: cd ~/public_html/rm/current unicorn_rails -D -E production -c config/unicorn.rb I determined the -c config/unicorn.rb was the problem. If I change it to: unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb everything works fine. My updated init script is located at http://gist.github.com/237958 and it seems to work fine. Best regards, Michael Guterl From normalperson at yhbt.net Wed Nov 18 12:21:19 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 18 Nov 2009 17:21:19 +0000 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> Message-ID: <20091118172118.GA17288@dcvr.yhbt.net> Michael Guterl wrote: > cd ~/public_html/rm/current > unicorn_rails -D -E production -c config/unicorn.rb > > I determined the -c config/unicorn.rb was the problem. If I change it to: > > unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb > > everything works fine. Hmm, maybe the config file path shouldn't be expanded then. Especially since it's only specified in the command-line. /me ponders a bit... -- Eric Wong From dstamat at elctech.com Wed Nov 18 13:17:53 2009 From: dstamat at elctech.com (Dylan Stamat) Date: Wed, 18 Nov 2009 10:17:53 -0800 Subject: Nginx Conf In-Reply-To: <489AA732-2AD2-4CF9-B330-4E4C37196EA4@berlin.ccc.de> References: <1258534700.7580.8.camel@hbartels-laptop> <489AA732-2AD2-4CF9-B330-4E4C37196EA4@berlin.ccc.de> Message-ID: <13215A21-5E6A-4C94-9495-14A27D1FC9E5@elctech.com> Hey Huet! On CentOS, I'm using something like this: http://pastie.org/704551 I'd like to be using the UDS's, but our architecture requires the use of TCP at the moment. John, how do you like having Varnish up front, and, what do you use it for? I was thinking about throwing it up simply for serving static assets, but I read somewhere that Nginx was definitely sufficient for doing so. On Nov 18, 2009, at 1:35 AM, John-Paul Bader wrote: > On a FreeBSD 7.2 > > varnish:80 -> nginx:8080 -> unicorn > > and for ssl > > nginx:443 -> varnish:80 -> nginx 8080 -> unicorn > > > Config: http://pastie.textmate.org/703919 > > > Kind regards, John > > > On 18.11.2009, at 09:58, huet bartels wrote: > >> Dear all, >> >> I know members of the list use nginx to forward to unicorn. Would some >> be so kind as to share a code snippet on how they have configured nginx >> to use unicorn. >> >> thanks in advance >> >> Huet Bartels >> >> >> >> _______________________________________________ >> mongrel-unicorn mailing list >> mongrel-unicorn at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-unicorn >> > > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn Dylan Stamat ELC Technologies (TM) 1921 State Street Santa Barbara, CA 93101 dstamat at elctech.com (866)863-7365 Tel (866)893-1902 Fax +44 (0)20 7504 1346 Tel - London Office +44 (0)20 7504 1347 Fax - London Office http://www.elctech.com From halogenandtoast at gmail.com Wed Nov 18 13:22:25 2009 From: halogenandtoast at gmail.com (Matt Mongeau) Date: Wed, 18 Nov 2009 13:22:25 -0500 Subject: Nginx Conf In-Reply-To: <489AA732-2AD2-4CF9-B330-4E4C37196EA4@berlin.ccc.de> References: <1258534700.7580.8.camel@hbartels-laptop> <489AA732-2AD2-4CF9-B330-4E4C37196EA4@berlin.ccc.de> Message-ID: <91d915350911181022s5cf3fa55p96f74a04b610e9f3@mail.gmail.com> On Wed, Nov 18, 2009 at 4:35 AM, John-Paul Bader wrote: > On ?a FreeBSD 7.2 > > varnish:80 -> nginx:8080 -> unicorn > > and for ssl > > nginx:443 -> varnish:80 -> nginx 8080 -> unicorn > > > Config: http://pastie.textmate.org/703919 > > > Kind regards, John > > > On 18.11.2009, at 09:58, huet bartels wrote: > >> Dear all, >> >> I know members of the list use nginx to forward to unicorn. ?Would some >> be so kind as to share a code snippet on how they have configured nginx >> to use unicorn. >> >> thanks in advance >> >> Huet Bartels >> >> >> >> _______________________________________________ >> mongrel-unicorn mailing list >> mongrel-unicorn at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-unicorn >> > > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > server { 47 listen 80; 48 server_name unicorn.local; 49 location / { 50 proxy_pass http://unicorn; 51 } 52 } 53 54 upstream unicorn { 55 server unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock; 56 } minus the line numbers of course From mguterl at gmail.com Wed Nov 18 14:47:35 2009 From: mguterl at gmail.com (Michael Guterl) Date: Wed, 18 Nov 2009 14:47:35 -0500 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <20091118172118.GA17288@dcvr.yhbt.net> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> <20091118172118.GA17288@dcvr.yhbt.net> Message-ID: <944a03770911181147i2e61a664t76d926ced6d48144@mail.gmail.com> On Wed, Nov 18, 2009 at 12:21 PM, Eric Wong wrote: > Michael Guterl wrote: >> cd ~/public_html/rm/current >> unicorn_rails -D -E production -c config/unicorn.rb >> >> I determined the -c config/unicorn.rb was the problem. ?If I change it to: >> >> unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb >> >> everything works fine. > > Hmm, maybe the config file path shouldn't be expanded then. ?Especially > since it's only specified in the command-line. ?/me ponders a bit... > I don't mean to hijack my own thread, but the problem seems to be closely related. My application code is not being reloaded, when I send HUP to the unicorn master process. I get the following output in my log: reloading config_file=/home/deploy/public_html/rm/current/config/unicorn.rb Refreshing Gem list done reloading config_file=/home/deploy/public_html/rm/current/config/unicorn.rb reaped # worker=0 reaped # worker=3 listening on addr=127.0.0.1:9293 fd=6 worker=0 ready listening on addr=127.0.0.1:9296 fd=6 worker=3 ready reaped # worker=1 reaped # worker=2 listening on addr=127.0.0.1:9294 fd=7 worker=1 ready listening on addr=127.0.0.1:9295 fd=6 worker=2 ready My unicorn config is here: http://gist.github.com/238167 I am using Rails 2.3.2 (shame, shame, I know), but there are areas of incompatibility we cannot currently fix. If there is anything else I can provide to help, please let me know. Best regards, Michael Guterl From normalperson at yhbt.net Wed Nov 18 20:05:37 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 19 Nov 2009 01:05:37 +0000 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <944a03770911181147i2e61a664t76d926ced6d48144@mail.gmail.com> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> <20091118172118.GA17288@dcvr.yhbt.net> <944a03770911181147i2e61a664t76d926ced6d48144@mail.gmail.com> Message-ID: <20091119010537.GA31570@dcvr.yhbt.net> Michael Guterl wrote: > On Wed, Nov 18, 2009 at 12:21 PM, Eric Wong wrote: > > Michael Guterl wrote: > >> cd ~/public_html/rm/current > >> unicorn_rails -D -E production -c config/unicorn.rb > >> > >> I determined the -c config/unicorn.rb was the problem. ?If I change it to: > >> > >> unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb > >> > >> everything works fine. > > > > Hmm, maybe the config file path shouldn't be expanded then. ?Especially > > since it's only specified in the command-line. ?/me ponders a bit... > > > > I don't mean to hijack my own thread, but the problem seems to be > closely related. My application code is not being reloaded, when I > send HUP to the unicorn master process. Hi Michael, Application code is only reloaded by HUP when your preload_app setting is false. Otherwise you have to use USR2 + QUIT to upgrade your entire process. This behavior is documented in the Configurator, but not in the SIGNALS documented, I'll fix the SIGNALS document in a few and disable expand_path in the launcher scripts. -- Eric Wong From normalperson at yhbt.net Wed Nov 18 20:46:28 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 18 Nov 2009 17:46:28 -0800 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <20091119010537.GA31570@dcvr.yhbt.net> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> <20091118172118.GA17288@dcvr.yhbt.net> <944a03770911181147i2e61a664t76d926ced6d48144@mail.gmail.com> <20091119010537.GA31570@dcvr.yhbt.net> Message-ID: <20091119014628.GA1935@dcvr.yhbt.net> Eric Wong wrote: > Michael Guterl wrote: > > I don't mean to hijack my own thread, but the problem seems to be > > closely related. My application code is not being reloaded, when I > > send HUP to the unicorn master process. > > Hi Michael, > > Application code is only reloaded by HUP when your preload_app setting > is false. Otherwise you have to use USR2 + QUIT to upgrade your entire > process. > > This behavior is documented in the Configurator, but not in the SIGNALS > documented, I'll fix the SIGNALS document in a few and disable > expand_path in the launcher scripts. Just pushed the following change out for the SIGNALS doc. http://unicorn.bogomips.org/SIGNALS.html diff --git a/SIGNALS b/SIGNALS index 5c7295e..be96892 100644 --- a/SIGNALS +++ b/SIGNALS @@ -9,7 +9,12 @@ Unicorn and nginx. === Master Process -* HUP - reload config file, app, and gracefully restart all workers +* HUP - reloads config file and gracefully restart all workers. + If the "preload_app" directive is false (the default), then workers + will also pick up any application code changes when restarted. If + "preload_app" is true, then application code changes will have no + effect; USR2 + QUIT (see below) must be used to load newer code in + this case. * INT/TERM - quick shutdown, kills all workers immediately -- Eric Wong From normalperson at yhbt.net Wed Nov 18 20:51:26 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 18 Nov 2009 17:51:26 -0800 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <20091118172118.GA17288@dcvr.yhbt.net> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> <20091118172118.GA17288@dcvr.yhbt.net> Message-ID: <20091119015126.GB1935@dcvr.yhbt.net> Eric Wong wrote: > Michael Guterl wrote: > > cd ~/public_html/rm/current > > unicorn_rails -D -E production -c config/unicorn.rb > > > > I determined the -c config/unicorn.rb was the problem. If I change it to: > > > > unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb > > > > everything works fine. > > Hmm, maybe the config file path shouldn't be expanded then. Especially > since it's only specified in the command-line. /me ponders a bit... Hi Michael, Just pushed this out, too. This should work better for you with: unicorn_rails -D -E production -c config/unicorn.rb Oddly, this has been a problem since the beginning of time and probably confused a good amount of people. Thanks Michael for finally bringing it to my attention. Most folks I know favor absolute ones (including myself), and don't hit this problem... Of course let me know if I broke something for anyone, I hope not... >From 15217fe1162a400fa1cd2216e395d9f17be8083e Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Wed, 18 Nov 2009 17:28:12 -0800 Subject: [PATCH] Do not expand paths given on the shell Shells already expand '~' before the executables see it, and relative paths inside symlinks can get set incorrectly to the actual directory name, and not the (usually desired) symlink name for things like Capistrano. Since our paths are now unexpanded, we must now check the "working_directory" directive and raise an error if the user specifies the config file in a way that makes the config file unreloadable. --- bin/unicorn | 4 +- bin/unicorn_rails | 2 +- lib/unicorn/configurator.rb | 7 ++++++ test/exec/test_exec.rb | 51 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 61 insertions(+), 3 deletions(-) diff --git a/bin/unicorn b/bin/unicorn index 225e819..325afb3 100755 --- a/bin/unicorn +++ b/bin/unicorn @@ -68,7 +68,7 @@ opts = OptionParser.new("", 24, ' ') do |opts| opts.on("-P", "--pid FILE", "DEPRECATED") do |f| warn %q{Use of --pid/-P is strongly discouraged} warn %q{Use the 'pid' directive in the Unicorn config file instead} - options[:pid] = File.expand_path(f) + options[:pid] = f end opts.on("-s", "--server SERVER", @@ -85,7 +85,7 @@ opts = OptionParser.new("", 24, ' ') do |opts| end opts.on("-c", "--config-file FILE", "Unicorn-specific config file") do |f| - options[:config_file] = File.expand_path(f) + options[:config_file] = f end # I'm avoiding Unicorn-specific config options on the command-line. diff --git a/bin/unicorn_rails b/bin/unicorn_rails index 36ed660..e46de70 100755 --- a/bin/unicorn_rails +++ b/bin/unicorn_rails @@ -75,7 +75,7 @@ opts = OptionParser.new("", 24, ' ') do |opts| end opts.on("-c", "--config-file FILE", "Unicorn-specific config file") do |f| - options[:config_file] = File.expand_path(f) + options[:config_file] = f end opts.on("-P PATH", "DEPRECATED") do |v| diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb index 2d92aa3..e809b22 100644 --- a/lib/unicorn/configurator.rb +++ b/lib/unicorn/configurator.rb @@ -372,6 +372,13 @@ module Unicorn def working_directory(path) # just let chdir raise errors path = File.expand_path(path) + if config_file && + config_file[0] != ?/ && + ! test(?r, "#{path}/#{config_file}") + raise ArgumentError, + "config_file=#{config_file} would not be accessible in" \ + " working_directory=#{path}" + end Dir.chdir(path) HttpServer::START_CTX[:cwd] = ENV["PWD"] = path end diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb index f6dfd6a..49762c0 100644 -- Eric Wong From preston at synergyeoc.com Wed Nov 18 21:06:05 2009 From: preston at synergyeoc.com (Preston Marshall) Date: Wed, 18 Nov 2009 20:06:05 -0600 Subject: Log File Location Message-ID: <0F1B76E7-F7F5-4A3C-8C15-668A5FAF5798@synergyeoc.com> This may be a dumb question, but where can I see the error logs that are usually displayed when unicorn is not daemonized? The production.log file is empty in my app. Everything is currently running as root. From mguterl at gmail.com Wed Nov 18 21:36:39 2009 From: mguterl at gmail.com (Michael Guterl) Date: Wed, 18 Nov 2009 21:36:39 -0500 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <20091119014628.GA1935@dcvr.yhbt.net> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> <20091118172118.GA17288@dcvr.yhbt.net> <944a03770911181147i2e61a664t76d926ced6d48144@mail.gmail.com> <20091119010537.GA31570@dcvr.yhbt.net> <20091119014628.GA1935@dcvr.yhbt.net> Message-ID: <944a03770911181836h41acc63n57d5fe8a5550306d@mail.gmail.com> On Wed, Nov 18, 2009 at 8:46 PM, Eric Wong wrote: > Eric Wong wrote: >> Michael Guterl wrote: >> > I don't mean to hijack my own thread, but the problem seems to be >> > closely related. ?My application code is not being reloaded, when I >> > send HUP to the unicorn master process. >> >> Hi Michael, >> >> Application code is only reloaded by HUP when your preload_app setting >> is false. ?Otherwise you have to use USR2 + QUIT to upgrade your entire >> process. >> >> This behavior is documented in the Configurator, but not in the SIGNALS >> documented, I'll fix the SIGNALS document in a few and disable >> expand_path in the launcher scripts. > > Just pushed the following change out for the SIGNALS doc. > ?http://unicorn.bogomips.org/SIGNALS.html > > diff --git a/SIGNALS b/SIGNALS > index 5c7295e..be96892 100644 > --- a/SIGNALS > +++ b/SIGNALS > @@ -9,7 +9,12 @@ Unicorn and nginx. > > ?=== Master Process > > -* HUP - reload config file, app, and gracefully restart all workers > +* HUP - reloads config file and gracefully restart all workers. > + ?If the "preload_app" directive is false (the default), then workers > + ?will also pick up any application code changes when restarted. ?If > + ?"preload_app" is true, then application code changes will have no > + ?effect; USR2 + QUIT (see below) must be used to load newer code in > + ?this case. > > ?* INT/TERM - quick shutdown, kills all workers immediately > This is great Eric, I had spent some time looking at Configurator, however, I was mostly looking at SIGNALS related to this problem. This adds much clarity to the docs! Best regards, Michael Guterl From mguterl at gmail.com Wed Nov 18 21:45:08 2009 From: mguterl at gmail.com (Michael Guterl) Date: Wed, 18 Nov 2009 21:45:08 -0500 Subject: unicorn, rails, and symlinked deployment In-Reply-To: <20091119015126.GB1935@dcvr.yhbt.net> References: <944a03770911170950i4fe79deay5f29b9904af58677@mail.gmail.com> <20091117222023.GA24006@dcvr.yhbt.net> <944a03770911180734m635549c9ua2efe5ef5b881b36@mail.gmail.com> <20091118172118.GA17288@dcvr.yhbt.net> <20091119015126.GB1935@dcvr.yhbt.net> Message-ID: <944a03770911181845o642aa33dk91946d6b02e09380@mail.gmail.com> On Wed, Nov 18, 2009 at 8:51 PM, Eric Wong wrote: > Eric Wong wrote: >> Michael Guterl wrote: >> > cd ~/public_html/rm/current >> > unicorn_rails -D -E production -c config/unicorn.rb >> > >> > I determined the -c config/unicorn.rb was the problem. ?If I change it to: >> > >> > unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb >> > >> > everything works fine. >> >> Hmm, maybe the config file path shouldn't be expanded then. ?Especially >> since it's only specified in the command-line. ?/me ponders a bit... > > Hi Michael, > > Just pushed this out, too. ?This should work better for you with: > > ?unicorn_rails -D -E production -c config/unicorn.rb > > Oddly, this has been a problem since the beginning of time and probably > confused a good amount of people. > > Thanks Michael for finally bringing it to my attention. ?Most folks I > know favor absolute ones (including myself), and don't hit this > problem... ?Of course let me know if I broke something for anyone, > I hope not... > Thank you Eric, it has been my pleasure to put to use such a great tool in my ruby arsenal. Your contribution to the community is greatly appreciated. Michael Guterl From mguterl at gmail.com Wed Nov 18 21:47:05 2009 From: mguterl at gmail.com (Michael Guterl) Date: Wed, 18 Nov 2009 21:47:05 -0500 Subject: Log File Location In-Reply-To: <0F1B76E7-F7F5-4A3C-8C15-668A5FAF5798@synergyeoc.com> References: <0F1B76E7-F7F5-4A3C-8C15-668A5FAF5798@synergyeoc.com> Message-ID: <944a03770911181847i20d5e886s28bf583d47a1c251@mail.gmail.com> On Wed, Nov 18, 2009 at 9:06 PM, Preston Marshall wrote: > This may be a dumb question, but where can I see the error logs that are > usually displayed when unicorn is not daemonized? ?The production.log file > is empty in my app. ?Everything is currently running as root. I'm not sure where the logs are directed by default, but I set their location in my unicorn config like so: stdout_path '/home/deploy/public_html/rm/current/log/unicorn.log' stderr_path '/home/deploy/public_html/rm/current/log/unicorn-error.log' More information here: http://unicorn.bogomips.org/Unicorn/Configurator.html Best, Michael Guterl From dstamat at elctech.com Wed Nov 18 22:25:38 2009 From: dstamat at elctech.com (Dylan Stamat) Date: Wed, 18 Nov 2009 19:25:38 -0800 Subject: Log File Location In-Reply-To: <944a03770911181847i20d5e886s28bf583d47a1c251@mail.gmail.com> References: <0F1B76E7-F7F5-4A3C-8C15-668A5FAF5798@synergyeoc.com> <944a03770911181847i20d5e886s28bf583d47a1c251@mail.gmail.com> Message-ID: On Nov 18, 2009, at 6:47 PM, Michael Guterl wrote: > On Wed, Nov 18, 2009 at 9:06 PM, Preston Marshall > wrote: >> This may be a dumb question, but where can I see the error logs that are >> usually displayed when unicorn is not daemonized? The production.log file >> is empty in my app. Everything is currently running as root. > > I'm not sure where the logs are directed by default, but I set their > location in my unicorn config like so: > > stdout_path '/home/deploy/public_html/rm/current/log/unicorn.log' > stderr_path '/home/deploy/public_html/rm/current/log/unicorn-error.log' > > More information here: http://unicorn.bogomips.org/Unicorn/Configurator.html > > Best, > Michael Guterl > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn I ended up doing something like: unicorn_logger = Logger.new("/home/xxxxxx/current/log/unicorn.log") unicorn_logger.formatter = Logger::Formatter.new logger unicorn_logger ... but, I think you can do this as well: Configurator::DEFAULTS[:logger].formatter = Logger::Formatter.new See: http://rubyforge.org/pipermail/mongrel-unicorn/2009-November/000147.html == Dylan From normalperson at yhbt.net Thu Nov 19 01:54:57 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 18 Nov 2009 22:54:57 -0800 Subject: Log File Location In-Reply-To: <944a03770911181847i20d5e886s28bf583d47a1c251@mail.gmail.com> References: <0F1B76E7-F7F5-4A3C-8C15-668A5FAF5798@synergyeoc.com> <944a03770911181847i20d5e886s28bf583d47a1c251@mail.gmail.com> Message-ID: <20091119065457.GA12592@dcvr.yhbt.net> Michael Guterl wrote: > On Wed, Nov 18, 2009 at 9:06 PM, Preston Marshall > wrote: > > This may be a dumb question, but where can I see the error logs that are > > usually displayed when unicorn is not daemonized? ?The production.log file > > is empty in my app. ?Everything is currently running as root. > > I'm not sure where the logs are directed by default, but I set their They go to /dev/null by default when daemonizing. But yes, it's recommended that you configure them to something like below: > location in my unicorn config like so: > > stdout_path '/home/deploy/public_html/rm/current/log/unicorn.log' > stderr_path '/home/deploy/public_html/rm/current/log/unicorn-error.log' From hbartels at i-neda.com Thu Nov 19 02:49:31 2009 From: hbartels at i-neda.com (huet bartels) Date: Thu, 19 Nov 2009 07:49:31 +0000 Subject: Nginx Conf Message-ID: <1258616971.7438.1.camel@hbartels-laptop> Thank you all very much for you time. I will have a look at the configuration examples today. regards Huet Bartels > Dear all, > > I know members of the list use nginx to forward to unicorn. Would some > be so kind as to share a code snippet on how they have configured nginx > to use unicorn. > > thanks in advance > > Huet Bartels > > > From hukl at h3q.com Thu Nov 19 03:11:17 2009 From: hukl at h3q.com (John-Paul Bader) Date: Thu, 19 Nov 2009 09:11:17 +0100 Subject: Nginx Conf In-Reply-To: <13215A21-5E6A-4C94-9495-14A27D1FC9E5@elctech.com> References: <1258534700.7580.8.camel@hbartels-laptop> <489AA732-2AD2-4CF9-B330-4E4C37196EA4@berlin.ccc.de> <13215A21-5E6A-4C94-9495-14A27D1FC9E5@elctech.com> Message-ID: Hey Dylan, I like varnish a lot as it is so flexible and fast. It really is! I've used Apaches and Nginx built in caching but they simply cannot match what varnish has to offer. You have full control and what i would call deep request introspection for debugging which helped me debug issues not related to caching. So yeah - If you need serious caching (and who does not) than I'd recommend trying varnish Kind regards, John On 18.11.2009, at 19:17, Dylan Stamat wrote: > Hey Huet! On CentOS, I'm using something like this: > http://pastie.org/704551 > > I'd like to be using the UDS's, but our architecture requires the use of TCP at the moment. > > John, how do you like having Varnish up front, and, what do you use it for? > I was thinking about throwing it up simply for serving static assets, but I read somewhere that > Nginx was definitely sufficient for doing so. > > > On Nov 18, 2009, at 1:35 AM, John-Paul Bader wrote: > >> On a FreeBSD 7.2 >> >> varnish:80 -> nginx:8080 -> unicorn >> >> and for ssl >> >> nginx:443 -> varnish:80 -> nginx 8080 -> unicorn >> >> >> Config: http://pastie.textmate.org/703919 >> >> >> Kind regards, John >> >> >> On 18.11.2009, at 09:58, huet bartels wrote: >> >>> Dear all, >>> >>> I know members of the list use nginx to forward to unicorn. Would some >>> be so kind as to share a code snippet on how they have configured nginx >>> to use unicorn. >>> >>> thanks in advance >>> >>> Huet Bartels >>> >>> >>> >>> _______________________________________________ >>> mongrel-unicorn mailing list >>> mongrel-unicorn at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn >>> >> >> _______________________________________________ >> mongrel-unicorn mailing list >> mongrel-unicorn at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-unicorn > > Dylan Stamat > > ELC Technologies (TM) > 1921 State Street > Santa Barbara, CA 93101 > dstamat at elctech.com > > (866)863-7365 Tel > (866)893-1902 Fax > > +44 (0)20 7504 1346 Tel - London Office > +44 (0)20 7504 1347 Fax - London Office > > http://www.elctech.com > > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From normalperson at yhbt.net Sat Nov 21 17:00:41 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 21 Nov 2009 14:00:41 -0800 Subject: [ANN] unicorn 0.95.1 - we <3 symlink deployments more Message-ID: <20091121220041.GA20599@dcvr.yhbt.net> Unicorn is an HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: Configuration files paths given on the command-line are no longer expanded. This should make configuration reloads possible when a non-absolute path is specified for --config-file and Unicorn was deployed to a symlink directories (as with Capistrano). Since deployments have always been strongly encouraged to use absolute paths in the config file, this change does not affect absolute path users. This is our first gem release using gemcutter. And we *just* got it working while I was writing this message :> Eric Wong (3): SIGNALS: HUP + preload_app cannot reload app code Do not expand paths given on the shell GNUmakefile: prep release process for gemcutter -- Eric Wong From normalperson at yhbt.net Sun Nov 22 17:53:50 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 22 Nov 2009 22:53:50 +0000 Subject: Nginx Conf In-Reply-To: <1258616971.7438.1.camel@hbartels-laptop> References: <1258616971.7438.1.camel@hbartels-laptop> Message-ID: <20091122225350.GC29926@dcvr.yhbt.net> huet bartels wrote: > Thank you all very much for you time. > > I will have a look at the configuration examples today. Sorry for the late reply, I forgot about this thread (and I'm lazy about following links). One general thing about the nginx configs I've seen is that they're missing the fail_timeout=0 directive in the "server" lines. I highly recommend setting it since it's a low cost to try an upstream for nginx, and you can avoid 502 errors in case there's a bug in your app that causes a Unicorn worker to not send a valid HTTP response (including hitting the app timeout). I actually have this in the Configurator documentation[1]: # # See http://wiki.nginx.org/NginxHttpUpstreamModule for more details # # on nginx upstream configuration: # upstream unicorn_backend { # # for UNIX domain socket setups: # server unix:/path/to/unicorn.sock fail_timeout=0; # # # for TCP setups # server 192.168.0.7:8080 fail_timeout=0; # server 192.168.0.8:8080 fail_timeout=0; # server 192.168.0.9:8080 fail_timeout=0; # } [1] - http://unicorn.bogomips.org/Unicorn/Configurator.html We've had fail_timeout=0 deployed to several places (many non-Unicorn servers) here and there and have experienced no negative effects (we're pretty good about keeping our backends up :) If anybody can recommend a better place in the Unicorn docs to put this, that'd be great, too... Maybe I'll drop something in the examples/ directory. -- Eric Wong From dstamat at elctech.com Sun Nov 22 22:57:02 2009 From: dstamat at elctech.com (Dylan Stamat) Date: Sun, 22 Nov 2009 19:57:02 -0800 Subject: Nginx Conf In-Reply-To: <20091122225350.GC29926@dcvr.yhbt.net> References: <1258616971.7438.1.camel@hbartels-laptop> <20091122225350.GC29926@dcvr.yhbt.net> Message-ID: <38731B72-8842-4F85-8A77-C0459052CC3D@elctech.com> On Nov 22, 2009, at 2:53 PM, Eric Wong wrote: > huet bartels wrote: >> Thank you all very much for you time. >> >> I will have a look at the configuration examples today. > > Sorry for the late reply, I forgot about this thread (and I'm lazy > about following links). > > One general thing about the nginx configs I've seen is that they're > missing the fail_timeout=0 directive in the "server" lines. > > I highly recommend setting it since it's a low cost to try an upstream > for nginx, and you can avoid 502 errors in case there's a bug in your > app that causes a Unicorn worker to not send a valid HTTP response > (including hitting the app timeout). > > I actually have this in the Configurator documentation[1]: > > # # See http://wiki.nginx.org/NginxHttpUpstreamModule for more details > # # on nginx upstream configuration: > # upstream unicorn_backend { > # # for UNIX domain socket setups: > # server unix:/path/to/unicorn.sock fail_timeout=0; > # > # # for TCP setups > # server 192.168.0.7:8080 fail_timeout=0; > # server 192.168.0.8:8080 fail_timeout=0; > # server 192.168.0.9:8080 fail_timeout=0; > # } > > [1] - http://unicorn.bogomips.org/Unicorn/Configurator.html > > We've had fail_timeout=0 deployed to several places (many non-Unicorn > servers) here and there and have experienced no negative effects > (we're pretty good about keeping our backends up :) > > If anybody can recommend a better place in the Unicorn docs to put this, > that'd be great, too... Maybe I'll drop something in the examples/ > directory. > > -- > Eric Wong > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn Hey Eric, Yeah, an nginx.conf in examples/ would be great. It's probably going to be the most widely used front for Unicorn/Rainbows!, so a sample config with some explanations here and there would be awesome. It was great setting up Unicorn and being able to just grab the init.sh out of there! In terms of the fail_timeout, I haven't seen a nginx.conf with that entry yet! Maybe I'm looking at configuration files on the wrong projects ;) So, since the upstream attempts are cheap, the combination of a max_fail of 1 and fail_timeout of 0 is ideal? If the fail_timeout was at 10, and all the upstreams timed out, wouldn't it restart at the beginning of the round-robin, and not block... or... would it actually not retry on any due to the inoperable state? Thanks! == Dylan Stamat ELC Technologies (TM) 1921 State Street Santa Barbara, CA 93101 dstamat at elctech.com (866)863-7365 Tel (866)893-1902 Fax +44 (0)20 7504 1346 Tel - London Office +44 (0)20 7504 1347 Fax - London Office http://www.elctech.com From normalperson at yhbt.net Sat Nov 21 17:00:42 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 22 Nov 2009 07:00:42 +0900 Subject: [ANN] unicorn 0.95.1 - we <3 symlink deployments more Message-ID: <20091121220041.GA20599@dcvr.yhbt.net> Unicorn is an HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: Configuration files paths given on the command-line are no longer expanded. This should make configuration reloads possible when a non-absolute path is specified for --config-file and Unicorn was deployed to a symlink directories (as with Capistrano). Since deployments have always been strongly encouraged to use absolute paths in the config file, this change does not affect absolute path users. This is our first gem release using gemcutter. And we *just* got it working while I was writing this message :> Eric Wong (3): SIGNALS: HUP + preload_app cannot reload app code Do not expand paths given on the shell GNUmakefile: prep release process for gemcutter -- Eric Wong From normalperson at yhbt.net Tue Nov 24 02:19:17 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 23 Nov 2009 23:19:17 -0800 Subject: Nginx Conf In-Reply-To: <38731B72-8842-4F85-8A77-C0459052CC3D@elctech.com> References: <1258616971.7438.1.camel@hbartels-laptop> <20091122225350.GC29926@dcvr.yhbt.net> <38731B72-8842-4F85-8A77-C0459052CC3D@elctech.com> Message-ID: <20091124071917.GA8233@dcvr.yhbt.net> Dylan Stamat wrote: > On Nov 22, 2009, at 2:53 PM, Eric Wong wrote: > > One general thing about the nginx configs I've seen is that they're > > missing the fail_timeout=0 directive in the "server" lines. > > > > I highly recommend setting it since it's a low cost to try an upstream > > for nginx, and you can avoid 502 errors in case there's a bug in your > > app that causes a Unicorn worker to not send a valid HTTP response > > (including hitting the app timeout). > > Hey Eric, > > Yeah, an nginx.conf in examples/ would be great. > It's probably going to be the most widely used front for > Unicorn/Rainbows!, so a sample config with some explanations here and > there would be awesome. It was great setting up Unicorn and being > able to just grab the init.sh out of there! OK, I'll push out an nginx example in a bit, need to make sure things are adequately explained and linked. > In terms of the fail_timeout, I haven't seen a nginx.conf with that > entry yet! Maybe I'm looking at configuration files on the wrong > projects ;) So, since the upstream attempts are cheap, the combination > of a max_fail of 1 and fail_timeout of 0 is ideal? If the > fail_timeout was at 10, and all the upstreams timed out, wouldn't it > restart at the beginning of the round-robin, and not block... or... > would it actually not retry on any due to the inoperable state? max_fails doesn't seem to have any effect when fail_timeout=0. Setting max_fails=0 means the same thing as fail_timeout=0 in nginx >=0.6.33 (it would segfault before :x) I've just been using fail_timeout=0 since what seems like forever in nginx... Reading ngx_http_upstream_round_robin.c again, it appears seems that fail_timeout/max_fails is supposed to get ignored if *all* upstreams are failing. So if you have a single upstream it looks like you're safe even if your Unicorn master nukes workers. On the other hand I remember this not being the case at some point a while back; an entire cluster got effectively shut down because of it. Just set fail_timeout=0 and stop worrying about it :) -- Eric Wong From sunaku at gmail.com Wed Nov 25 23:50:56 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Wed, 25 Nov 2009 20:50:56 -0800 Subject: feature request - when_ready() hook Message-ID: Hello, I've been trying to achieve truly transparent zero downtime deploys with Unicorn and Rails for some time now (using SIGUSR2 and SIGQUIT strategy) and I've always hit the problem of my "last worker sends SIGQUIT to the old master" logic being executed way too soon. In particular, I tried killing the old master in: * before_fork() -- approx. 2 minute downtime * after_fork() -- approx. 2 minute downtime * storing the old-master-killing logic inside a lambda in after_fork() (for the last worker only) and later executing that lambda in Rails' config.after_initialize() hook -- approx. 20 second downtime As you can see, the more I delayed the execution of that "killing the old master" logic, the closer I got to zero downtime deploys. In this manner, I request the addition of a when_ready() hook which is executed just after Unicorn prints "worker=# ready!" to its error log inside Unicorn::HttpServer#worker_loop(). I am happy to implement this (with tests) and submit a patch, but I first wanted to know your opinion on this approach. (I should note that my unicorn setup does not run very close to the memory limit of its host; instead, the amount of free memory is more than double of the current unicorn memory footprint, so I can safely spawn a second set of Unicorn master + workers (via SIGUSR2) without worrying about the SIGTTOU before_fork() strategy shown in the Unicorn configuration example.) Thanks for your consideration. From normalperson at yhbt.net Thu Nov 26 01:05:19 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 25 Nov 2009 22:05:19 -0800 Subject: feature request - when_ready() hook In-Reply-To: References: Message-ID: <20091126060519.GC22762@dcvr.yhbt.net> Suraj Kurapati wrote: > Hello, > > I've been trying to achieve truly transparent zero downtime deploys > with Unicorn and Rails for some time now (using SIGUSR2 and SIGQUIT > strategy) and I've always hit the problem of my "last worker sends > SIGQUIT to the old master" logic being executed way too soon. > > In particular, I tried killing the old master in: > > * before_fork() -- approx. 2 minute downtime > * after_fork() -- approx. 2 minute downtime > * storing the old-master-killing logic inside a lambda in after_fork() > (for the last worker only) and later executing that lambda in Rails' > config.after_initialize() hook -- approx. 20 second downtime Hi Suraj, I'm looking at those times and can't help but wonder if there's something very weird/broken with your setup.. 20 seconds is already an eternity (even with preload_app=false), but 2 minutes?(!). Are you doing per-process listeners and retrying? The new ones could be fighting for a port held by the old workers... Other than that... I have many questions, because those times look extremely scary to me and I wonder if such a hook would only be masking the symptoms of a bigger problem. What kind of software/hardware stack are you running? (please don't say NSLU2 :) How many workers? How heavy is traffic on the site when you're deploying? How long does it take for a single worker to get ready and start serving requests? Are you using preload_app? It should be faster if you do, but there really appears to be something else wrong based on those times. Thanks in advance. > As you can see, the more I delayed the execution of that "killing the > old master" logic, the closer I got to zero downtime deploys. In this > manner, I request the addition of a when_ready() hook which is > executed just after Unicorn prints "worker=# ready!" to its error log > inside Unicorn::HttpServer#worker_loop(). At this stage, maybe even implementing something as middleware and making it hook into request processing (that way you really know the worker is really responding to requests) is the way to go... > I am happy to implement this (with tests) and submit a patch, but I > first wanted to know your opinion on this approach. (I should note > that my unicorn setup does not run very close to the memory limit of > its host; instead, the amount of free memory is more than double of > the current unicorn memory footprint, so I can safely spawn a second > set of Unicorn master + workers (via SIGUSR2) without worrying about > the SIGTTOU before_fork() strategy shown in the Unicorn configuration > example.) Given your memory availability, I wouldn't even worry about the automated killing of the old workers. Automatically killing old workers means you need a redeploy to roll back changes, whereas if you SIGWINCH the old set away, you can HUP the old master to bring them back in case the new set is having problems. -- Eric Wong From dvdplm at gmail.com Thu Nov 26 09:39:32 2009 From: dvdplm at gmail.com (David Palm) Date: Thu, 26 Nov 2009 15:39:32 +0100 Subject: GC settings for unicorn? Message-ID: <20091126153932341234.878b7f4e@gmail.com> Hi all, we have a batch of unicorn workers running behind nginx in production since a week now. Works awesomely. We're using REE 1.8.7 and I'd like to tune the GC settings to something like: RUBY_HEAP_FREE_MIN=10000 RUBY_HEAP_MIN_SLOTS=40000 RUBY_HEAP_SLOTS_INCREMENT=20000 RUBY_GC_MALLOC_LIMIT=40000000 RUBY_HEAP_SLOTS_GROWTH_FACTOR=1.1 Now, how should I go about getting unicorn to pick that up the ENV? I tried sticking the above exports into /etc/profile.d/ree_gc_tuning.sh but when unicorn spawns workers they don't get the environment from the spawning user (root). Our unicorn.rb config file has this in the after_fork section: # Unicorn master is started as root, which is fine, but let's # drop the workers to mongrel:mongrel begin uid, gid = Process.euid, Process.egid user, group = 'mongrel', 'mongrel' target_uid = Etc.getpwnam(user).uid target_gid = Etc.getgrnam(group).gid worker.tmp.chown(target_uid, target_gid) if uid != target_uid || gid != target_gid Process.initgroups(user, target_gid) Process::GID.change_privilege(target_gid) Process::UID.change_privilege(target_uid) end rescue => e raise e end To verify the contents of the env at this stage of the init I inserted a: File.open('/tmp/unicorn_init.log','w'){|f| f << "Who am I? #{`whoami`}\n#{`env`}"} The user is really "mongrel" at this stage and the env does not contain any of the GC tunings. Is there a best practise regarding this? Where should env var setting go? Is tuning the GC something we'd want to be able to do in the unicorn config file directly (feature request...)? Thanks for any pointers! David From normalperson at yhbt.net Thu Nov 26 11:55:41 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 26 Nov 2009 16:55:41 +0000 Subject: GC settings for unicorn? In-Reply-To: <20091126153932341234.878b7f4e@gmail.com> References: <20091126153932341234.878b7f4e@gmail.com> Message-ID: <20091126165541.GA26561@dcvr.yhbt.net> David Palm wrote: > Hi all, > we have a batch of unicorn workers running behind nginx in production since a week now. Works awesomely. > > We're using REE 1.8.7 and I'd like to tune the GC settings to something like: > RUBY_HEAP_FREE_MIN=10000 > RUBY_HEAP_MIN_SLOTS=40000 > RUBY_HEAP_SLOTS_INCREMENT=20000 > RUBY_GC_MALLOC_LIMIT=40000000 > RUBY_HEAP_SLOTS_GROWTH_FACTOR=1.1 > > Now, how should I go about getting unicorn to pick that up the ENV? I > tried sticking the above exports into /etc/profile.d/ree_gc_tuning.sh > but when unicorn spawns workers they don't get the environment from > the spawning user (root). Hi David, I would set those in the init/deploy script you're using. REE picks up those variables at startup before it ever sees Unicorn. However, if you want to change/modify them once your site is live without downtime, then you should be able to change them anywhere in the Unicorn config file (any valid Ruby can be in there, so just ENV["FOO"] = "BAR") and then SIGUSR2 to upgrade the process (and SIGQUIT the old master). It doesn't look like you can affect the running process with those GC settings, but environment variables are carried across fork + exec so the new one can pick them up at interpreter startup. > Our unicorn.rb config file has this in the after_fork section: > # Unicorn master is started as root, which is fine, but let's > # drop the workers to mongrel:mongrel Actually, with the latest versions you should be able to just do this: worker.user "mongrel", "mongrel" > Is there a best practise regarding this? Where should env var setting > go? Is tuning the GC something we'd want to be able to do in the > unicorn config file directly (feature request...)? > > Thanks for any pointers! No problem, let us know how it goes! -- Eric Wong From sunaku at gmail.com Thu Nov 26 14:05:08 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Thu, 26 Nov 2009 11:05:08 -0800 Subject: feature request - when_ready() hook In-Reply-To: <20091126060519.GC22762@dcvr.yhbt.net> References: <20091126060519.GC22762@dcvr.yhbt.net> Message-ID: On Wed, Nov 25, 2009 at 10:05 PM, Eric Wong wrote: > Suraj Kurapati wrote: >> * before_fork() -- approx. 2 minute downtime >> * after_fork() -- approx. 2 minute downtime >> * storing the old-master-killing logic inside a lambda in after_fork() >> (for the last worker only) and later executing that lambda in Rails' >> config.after_initialize() hook -- approx. 20 second downtime > > I'm looking at those times and can't help but wonder if there's > something very weird/broken with your setup.. ?20 seconds is already an > eternity (even with preload_app=false), but 2 minutes?(!). Yes, I am using preload_app=false. These delays mainly come from establishing DB connections and loading XML datasets into the Rails app. Our production DBs are pretty slow to give out new connections. The startup time is much faster in development, where I use SQLite. Please note that the reported downtimes are shocking only because they were *visible* downtimes, where the last worker of the new Unicorn master killed the old Unicorn master too soon. IMHO, it doesn't matter how long it takes for the Rails app to become ready, so long as the old Unicorn master + workers continue to exist & service requests until the new Unicorn master + workers take over. > Are you doing per-process listeners and retrying? ?The new ones could be > fighting for a port held by the old workers... ?Other than that... No, I have one listen() call (on a UNIX socket) at the top level of my Unicorn configuration file. Nothing fancy. > I have many questions, because those times look extremely scary to me > and I wonder if such a hook would only be masking the symptoms of > a bigger problem. > > What kind of software/hardware stack are you running? > (please don't say NSLU2 :) The hardware is some kind of VM running on a VM server farm, running CentOS 4. The software is Ruby 1.9.1-p243 with Rails 2.3.3, running on Unicorn 0.95.1, behind Nginx, behind M$ IIS. > How many workers? Three. > How heavy is traffic on the site when you're deploying? About 15 to 20 users. > How long does it take for a single worker to get ready and start > serving requests? Approximately 2 minutes. > Are you using preload_app? ?It should be faster if you do, but there > really appears to be something else wrong based on those times. I was for a few weeks, but I stopped because the XML dataset loading (see above) kept increasing the master's (and the new set of workers') memory footprint by 1.5x every time Unicorn was restarted via SIGUSR2. >> As you can see, the more I delayed the execution of that "killing the >> old master" logic, the closer I got to zero downtime deploys. ?In this >> manner, I request the addition of a when_ready() hook which is >> executed just after Unicorn prints "worker=# ready!" to its error log >> inside Unicorn::HttpServer#worker_loop(). > > At this stage, maybe even implementing something as middleware and > making it hook into request processing (that way you really know the > worker is really responding to requests) is the way to go... Hmm, but that would incur a penalty on each request (check if I've already killed the old master and do it if necessary). I'm pretty confident that killing the old master in the when_ready() hook will be Good Enough for my setup (at most I expect to see 1-2 second "down"time). Let me try this out and I'll tell you the results & submit a patch. >> my unicorn setup does not run very close to the memory limit of >> its host; instead, the amount of free memory is more than double of >> the current unicorn memory footprint, so I can safely spawn a second >> set of Unicorn master + workers (via SIGUSR2) without worrying about >> the SIGTTOU before_fork() strategy shown in the Unicorn configuration >> example.) > > Given your memory availability, I wouldn't even worry about the > automated killing of the old workers. > > Automatically killing old workers means you need a redeploy to roll back > changes, whereas if you SIGWINCH the old set away, you can HUP the old > master to bring them back in case the new set is having problems. Wow this is cool. Perhaps this strategy could be mentioned in the documentation? Thanks for your consideration. From normalperson at yhbt.net Thu Nov 26 14:53:48 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 26 Nov 2009 19:53:48 +0000 Subject: feature request - when_ready() hook In-Reply-To: References: <20091126060519.GC22762@dcvr.yhbt.net> Message-ID: <20091126195348.GB26561@dcvr.yhbt.net> Suraj Kurapati wrote: > On Wed, Nov 25, 2009 at 10:05 PM, Eric Wong wrote: > > Suraj Kurapati wrote: > >> * before_fork() -- approx. 2 minute downtime > >> * after_fork() -- approx. 2 minute downtime > >> * storing the old-master-killing logic inside a lambda in after_fork() > >> (for the last worker only) and later executing that lambda in Rails' > >> config.after_initialize() hook -- approx. 20 second downtime > > > > I'm looking at those times and can't help but wonder if there's > > something very weird/broken with your setup.. ?20 seconds is already an > > eternity (even with preload_app=false), but 2 minutes?(!). > > Yes, I am using preload_app=false. These delays mainly come from > establishing DB connections and loading XML datasets into the Rails > app. Our production DBs are pretty slow to give out new connections. > The startup time is much faster in development, where I use SQLite. Yikes. Is there some sort of misconfiguration in the DBs? Perhaps they're trying to do reverse DNS on every client connection, that can really ruin your day if your app servers don't have reverse DNS. > Please note that the reported downtimes are shocking only because they > were *visible* downtimes, where the last worker of the new Unicorn > master killed the old Unicorn master too soon. IMHO, it doesn't > matter how long it takes for the Rails app to become ready, so long as > the old Unicorn master + workers continue to exist & service requests > until the new Unicorn master + workers take over. > > Are you using preload_app? ?It should be faster if you do, but there > > really appears to be something else wrong based on those times. > > I was for a few weeks, but I stopped because the XML dataset loading > (see above) kept increasing the master's (and the new set of workers') > memory footprint by 1.5x every time Unicorn was restarted via SIGUSR2. Side problem, but another thing that makes me go "Huh?" Did the new master's footprint increase? Are you mmap()-ing the XML dataset? Is RSS increasing or just VmSize? Unicorn sets FD_CLOEXEC on the first 1024 (non-listener) file descriptors, so combined with exec(), that should give the new master (and subsequent workers) a clean memory footprint. > >> As you can see, the more I delayed the execution of that "killing the > >> old master" logic, the closer I got to zero downtime deploys. ?In this > >> manner, I request the addition of a when_ready() hook which is > >> executed just after Unicorn prints "worker=# ready!" to its error log > >> inside Unicorn::HttpServer#worker_loop(). > > > > At this stage, maybe even implementing something as middleware and > > making it hook into request processing (that way you really know the > > worker is really responding to requests) is the way to go... > > Hmm, but that would incur a penalty on each request (check if I've > already killed the old master and do it if necessary). I'm pretty > confident that killing the old master in the when_ready() hook will be > Good Enough for my setup (at most I expect to see 1-2 second > "down"time). Let me try this out and I'll tell you the results & > submit a patch. I don't think a runtime condition would be any more expensive than all the routing/filters/checks that any Rails app already does and you can cache the result into a global variable. As you may have noticed, I'm quite hesitant to add new features, especially for uncommon/rare cases. Things like supporting the "working_directory" directive and user-switching took *months* of debating with myself before they were finally added. > >> my unicorn setup does not run very close to the memory limit of > >> its host; instead, the amount of free memory is more than double of > >> the current unicorn memory footprint, so I can safely spawn a second > >> set of Unicorn master + workers (via SIGUSR2) without worrying about > >> the SIGTTOU before_fork() strategy shown in the Unicorn configuration > >> example.) > > > > Given your memory availability, I wouldn't even worry about the > > automated killing of the old workers. > > > > Automatically killing old workers means you need a redeploy to roll back > > changes, whereas if you SIGWINCH the old set away, you can HUP the old > > master to bring them back in case the new set is having problems. > > Wow this is cool. Perhaps this strategy could be mentioned in the > documentation? It's already at the bottom of the SIGNALS (http://unicorn.bogomips.org/SIGNALS.html) document > Thanks for your consideration. No problem, let us know if it's the DB doing reverse DNS because I've seen that to be a problem in a lot of cases. -- Eric Wong From dvdplm at gmail.com Mon Nov 30 06:41:17 2009 From: dvdplm at gmail.com (David Palm) Date: Mon, 30 Nov 2009 12:41:17 +0100 Subject: Snow Leopard 32bit kernel gotcha Message-ID: <20091130124117149012.b5485116@gmail.com> Hi, just thought I'd signal an issue I had this morning installing unicorn under Ruby 1.9.1. I'm using the exellent rvm to handle my rubies and made a mistake when I first installed it after passing to Snow Leopard. In order to build libraries with 64bit support you have to stick the following in your ~/.rvmrc: rvm_archflags="-arch x86_64" So my mistake was I built my ruby 1.9 before passing to a 64bit kernel. I got a pretty weird error from unicorn: In file included from unicorn_http.rl:7: ext_help.h: In function ?rb_18_str_set_len?: ext_help.h:20: error: ?struct RString? has no member named ?len? ext_help.h:21: error: ?struct RString? has no member named ?ptr? make: *** [unicorn_http.o] Error 1 That threw me off in the wrong direction, thinking it was yet another extension that didn't get the RSTRING struct changes right. Not so. Looking more carefully at mkmf.log it all became obvious: In file included from /Users/david/.rvm/ruby-1.9.1-p243/include/ruby-1.9.1/ruby.h:32, from conftest.c:1: /Users/david/.rvm/ruby-1.9.1-p243/include/ruby-1.9.1/ruby/ruby.h: In function ?INT2NUM?: /Users/david/.rvm/ruby-1.9.1-p243/include/ruby-1.9.1/ruby/ruby.h:464: warning: comparison is always true due to limited range of data type /Users/david/.rvm/ruby-1.9.1-p243/include/ruby-1.9.1/ruby/ruby.h:464: warning: comparison is always true due to limited range of data type /Users/david/.rvm/ruby-1.9.1-p243/include/ruby-1.9.1/ruby/ruby.h: In function ?UINT2NUM?: /Users/david/.rvm/ruby-1.9.1-p243/include/ruby-1.9.1/ruby/ruby.h:472: warning: comparison is always true due to limited range of data type ld: warning: in /Users/david/.rvm/ruby-1.9.1-p243/lib/libruby-static.a, file is not of required architecture Lipo to the rescue: lipo -info ~/.rvm/ruby-1.9.1-p243/lib/libruby-static.a input file /Users/david/.rvm/ruby-1.9.1-p243/lib/libruby-static.a is not a fat file Non-fat file: /Users/david/.rvm/ruby-1.9.1-p243/lib/libruby-static.a is architecture: i386 Solution? Remove old 1.9 install and reinstall everything (at least all C extensions). :) From sunaku at gmail.com Mon Nov 30 17:34:19 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Mon, 30 Nov 2009 14:34:19 -0800 Subject: where to chmod socket file? In-Reply-To: References: <20091113020351.GA5577@dcvr.yhbt.net> <20091115002433.GA29378@dcvr.yhbt.net> Message-ID: Eric Wong wrote: > [PATCH] configurator: listen :umask parameter for UNIX sockets Hi Eric, I'm using Unicorn 0.95.1 and I think that the :umask option is behaving inversely. For example, if I specify :umask => 0600, the socket file ends up with s---rwxrwx (same as chmod 0077) permissions. In contrast, if I specify :umask => 0177, the socket file ends up with srw------- (same as chmod 0600) permissions. Is umask normally specified as the inverse of a desired chmod? Thanks for your consideration. From sunaku at gmail.com Mon Nov 30 18:47:59 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Mon, 30 Nov 2009 15:47:59 -0800 Subject: feature request - when_ready() hook In-Reply-To: <20091126195348.GB26561@dcvr.yhbt.net> References: <20091126060519.GC22762@dcvr.yhbt.net> <20091126195348.GB26561@dcvr.yhbt.net> Message-ID: On Thu, Nov 26, 2009 at 11:53 AM, Eric Wong wrote: > Suraj Kurapati wrote: >> the XML dataset loading >> (see above) kept increasing the master's (and the new set of workers') >> memory footprint by 1.5x every time Unicorn was restarted via SIGUSR2. > > Side problem, but another thing that makes me go "Huh?" > > Did the new master's footprint increase? Yes, but this seems to have been my fault. I programmed the master (via the Unicorn configuration file) to accept a SIGPWR signal which made it (1) reload the XML dataset (i.e. added memory bloat) and (2) send SIGUSR2 to itself (thereby providing the new XML data to the new Unicorn master + workers). The idea was to cut down the time required for loading the XML dataset. Unfortunately, the extra memory bloat added by reloading the XML dataset carried over to the new Unicorn generation as a side-effect. >?Are you mmap()-ing the XML dataset? Nope, nothing fancy like that. > Is RSS increasing or just VmSize? Hmm, I did not pay attention to these individual stats. I just saw that the memory% statistic in `ps xv` would increase by 10% every time Unicorn was restarted through my SIGPWR handler. Again, this was my fault for using such a non-standard approach. >?Unicorn sets FD_CLOEXEC on > the first 1024 (non-listener) file descriptors, so combined with exec(), > that should give the new master (and subsequent workers) a clean memory > footprint. Thanks. This is good to know, now that I'm using the standard SIGUSR2/QUIT method. >> > At this stage, maybe even implementing something as middleware and >> > making it hook into request processing (that way you really know the >> > worker is really responding to requests) is the way to go... >> >> Hmm, but that would incur a penalty on each request (check if I've >> already killed the old master and do it if necessary). > > I don't think a runtime condition would be any more expensive than all > the routing/filters/checks that any Rails app already does and you can > cache the result into a global variable. Okay. > As you may have noticed, I'm quite hesitant to add new features, > especially for uncommon/rare cases. ?Things like supporting the > "working_directory" directive and user-switching took *months* of > debating with myself before they were finally added. No problem. I ended up using a simple workaround for this whole problem: from Capistrano, I send SIGUSR2 to the existing Unicorn master (which will become the old Unicorn master), wait 90 seconds, and then send SIGQUIT to the old Unicorn master. There's nothing fancy in my Unicorn configuration file anymore --- no before/after hooks at all; just a number of workers + listen directive. This configuration is working out pretty well, and I have finally achieved zero downtime deploys. (Yay! :-) The only thing I'm worried about is that I'll have to keep adjusting this timeout as the infrastructure my app depends upon becomes slower/faster. A when_ready() hook would really do wonders for me, and I will implement and try it as planned when I get some time. > let us know if it's the DB doing reverse DNS because > I've seen that to be a problem in a lot of cases. I'll ask about this and let you know. Thanks for your consideration. From normalperson at yhbt.net Mon Nov 30 19:41:35 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 30 Nov 2009 16:41:35 -0800 Subject: where to chmod socket file? In-Reply-To: References: <20091113020351.GA5577@dcvr.yhbt.net> <20091115002433.GA29378@dcvr.yhbt.net> Message-ID: <20091201004135.GA26299@dcvr.yhbt.net> Suraj Kurapati wrote: > Eric Wong wrote: > > [PATCH] configurator: listen :umask parameter for UNIX sockets > > Hi Eric, > > I'm using Unicorn 0.95.1 and I think that the :umask option is > behaving inversely. > > For example, if I specify :umask => 0600, the socket file ends up with > s---rwxrwx (same as chmod 0077) permissions. In contrast, if I > specify :umask => 0177, the socket file ends up with srw------- (same > as chmod 0600) permissions. > > Is umask normally specified as the inverse of a desired chmod? Hi Suraj, that's how umask works. "man 2 umask" -- Eric Wong From sunaku at gmail.com Mon Nov 30 20:36:05 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Mon, 30 Nov 2009 17:36:05 -0800 Subject: where to chmod socket file? In-Reply-To: <20091201004135.GA26299@dcvr.yhbt.net> References: <20091113020351.GA5577@dcvr.yhbt.net> <20091115002433.GA29378@dcvr.yhbt.net> <20091201004135.GA26299@dcvr.yhbt.net> Message-ID: On Mon, Nov 30, 2009 at 4:41 PM, Eric Wong wrote: > Suraj Kurapati wrote: >> Is umask normally specified as the inverse of a desired chmod? > > Hi Suraj, that's how umask works. ?"man 2 umask" Thanks!