From normalperson at yhbt.net Thu Oct 1 04:13:58 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 1 Oct 2009 01:13:58 -0700 Subject: draft release notes (so far) for upcoming 0.93.0 Message-ID: <20091001081358.GA13562@dcvr.yhbt.net> The one minor bugfix is only for users who set RAILS_RELATIVE_URL_ROOT in a config file. Users of the "--path" switch or those who set the environment variable in the shell were unaffected by this bug. Note that we still don't have relative URL root support for Rails < 2.3, and are unlikely to bother with it unless there is visible demand for it. I didn't even know people used/cared for relative URL roots before today (it was a Rails 2.3.4 user). New features (so far) includes support for :tries and :delay when specifying a "listen" in an after_fork hook. This was inspired by Chris Wanstrath's example of binding per-worker listen sockets in a loop while migrating (or upgrading) Unicorn. Setting a negative value for :tries means we'll retry the listen indefinitely until the socket becomes available. So you can do something like this in an after_fork hook: after_fork do |server,worker| addr = "127.0.0.1:#{9293 + worker.nr}" server.listen(addr, :tries => -1, :delay => 5) end There's also the usual round of added documentation, packaging fixes, code cleanups and minor performance improvements that are viewable in the git log.... Shortlog since v0.92.0 (so far) below: Eric Wong (45): build: hardcode the canonical git URL build: manifest dropped manpages build: smaller ChangeLog doc/LATEST: remove trailing newline http: don't force -fPIC if it can't be used .gitignore on *.rbc files Rubinius generates README/gemspec: a better description, hopefully GNUmakefile: add missing .manifest dep on test installs Add HACKING document configurator: fix user switch example in RDoc local.mk.sample: time and perms enforcement unicorn_rails: show "RAILS_ENV" in help message gemspec: compatibility with older Rubygems Split out KNOWN_ISSUES document KNOWN_ISSUES: add notes about the "isolate" gem gemspec: fix test_files regexp match gemspec: remove tests that fork from test_files test_signals: ensure we can parse pids in response GNUmakefile: cleanup test/manifest generation util: remove APPEND_FLAGS constant http_request: simplify and remove handle_body method http_response: simplify and remove const dependencies local.mk.sample: fix .js times TUNING: notes about benchmarking a high :backlog HttpServer#listen accepts :tries and :delay parameters "make install" avoids installing multiple .so objects Use Configurator#expand_addr in HttpServer#listen configurator: move initialization stuff to #initialize Remove "Z" constant for binary strings cgi_wrapper: don't warn about stdoutput usage cgi_wrapper: simplify status handling in response cgi_wrapper: use Array#concat instead of += server: correctly unset reexec_pid on child death configurator: update and modernize examples configurator: add colons in front of listen() options configurator: remove DEFAULT_LOGGER constant gemspec: clarify commented-out licenses section Add makefile targets for non-release installs cleanup: use question mark op for 1-byte comparisons RDoc for Unicorn::HttpServer::Worker small cleanup to pid file handling + documentation rails: RAILS_RELATIVE_URL_ROOT may be set in Unicorn config unicorn_rails: undeprecate --path switch manpages: document environment variables README: remove reference to different versions -- Eric Wong From normalperson at yhbt.net Thu Oct 1 13:49:11 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 1 Oct 2009 10:49:11 -0700 Subject: non-Linux users out there? Message-ID: <20091001174911.GA19155@dcvr.yhbt.net> Does anybody here run Unicorn on a non-Linux system such as FreeBSD/OSX/Solaris/NetBSD/OpenBSD? Does everything work as expected? I'm mainly concerned about the signal handling wrt log cycling and transparent upgrades, but there may be other things that don't work as expected... So if there's anything broken on a modern *nix kernel, please let us know. If you could provide shell access via ssh so we can fix and test directly on the box, even better. I currently only have Linux 2.6.x machines to test on, but I have been thinking about getting an inexpensive FreeBSD jail/VPS as a playground for Unicorn and other things anyways. If there are any recommendations (or shady places to avoid) for inexpensive non-Linux hosting providers, I'm welcome to suggestions. I very strongly prefer a non-Xen provider outside of California for disaster scenarios. -- Eric Wong From chris at ozmm.org Thu Oct 1 19:52:09 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Thu, 1 Oct 2009 16:52:09 -0700 Subject: TERM vs QUIT Message-ID: <8b73aaca0910011652y68ea41cai8685abfe1b1ec735@mail.gmail.com> Hi list, According to http://en.wikipedia.org/wiki/SIGTERM, SIGTERM should be used for graceful termination. According to http://www.cons.org/cracauer/sigint.html, SIGQUIT should be used for immediate termination. Is there any reason Unicorn seems to flip these two and use TERM for immediate termination and QUIT for graceful termination? We're running into a situation where some our daemons respond to the same signals in different ways and it'd be great if they were consistent. Thanks! -- Chris Wanstrath http://github.com/defunkt From chris at ozmm.org Thu Oct 1 19:56:42 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Thu, 1 Oct 2009 16:56:42 -0700 Subject: TERM vs QUIT In-Reply-To: <8b73aaca0910011652y68ea41cai8685abfe1b1ec735@mail.gmail.com> References: <8b73aaca0910011652y68ea41cai8685abfe1b1ec735@mail.gmail.com> Message-ID: <8b73aaca0910011656k4740f079v1f3f8def2b196ca1@mail.gmail.com> On Thu, Oct 1, 2009 at 4:52 PM, Chris Wanstrath wrote: > According to http://en.wikipedia.org/wiki/SIGTERM, SIGTERM should be > used for graceful termination. > > According to http://www.cons.org/cracauer/sigint.html, SIGQUIT should > be used for immediate termination. > > Is there any reason Unicorn seems to flip these two and use TERM for > immediate termination and QUIT for graceful termination? Looks like nginx flips them, too, and Unicorn's signal handling is based on nginx's. We will defer to the Russians and make our stuff consistent with nginx! Thanks, Chris From normalperson at yhbt.net Thu Oct 1 21:53:01 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 1 Oct 2009 18:53:01 -0700 Subject: TERM vs QUIT In-Reply-To: <8b73aaca0910011656k4740f079v1f3f8def2b196ca1@mail.gmail.com> References: <8b73aaca0910011652y68ea41cai8685abfe1b1ec735@mail.gmail.com> <8b73aaca0910011656k4740f079v1f3f8def2b196ca1@mail.gmail.com> Message-ID: <20091002015301.GA7353@dcvr.yhbt.net> Chris Wanstrath wrote: > On Thu, Oct 1, 2009 at 4:52 PM, Chris Wanstrath wrote: > > > According to http://en.wikipedia.org/wiki/SIGTERM, SIGTERM should be > > used for graceful termination. > > > > According to http://www.cons.org/cracauer/sigint.html, SIGQUIT should > > be used for immediate termination. > > > > Is there any reason Unicorn seems to flip these two and use TERM for > > immediate termination and QUIT for graceful termination? > > Looks like nginx flips them, too, and Unicorn's signal handling is > based on nginx's. > > We will defer to the Russians and make our stuff consistent with nginx! Yup, I based the current user interface around the one in nginx to capitalize on some of my target users' existing familiarity with nginx and also to (hopefully) lower the barrier of entry for folks new to both. Of course I'm not happy with some of the choices nginx made. Using "pid" instead of "pid_file" in the configuration language really bothers me, and I completely agree TERM is more logical for graceful shutdowns. But in the end the nginx UI was "good enough" and consistency won me over. So things like the init script in examples/init.sh can be adapted to work for both unicorn and nginx without changing anything other than path names and some (hopefully few) command-line arguments. Making user interface choices for for other people isn't something I'm comfortable with, so I just try to take advantage of what people already know/expect as much as possible. I'm lazy in that way :) For a few weeks (or months now?) I've been debating/avoiding the "--kill" switch item I have in the TODO (it was requested by a user in Real-Life(TM)). I'm still not sure where/how/if it would fit, and maybe it just doesn't fit. /me goes on to ponder it a bit more ... -- Eric Wong From normalperson at yhbt.net Fri Oct 2 04:11:02 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 2 Oct 2009 01:11:02 -0700 Subject: [PATCH] configurator: update some migration examples Message-ID: <20091002081102.GA11179@dcvr.yhbt.net> We now give an example of how a before_fork hook can be used to incrementally migrate off the old code base without hitting a thundering herd (especially in the "preload_app false") case. Also comment on the per-worker listen usage in the RDoc, not just a hidden comment. --- I just pushed this out earlier, this example was inspired by Chris Wanstrath's before_fork hook but modified to use SIGTTOU instead of SIGQUIT for all but the last worker to minimize memory explosions during upgrades. The :tries and :delay options were also inspired by Chris and will be in v0.93.0 which will probably be released. lib/unicorn/configurator.rb | 21 +++++++++++++++++---- 1 files changed, 17 insertions(+), 4 deletions(-) diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb index 7e66f60..bff8f7e 100644 --- a/lib/unicorn/configurator.rb +++ b/lib/unicorn/configurator.rb @@ -25,9 +25,26 @@ module Unicorn # # as there's no need for the master process to hold a connection # defined?(ActiveRecord::Base) and # ActiveRecord::Base.connection.disconnect! + # + # # the following allows a new master process to incrementally + # # phase out the old master process with SIGTTOU to avoid a + # # thundering herd (especially in the "preload_app false" case) + # # when doing a transparent upgrade. The last worker spawned + # # will then kill off the old master process with a SIGQUIT. + # old_pid = "#{server.config[:pid]}.oldbin" + # if old_pid != server.pid + # begin + # sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU + # Process.kill(sig, File.read(old_pid).to_i) + # rescue Errno::ENOENT, Errno::ESRCH + # end + # + # # optionally throttle the master from forking too quickly by sleeping + # sleep 1 # end # # after_fork do |server, worker| + # # per-process listener ports for debugging/admin/migrations # addr = "127.0.0.1:#{9293 + worker.nr}" # server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true) # @@ -50,10 +67,6 @@ module Unicorn :worker_processes => 1, :after_fork => lambda { |server, worker| server.logger.info("worker=#{worker.nr} spawned pid=#{$$}") - - # per-process listener ports for debugging/admin: - # addr = "127.0.0.1:#{8081 + worker.nr}" - # server.listen(addr, :tries => -1, :delay => 5) }, :before_fork => lambda { |server, worker| server.logger.info("worker=#{worker.nr} spawning...") -- Eric Wong From chris at ozmm.org Fri Oct 2 14:35:22 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Fri, 2 Oct 2009 11:35:22 -0700 Subject: TERM vs QUIT In-Reply-To: <20091002015301.GA7353@dcvr.yhbt.net> References: <8b73aaca0910011652y68ea41cai8685abfe1b1ec735@mail.gmail.com> <8b73aaca0910011656k4740f079v1f3f8def2b196ca1@mail.gmail.com> <20091002015301.GA7353@dcvr.yhbt.net> Message-ID: <8b73aaca0910021135t322f9686lee40d85ea69ad694@mail.gmail.com> On Thu, Oct 1, 2009 at 6:53 PM, Eric Wong wrote: > Yup, I based the current user interface around the one in nginx to > capitalize on some of my target users' existing familiarity with nginx > and also to (hopefully) lower the barrier of entry for folks new to > both. I think this is a great decision. Saying "Unicorn behaves just like nginx when sent HUP," for example, is really useful when learning a new technology. Thanks for the response! -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Fri Oct 2 17:44:21 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 2 Oct 2009 14:44:21 -0700 Subject: [ANN] unicorn 0.93.0 Message-ID: <20091002214421.GB13477@dcvr.yhbt.net> More small fixes and documentation improvements in this release, things have settled down a lot since 0.90.0 in August and we've mainly been focusing on documentation, minor UI improvements and the occasional minor bugfix as they get reported. Changes: The one minor bugfix is only for Rails 2.3.x+ users who set the RAILS_RELATIVE_URL_ROOT environment variable in a config file. Users of the "--path" switch or those who set the environment variable in the shell were unaffected by this bug. Note that we still don't have relative URL root support for Rails < 2.3, and are unlikely to bother with it unless there is visible demand for it. New features includes support for :tries and :delay when specifying a "listen" in an after_fork hook. This was inspired by Chris Wanstrath's example of binding per-worker listen sockets in a loop while migrating (or upgrading) Unicorn. Setting a negative value for :tries means we'll retry the listen indefinitely until the socket becomes available. So you can do something like this in an after_fork hook: after_fork do |server, worker| addr = "127.0.0.1:#{9293 + worker.nr}" server.listen(addr, :tries => -1, :delay => 5) end There's also the usual round of added documentation, packaging fixes, code cleanups, small fixes and minor performance improvements that are viewable in the "git log" output. * site: http://unicorn.bogomips.org/ * git: git://git.bogomips.org/unicorn.git * cgit: http://git.bogomips.org/cgit/unicorn.git/ * list: mongrel-unicorn at rubyforge.org * nntp: nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general * finger: unicorn at bogomips.org -- Eric Wong From jreitz at gmail.com Fri Oct 2 19:54:48 2009 From: jreitz at gmail.com (Jay Reitz) Date: Fri, 2 Oct 2009 16:54:48 -0700 Subject: 0.93.0 gem bug report: permissions snafu Message-ID: <850c32e30910021654j544c19afn2d482ceb451acee6@mail.gmail.com> Hey all- I'm not embarrassed to say that I love the unicorn, no matter how weird that sounds. ?When I first read the README, my hopes soared because the philosophy was exactly in line with how I'd imagined a ruby application server should behave. ?And so far I have been pleased with our initial results on a portion of traffic in production. I found some permissions issues with the 0.93.0 gem release. ?In short, the installed permissions are excessively strict and incorrect. ?The following files have the mode 700 rather than the correct 755 mode: lib/unicorn.rb lib/unicorn/configurator.rb lib/unicorn/http_request.rb lib/unicorn/http_response.rb lib/unicorn/socket_helper.rb This prevents unicorn from being started by a non-super user, as many people are wont to do. ?There are also a smattering of document files and the gemspec with incorrect permissions but this it less of an issue. Thanks again and I've subscribed to the mailing list. >j. From normalperson at yhbt.net Fri Oct 2 21:23:01 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 2 Oct 2009 18:23:01 -0700 Subject: 0.93.0 gem bug report: permissions snafu In-Reply-To: <850c32e30910021654j544c19afn2d482ceb451acee6@mail.gmail.com> References: <850c32e30910021654j544c19afn2d482ceb451acee6@mail.gmail.com> Message-ID: <20091003012301.GA9191@dcvr.yhbt.net> Jay Reitz wrote: > I found some permissions issues with the 0.93.0 gem release. ?In > short, the installed permissions are excessively strict and incorrect. > ?The following files have the mode 700 rather than the correct 755 > mode: > This prevents unicorn from being started by a non-super user, as many > people are wont to do. ?There are also a smattering of document files > and the gemspec with incorrect permissions but this it less of an > issue. Oops, I just uploaded v0.93.1 which should fix this. Huge thanks for reporting this (and using Unicorn :) I never would've noticed as all my Ruby and Gem installations are done as regular users -- Eric Wong From normalperson at yhbt.net Fri Oct 2 21:26:48 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 2 Oct 2009 18:26:48 -0700 Subject: [ANN] unicorn 0.93.1 Message-ID: <20091003012648.GB29768@dcvr.yhbt.net> Changes: Fix permissions for release tarballs/gems, no other changes. Thanks to Jay Reitz for reporting this. * site: http://unicorn.bogomips.org/ * git: git://git.bogomips.org/unicorn.git * cgit: http://git.bogomips.org/cgit/unicorn.git/ * list: mongrel-unicorn at rubyforge.org * nntp: nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general * finger: unicorn at bogomips.org -- Eric Wong From normalperson at yhbt.net Mon Oct 5 07:00:13 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 5 Oct 2009 04:00:13 -0700 Subject: [ANN] Rainbows! 0.1.0 initial release Message-ID: <20091005110013.GB19193@dcvr.yhbt.net> Rainbows! Unicorn for slow apps and slow clients Rainbows! is a HTTP server for Rack applications. It is based on Unicorn, but designed to handle applications that expect long request/response times and/or slow clients. For Rack applications not heavily bound by slow external dependencies, consider Unicorn instead as it simpler and easier to debug. Rainbows! includes previous work from the "gossamer" and "rainbows" branches of Unicorn and will support additional network concurrency models. It is basically for all the things that Unicorn sucks at :) * http://rainbows.rubyforge.org/ * rainbows-talk at rubyforge.org * git://git.bogomips.org/rainbows.git Changes: Initial release This release is currently highly experimental and is still missing a lot of test coverage. -- Eric Wong From chris at ozmm.org Tue Oct 6 17:13:10 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Tue, 6 Oct 2009 14:13:10 -0700 Subject: POST Body Truncated Message-ID: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> Hello list, I'm running into an issue where POST bodies are truncated by the time they hit Rails. It occurs with both TCP and UDS (both behind nginx). This problem does not happen when I'm running TCP Thin behind nginx. I have Unicorn 0.93.1 installed. I've tried tweaking the rcvbuf setting but it seems to do nothing. Any ideas? Thanks! -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Tue Oct 6 17:22:04 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 6 Oct 2009 14:22:04 -0700 Subject: POST Body Truncated In-Reply-To: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> Message-ID: <20091006212204.GA13411@dcvr.yhbt.net> Chris Wanstrath wrote: > Hello list, > > I'm running into an issue where POST bodies are truncated by the time > they hit Rails. It occurs with both TCP and UDS (both behind nginx). > > This problem does not happen when I'm running TCP Thin behind nginx. > > I have Unicorn 0.93.1 installed. I've tried tweaking the rcvbuf > setting but it seems to do nothing. > > Any ideas? Thanks! This is not good :x It could be a bug in the relatively new TeeInput class. How large are the POST bodies that are getting truncated? Which version of Rails is this and which middleware(s), if any, are you running? I'd like to get to the bottom of this ASAP, thanks! -- Eric Wong From chris at ozmm.org Tue Oct 6 17:26:45 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Tue, 6 Oct 2009 14:26:45 -0700 Subject: POST Body Truncated In-Reply-To: <20091006212204.GA13411@dcvr.yhbt.net> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> Message-ID: <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> On Tue, Oct 6, 2009 at 2:22 PM, Eric Wong wrote: > This is not good :x > > It could be a bug in the relatively new TeeInput class. ?How large are > the POST bodies that are getting truncated? ?Which version of Rails is > this and which middleware(s), if any, are you running? ?I'd like to get > to the bottom of this ASAP, thanks! My current test payload is 25740 bytes, consistently truncated to 12945 bytes when using Unicorn with TCP. We're on Rails 2.2.2 - no middlewares, no Rack. -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Tue Oct 6 17:30:12 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 6 Oct 2009 14:30:12 -0700 Subject: POST Body Truncated In-Reply-To: <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> Message-ID: <20091006213012.GA31036@dcvr.yhbt.net> Chris Wanstrath wrote: > On Tue, Oct 6, 2009 at 2:22 PM, Eric Wong wrote: > > > This is not good :x > > > > It could be a bug in the relatively new TeeInput class. ?How large are > > the POST bodies that are getting truncated? ?Which version of Rails is > > this and which middleware(s), if any, are you running? ?I'd like to get > > to the bottom of this ASAP, thanks! > > My current test payload is 25740 bytes, consistently truncated to > 12945 bytes when using Unicorn with TCP. > > We're on Rails 2.2.2 - no middlewares, no Rack. Thanks. I'll take a look. Do you have any example controllers/actions to reproduce this with so I can track this down easier? (I completely understand if you can't share secrets, too :). Thanks. -- Eric Wong From chris at ozmm.org Tue Oct 6 17:46:39 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Tue, 6 Oct 2009 14:46:39 -0700 Subject: POST Body Truncated In-Reply-To: <20091006213012.GA31036@dcvr.yhbt.net> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> Message-ID: <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> On Tue, Oct 6, 2009 at 2:30 PM, Eric Wong wrote: > Thanks. ?I'll take a look. ?Do you have any example controllers/actions > to reproduce this with so I can track this down easier? ?(I completely > understand if you can't share secrets, too :). ?Thanks. It happens everywhere for us - big comments, big wikis, big file edits, big gists. I'm not sure what code I could share because it happens between the browser sending a POST and Rails actually receiving the request. Let me know if you have a suggestion for what you want and I'll whip it up. -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Tue Oct 6 17:50:14 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 6 Oct 2009 14:50:14 -0700 Subject: POST Body Truncated In-Reply-To: <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> Message-ID: <20091006215014.GA5460@dcvr.yhbt.net> Chris Wanstrath wrote: > On Tue, Oct 6, 2009 at 2:30 PM, Eric Wong wrote: > > > Thanks. ?I'll take a look. ?Do you have any example controllers/actions > > to reproduce this with so I can track this down easier? ?(I completely > > understand if you can't share secrets, too :). ?Thanks. > > It happens everywhere for us - big comments, big wikis, big file > edits, big gists. > > I'm not sure what code I could share because it happens between the > browser sending a POST and Rails actually receiving the request. Let > me know if you have a suggestion for what you want and I'll whip it > up. No worries, I'm working on a a reproducible test case. Can you try and see if you're using curl's -F to POST, you _don't_ hit it? Thanks. -- Eric Wong From chris at ozmm.org Tue Oct 6 17:58:30 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Tue, 6 Oct 2009 14:58:30 -0700 Subject: POST Body Truncated In-Reply-To: <20091006215014.GA5460@dcvr.yhbt.net> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> <20091006215014.GA5460@dcvr.yhbt.net> Message-ID: <8b73aaca0910061458x3b70ef51hd166fb03b7e5bba@mail.gmail.com> On Tue, Oct 6, 2009 at 2:50 PM, Eric Wong wrote: > No worries, I'm working on a a reproducible test case. ?Can you try > and see if you're using curl's -F to POST, you _don't_ hit it? `curl -F` does NOT trigger POST body truncation. -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Tue Oct 6 18:32:59 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 6 Oct 2009 15:32:59 -0700 Subject: POST Body Truncated In-Reply-To: <8b73aaca0910061458x3b70ef51hd166fb03b7e5bba@mail.gmail.com> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> <20091006215014.GA5460@dcvr.yhbt.net> <8b73aaca0910061458x3b70ef51hd166fb03b7e5bba@mail.gmail.com> Message-ID: <20091006223259.GB5460@dcvr.yhbt.net> Chris Wanstrath wrote: > On Tue, Oct 6, 2009 at 2:50 PM, Eric Wong wrote: > > > No worries, I'm working on a a reproducible test case. ?Can you try > > and see if you're using curl's -F to POST, you _don't_ hit it? > > `curl -F` does NOT trigger POST body truncation. Still trying to reproduce this, I think you're hitting a buffer boundary in nginx and triggering a bug in Unicorn::TeeInput somewhere. You may be able to increase some userspace buffer sizes in nginx to workaround it until I (or somebody else) can fix it (hopefully very soon now). -- Eric Wong From normalperson at yhbt.net Tue Oct 6 18:52:19 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 6 Oct 2009 15:52:19 -0700 Subject: POST Body Truncated In-Reply-To: <20091006223259.GB5460@dcvr.yhbt.net> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> <20091006215014.GA5460@dcvr.yhbt.net> <8b73aaca0910061458x3b70ef51hd166fb03b7e5bba@mail.gmail.com> <20091006223259.GB5460@dcvr.yhbt.net> Message-ID: <20091006225219.GA7831@dcvr.yhbt.net> Eric Wong wrote: > Chris Wanstrath wrote: > > On Tue, Oct 6, 2009 at 2:50 PM, Eric Wong wrote: > > > > > No worries, I'm working on a a reproducible test case. ?Can you try > > > and see if you're using curl's -F to POST, you _don't_ hit it? > > > > `curl -F` does NOT trigger POST body truncation. > > Still trying to reproduce this, I think you're hitting a buffer boundary > in nginx and triggering a bug in Unicorn::TeeInput somewhere. You may > be able to increase some userspace buffer sizes in nginx to > workaround it until I (or somebody else) can fix it (hopefully very soon > now). OK, here's a workaround that should work for now. I have to hit the road in a few minutes but will be back on a computer in a few hours. This only affects older Rails (and I'm supposed to still be supporting 1.2.x!) and its interaction with a wrapped CGI.stdinput somewhere is going bad... diff --git a/lib/unicorn/app/old_rails.rb b/lib/unicorn/app/old_rails.rb index ba1260a..51bda52 100644 --- a/lib/unicorn/app/old_rails.rb +++ b/lib/unicorn/app/old_rails.rb @@ -14,6 +14,8 @@ module Unicorn; module App; end; end class Unicorn::App::OldRails def call(env) + env['rack.input'].each { |x| } + env['rack.input'].rewind cgi = Unicorn::CGIWrapper.new(env) begin Dispatcher.dispatch(cgi, --- I'll have a better diagnosis+fix+test when I return. Sorry for the bug :x -- Eric Wong From chris at ozmm.org Tue Oct 6 18:57:55 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Tue, 6 Oct 2009 15:57:55 -0700 Subject: POST Body Truncated In-Reply-To: <20091006225219.GA7831@dcvr.yhbt.net> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> <20091006215014.GA5460@dcvr.yhbt.net> <8b73aaca0910061458x3b70ef51hd166fb03b7e5bba@mail.gmail.com> <20091006223259.GB5460@dcvr.yhbt.net> <20091006225219.GA7831@dcvr.yhbt.net> Message-ID: <8b73aaca0910061557l7bed19cu1c122e1f1b5a68d9@mail.gmail.com> On Tue, Oct 6, 2009 at 3:52 PM, Eric Wong wrote: > OK, here's a workaround that should work for now. ?I have to hit the > road in a few minutes but will be back on a computer in a few hours. > > This only affects older Rails (and I'm supposed to still be supporting > 1.2.x!) and its interaction with a wrapped CGI.stdinput somewhere is > going bad... This works! I suppose we should upgrade to a newer Rails :) As usual, thanks a million. -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Tue Oct 6 23:04:51 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 6 Oct 2009 20:04:51 -0700 Subject: POST Body Truncated In-Reply-To: <8b73aaca0910061557l7bed19cu1c122e1f1b5a68d9@mail.gmail.com> References: <8b73aaca0910061413y51145254s8def0904080562a5@mail.gmail.com> <20091006212204.GA13411@dcvr.yhbt.net> <8b73aaca0910061426p1d949db2ga1d0241a0d5192c8@mail.gmail.com> <20091006213012.GA31036@dcvr.yhbt.net> <8b73aaca0910061446ud7e7ff8ia2cc1a711766dc3b@mail.gmail.com> <20091006215014.GA5460@dcvr.yhbt.net> <8b73aaca0910061458x3b70ef51hd166fb03b7e5bba@mail.gmail.com> <20091006223259.GB5460@dcvr.yhbt.net> <20091006225219.GA7831@dcvr.yhbt.net> <8b73aaca0910061557l7bed19cu1c122e1f1b5a68d9@mail.gmail.com> Message-ID: <20091007030451.GA20541@dcvr.yhbt.net> Chris Wanstrath wrote: > On Tue, Oct 6, 2009 at 3:52 PM, Eric Wong wrote: > > > OK, here's a workaround that should work for now. ?I have to hit the > > road in a few minutes but will be back on a computer in a few hours. > > > > This only affects older Rails (and I'm supposed to still be supporting > > 1.2.x!) and its interaction with a wrapped CGI.stdinput somewhere is > > going bad... > > This works! > > I suppose we should upgrade to a newer Rails :) > > As usual, thanks a million. Here's a real patch with lots of documentation I just pushed out, still working on automated test cases. Can you let me know how it works? Thanks. >From 438c99aec2d74489fa89b3a6c60d1fb41bb2f7e6 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Tue, 6 Oct 2009 19:45:05 -0700 Subject: [PATCH] more-compatible TeeInput#read for POSTs with Content-Length There are existing applications and libraries that don't check the return value of env['rack.input'].read(length) (like Rails :x). Those applications became broken under the IO#readpartial semantics of TeeInput#read when handling larger request bodies. We'll preserve the IO#readpartial semantics _only_ when handling chunked requests (as long as Rack allows it, it's useful for real-time processing of audio/video streaming uploads, especially with Rainbows! and mobile clients) but use read-in-full semantics for TeeInput#read on requests with a known Content-Length. --- lib/unicorn/tee_input.rb | 43 +++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 41 insertions(+), 2 deletions(-) diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb index 96a053a..188e2ea 100644 --- a/lib/unicorn/tee_input.rb +++ b/lib/unicorn/tee_input.rb @@ -41,6 +41,26 @@ module Unicorn @size = tmp_size end + # call-seq: + # ios = env['rack.input'] + # ios.read([length [, buffer ]]) => string, buffer, or nil + # + # Reads at most length bytes from the I/O stream, or to the end of + # file if length is omitted or is nil. length must be a non-negative + # integer or nil. If the optional buffer argument is present, it + # must reference a String, which will receive the data. + # + # At end of file, it returns nil or "" depend on length. + # ios.read() and ios.read(nil) returns "". + # ios.read(length [, buffer]) returns nil. + # + # If the Content-Length of the HTTP request is known (as is the common + # case for POST requests), then ios.read(length [, buffer]) will block + # until the specified length is read (or it is the last chunk). + # Otherwise, for uncommon "Transfer-Encoding: chunked" requests, + # ios.read(length [, buffer]) will return immediately if there is + # any data and only block when nothing is available (providing + # IO#readpartial semantics). def read(*args) socket or return @tmp.read(*args) @@ -55,9 +75,9 @@ module Unicorn rv = args.shift || @buf2.dup diff = tmp_size - @tmp.pos if 0 == diff - tee(length, rv) + ensure_length(tee(length, rv), length) else - @tmp.read(diff > length ? length : diff, rv) + ensure_length(@tmp.read(diff > length ? length : diff, rv), length) end end end @@ -130,5 +150,24 @@ module Unicorn StringIO === @tmp ? @tmp.size : @tmp.stat.size end + # tee()s into +buf+ until it is of +length+ bytes (or until + # we've reached the Content-Length of the request body). + # Returns +buf+ (the exact object, not a duplicate) + # To continue supporting applications that need near-real-time + # streaming input bodies, this is a no-op for + # "Transfer-Encoding: chunked" requests. + def ensure_length(buf, length) + # @size is nil for chunked bodies, so we can't ensure length for those + # since they could be streaming bidirectionally and we don't want to + # block the caller in that case. + return buf if buf.nil? || @size.nil? + + while buf.size < length && @size != @tmp.pos + buf << tee(length - buf.size, @buf2) + end + + buf + end + end end -- Eric Wong From normalperson at yhbt.net Wed Oct 7 04:58:38 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 7 Oct 2009 01:58:38 -0700 Subject: [ANN] unicorn 0.93.2 - more compatible with Rails Message-ID: <20091007085838.GC20541@dcvr.yhbt.net> Unicorn is a HTTP server for Rack applications designed to take advantage of features in Unix/Unix-like kernels and only serve fast clients on low-latency, high-bandwidth connections. Slow clients should only be served by placing a reverse proxy capable of fully-buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Thanks to Chris Wanstrath for reporting issues with large POST bodies and for helping me test. Changes: Avoid truncated POST bodies with URL-encoded forms in Rails by switching TeeInput to use read-in-full semantics (only) when a Content-Length: header exists. Chunked request bodies continue to exhibit readpartial semantics to support simultaneous bidirectional chunking. The lack of return value checking in Rails to protect against a short ios.read(length) is entirely reasonable even if not pedantically correct. Most ios.read(length) implementations return the full amount requested except right before EOF. A ticket has been opened here to track the issue: https://rails.lighthouseapp.com/projects/8994/tickets/3343 Also there are some minor documentation improvements. Eric Wong (8): Fix NEWS generation on single-paragraph tag messages Include GPLv2 in docs doc: make it clear contributors retain copyrights TODO: removed Rainbows! (see rainbows.rubyforge.org) Document the START_CTX hash contents more-compatible TeeInput#read for POSTs with Content-Length tests for read-in-full vs readpartial semantics unicorn 0.93.2 -- Eric Wong From chris at ozmm.org Wed Oct 7 14:04:43 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Wed, 7 Oct 2009 11:04:43 -0700 Subject: [ANN] unicorn 0.93.2 - more compatible with Rails In-Reply-To: <20091007085838.GC20541@dcvr.yhbt.net> References: <20091007085838.GC20541@dcvr.yhbt.net> Message-ID: <8b73aaca0910071104o2502f6b7o124be0f0dda4fbfd@mail.gmail.com> On Wed, Oct 7, 2009 at 1:58 AM, Eric Wong wrote: > Changes: > > Avoid truncated POST bodies with URL-encoded forms in Rails > by switching TeeInput to use read-in-full semantics (only) when > a Content-Length: header exists. ?Chunked request bodies > continue to exhibit readpartial semantics to support > simultaneous bidirectional chunking. > > The lack of return value checking in Rails to protect against a > short ios.read(length) is entirely reasonable even if not > pedantically correct. ?Most ios.read(length) implementations > return the full amount requested except right before EOF. > > A ticket has been opened here to track the issue: > ?https://rails.lighthouseapp.com/projects/8994/tickets/3343 This release fixes the POST issue we saw. Thanks! -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Wed Oct 7 18:05:48 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 7 Oct 2009 15:05:48 -0700 Subject: [PATCH] doc: expand on the SELF_PIPE description Message-ID: <20091007220548.GA30473@dcvr.yhbt.net> There seems to be a small amount of confusion regarding how it's used (and some of the code is not very obvious). So explain our usage of it and distinguish its use in the master vs worker(s). --- Hi Ryan, Great article[1] on Unicorn! I just noticed a small misunderstanding in how SELF_PIPE is used by the workers, so I just updated the RDoc to hopefully better describe it. I believe Unicorn is a great way to introduce Unix systems programming and you writing a great article is helping a lot :> As far as workers going down quickly in case the master dies unexpectedly: this is usually handled by checking the parent PID in the worker loop (which has a normal[2] upper bound of timeout(=default=60)/2.0 seconds per iteration on an idle server). Needing to get the workers to go down more quickly than that wasn't too important to us, so I made the decision to not get fancier. [1] - for those on the mailing list who haven't seen it: http://tomayko.com/writings/unicorn-is-unix [2] - for the case where speculative accept() is constantly successful, then this could be infinity as long as requests keep flowing in faster than they can be processed. Not that I expect anybody to care or hit this case (unexpectedly dead masters are very uncommon already :>). lib/unicorn.rb | 18 +++++++++++++++++- 1 files changed, 17 insertions(+), 1 deletions(-) diff --git a/lib/unicorn.rb b/lib/unicorn.rb index ddec8e9..d63567f 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -42,7 +42,23 @@ module Unicorn # This hash maps PIDs to Workers WORKERS = {} - # See: http://cr.yp.to/docs/selfpipe.html + # We use SELF_PIPE differently in the master and worker processes: + # + # * The master process never closes or reinitializes this once + # initialized. Signal handlers in the master process will write to + # it to wake up the master from IO.select in exactly the same manner + # djb describes in http://cr.yp.to/docs/selfpipe.html + # + # * The workers immediately close the pipe they inherit from the + # master and replace it with a new pipe after forking. This new + # pipe is also used to wakeup from IO.select from inside (worker) + # signal handlers. However, workers *close* the pipe descriptors in + # the signal handlers to raise EBADF in IO.select instead of writing + # like we do in the master. We cannot easily use the reader set for + # IO.select because LISTENERS is already that set, and it's extra + # work (and cycles) to distinguish the pipe FD from the reader set + # once IO.select returns. So we're lazy and just close the pipe when + # a (rare) signal arrives in the worker and reinitialize the pipe later. SELF_PIPE = [] # signal queue used for self-piping -- Eric Wong From normalperson at yhbt.net Wed Oct 7 20:09:42 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 7 Oct 2009 17:09:42 -0700 Subject: why Unicorn doesn't do slow clients Message-ID: <20091008000942.GA25054@dcvr.yhbt.net> You may have heard of Slowloris and Nkiller2 (if not check them out), but there is also david: http://git.bogomips.org/cgit/david.git I wasn't comfortable with announcing this two years ago when I wrote it. I'm OK now since Slowloris and Nkiller2 are similar (ok, Nkiller2 is lower-level and meaner) and attacks of this type should be well-known by now. I'm sure some folks have known about these types of attack since the 1990s, even. I haven't touched david in over two years and don't have any desire to maintain or support it. Feel free to take and hack on it for your own testing or even make it into a real project, but please be nice and don't hurt people with it. This is different from Slowloris in that it throttles the entire request including the POST/PUT body. Most proxies (including haproxy) don't buffer bodies before sending them on whereas nginx will buffer large ones to temporary files. I originally wrote david to convince some friends to stick nginx in front of anything that wasn't nginx; including Apache + mod_perl|mod_php|mongrel and Tomcat setups. And it was quite convincing at the time :> Implementation details: * david combines fork() with select() to multiplex file descriptors and get around select() and non-priviledged process FD limits. * The defaults at the top of the davic.c file are all pretty tame, but they're all commented on and you can tweak them. * david mmaps a raw file that is an HTTP request, some small samples are provided but you can/should generate your own large POST/PUT requests to really stress request body handling. -- Eric Wong From normalperson at yhbt.net Wed Oct 7 20:31:44 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 7 Oct 2009 17:31:44 -0700 Subject: Unicorn - "Still a couple bugs to knock out." Message-ID: <20091008003144.GB31036@dcvr.yhbt.net> Hi there, We noticed your blog entry[1] and are pleasantly surprised with the performance compared to existing servers. However, this bit has us worried, care to elaborate so we can track down and fix them? > We aren't going to be jumping to unicorn in the near future. > Still a couple bugs to knock out. We released a fix in v0.93.2 last night to work around truncated POST bodies in Rails. Did you encounter something else? Thanks! [1] http://labs.revelationglobal.com/2009/10/06/mongrel_passenger_unicorn.html -- Eric Wong From chris at ozmm.org Wed Oct 7 20:59:14 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Wed, 7 Oct 2009 17:59:14 -0700 Subject: Unicorn - "Still a couple bugs to knock out." In-Reply-To: <20091008003144.GB31036@dcvr.yhbt.net> References: <20091008003144.GB31036@dcvr.yhbt.net> Message-ID: <8b73aaca0910071759r5afe9ad7t7b504d1a8d11229f@mail.gmail.com> On Wed, Oct 7, 2009 at 5:31 PM, Eric Wong wrote: > We noticed your blog entry[1] and are pleasantly surprised with > the performance compared to existing servers. Note that the blog post features an embedded Gist, served via Unicorn ;) -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Thu Oct 8 00:47:33 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 7 Oct 2009 21:47:33 -0700 Subject: Unicorn - "Still a couple bugs to knock out." In-Reply-To: References: <20091008003144.GB31036@dcvr.yhbt.net> Message-ID: <20091008044733.GA2112@dcvr.yhbt.net> Dan Herrera wrote: > Hi Eric, > > Thanks for getting in touch. We were also pleasantly surprised by the > performance of unicorn! > > As to the blog post, we misspoke and will revise our language on the post to > reflect that what we encountered weren't bugs, but issues with our own > setup. Thanks for the clarification, I was worried :x > Our app has a memory leak, and we haven't yet settled on a monit recipe to > restart unicorn when the sub processes that are started run over a certain > memory threshold. There were also some changes we needed to make with our > nginx configuration, but again, they didn't reflect any problems with > unicorn. You can probably have monit just SIGQUIT the workers that run over the memory threshold and just let the unicorn master restart them (note: I'm not a monit user). If your OS supports it, you can try setrlimit in your after_fork hook. Modern Linux 2.6 doesn't support Process::RLIMIT_RSS, but you can still do Process::RLIMIT_AS which is still somewhat useful[1]. after_fork do |server, worker| Process.setrlimit(Process::RLIMIT_AS, size_in_kilobytes) end Then have a simple Rack middleware that traps NoMemoryError: # XXX totally untested!, I'm not sure what to do if Process.kill # here hits NoMemoryError, either... class ExitOnOOM < Struct.new(:app) def call(env) begin app.call(env) rescue NoMemoryError Process.kill(:QUIT, 0) # graceful exit signal # or maybe just: exit! end end end [1] On modern GNU/Linux systems that don't have RLIMIT_RSS, the VM size taken by the mmap()'ed shared libraries usually doesn't change over the lifetime of the process, so you can use that as a baseline and figure out how far you want to go from there.... Of course, your Unicorns shouldn't be swapping so swap usage won't have to be accounted for. Of course the proper solution to all this is to fix your memory leaks :) I posted some notes here a few months back on the Mongrel list: http://thread.gmane.org/gmane.comp.lang.ruby.mongrel.general/5760/focus=5766 > Engine Yard doesn't currently support unicorn on their slices, but once we > have a satisfactory monit recipe and no other issues with our setup, we plan > to work with them so that they'll hopefully change their mind. I think > this, and the monit recipe is what James meant when he said we wouldn't be > using unicorn in the near future. Ah, didn't know that. I guess github was a special case where they thought: "oh well, these guys are leaving us for another host anyways, they can be our guinea pig!" :) > Anyway, we are all great fans, and would like to thank all of you for all of > your hard work. No problem :) -- Eric Wong From jeremyevans0 at gmail.com Thu Oct 8 19:57:38 2009 From: jeremyevans0 at gmail.com (Jeremy Evans) Date: Thu, 8 Oct 2009 16:57:38 -0700 Subject: alive.chmod(Time.now.to_i) raises Errno::EINVAL on OpenBSD Message-ID: <5c548e460910081657t7df3403du98f2f479b52536e9@mail.gmail.com> On OpenBSD: $ ruby -e "File.new('TODO').chmod(Time.now.to_i)" -e:1:in `chmod': Invalid argument - TODO (Errno::EINVAL) from -e:1 This is explained in the man page: int fchmod(int fd, mode_t mode); ... [EINVAL] mode contains bits other than the file type and those de- scribed above. I think 04777 is the highest allowed mode on OpenBSD. Time.now.to_i is obviously higher than that. Here's a diff that should fix the problem. At the very least it allows the workers to start without crashing: diff --git a/lib/unicorn.rb b/lib/unicorn.rb index ddec8e9..092f500 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -579,13 +579,13 @@ module Unicorn # changes with chmod doesn't update ctime on all filesystems; so # we change our counter each and every time (after process_client # and before IO.select). - t == (ti = Time.now.to_i) or alive.chmod(t = ti) + t == (ti = Time.now.to_i) or (t = ti; alive.chmod(alive.stat.mode ^ 0100)) ready.each do |sock| begin process_client(sock.accept_nonblock) nr += 1 - t == (ti = Time.now.to_i) or alive.chmod(t = ti) + t == (ti = Time.now.to_i) or (t = ti; alive.chmod(alive.stat.mode ^ 0100)) rescue Errno::EAGAIN, Errno::ECONNABORTED end break if nr < 0 There are definitely other ways that will work, as long as the mode is kept between 0 and 04777. Jeremy From normalperson at yhbt.net Thu Oct 8 23:22:39 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 8 Oct 2009 20:22:39 -0700 Subject: alive.chmod(Time.now.to_i) raises Errno::EINVAL on OpenBSD In-Reply-To: <5c548e460910081657t7df3403du98f2f479b52536e9@mail.gmail.com> References: <5c548e460910081657t7df3403du98f2f479b52536e9@mail.gmail.com> Message-ID: <20091009032239.GA2695@dcvr.yhbt.net> Jeremy Evans wrote: > On OpenBSD: > > $ ruby -e "File.new('TODO').chmod(Time.now.to_i)" > -e:1:in `chmod': Invalid argument - TODO (Errno::EINVAL) > from -e:1 > > This is explained in the man page: > > int > fchmod(int fd, mode_t mode); > > ... > > [EINVAL] mode contains bits other than the file type and those de- > scribed above. > > I think 04777 is the highest allowed mode on OpenBSD. Time.now.to_i > is obviously higher than that. Yikes. I was worried about the portability of this. I actually redid this in the Revactor model of Rainbows! to flip between 0 and 1, I'll do that in Unicorn, too. > Here's a diff that should fix the problem. At the very least it > allows the workers to start without crashing: > > diff --git a/lib/unicorn.rb b/lib/unicorn.rb > index ddec8e9..092f500 100644 > --- a/lib/unicorn.rb > +++ b/lib/unicorn.rb > @@ -579,13 +579,13 @@ module Unicorn > # changes with chmod doesn't update ctime on all filesystems; so > # we change our counter each and every time (after process_client > # and before IO.select). > - t == (ti = Time.now.to_i) or alive.chmod(t = ti) > + t == (ti = Time.now.to_i) or (t = ti; > alive.chmod(alive.stat.mode ^ 0100)) I think the Time.now.to_i bit is overkill (and probably detrimental to performance on non-x86_64, non-VDSO-enabled, non-GNU/Linux machines). > There are definitely other ways that will work, as long as the mode is > kept between 0 and 04777. Here's what I committed and pushed out, it should work for all *nix now since it only alternates between 0 and 1, but be sure to let me know if it doesn't for some reason. Thanks! >From 24a1b4c6b5fcb948e4fdee04e286c044d6d45f98 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Thu, 8 Oct 2009 19:56:29 -0700 Subject: [PATCH] fchmod heartbeat flips between 0/1 for compatibility This removes the Time.now.to_i comparison that was used to avoid multiple, no-op fchmod() syscalls[1] within the same second. This should allow us to run on OpenBSD where it can raise EINVAL when Time.now.to_i is passed to it. Reported-by: Jeremy Evans [1] - gettimeofday() from Time.now is not a real syscall on VDSO-enabled x86_64 GNU/Linux systems where Unicorn is primarily developed. --- lib/unicorn.rb | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/unicorn.rb b/lib/unicorn.rb index d63567f..13c203a 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -575,13 +575,13 @@ module Unicorn nr = 0 # this becomes negative if we need to reopen logs alive = worker.tmp # tmp is our lifeline to the master process ready = LISTENERS - t = ti = 0 # closing anything we IO.select on will raise EBADF trap(:USR1) { nr = -65536; SELF_PIPE.first.close rescue nil } trap(:QUIT) { alive = nil; LISTENERS.each { |s| s.close rescue nil } } [:TERM, :INT].each { |sig| trap(sig) { exit!(0) } } # instant shutdown logger.info "worker=#{worker.nr} ready" + m = 0 begin nr < 0 and reopen_worker_logs(worker.nr) @@ -595,13 +595,13 @@ module Unicorn # changes with chmod doesn't update ctime on all filesystems; so # we change our counter each and every time (after process_client # and before IO.select). - t == (ti = Time.now.to_i) or alive.chmod(t = ti) + alive.chmod(m = 0 == m ? 1 : 0) ready.each do |sock| begin process_client(sock.accept_nonblock) nr += 1 - t == (ti = Time.now.to_i) or alive.chmod(t = ti) + alive.chmod(m = 0 == m ? 1 : 0) rescue Errno::EAGAIN, Errno::ECONNABORTED end break if nr < 0 @@ -614,7 +614,7 @@ module Unicorn redo unless nr == 0 # (nr < 0) => reopen logs ppid == Process.ppid or return - alive.chmod(t = 0) + alive.chmod(m = 0 == m ? 1 : 0) begin # timeout used so we can detect parent death: ret = IO.select(LISTENERS, nil, SELF_PIPE, timeout) or redo -- Eric Wong From chris at ozmm.org Fri Oct 9 15:42:45 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Fri, 9 Oct 2009 12:42:45 -0700 Subject: Our Unicorn Setup Message-ID: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> Hi list, I've just published a post detailing our Unicorn setup, migration process, and reasons for choosing it: http://github.com/blog/517-unicorn Thanks again! -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Fri Oct 9 16:30:25 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 9 Oct 2009 13:30:25 -0700 Subject: Our Unicorn Setup In-Reply-To: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> Message-ID: <20091009203025.GA23205@dcvr.yhbt.net> Chris Wanstrath wrote: > Hi list, > > I've just published a post detailing our Unicorn setup, migration > process, and reasons for choosing it: > http://github.com/blog/517-unicorn Cool! "The Unicorn master manages the workers and balancing" Actually, the master never manages balancing, just the workers. The diagram is a little inaccurate as it looks like the master sees the requests, it never does. The request flow is like this: requests | | | shared socket(s) /|\ / | \ | | | | | | worker pool While the shared socket is opened and configured by the master, but the master does nothing else with the sockets. You're completely right about the pull balancing, it's one of the most distinctive differences between Unicorn and other setups. Also, for the 502s, do you get more 502s after the initial worker times out? I think nginx may (still)[1] mark a backend as completely dead/down when a backend dies. That may cause nginx to stop forwarding to that backend entirely and throw more 502s for a few seconds until nginx decides to actually try that backend again. Setting fail_timeout=0 causes nginx to never mark backends as down and always try to send a request to them: upstream github { server unix:/data/github/current/tmp/sockets/unicorn.sock fail_timeout=0; } I'll add this bit somewhere in the Unicorn docs and release 0.93.3 with the OpenBSD fix for Jeremy in a bit. > Thanks again! No problem :) [1] - I'm not 100% sure if nginx still does this, but I don't see anything in the 0.6.x changelog that indicates otherwise. I don't see a huge amount of harm in doing this always, we've been using fail_timeout=0 for ~2 years now regardless of the backend. -- Eric Wong From unicorn at dusty.name Fri Oct 9 17:03:24 2009 From: unicorn at dusty.name (Dusty Doris) Date: Fri, 9 Oct 2009 17:03:24 -0400 Subject: Our Unicorn Setup In-Reply-To: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> Message-ID: Thanks for this post Chris, it was very informative and has answered a few questions that I've had in my head over the last couple of days. I've been testing unicorn with a few apps for a couple days and actually already moved one over to it. I have a question for list. We are currently setup with a load balancer that runs nginx and haproxy. Nginx, simply proxies to haproxy, which then balances that across multiple mongrel or thin instances that span several servers. We simply include the public directory on our load balancer so nginx can serve static files right there. We don't have nginx running on the app servers, they are just mongrel or thin. So, my question. How would you do a Unicorn deployment when you have multiple app servers? 1. Simply use mongrels upstream and let it round-robin between all the unicorn instances on the different servers? Or, perhaps use the fair-upstream plugin? nginx -> [unicorns] 2. Keep haproxy in the middle? nginx -> haproxy -> [unicorns] 3. Stick haproxy in front and have it balance between the app servers that run their own nginx? haproxy -> [nginxs] -> unicorn # could use socket instead of tcp in this case I would love to hear any opinions. Thanks! Dusty Doris On Fri, Oct 9, 2009 at 3:42 PM, Chris Wanstrath wrote: > Hi list, > > I've just published a post detailing our Unicorn setup, migration > process, and reasons for choosing it: > http://github.com/blog/517-unicorn > > Thanks again! > > -- > Chris Wanstrath > http://github.com/defunkt > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From chris at ozmm.org Fri Oct 9 17:25:15 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Fri, 9 Oct 2009 14:25:15 -0700 Subject: Our Unicorn Setup In-Reply-To: <20091009203025.GA23205@dcvr.yhbt.net> References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> <20091009203025.GA23205@dcvr.yhbt.net> Message-ID: <8b73aaca0910091425y6d688b5raa9fec7c45d6c88c@mail.gmail.com> On Fri, Oct 9, 2009 at 1:30 PM, Eric Wong wrote: > "The Unicorn master manages the workers and balancing" > > Actually, the master never manages balancing, just the workers. ?The > diagram is a little inaccurate as it looks like the master sees the > requests, it never does. > > The request flow is like this: > > ? ? ? ? ? requests > ? ? ? ? ? ? ?| > ? ? ? ? ? ? ?| > ? ? ? ? ? ? ?| > ? ? ? ?shared socket(s) > ? ? ? ? ? ? /|\ > ? ? ? ? ? ?/ | \ > ? ? ? ? ? | ?| ?| > ? ? ? ? ? | ?| ?| > ? ? ? ? worker pool > > While the shared socket is opened and configured by the master, but the > master does nothing else with the sockets. ?You're completely right > about the pull balancing, it's one of the most distinctive differences > between Unicorn and other setups. Thanks! I've updated the diagram and some of the language to be accurate. > Also, for the 502s, do you get more 502s after the initial worker times > out? ?I think nginx may (still)[1] mark a backend as completely dead/down > when a backend dies. ?That may cause nginx to stop forwarding to that > backend entirely and throw more 502s for a few seconds until nginx > decides to actually try that backend again. > > Setting fail_timeout=0 causes nginx to never mark backends as down and > always try to send a request to them: > > ?upstream github { > ? ?server unix:/data/github/current/tmp/sockets/unicorn.sock fail_timeout=0; > ?} We'll try this out, thanks for letting us know about it. Cheers, -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Fri Oct 9 18:01:10 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 9 Oct 2009 15:01:10 -0700 Subject: Our Unicorn Setup In-Reply-To: References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> Message-ID: <20091009220110.GB14137@dcvr.yhbt.net> Dusty Doris wrote: > Thanks for this post Chris, it was very informative and has answered a > few questions that I've had in my head over the last couple of days. > I've been testing unicorn with a few apps for a couple days and > actually already moved one over to it. > > I have a question for list. First off, please don't top post, thanks :) > We are currently setup with a load balancer that runs nginx and > haproxy. Nginx, simply proxies to haproxy, which then balances that > across multiple mongrel or thin instances that span several servers. > We simply include the public directory on our load balancer so nginx > can serve static files right there. We don't have nginx running on > the app servers, they are just mongrel or thin. > > So, my question. How would you do a Unicorn deployment when you have > multiple app servers? For me, it depends on the amount of static files you serve with nginx and also the traffic you hit. Can I assume you're running Linux 2.6 (with epoll + awesome VFS layer)? May I also assume your load balancer box is not very stressed right now? > 1. Simply use mongrels upstream and let it round-robin between all > the unicorn instances on the different servers? Or, perhaps use the > fair-upstream plugin? > > nginx -> [unicorns] Based on your description of your current setup, this would be the best way to go. I would configure a lowish listen() :backlog for the Unicorns, fail_timeout=0 in nginx for every server This setup means round-robin by default, but if one machine gets a :backlog overflow, then nginx will automatically retry on a different backend. > 2. Keep haproxy in the middle? > > nginx -> haproxy -> [unicorns] This is probably not necessary, but it can't hurt a whole lot either. Also an option for balancing. If you're uncomfortable with the first approach you can also configure haproxy as a backup server: upstream unicorn_failover { # round-robin between unicorn app servers on the LAN: server 192.168.0.1:8080 fail_timeout=0; server 192.168.0.2:8080 fail_timeout=0; server 192.168.0.3:8080 fail_timeout=0; # haproxy, configured the same way as you do now # the "backup" parameter means nginx won't hit haproxy unless # all the direct unicorn connections have backlog overflows # or other issues server 127.0.0.1:8080 fail_timeout=0 backup; # haproxy backup } So your traffic flow may look like the first for the common case, but you may have a slightly more balanced/queueing solution in case you're completely overloaded. > 3. Stick haproxy in front and have it balance between the app servers > that run their own nginx? > > haproxy -> [nginxs] -> unicorn # could use socket instead of tcp in this case This is probably only necessary if: 1) you have a lot of static files that don't all fit in the VFS caches 2) you handle a lot of large uploads/responses and nginx buffering will thrash one box I know some sites that run this (or similar) config, but it's mainly because this is what they've had for 5-10 years and don't have time/resources to test new setups. > I would love to hear any opinions. You can also try the following, which is similar to what I describe in: http://article.gmane.org/gmane.comp.lang.ruby.unicorn.general/31 Pretty much all the above setups are valid. The important part is that nginx must sit *somewhere* in between Unicorn and the rest of the world. -- Eric Wong From unicorn at dusty.name Fri Oct 9 18:44:17 2009 From: unicorn at dusty.name (Dusty Doris) Date: Fri, 9 Oct 2009 18:44:17 -0400 Subject: Our Unicorn Setup In-Reply-To: <20091009220110.GB14137@dcvr.yhbt.net> References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> <20091009220110.GB14137@dcvr.yhbt.net> Message-ID: On Fri, Oct 9, 2009 at 6:01 PM, Eric Wong wrote: > Dusty Doris wrote: >> Thanks for this post Chris, it was very informative and has answered a >> few questions that I've had in my head over the last couple of days. >> I've been testing unicorn with a few apps for a couple days and >> actually already moved one over to it. >> >> I have a question for list. > > First off, please don't top post, thanks :) Sorry about that. > >> We are currently setup with a load balancer that runs nginx and >> haproxy. ?Nginx, simply proxies to haproxy, which then balances that >> across multiple mongrel or thin instances that span several servers. >> We simply include the public directory on our load balancer so nginx >> can serve static files right there. ?We don't have nginx running on >> the app servers, they are just mongrel or thin. >> >> So, my question. ?How would you do a Unicorn deployment when you have >> multiple app servers? > > For me, it depends on the amount of static files you serve with nginx > and also the traffic you hit. > > Can I assume you're running Linux 2.6 (with epoll + awesome VFS layer)? > > May I also assume your load balancer box is not very stressed right now? > Yep. We server our css, javascript, and some images out of the load balancer. But for the majority of our images and all the dynamically created ones, we serve those from dedicated image servers that have their own nginx instance running on each one. >> 1. ?Simply use mongrels upstream and let it round-robin between all >> the unicorn instances on the different servers? ?Or, perhaps use the >> fair-upstream plugin? >> >> nginx -> [unicorns] > > Based on your description of your current setup, this would be the best > way to go. ?I would configure a lowish listen() :backlog for the > Unicorns, fail_timeout=0 in nginx for every server ?This setup means > round-robin by default, but if one machine gets a :backlog overflow, > then nginx will automatically retry on a different backend. > Thanks for the recommendation. I was going to give that a shot first to see how it went, as it would also be the easiest to manage. When you say a lowish backlog? What kind of numbers are you talking about? Say, we had 8 workers running that stayed pretty active. They are usually quick to respond, with an occasional 2 second response (say 1/100) due to a bad sql query that we need to fix. Would lowish be 16, 32, 64, 128, 1024? Oh and thanks for the tip on the fail_timeout. >> 2. ?Keep haproxy in the middle? >> >> nginx -> haproxy -> [unicorns] > > This is probably not necessary, but it can't hurt a whole lot either. > > Also an option for balancing. ?If you're uncomfortable with the first > approach you can also configure haproxy as a backup server: > > ?upstream unicorn_failover { > ? ?# round-robin between unicorn app servers on the LAN: > ? ?server 192.168.0.1:8080 fail_timeout=0; > ? ?server 192.168.0.2:8080 fail_timeout=0; > ? ?server 192.168.0.3:8080 fail_timeout=0; > > ? ?# haproxy, configured the same way as you do now > ? ?# the "backup" parameter means nginx won't hit haproxy unless > ? ?# all the direct unicorn connections have backlog overflows > ? ?# or other issues > ? ?server 127.0.0.1:8080 fail_timeout=0 backup; # haproxy backup > ?} > > So your traffic flow may look like the first for the common case, but > you may have a slightly more balanced/queueing solution in case you're > completely overloaded. > >> 3. ?Stick haproxy in front and have it balance between the app servers >> that run their own nginx? >> >> haproxy -> [nginxs] -> unicorn # could use socket instead of tcp in this case > > This is probably only necessary if: > > ?1) you have a lot of static files that don't all fit in the VFS caches > > ?2) you handle a lot of large uploads/responses and nginx buffering will > ? ? thrash one box > > I know some sites that run this (or similar) config, but it's mainly > because this is what they've had for 5-10 years and don't have > time/resources to test new setups. > >> I would love to hear any opinions. > > You can also try the following, which is similar to what I describe in: > > ?http://article.gmane.org/gmane.comp.lang.ruby.unicorn.general/31 > Thats an interesting idea, thanks for sharing it. I like how the individual server also acts as a load balancer, but only if its having trouble itself. Otherwise, it just handles the requests through the socket connection. > Pretty much all the above setups are valid. ?The important part is that > nginx must sit *somewhere* in between Unicorn and the rest of the world. > > -- > Eric Wong > I appreciate your reply and especially for Unicorn. Thanks! From bketelsen at gmail.com Fri Oct 9 18:59:34 2009 From: bketelsen at gmail.com (Brian Ketelsen) Date: Fri, 9 Oct 2009 18:59:34 -0400 Subject: While we're on the topic Message-ID: <54A2628C-A2FE-43A6-B35A-B124275CC959@gmail.com> First - Thanks! We're loving Unicorn. I'm serving about 200K API requests a day on a few linux servers running Unicorn. We have the API calls, and also some admin stuff that has a UI. I have a pair of F5 load balancers in front of the setup with two pools. The API pool passes directly to the unicorns, while the UI pool passes to nginx which passes to unicorn. At the time it seemed to make sense that the API calls were stateless and waiting only on our backend services to process. The UI was a more complex story with web browsers and persistence. I'm curious -- is the F5 straight to unicorn plan a good one? We've had no complaints from our customers and great performance, so it's more of a rhetorical question, my average API response time is a tenth of a second. Are there any traps here? Thanks! Brian Ketelsen From normalperson at yhbt.net Fri Oct 9 19:11:34 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 9 Oct 2009 16:11:34 -0700 Subject: Our Unicorn Setup In-Reply-To: References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> <20091009220110.GB14137@dcvr.yhbt.net> Message-ID: <20091009231133.GC14137@dcvr.yhbt.net> Dusty Doris wrote: > On Fri, Oct 9, 2009 at 6:01 PM, Eric Wong wrote: > > Dusty Doris wrote: > >> 1. ?Simply use mongrels upstream and let it round-robin between all > >> the unicorn instances on the different servers? ?Or, perhaps use the > >> fair-upstream plugin? > >> > >> nginx -> [unicorns] > > > > Based on your description of your current setup, this would be the best > > way to go. ?I would configure a lowish listen() :backlog for the > > Unicorns, fail_timeout=0 in nginx for every server ?This setup means > > round-robin by default, but if one machine gets a :backlog overflow, > > then nginx will automatically retry on a different backend. > > Thanks for the recommendation. I was going to give that a shot first > to see how it went, as it would also be the easiest to manage. > > When you say a lowish backlog? What kind of numbers are you talking > about? Say, we had 8 workers running that stayed pretty active. They > are usually quick to respond, with an occasional 2 second response > (say 1/100) due to a bad sql query that we need to fix. Would lowish > be 16, 32, 64, 128, 1024? 1024 is the default in Mongrel and Unicorn which is very generous. 5 is the default value that Ruby initializes the sockets at, so picking something in between is recommended. It really depends on your app and comfort level. You can also tune and refine it over time safely without worrying too much about dropping connections by configuring multiple listeners per-instance (see below). Keep in mind the backlog is rarely an exact setting, it's more of a recommendation to the kernel (and the actual value is often higher than specified). > Oh and thanks for the tip on the fail_timeout. No problem, I somehow thought it was widely-known by now... > > You can also try the following, which is similar to what I describe in: > > > > ?http://article.gmane.org/gmane.comp.lang.ruby.unicorn.general/31 > > > > Thats an interesting idea, thanks for sharing it. I like how the > individual server also acts as a load balancer, but only if its having > trouble itself. Otherwise, it just handles the requests through the > socket connection. > I appreciate your reply and especially for Unicorn. You can also try a combination of (1) above and my proposed idea in $gmane/31 by configuring two listeners per-Unicorn instance: # primary listen 8080, :backlog => 10, :tcp_nopush => true # only when all servers overflow the backlog=10 above listen 8081, :backlog => 1024, :tcp_nopush => true And then putting the 8081s as a backup in nginx like this: ?upstream unicorn_failover { ? ?# round-robin between unicorns with small backlogs # as the primary option ? ?server 192.168.0.1:8080 fail_timeout=0; ? ?server 192.168.0.2:8080 fail_timeout=0; ? ?server 192.168.0.3:8080 fail_timeout=0; ? ?# the "backup" parameter means nginx won't ever try these # unless the set of listeners above fail. ? ?server 192.168.0.1:8081 fail_timeout=0 backup; ? ?server 192.168.0.2:8081 fail_timeout=0 backup; ? ?server 192.168.0.3:8081 fail_timeout=0 backup; ?} You can monitor the nginx error logs and see how often it fails on the low backlog listener, and then increment/decrement the backlog of the primary listeners as needed to get better load-balancing. -- Eric Wong From unicorn at dusty.name Fri Oct 9 19:17:36 2009 From: unicorn at dusty.name (Dusty Doris) Date: Fri, 9 Oct 2009 19:17:36 -0400 Subject: Our Unicorn Setup In-Reply-To: <20091009231133.GC14137@dcvr.yhbt.net> References: <8b73aaca0910091242u78ac787aj48fb1b63b5bf55bc@mail.gmail.com> <20091009220110.GB14137@dcvr.yhbt.net> <20091009231133.GC14137@dcvr.yhbt.net> Message-ID: On Fri, Oct 9, 2009 at 7:11 PM, Eric Wong wrote: > Dusty Doris wrote: >> On Fri, Oct 9, 2009 at 6:01 PM, Eric Wong wrote: >> > Dusty Doris wrote: >> >> 1. ?Simply use mongrels upstream and let it round-robin between all >> >> the unicorn instances on the different servers? ?Or, perhaps use the >> >> fair-upstream plugin? >> >> >> >> nginx -> [unicorns] >> > >> > Based on your description of your current setup, this would be the best >> > way to go. ?I would configure a lowish listen() :backlog for the >> > Unicorns, fail_timeout=0 in nginx for every server ?This setup means >> > round-robin by default, but if one machine gets a :backlog overflow, >> > then nginx will automatically retry on a different backend. >> >> Thanks for the recommendation. ?I was going to give that a shot first >> to see how it went, as it would also be the easiest to manage. >> >> When you say a lowish backlog? ?What kind of numbers are you talking >> about? ?Say, we had 8 workers running that stayed pretty active. ?They >> are usually quick to respond, with an occasional 2 second response >> (say 1/100) due to a bad sql query that we need to fix. ?Would lowish >> be 16, 32, 64, 128, 1024? > > 1024 is the default in Mongrel and Unicorn which is very generous. ?5 is > the default value that Ruby initializes the sockets at, so picking > something in between is recommended. ?It really depends on your app and > comfort level. ?You can also tune and refine it over time safely > without worrying too much about dropping connections by configuring > multiple listeners per-instance (see below). > > Keep in mind the backlog is rarely an exact setting, it's more of a > recommendation to the kernel (and the actual value is often higher > than specified). > >> Oh and thanks for the tip on the fail_timeout. > > No problem, I somehow thought it was widely-known by now... > >> > You can also try the following, which is similar to what I describe in: >> > >> > ?http://article.gmane.org/gmane.comp.lang.ruby.unicorn.general/31 >> > >> >> Thats an interesting idea, thanks for sharing it. ?I like how the >> individual server also acts as a load balancer, but only if its having >> trouble itself. ?Otherwise, it just handles the requests through the >> socket connection. >> I appreciate your reply and especially for Unicorn. > > You can also try a combination of (1) above and my proposed idea in > $gmane/31 by configuring two listeners per-Unicorn instance: > > ? # primary > ? listen 8080, :backlog => 10, :tcp_nopush => true > > ? # only when all servers overflow the backlog=10 above > ? listen 8081, :backlog => 1024, :tcp_nopush => true > > And then putting the 8081s as a backup in nginx like this: > > ??upstream unicorn_failover { > ?? ?# round-robin between unicorns with small backlogs > ? ?# as the primary option > ?? ?server 192.168.0.1:8080 fail_timeout=0; > ?? ?server 192.168.0.2:8080 fail_timeout=0; > ?? ?server 192.168.0.3:8080 fail_timeout=0; > > ?? ?# the "backup" parameter means nginx won't ever try these > ? ?# unless the set of listeners above fail. > ?? ?server 192.168.0.1:8081 fail_timeout=0 backup; > ?? ?server 192.168.0.2:8081 fail_timeout=0 backup; > ?? ?server 192.168.0.3:8081 fail_timeout=0 backup; > ??} > > You can monitor the nginx error logs and see how often it fails on the > low backlog listener, and then increment/decrement the backlog of > the primary listeners as needed to get better load-balancing. > > -- > Eric Wong > Awesome! I am going to give that a shot. From normalperson at yhbt.net Fri Oct 9 19:26:32 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 9 Oct 2009 16:26:32 -0700 Subject: While we're on the topic In-Reply-To: <54A2628C-A2FE-43A6-B35A-B124275CC959@gmail.com> References: <54A2628C-A2FE-43A6-B35A-B124275CC959@gmail.com> Message-ID: <20091009232632.GD14137@dcvr.yhbt.net> Brian Ketelsen wrote: > First - Thanks! We're loving Unicorn. Hi Brian, thanks for the feedback! > I'm serving about 200K API requests a day on a few linux servers running > Unicorn. We have the API calls, and also some admin stuff that has a UI. > I have a pair of F5 load balancers in front of the setup with two pools. > The API pool passes directly to the unicorns, while the UI pool passes > to nginx which passes to unicorn. At the time it seemed to make sense > that the API calls were stateless and waiting only on our backend > services to process. The UI was a more complex story with web browsers > and persistence. > > I'm curious -- is the F5 straight to unicorn plan a good one? We've had > no complaints from our customers and great performance, so it's more of a > rhetorical question, my average API response time is a tenth of a second. > Are there any traps here? It depends heavily on several factors: * Responses sizes, are they smaller than the outgoing network buffers? As long as Unicorn can write to their outgoing network buffers without the receiving end (the F5) blocking it, the response should be fine since it's basically fire-and-forget. * Do you handle large requests? This is the troublesome part with non-nginx frontends... I think the F5 can buffer headers entirely before sending them on if it's told to do layer 7, but it never buffers request bodies. That said I've never been able to study F5s (or any non-Free frontends) in depth... * Do your clients want/need/benefit from HTTP keepalive and pipelining? I'm not sure if the F5 is capable of mapping keepalive/pipelining from the client to the non-keepalive Unicorn backends. -- Eric Wong From normalperson at yhbt.net Fri Oct 9 19:28:12 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 9 Oct 2009 16:28:12 -0700 Subject: [ANN] unicorn 0.93.3 - OpenBSD compatibility Message-ID: <20091009232812.GA26231@dcvr.yhbt.net> Unicorn is a HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: This release fixes compatibility with OpenBSD (and possibly other Unices with stricter fchmod(2) implementations) thanks to Jeremy Evans. Additionally there are small documentation changes all around. Eric Wong (12): doc: expand on the SELF_PIPE description fchmod heartbeat flips between 0/1 for compatibility examples/init.sh: remove "set -u" configurator: update with nginx fail_timeout=0 example PHILOSOPHY: clarify experience other deployments PHILOSOPHY: plug the Rainbows! spin-off project README: remove unnecessary and extraneous dash DESIGN: clarification and possibly improve HTML validity README: remove the "non-existent" part README: emphasize the "fast clients"-only part drop the whitespace cleaner for Ragel->C unicorn 0.93.3 -- Eric Wong From jeremyevans0 at gmail.com Sun Oct 11 20:55:48 2009 From: jeremyevans0 at gmail.com (Jeremy Evans) Date: Sun, 11 Oct 2009 17:55:48 -0700 Subject: Worker restarts even when idle Message-ID: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> Eric, Thanks for the 0.93.3 release. I'm now able to test my apps with unicorn. If I have an idle application, the master program restarts the workers every timeout seconds. Is this the expected behavior, and if so, is there a reason for it? Would you be opposed to changing it so that the master doesn't restart workers that haven't had any activity in the last timeout seconds? Jeremy From normalperson at yhbt.net Sun Oct 11 21:32:31 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 11 Oct 2009 18:32:31 -0700 Subject: Worker restarts even when idle In-Reply-To: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> References: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> Message-ID: <20091012013231.GA17768@dcvr.yhbt.net> Jeremy Evans wrote: > Eric, > > Thanks for the 0.93.3 release. I'm now able to test my apps with unicorn. > > If I have an idle application, the master program restarts the workers > every timeout seconds. Is this the expected behavior, and if so, is > there a reason for it? Would you be opposed to changing it so that > the master doesn't restart workers that haven't had any activity in > the last timeout seconds? Idle as in no clients connected (and not long-poll/Comet idle)? Definitely not expected behavior... Can you try playing the following on OpenBSD and seeing if the ctimes match for before and after? require 'tempfile' foo = Tempfile.new nil before = foo.stat foo.chmod 1 sleep 1 foo.chmod 0 after = foo.stat p [ before.ctime, after.ctime ] If they're the same, can you also try syswrite(".") instead of chmod and see if that does anything? I wonder if OpenBSD is not implementing st_ctime changes properly, I understanding is that st_ctime should change for any FS-related operations, from the Linux stat(2) manpage: http://linux.die.net/man/2/stat The field st_ctime is changed by writing or by setting inode information (i.e., owner, group, link count, mode, etc.). Likewise, the OpenBSD manpage: http://www.openbsd.org/cgi-bin/man.cgi?query=stat&apropos=0&sektion=2&manpath=OpenBSD+Current&arch=i386&format=html st_ctime Time when file status was last changed (inode data modification). Changed by the chmod(2), chown(2), link(2), rename(2), unlink(2), utimes(2), and write(2) system calls. Note that the OpenBSD manpage doesn't explicitly say fchmod(2), just chmod(2), so if the above didn't work, maybe the following does? require 'tempfile' foo = Tempfile.new nil before = foo.stat File.chmod(1, foo.path) sleep 1 File.chmod(0, foo.path) after = foo.stat p [ before.ctime, after.ctime ] This might be a bug in OpenBSD, but I'd be willing to make a workaround -- Eric Wong From normalperson at yhbt.net Sun Oct 11 22:31:50 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 11 Oct 2009 19:31:50 -0700 Subject: Worker restarts even when idle In-Reply-To: <5c548e460910111911j3864b6b4pdc4a7d811c9c1e01@mail.gmail.com> References: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> <20091012013231.GA17768@dcvr.yhbt.net> <5c548e460910111911j3864b6b4pdc4a7d811c9c1e01@mail.gmail.com> Message-ID: <20091012023150.GA5509@dcvr.yhbt.net> Jeremy Evans wrote: > On Sun, Oct 11, 2009 at 6:32 PM, Eric Wong wrote: > > Jeremy Evans wrote: > > Can you try playing the following on OpenBSD and seeing if > > the ctimes match for before and after? > The times are different: > > [Sun Oct 11 19:08:42 -0700 2009, Sun Oct 11 19:08:43 -0700 2009] OK. Is this only on a certain app/library that might use IO.select in a different thread? Can you reproduce it on a dead simple "Hello World" app that you're certain is single-threaded? It could be related to this bug someone off list hit the other day: http://net-ssh.lighthouseapp.com/projects/36253/tickets/1 (redmine.ruby-lang.org is dead right now :<) http://redmine.ruby-lang.org/issues/show/2039 http://redmine.ruby-lang.org/issues/show/1993 Can you ktrace(? (or any other) syscall tracer) the worker immediately after spawning and see what it's doing up until the master kills it? The other possible culprit is that the accept_nonblock is blocking, somehow... -- Eric Wong From jeremyevans0 at gmail.com Sun Oct 11 22:50:36 2009 From: jeremyevans0 at gmail.com (Jeremy Evans) Date: Sun, 11 Oct 2009 19:50:36 -0700 Subject: Worker restarts even when idle In-Reply-To: <5c548e460910111946l4269750kb252044abbb10bd3@mail.gmail.com> References: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> <20091012013231.GA17768@dcvr.yhbt.net> <5c548e460910111911j3864b6b4pdc4a7d811c9c1e01@mail.gmail.com> <20091012023150.GA5509@dcvr.yhbt.net> <5c548e460910111946l4269750kb252044abbb10bd3@mail.gmail.com> Message-ID: <5c548e460910111950u53b3d61ayd6eceeca7ad02e36@mail.gmail.com> On Sun, Oct 11, 2009 at 7:46 PM, Jeremy Evans wrote: > On Sun, Oct 11, 2009 at 7:31 PM, Eric Wong wrote: >> Jeremy Evans wrote: >>> On Sun, Oct 11, 2009 at 6:32 PM, Eric Wong wrote: >>> > Jeremy Evans wrote: >>> > Can you try playing the following on OpenBSD and seeing if >>> > the ctimes match for before and after? >> >>> The times are different: >>> >>> ? [Sun Oct 11 19:08:42 -0700 2009, Sun Oct 11 19:08:43 -0700 2009] >> >> OK. >> >> Is this only on a certain app/library that might use IO.select in a >> different thread? ?Can you reproduce it on a dead simple "Hello World" >> app that you're certain is single-threaded? ?It could be related to this >> bug someone off list hit the other day: > > This is completely my fault. ?I wasn't using a config.ru file > previously, so when I first ran unicorn, it asked me to create one. ?I > added one that did: > > ?require 'start' # start.rb being the file that started the application > > Near the end of start.rb: > > ?Rack::Handler.get('mongrel').run ... > > Things are working fine now that I've fixed my code. ?Sorry for the noise. > > Jeremy > Eric, This and earlier replies were meant for the list. Is it possible for the list's reply-to address to be the list address itself? Jeremy From normalperson at yhbt.net Sun Oct 11 22:54:12 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 11 Oct 2009 19:54:12 -0700 Subject: Worker restarts even when idle In-Reply-To: <5c548e460910111946l4269750kb252044abbb10bd3@mail.gmail.com> References: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> <20091012013231.GA17768@dcvr.yhbt.net> <5c548e460910111911j3864b6b4pdc4a7d811c9c1e01@mail.gmail.com> <20091012023150.GA5509@dcvr.yhbt.net> <5c548e460910111946l4269750kb252044abbb10bd3@mail.gmail.com> Message-ID: <20091012025412.GA5603@dcvr.yhbt.net> Jeremy Evans wrote: > On Sun, Oct 11, 2009 at 7:31 PM, Eric Wong wrote: > > Jeremy Evans wrote: > >> On Sun, Oct 11, 2009 at 6:32 PM, Eric Wong wrote: > >> > Jeremy Evans wrote: > This is completely my fault. I wasn't using a config.ru file > previously, so when I first ran unicorn, it asked me to create one. I > added one that did: > > require 'start' # start.rb being the file that started the application > > Near the end of start.rb: > > Rack::Handler.get('mongrel').run ... > > Things are working fine now that I've fixed my code. Sorry for the noise. No worries :) I hit a similar issue with older Sinatra, too. Btw, the convention here should be to Cc: the list even when replying to a person (I prefer the same conventions as git ML and LKML). -- Eric Wong From normalperson at yhbt.net Sun Oct 11 22:56:12 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 11 Oct 2009 19:56:12 -0700 Subject: Worker restarts even when idle In-Reply-To: <5c548e460910111950u53b3d61ayd6eceeca7ad02e36@mail.gmail.com> References: <5c548e460910111755k4904ae7ei2672221c8519b38b@mail.gmail.com> <20091012013231.GA17768@dcvr.yhbt.net> <5c548e460910111911j3864b6b4pdc4a7d811c9c1e01@mail.gmail.com> <20091012023150.GA5509@dcvr.yhbt.net> <5c548e460910111946l4269750kb252044abbb10bd3@mail.gmail.com> <5c548e460910111950u53b3d61ayd6eceeca7ad02e36@mail.gmail.com> Message-ID: <20091012025612.GB5603@dcvr.yhbt.net> Jeremy Evans wrote: > Eric, > > This and earlier replies were meant for the list. Is it possible for > the list's reply-to address to be the list address itself? Otoh, this list is much less traffic than the other mailing lists I follow. I'll consider changing it in 24 hours unless there are objections... -- Eric Wong From normalperson at yhbt.net Tue Oct 13 02:41:23 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 12 Oct 2009 23:41:23 -0700 Subject: [PATCH] README: alter reply conventions for the mailing list Message-ID: <20091013064123.GA15931@dcvr.yhbt.net> Mailman is now configured to munge Reply-To: to point back to the mailing list. This might make things easier for folks on low traffic mailing lists like ours. --- Eric Wong wrote: > Jeremy Evans wrote: > > Eric, > > > > This and earlier replies were meant for the list. Is it possible for > > the list's reply-to address to be the list address itself? > > Otoh, this list is much less traffic than the other mailing lists I > follow. I'll consider changing it in 24 hours unless there are > objections... README | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README b/README index 674d581..09848bb 100644 --- a/README +++ b/README @@ -143,8 +143,8 @@ regarding this. All feedback (bug reports, user/development dicussion, patches, pull requests) go to the mailing list/newsgroup. Patches must be sent inline (git format-patch -M + git send-email). No subscription is necessary -to post on the mailing list. No top posting. Address replies +To:+ (or -+Cc:+) the original sender and +Cc:+ the mailing list. +to post on the mailing list. No top posting. Address replies +To:+ +the mailing list. * email: mailto:mongrel-unicorn at rubyforge.org * nntp: nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general -- Eric Wong From halogenandtoast at gmail.com Tue Oct 13 12:57:21 2009 From: halogenandtoast at gmail.com (Matt Mongeau) Date: Tue, 13 Oct 2009 12:57:21 -0400 Subject: Unicorn Nginx Issue Message-ID: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> I've setup nginx with this server { listen 80; server_name unicorn.local; location / { proxy_pass http://unicorn; } } upstream unicorn { server unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock; } When I run unicorn_rails -c config/unicorn.rb -E development I can access the application just fine on unicorn.local When I run unicorn_rails -c config/unicorn.rb -E development -D I get this issue 2009/10/13 12:34:22 [error] 1007#0: *62 connect() to unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock failed (61: Connection refused) while connecting to upstream, client: 127.0.0.1, server: unicorn.local, request: "GET / HTTP/1.1", upstream: "http://unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock:/", host: "unicorn.local" ps -aux shows that the workers are created and ls -la show srwxrwxrwx 1 mattmongeau staff 0 Oct 13 12:51 unicorn.sock= Any suggestions on what I am doing wrong. I running OSX 10.6.1, with nginx installed from passenger (nginx version: nginx/0.6.37) Thanks, Matt From bketelsen at gmail.com Tue Oct 13 13:20:42 2009 From: bketelsen at gmail.com (Brian Ketelsen) Date: Tue, 13 Oct 2009 13:20:42 -0400 Subject: Unicorn Nginx Issue In-Reply-To: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> Message-ID: <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> On Oct 13, 2009, at 12:57 PM, Matt Mongeau wrote: > 2009/10/13 12:34:22 [error] 1007#0: *62 connect() to > unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock failed > (61: Connection refused) while connecting to upstream, client: > 127.0.0.1, server: unicorn.local, request: "GET / HTTP/1.1", upstream: > "http://unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock:/ > ", > host: "unicorn.local" > > ps -aux shows that the workers are created > and ls -la show srwxrwxrwx 1 mattmongeau staff 0 Oct 13 12:51 > unicorn.sock= > > Any suggestions on what I am doing wrong. I running OSX 10.6.1, with > nginx installed from passenger (nginx version: nginx/0.6.37) > > Thanks, Matt > _______________________________________________ You must have a host entry in /etc/hosts or in NetInfo for unicorn.local - do you have it set to "localhost" or to 127.0.0.1? I've seen issues in Mac OS where I had to explicitly use 127.0.0.1 instead of localhost, and the error messages looked similar. Brian From halogenandtoast at gmail.com Tue Oct 13 13:27:39 2009 From: halogenandtoast at gmail.com (Matt Mongeau) Date: Tue, 13 Oct 2009 13:27:39 -0400 Subject: Unicorn Nginx Issue In-Reply-To: <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> Message-ID: <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> Hi Brian, my /etc/hosts file has an entry like this: 127.0.0.1 unicorn.local some further debugging... this works: unicorn_rails -E production -l unix:tmp/sockets/unicorn.sock -D this does not: unicorn_rails -E development -l unix:tmp/sockets/unicorn.sock -D On Tue, Oct 13, 2009 at 1:20 PM, Brian Ketelsen wrote: > > > On Oct 13, 2009, at 12:57 PM, Matt Mongeau wrote: > >> 2009/10/13 12:34:22 [error] 1007#0: *62 connect() to >> unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock failed >> (61: Connection refused) while connecting to upstream, client: >> 127.0.0.1, server: unicorn.local, request: "GET / HTTP/1.1", upstream: >> >> "http://unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock:/", >> host: "unicorn.local" >> >> ps -aux shows that the workers are created >> and ls -la show srwxrwxrwx ?1 mattmongeau ?staff ? ?0 Oct 13 12:51 >> unicorn.sock= >> >> Any suggestions on what I am doing wrong. I running OSX 10.6.1, with >> nginx installed from passenger (nginx version: nginx/0.6.37) >> >> Thanks, Matt >> _______________________________________________ > > > > You must have a host entry in /etc/hosts or in NetInfo for unicorn.local - > do you have it set to "localhost" or to 127.0.0.1? ?I've seen issues in Mac > OS where I had to explicitly use 127.0.0.1 instead of localhost, and the > error messages looked similar. > > Brian > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From halogenandtoast at gmail.com Tue Oct 13 13:34:21 2009 From: halogenandtoast at gmail.com (Matt Mongeau) Date: Tue, 13 Oct 2009 13:34:21 -0400 Subject: Unicorn Nginx Issue In-Reply-To: <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> Message-ID: <91d915350910131034t5dabb51eo5cde67397612f9f4@mail.gmail.com> Got it working, I had a bad value in my config. I had listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', :backlog => 1024 I guess I needed listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', :backlog => 1024 Thanks for the help. On Tue, Oct 13, 2009 at 1:27 PM, Matt Mongeau wrote: > Hi Brian, > > my /etc/hosts file has an entry like this: > > 127.0.0.1 unicorn.local > > some further debugging... > this works: > unicorn_rails ?-E production -l unix:tmp/sockets/unicorn.sock -D > this does not: > unicorn_rails ?-E development -l unix:tmp/sockets/unicorn.sock -D > > On Tue, Oct 13, 2009 at 1:20 PM, Brian Ketelsen wrote: >> >> >> On Oct 13, 2009, at 12:57 PM, Matt Mongeau wrote: >> >>> 2009/10/13 12:34:22 [error] 1007#0: *62 connect() to >>> unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock failed >>> (61: Connection refused) while connecting to upstream, client: >>> 127.0.0.1, server: unicorn.local, request: "GET / HTTP/1.1", upstream: >>> >>> "http://unix:/Users/user/projects/test/unicorn/tmp/sockets/unicorn.sock:/", >>> host: "unicorn.local" >>> >>> ps -aux shows that the workers are created >>> and ls -la show srwxrwxrwx ?1 mattmongeau ?staff ? ?0 Oct 13 12:51 >>> unicorn.sock= >>> >>> Any suggestions on what I am doing wrong. I running OSX 10.6.1, with >>> nginx installed from passenger (nginx version: nginx/0.6.37) >>> >>> Thanks, Matt >>> _______________________________________________ >> >> >> >> You must have a host entry in /etc/hosts or in NetInfo for unicorn.local - >> do you have it set to "localhost" or to 127.0.0.1? ?I've seen issues in Mac >> OS where I had to explicitly use 127.0.0.1 instead of localhost, and the >> error messages looked similar. >> >> Brian >> _______________________________________________ >> mongrel-unicorn mailing list >> mongrel-unicorn at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mongrel-unicorn >> > From normalperson at yhbt.net Tue Oct 13 14:53:15 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 13 Oct 2009 11:53:15 -0700 Subject: Unicorn Nginx Issue In-Reply-To: <91d915350910131034t5dabb51eo5cde67397612f9f4@mail.gmail.com> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> <91d915350910131034t5dabb51eo5cde67397612f9f4@mail.gmail.com> Message-ID: <20091013185314.GA15026@dcvr.yhbt.net> Matt Mongeau wrote: Hi Matt, please don't top post, thanks. > Got it working, I had a bad value in my config. > > I had > listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', > :backlog => 1024 > I guess I needed > listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', > :backlog => 1024 Both values you had above really should just work. Assuming you've installed unicorn (>= 0.93.0) as a gem, can you run: gem check -t unicorn and tell me if it passes? If you installed from the tgz or git, you can also try running the following from the source tree: make test/unit/test_configurator.rb Thanks. -- Eric Wong From halogenandtoast at gmail.com Tue Oct 13 15:03:14 2009 From: halogenandtoast at gmail.com (Matt Mongeau) Date: Tue, 13 Oct 2009 15:03:14 -0400 Subject: Unicorn Nginx Issue In-Reply-To: <20091013185314.GA15026@dcvr.yhbt.net> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> <91d915350910131034t5dabb51eo5cde67397612f9f4@mail.gmail.com> <20091013185314.GA15026@dcvr.yhbt.net> Message-ID: <91d915350910131203w2b67ddbcjba9189c2eac303d2@mail.gmail.com> sudo gem check -t unicorn fails Failure: test_rack_lint_big_put(RequestTest) [./test/unit/test_request.rb:178]: expected but was <" followed by lots whitespace ">. Error: test_expand_addr(TestConfigurator): SocketError: getaddrinfo: nodename nor servname provided, or not known /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in `pack_sockaddr_in' /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in `expand_addr' ./test/unit/test_configurator.rb:35:in `call' ./test/unit/test_configurator.rb:35:in `test_expand_addr' ERROR: 98 tests, 4748 assertions, 1 failures, 1 errors On Tue, Oct 13, 2009 at 2:53 PM, Eric Wong wrote: > Matt Mongeau wrote: > > Hi Matt, please don't top post, thanks. > >> Got it working, I had a bad value in my config. >> >> I had >> listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', >> :backlog => 1024 >> I guess I needed >> listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', >> :backlog => 1024 > > Both values you had above really should just work. ?Assuming you've > installed unicorn (>= 0.93.0) as a gem, can you run: > > ?gem check -t unicorn > > and tell me if it passes? ?If you installed from the tgz or git, you can > also try running the following from the source tree: > > ? make test/unit/test_configurator.rb > > Thanks. > > -- > Eric Wong > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From normalperson at yhbt.net Tue Oct 13 15:43:24 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 13 Oct 2009 12:43:24 -0700 Subject: Unicorn Nginx Issue In-Reply-To: <91d915350910131203w2b67ddbcjba9189c2eac303d2@mail.gmail.com> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> <91d915350910131034t5dabb51eo5cde67397612f9f4@mail.gmail.com> <20091013185314.GA15026@dcvr.yhbt.net> <91d915350910131203w2b67ddbcjba9189c2eac303d2@mail.gmail.com> Message-ID: <20091013194324.GA1676@dcvr.yhbt.net> Matt Mongeau wrote: > sudo gem check -t unicorn Again, please don't top post on this mailing list (nor on other technical mailing lists in general). > fails > > Failure: > test_rack_lint_big_put(RequestTest) [./test/unit/test_request.rb:178]: > expected but was > <" > > followed by lots whitespace > > ">. Unrelated to the problem we were having, but this really should be working, especially with 0.93.2 or later. Is anybody else out there hitting this? > Error: > test_expand_addr(TestConfigurator): > SocketError: getaddrinfo: nodename nor servname provided, or not known > /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in > `pack_sockaddr_in' > /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in > `expand_addr' > ./test/unit/test_configurator.rb:35:in `call' > ./test/unit/test_configurator.rb:35:in `test_expand_addr' This looks like a portability issue. I'll probably rip those tests out since a good chunk of systems don't addresses like this. But above this test failure, the other test_expand_addr assertions manage to pass which is strange, namely the following: meth = Unicorn::Configurator.new.method(:expand_addr) assert_equal "/var/run/unicorn.sock", meth.call("/var/run/unicorn.sock") assert_equal "#{Dir.pwd}/foo/bar.sock", meth.call("unix:foo/bar.sock") Your original paths were under 104 bytes, too[1] > >> I had > >> listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', > >> :backlog => 1024 > >> I guess I needed > >> listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', > >> :backlog => 1024 Does using a shorter path help at all? Shorter (and shallower) paths are even a small bit faster because the filesystem has to do less work to resolve it for every connection :) [1] - http://portabilityblog.com/blog/archives/4-UNIX-domain-sockets.html -- Eric Wong From halogenandtoast at gmail.com Tue Oct 13 16:11:44 2009 From: halogenandtoast at gmail.com (Matt Mongeau) Date: Tue, 13 Oct 2009 16:11:44 -0400 Subject: Unicorn Nginx Issue In-Reply-To: <20091013194324.GA1676@dcvr.yhbt.net> References: <91d915350910130957w6a5412f9w8e13ef1d0167ee40@mail.gmail.com> <0AFCBD06-4CB0-42FE-B3D8-0A6AFCF89962@gmail.com> <91d915350910131027g6a3e9704r5becf0d2e7402e39@mail.gmail.com> <91d915350910131034t5dabb51eo5cde67397612f9f4@mail.gmail.com> <20091013185314.GA15026@dcvr.yhbt.net> <91d915350910131203w2b67ddbcjba9189c2eac303d2@mail.gmail.com> <20091013194324.GA1676@dcvr.yhbt.net> Message-ID: <91d915350910131311i78473fd5yb4782e6fba60c9d9@mail.gmail.com> On Tue, Oct 13, 2009 at 3:43 PM, Eric Wong wrote: > Matt Mongeau wrote: >> sudo gem check -t unicorn > > Again, please don't top post on this mailing list (nor on other technical > mailing lists in general). > >> fails >> >> Failure: >> test_rack_lint_big_put(RequestTest) [./test/unit/test_request.rb:178]: >> expected but was >> <" >> >> followed by lots whitespace >> >> ">. > > Unrelated to the problem we were having, but this really should be > working, especially with 0.93.2 or later. ?Is anybody else out there > hitting this? > >> Error: >> test_expand_addr(TestConfigurator): >> SocketError: getaddrinfo: nodename nor servname provided, or not known >> ? ? /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in >> `pack_sockaddr_in' >> ? ? /opt/local/lib/ruby/gems/1.8/gems/unicorn-0.93.2/lib/unicorn/configurator.rb:346:in >> `expand_addr' >> ? ? ./test/unit/test_configurator.rb:35:in `call' >> ? ? ./test/unit/test_configurator.rb:35:in `test_expand_addr' > > This looks like a portability issue. ?I'll probably rip those tests out > since a good chunk of systems don't addresses like this. > > But above this test failure, the other test_expand_addr assertions > manage to pass which is strange, namely the following: > > ? ?meth = Unicorn::Configurator.new.method(:expand_addr) > > ? ?assert_equal "/var/run/unicorn.sock", meth.call("/var/run/unicorn.sock") > ? ?assert_equal "#{Dir.pwd}/foo/bar.sock", meth.call("unix:foo/bar.sock") > > Your original paths were under 104 bytes, too[1] > >> >> I had >> >> listen '/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', >> >> :backlog => 1024 >> >> I guess I needed >> >> listen 'unix:/Users/mattmongeau/projects/test/unicorn/tmp/sockets/unicorn.sock', >> >> :backlog => 1024 > > Does using a shorter path help at all? > > Shorter (and shallower) paths are even a small bit faster because the > filesystem has to do less work to resolve it for every connection :) > > [1] - http://portabilityblog.com/blog/archives/4-UNIX-domain-sockets.html > > -- > Eric Wong > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > Sorry for top posting, I actually didn't know what you meant by that... had to have someone else explain it to me. Ok my previous problem seems to be from running unicorn_rails multiple times so if I run unicorn_rails -c config/unicorn.rb -E development -D it works fine (with and without the unix prefix on the socket) then if I just run unicorn_rails -c config/unicorn.rb -E development without killing the current master everything seems to work if I issue a term signal ^C to it and try unicorn_rails -c config/unicorn.rb -E development -D it no longer works seems like I shouldn't issue these commands in tandem without first killing the previous unicorn instance From sunaku at gmail.com Tue Oct 13 22:34:21 2009 From: sunaku at gmail.com (Suraj N. Kurapati) Date: Tue, 13 Oct 2009 19:34:21 -0700 Subject: [PATCH] show configuration file path in errors instead of '(eval)' Message-ID: <1255487661-20498-1-git-send-email-sunaku@gmail.com> also __FILE__ did not reflect configuration file path --- lib/unicorn/configurator.rb | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb index c7afece..80584f0 100644 --- a/lib/unicorn/configurator.rb +++ b/lib/unicorn/configurator.rb @@ -91,7 +91,7 @@ module Unicorn end def reload #:nodoc: - instance_eval(File.read(config_file)) if config_file + instance_eval(File.read(config_file), config_file) if config_file end def commit!(server, options = {}) #:nodoc: -- 1.6.3.1 ----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ----------------------------------------------------------------------------------- From sunaku at gmail.com Tue Oct 13 22:38:50 2009 From: sunaku at gmail.com (Suraj Kurapati) Date: Tue, 13 Oct 2009 19:38:50 -0700 Subject: [PATCH] show configuration file path in errors instead of '(eval)' In-Reply-To: <1255487661-20498-1-git-send-email-sunaku@gmail.com> References: <1255487661-20498-1-git-send-email-sunaku@gmail.com> Message-ID: On Tue, Oct 13, 2009 at 7:34 PM, Suraj N. Kurapati wrote: > This email message is for the sole use of the intended recipient(s) and may contain > confidential information. ?Any unauthorized review, use, disclosure or distribution > is prohibited. ?If you are not the intended recipient, please contact the sender by > reply email and destroy all copies of the original message. Sigh, please ignore the above automatically-added legal disclaimer notice. :-( Thanks for your consideration. From normalperson at yhbt.net Wed Oct 14 02:17:47 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 13 Oct 2009 23:17:47 -0700 Subject: [PATCH] show configuration file path in errors instead of '(eval)' In-Reply-To: <1255487661-20498-1-git-send-email-sunaku@gmail.com> References: <1255487661-20498-1-git-send-email-sunaku@gmail.com> Message-ID: <20091014061747.GA15010@dcvr.yhbt.net> "Suraj N. Kurapati" wrote: > also __FILE__ did not reflect configuration file path > --- > lib/unicorn/configurator.rb | 2 +- > 1 files changed, 1 insertions(+), 1 deletions(-) Thanks Suraj! Applied and pushed out. -- Eric Wong From normalperson at yhbt.net Wed Oct 14 18:15:52 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 14 Oct 2009 15:15:52 -0700 Subject: weird unicorn_rails issue... Message-ID: <20091014221552.GA30624@dcvr.yhbt.net> Hi all, I just had a user on Rails v2.3.2 that had trouble[1] with the out-of-the-box unicorn_rails, but was worked around by using the following RAILS_ROOT/config.ru file with plain "unicorn" and manually setting RAILS_ENV in the shell environment require 'config/environment' use Rails::Rack::LogTailer map("/") do use Rails::Rack::Static run ActionController::Dispatcher.new end script/server + WEBrick worked out-of-the-box, as well. Oddly, the same config.ru file does not work with "unicorn_rails", either (even when the "config.ru" file is explicitly specified); only with "unicorn". So I'm a bit perplexed... [1] - by "trouble", I mean the app became very subtly broken. Query parameters (it was a GET request) appeared to be handled correctly, but the app was not returning the same results. I looked briefly at the app and noticed *something* was a bit suspicious: -------------- app/controllers/foo_controller.rb ------------- class FooController < ApplicationController def index all_params = some_weird_params_generated results = BarController.new.action(all_params) end end -------------- app/controllers/bar_controller.rb ------------- class BarController < ApplicationController def action(all_params) do_something end end -------------------------------------------------------------- That is, it creates a new controller from within one controller inside one action. Note that I'm not 100% certain this responsible for the breakage we were seeing, but it certainly does look like suspicious Rails code to me. I haven't decided if I'll spend time to fix/debug this, but at least I'll document it here if somebody wants to look into it further. -- Eric Wong From normalperson at yhbt.net Fri Oct 16 16:57:21 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 16 Oct 2009 13:57:21 -0700 Subject: [PATCH] KNOWN_ISSUES: document Rack gem issue w/Rails 2.3.2 In-Reply-To: <20091014221552.GA30624@dcvr.yhbt.net> References: <20091014221552.GA30624@dcvr.yhbt.net> Message-ID: <20091016205720.GA13223@dcvr.yhbt.net> In short: upgrade to Rails 2.3.4 (or later) ref: http://mid.gmane.org/20091014221552.GA30624 at dcvr.yhbt.net Note: the workaround described in the article above only made the issue more subtle and we didn't notice them immediately. --- Eric Wong wrote: > Hi all, > > I just had a user on Rails v2.3.2 that had trouble[1] with the > out-of-the-box unicorn_rails, but was worked around by using the > following RAILS_ROOT/config.ru file with plain "unicorn" and > manually setting RAILS_ENV in the shell environment > > require 'config/environment' > use Rails::Rack::LogTailer > map("/") do > use Rails::Rack::Static > run ActionController::Dispatcher.new > end > > script/server + WEBrick worked out-of-the-box, as well. > > Oddly, the same config.ru file does not work with "unicorn_rails", > either (even when the "config.ru" file is explicitly specified); > only with "unicorn". > > So I'm a bit perplexed... > > > [1] - by "trouble", I mean the app became very subtly broken. Query > parameters (it was a GET request) appeared to be handled correctly, but > the app was not returning the same results. I looked briefly at the > app and noticed *something* was a bit suspicious: > > -------------- app/controllers/foo_controller.rb ------------- > class FooController < ApplicationController > def index > all_params = some_weird_params_generated > results = BarController.new.action(all_params) > end > end > -------------- app/controllers/bar_controller.rb ------------- > class BarController < ApplicationController > def action(all_params) > do_something > end > end > -------------------------------------------------------------- > > That is, it creates a new controller from within one controller inside > one action. Note that I'm not 100% certain this responsible for the > breakage we were seeing, but it certainly does look like suspicious > Rails code to me. > > > I haven't decided if I'll spend time to fix/debug this, but at least > I'll document it here if somebody wants to look into it further. KNOWN_ISSUES | 13 +++++++++++++ unicorn.gemspec | 4 ++++ 2 files changed, 17 insertions(+), 0 deletions(-) diff --git a/KNOWN_ISSUES b/KNOWN_ISSUES index 436997d..979ac9d 100644 --- a/KNOWN_ISSUES +++ b/KNOWN_ISSUES @@ -1,5 +1,18 @@ = Known Issues +* Rails 2.3.2 bundles its own version of Rack. This may cause subtle + bugs when simultaneously loaded with the system-wide Rack Rubygem + which Unicorn depends on. Upgrading to Rails 2.3.4 (or later) is + strongly recommended for all Rails 2.3.x users for this (and security + reasons). Rails 2.2.x series (or before) did not bundle Rack and are + should be unnaffected. If there is any reason which forces your + application to use Rails 2.3.2 and you have no other choice, then + you may edit your Unicorn gemspec and remove the Rack dependency. + + ref: http://mid.gmane.org/20091014221552.GA30624 at dcvr.yhbt.net + Note: the workaround described in the article above only made + the issue more subtle and we didn't notice them immediately. + * Installing "unicorn" as a system-wide Rubygem and using the {isolate}[http://github.com/jbarnette/isolate] gem may cause issues if you're using any of the bundled application-level libraries in diff --git a/unicorn.gemspec b/unicorn.gemspec index c5b4422..063b313 100644 --- a/unicorn.gemspec +++ b/unicorn.gemspec @@ -43,6 +43,10 @@ Gem::Specification.new do |s| s.test_files = test_files + # for people that are absolutely stuck on Rails 2.3.2 and can't + # up/downgrade to any other version, the Rack dependency may be + # commented out. Nevertheless, upgrading to Rails 2.3.4 or later is + # *strongly* recommended for security reasons. s.add_dependency(%q) # s.licenses = %w(GPLv2 Ruby) # licenses= method is not in older Rubygems -- Eric Wong From normalperson at yhbt.net Sat Oct 17 11:58:52 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 17 Oct 2009 08:58:52 -0700 Subject: [RFH] test failure Message-ID: <20091017155852.GA29388@dcvr.yhbt.net> Btw, Matt's post also had this test failure. Has anybody else hit this? If so, which OS/Ruby version/arch? Matt Mongeau wrote: > sudo gem check -t unicorn > > fails > > Failure: > test_rack_lint_big_put(RequestTest) [./test/unit/test_request.rb:178]: > expected but was > <" > > followed by lots whitespace > > ">. I (obviously) can't reproduce myself[1]. Thanks! [1] - Though I did notice some redundant code now that env['rack.input'].read(bytes) always returns the requested size... -- Eric Wong From johnwulff at gmail.com Wed Oct 21 21:31:50 2009 From: johnwulff at gmail.com (John Wulff) Date: Wed, 21 Oct 2009 18:31:50 -0700 Subject: Metrics Message-ID: I'm a Munin-aholic (http://github.com/jwulff/munin_plugins) and I'd like to write a plugin for Unicorn. Does Unicorn keep any interesting metrics? Requests handled, etc.? If so, where can I find/access them? Thanks From normalperson at yhbt.net Thu Oct 22 01:13:13 2009 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 21 Oct 2009 22:13:13 -0700 Subject: Metrics In-Reply-To: References: Message-ID: <20091022051313.GA7022@dcvr.yhbt.net> John Wulff wrote: > I'm a Munin-aholic (http://github.com/jwulff/munin_plugins) and I'd > like to write a plugin for Unicorn. Does Unicorn keep any interesting > metrics? Requests handled, etc.? If so, where can I find/access > them? Nope, Unicorn tries to avoid anything that can be better implemented as a Rack handler. You could probably start with something like this: # this is totally untested of course :) class RequestCounter < Struct.new(:app, :nr) def initialize(app) super(app, 0) end def call(env) if env["PATH_INFO"] == "/NR" [200, { "Content-Type" => "text/plain" }, [ "#{nr}\n" ] ] else self.nr += 1 app.call(env) end end end -- Eric Wong From vadim at spivak.net Fri Oct 23 10:38:27 2009 From: vadim at spivak.net (Vadim Spivak) Date: Fri, 23 Oct 2009 07:38:27 -0700 Subject: Truncated request bodies In-Reply-To: References: Message-ID: There seems to be a ruby bug in 1.8.7 on OS X that's causing request bodies to be truncated in unicorn when they're bigger than the MAX_BODY (backed by a temp file instead of stringio). Example of truncation: http://gist.github.com/211431 Smaller ruby bug example: http://gist.github.com/216703 I couldn't reproduce this on 1.9.1 on OS X or 1.8.7 on Linux. Thanks, Vadim From normalperson at yhbt.net Fri Oct 23 12:15:58 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 23 Oct 2009 09:15:58 -0700 Subject: Truncated request bodies In-Reply-To: References: Message-ID: <20091023161558.GA21242@dcvr.yhbt.net> Vadim Spivak wrote: > There seems to be a ruby bug in 1.8.7 on OS X that's causing request > bodies to be truncated in unicorn when they're bigger than the > MAX_BODY (backed by a temp file instead of stringio). > > Example of truncation: http://gist.github.com/211431 > > Smaller ruby bug example: http://gist.github.com/216703 Hi Vadim, What's the output of that standalone ruby example for you? Is there a bug filed with ruby-core I could look at? Also, does using f << or f.syswrite change things? f.sync = true should really make it unnecessary... f = File.open("bar", File::RDWR|File::CREAT, 0600) f.sync=true f.read # why is this here? f << "Hello" # or f.syswrite # would an explicit f.flush here work? shouldn't be needed # with f.sync = true puts "Should be 5: #{f.pos}" f.close > I couldn't reproduce this on 1.9.1 on OS X or 1.8.7 on Linux. I'll definitely need help with testing this then since I only have Linux. Which 1.8.7 patchlevel is your OS X Ruby at? Do you know if the OS X packagers apply any vendor patches on top of the stock Ruby distro that could be causing it? Are those patches downloadable anywhere? Thanks! -- Eric Wong From normalperson at yhbt.net Sat Oct 24 12:40:19 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 24 Oct 2009 09:40:19 -0700 Subject: Fwd: Doc Examples In-Reply-To: <5976e8970910240835i760a0736xe6ae9c832868a57@mail.gmail.com> References: <5976e8970910240835i760a0736xe6ae9c832868a57@mail.gmail.com> Message-ID: <20091024164018.GB21242@dcvr.yhbt.net> Greg Melton wrote: > Eric, tried sending this one to the Unicorn list last night. I forgot to > subscribe to the list before sending and it bounced... Hi Greg, that's because HTML mail is not allowed on the list. > ---------- Forwarded message ---------- > From: Greg Melton > To: mongrel-unicorn at rubyforge.org > Date: Fri, 23 Oct 2009 23:11:05 -0700 > Subject: Doc Examples > I'm in the process of adding support for Unicorn to a Rack MVC framework > I've been noodling with => http://github.com/grippy/doozer. > > I copied the example conf from > http://unicorn.bogomips.org/Unicorn/Configurator.html and it's missing an > 'end'. > > No biggie, though. Thanks, updated and pushed out to the site. > One thing I was wondering: is there anyway Unicorn could come with some sort > init.d file (similar to nginx) to make starts/stops/restarts easier? There's examples/init.sh in the source tree: http://git.bogomips.org/cgit/unicorn.git/tree/examples/init.sh It should work with nginx, too since nginx and Unicorn accept the same sets of signals. > Here's the update for the DOCS file. > > if old_pid != server.pid > begin > sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU > Process.kill(sig, File.read(old_pid).to_i) > rescue Errno::ENOENT, Errno::ESRCH > > end > end From vadim at spivak.net Sat Oct 24 18:05:47 2009 From: vadim at spivak.net (Vadim Spivak) Date: Sat, 24 Oct 2009 15:05:47 -0700 Subject: Truncated request bodies In-Reply-To: References: Message-ID: > Hi Vadim, > > What's the output of that standalone ruby example for you? Is there a > bug filed with ruby-core I could look at? "Should be 5: 0" I haven't had a chance to file a bug yet and couldn't find Also, if you don't set sync to true, then you get the expected output 5 instead of 0. > > Also, does using f << or f.syswrite change things? > f.sync = true should really make it unnecessary... I tried both and they didn't make a difference > > f = File.open("bar", File::RDWR|File::CREAT, 0600) > f.sync=true > f.read # why is this here? > f << "Hello" # or f.syswrite > > # would an explicit f.flush here work? shouldn't be needed > # with f.sync = true > puts "Should be 5: #{f.pos}" > f.close > > > I couldn't reproduce this on 1.9.1 on OS X or 1.8.7 on Linux. > > I'll definitely need help with testing this then since I only have > Linux. > > Which 1.8.7 patchlevel is your OS X Ruby at? Do you know if the > OS X packagers apply any vendor patches on top of the stock Ruby distro > that could be causing it? Are those patches downloadable anywhere? > Thanks! I've tested both on 1.8.7 patch level 72. Also, just to make sure it wasn't a vendor patch, I just downloaded and compiled the source for 1.8.7 p72 and saw the same issue. # ships with OS X ruby 1.8.7 (2008-08-11 patchlevel 72) [universal-darwin10.0] # compiled from source on OS X ruby 1.8.7 (2008-08-11 patchlevel 72) [i686-darwin10.0.0] # linux install ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux] Also, I just tried it on a FreeBSD 7.2 VM and saw the same results as on OS X. (VM downloaded from http://www.thoughtpolice.co.uk/vmware/) Thanks, Vadim > > -- > Eric Wong From normalperson at yhbt.net Sat Oct 24 20:04:35 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 24 Oct 2009 17:04:35 -0700 Subject: Truncated request bodies In-Reply-To: References: Message-ID: <20091025000435.GA4130@dcvr.yhbt.net> Vadim Spivak wrote: > > Hi Vadim, > > > > What's the output of that standalone ruby example for you? Is there a > > bug filed with ruby-core I could look at? > > "Should be 5: 0" > > I haven't had a chance to file a bug yet and couldn't find > > Also, if you don't set sync to true, then you get the expected output > 5 instead of 0. Interesting.... I bet it doesn't fail if you skip the f.read in there. > > Also, does using f << or f.syswrite change things? > > f.sync = true should really make it unnecessary... > > I tried both and they didn't make a difference sysread (instead of read) + syswrite() would've made a difference, I very much hope, at least. > > f = File.open("bar", File::RDWR|File::CREAT, 0600) ^-- btw, I would put File::TRUNC there just in case because I got 10 the 2nd time running it :x > > f.sync=true > > f.read # why is this here? > > f << "Hello" # or f.syswrite > > > > # would an explicit f.flush here work? shouldn't be needed > > # with f.sync = true > > puts "Should be 5: #{f.pos}" > > f.close > > > > > I couldn't reproduce this on 1.9.1 on OS X or 1.8.7 on Linux. This could be a stdio bug... 1.9.1 uses only the saner unistd.h functions so it's immune to stdio bugs, but 1.8.7 uses stdio with explicit fflush() when sync=true instead of calling setvbuf(..._IONBF...) to disable buffering like a normal app would. Looking at io.c in 1.8.7-p72, it even seems to call setvbuf(..._IOFBF...) for full buffering on FreeBSD regardless of sync mode (!) > > I'll definitely need help with testing this then since I only have > > Linux. Does the following patch fix things for you? diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb index 6b35426..0f517e9 100644 --- a/lib/unicorn/util.rb +++ b/lib/unicorn/util.rb @@ -54,7 +54,6 @@ module Unicorn end File.unlink(fp.path) fp.binmode - fp.sync = true fp end --- On the flip side, I'm once again tempted to use sysread/sysseek/syswrite because I have a very intense and long-standing distrust of luserspace IO buffering libraries. -- Eric Wong From normalperson at yhbt.net Sat Oct 24 23:49:37 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 24 Oct 2009 20:49:37 -0700 Subject: Truncated request bodies In-Reply-To: <20091025000435.GA4130@dcvr.yhbt.net> References: <20091025000435.GA4130@dcvr.yhbt.net> Message-ID: <20091025034936.GA25264@dcvr.yhbt.net> Eric Wong wrote: > > Does the following patch fix things for you? > > diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb > index 6b35426..0f517e9 100644 > --- a/lib/unicorn/util.rb > +++ b/lib/unicorn/util.rb > @@ -54,7 +54,6 @@ module Unicorn > end > File.unlink(fp.path) > fp.binmode > - fp.sync = true > fp > end > --- Actually, that may not work since I need to call IO#stat.size, in TeeInput, and if I explicitly call IO#flush then bad things still end up happening :/ -- Eric Wong From normalperson at yhbt.net Sun Oct 25 17:31:31 2009 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 25 Oct 2009 14:31:31 -0700 Subject: Truncated request bodies In-Reply-To: <20091025034936.GA25264@dcvr.yhbt.net> References: <20091025000435.GA4130@dcvr.yhbt.net> <20091025034936.GA25264@dcvr.yhbt.net> Message-ID: <20091025213131.GB2505@dcvr.yhbt.net> Eric Wong wrote: > Eric Wong wrote: > > > > Does the following patch fix things for you? Hi Vadim, I actually just got a better patch offlist that looks more reasonable than mine: diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb index 188e2ea..7e77cdf 100644 --- a/lib/unicorn/tee_input.rb +++ b/lib/unicorn/tee_input.rb @@ -131,6 +131,7 @@ module Unicorn begin if parser.filter_body(dst, socket.readpartial(length, buf)).nil? @tmp.write(dst) + @tmp.seek(0, IO::SEEK_END) # workaround FreeBSD/OSX + MRI 1.8.x bug return dst end rescue EOFError --- Also pushed out to git://git.bogomips.org/unicorn Upon further inspection of the Ruby 1.8.7 source, I'm surprised it worked anywhere, glibc + Linux included :x I've managed to open a ticket on the issue for ruby-core: http://redmine.ruby-lang.org/issues/show/2267 -- Eric Wong From port001 at gmail.com Mon Oct 26 11:04:56 2009 From: port001 at gmail.com (Ian Leitch) Date: Mon, 26 Oct 2009 11:04:56 -0400 Subject: RAILS_ROOT and USR2 restarts Message-ID: Hi, I've just started using Unicorn in production behind Nginx with a Rails app, for the most part its been a smooth transition apart from this one issue. When I do an?in place?restart (USR2), the restart process works but the workers load my old Rails release, not the new one. I'm guessing that because the master process is currently in the old release directory, when I signal USR2 it's just reloading inside the current directory. I'm deploying with Capistrano, so app/current is a symlink to a specific release. I've tried changing to the new release directory in before_work but that made no difference. Is there a callback that fires before the master preloads the app? worker_processes 6 preload_app true timeout 60 listen "/tmp/unicorn.production.sock" pid "/tmp/unicorn.production.pid" before_fork do |server, worker| ??Dir.chdir("/var/www/apps/systino_production/current") ??old_pid = "/tmp/unicorn.#{RAILS_ENV}.pid.oldbin" ??if File.exists?(old_pid) && server.pid != old_pid ?? ?begin ?? ? ?Process.kill("QUIT", File.read(old_pid).to_i) ?? ?rescue Errno::ENOENT, Errno::ESRCH ?? ? ?# someone else did our job for us ?? ?end ??end end after_fork do |server, worker| ??ActiveRecord::Base.establish_connection end I also have a little launcher script (I'm using Solaris - SMF doesn't let me provide a custom restart routine, so I have to bypass it and use this script): #!/usr/bin/ruby $rails_env = ARGV[0] $pid_file = "/tmp/unicorn.#{$rails_env}.pid" def restart(pid) ??system("kill -USR2 #{pid}") end def start ??system("/opt/local/bin/unicorn_rails -c config/unicorn.rb -E #{$rails_env} -D") end if File.exists?($pid_file) ??pid = File.read($pid_file).strip.to_i ??begin ?? ?Process.kill(0, pid) ??rescue Errno::ESRCH ?? ?# Not running ?? ?start ??else ?? ?restart(pid) ??end else ??start end At the end of the deploy I execute: task :restart_unicorn do ??run "cd #{current_path}; ruby config/unicorn_launcher.rb #{rails_env}" end From normalperson at yhbt.net Mon Oct 26 12:01:58 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 26 Oct 2009 09:01:58 -0700 Subject: RAILS_ROOT and USR2 restarts In-Reply-To: References: Message-ID: <20091026160158.GA16220@dcvr.yhbt.net> Ian Leitch wrote: > Hi, > > I've just started using Unicorn in production behind Nginx with a > Rails app, for the most part its been a smooth transition apart from > this one issue. > When I do an?in place?restart (USR2), the restart process works but > the workers load my old Rails release, not the new one. I'm guessing > that because the master process is currently in the old release > directory, when I signal USR2 it's just reloading inside the current > directory. I'm deploying with Capistrano, so app/current is a symlink > to a specific release. I've tried changing to the new release > directory in before_work but that made no difference. Is there a > callback that fires before the master preloads the app? Hi Ian, Unicorn chdirs to the directory returned by `/bin/sh -c pwd` when it was originally started. The output of `pwd` *should* be symlink-aware, but then Solaris /bin/sh is weird. Can you confirm that it's broken? You can put the following in your config (outside of the hooks): app_root = "/var/www/apps/systino_production/current" Dir.chdir(Unicorn::HttpServer::START_CTX[:cwd] = app_root) I've been pondering adding a "working_directory" directive to Configurator, too. However the START_CTX hash is considered a stable interface and I recently documented it in http://unicorn.bogomips.org/Unicorn.html START_CTX even allows you to switch between different installation paths for Unicorn[1] and alter command line options that were originally passed. Lots of rope there :> [1] for different Ruby installs/versions, or even swap in Rainbows! or vice versa -- Eric Wong From port001 at gmail.com Mon Oct 26 12:13:31 2009 From: port001 at gmail.com (Ian Leitch) Date: Mon, 26 Oct 2009 12:13:31 -0400 Subject: RAILS_ROOT and USR2 restarts In-Reply-To: <20091026160158.GA16220@dcvr.yhbt.net> References: <20091026160158.GA16220@dcvr.yhbt.net> Message-ID: Apologies for the noise, I discovered START_CTX[:cwd] and that fixed it. Solaris sh is indeed the culprit: $ cd /var/www/apps/production/current (admin at production:/var/www/apps/production/current) $ sh -c pwd /var/www/apps/production/releases/20091026160154 Thanks for creating Unicorn! 2009/10/26 Eric Wong : > Ian Leitch wrote: >> Hi, >> >> I've just started using Unicorn in production behind Nginx with a >> Rails app, for the most part its been a smooth transition apart from >> this one issue. >> When I do an?in place?restart (USR2), the restart process works but >> the workers load my old Rails release, not the new one. I'm guessing >> that because the master process is currently in the old release >> directory, when I signal USR2 it's just reloading inside the current >> directory. I'm deploying with Capistrano, so app/current is a symlink >> to a specific release. I've tried changing to the new release >> directory in before_work but that made no difference. Is there a >> callback that fires before the master preloads the app? > > Hi Ian, > > Unicorn chdirs to the directory returned by `/bin/sh -c pwd` when it was > originally started. ?The output of `pwd` *should* be symlink-aware, but > then Solaris /bin/sh is weird. ?Can you confirm that it's broken? > > You can put the following in your config (outside of the hooks): > > ?app_root = "/var/www/apps/systino_production/current" > ?Dir.chdir(Unicorn::HttpServer::START_CTX[:cwd] = app_root) > > I've been pondering adding a "working_directory" directive to > Configurator, too. ?However the START_CTX hash is considered a stable > interface and I recently documented it in > http://unicorn.bogomips.org/Unicorn.html > > START_CTX even allows you to switch between different installation paths > for Unicorn[1] and alter command line options that were originally > passed. ?Lots of rope there :> > > [1] for different Ruby installs/versions, > ? ?or even swap in Rainbows! or vice versa > > -- > Eric Wong > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From dstamat at elctech.com Mon Oct 26 15:34:00 2009 From: dstamat at elctech.com (Dylan Stamat) Date: Mon, 26 Oct 2009 12:34:00 -0700 Subject: multi server failover setup Message-ID: Eric, have you heard of anybody using the idea you laid out here? http://article.gmane.org/gmane.comp.lang.ruby.unicorn.general/31 I really like the approach, and hoping I'm not going to be the first one trying it ;) I noticed that Mr. Wanstrath is using UDS in his deployment here: http://github.com/blog/517-unicorn Wondering if they're running with the same or similar strategy as your suggestion? Our application is long-running-request heavy, so we may actually be a Rainbows! candidate. Thanks! == Dylan From chris at ozmm.org Mon Oct 26 15:42:53 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Mon, 26 Oct 2009 12:42:53 -0700 Subject: multi server failover setup In-Reply-To: References: Message-ID: <8b73aaca0910261242r6323f664s46536567f584d8d0@mail.gmail.com> On Mon, Oct 26, 2009 at 12:34 PM, Dylan Stamat wrote: > I noticed that Mr. Wanstrath is using UDS in his deployment here: > ?http://github.com/blog/517-unicorn > Wondering if they're running with the same or similar strategy as your > suggestion? We're currently not using this strategy. Perhaps in the future, but currently we're still trying to completely isolate the various services within our app - when nginx or Unicorn chokes presently, they all do. Once that's fixed we'll begin looking into this or something similar. -- Chris Wanstrath http://github.com/defunkt From normalperson at yhbt.net Mon Oct 26 17:42:54 2009 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 26 Oct 2009 14:42:54 -0700 Subject: multi server failover setup In-Reply-To: <8b73aaca0910261242r6323f664s46536567f584d8d0@mail.gmail.com> References: <8b73aaca0910261242r6323f664s46536567f584d8d0@mail.gmail.com> Message-ID: <20091026214253.GA23833@dcvr.yhbt.net> Chris Wanstrath wrote: > On Mon, Oct 26, 2009 at 12:34 PM, Dylan Stamat wrote: > > I noticed that Mr. Wanstrath is using UDS in his deployment here: > > ?http://github.com/blog/517-unicorn > > Wondering if they're running with the same or similar strategy as your > > suggestion? > > We're currently not using this strategy. Perhaps in the future, but > currently we're still trying to completely isolate the various > services within our app - when nginx or Unicorn chokes presently, they > all do. Once that's fixed we'll begin looking into this or something > similar. By Unicorn choking you mean something in your app chokes, right? :) Is this something adding fail_timeout=0 in nginx can fix/workaround? Not sure where nginx itself would choke besides uncached disk reads or somehow writing a lot of logs. That said I myself haven't gotten a chance to test that setup I proposed for a while (except the nginx fail_timeout=0 part, I've gotten folks running with that for ages now). -- Eric Wong From chris at ozmm.org Mon Oct 26 17:47:29 2009 From: chris at ozmm.org (Chris Wanstrath) Date: Mon, 26 Oct 2009 14:47:29 -0700 Subject: multi server failover setup In-Reply-To: <20091026214253.GA23833@dcvr.yhbt.net> References: <8b73aaca0910261242r6323f664s46536567f584d8d0@mail.gmail.com> <20091026214253.GA23833@dcvr.yhbt.net> Message-ID: <8b73aaca0910261447v4688de9cy9ea909c75b3448d9@mail.gmail.com> On Mon, Oct 26, 2009 at 2:42 PM, Eric Wong wrote: >> We're currently not using this strategy. Perhaps in the future, but >> currently we're still trying to completely isolate the various >> services within our app - when nginx or Unicorn chokes presently, they >> all do. Once that's fixed we'll begin looking into this or something >> similar. > > By Unicorn choking you mean something in your app chokes, right? :) > Is this something adding fail_timeout=0 in nginx can fix/workaround? Yes, our app itself. Unicorn has been amazingly solid for us, as has nginx. We've added fail_timeout=0 into our config as per your suggestion and it's helped a lot. Still doesn't fix bugs in our app code though :) -- Chris Wanstrath http://github.com/defunkt From vadim at spivak.net Mon Oct 26 22:48:35 2009 From: vadim at spivak.net (Vadim Spivak) Date: Mon, 26 Oct 2009 19:48:35 -0700 Subject: Truncated request bodies In-Reply-To: <20091025213131.GB2505@dcvr.yhbt.net> References: <20091025000435.GA4130@dcvr.yhbt.net> <20091025034936.GA25264@dcvr.yhbt.net> <20091025213131.GB2505@dcvr.yhbt.net> Message-ID: Thanks Eric, I verified that it's working now. On Sun, Oct 25, 2009 at 2:31 PM, Eric Wong wrote: > Eric Wong wrote: >> Eric Wong wrote: >> > >> > Does the following patch fix things for you? > > Hi Vadim, I actually just got a better patch offlist that > looks more reasonable than mine: > > diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb > index 188e2ea..7e77cdf 100644 > --- a/lib/unicorn/tee_input.rb > +++ b/lib/unicorn/tee_input.rb > @@ -131,6 +131,7 @@ module Unicorn > ? ? ? ? begin > ? ? ? ? ? if parser.filter_body(dst, socket.readpartial(length, buf)).nil? > ? ? ? ? ? ? @tmp.write(dst) > + ? ? ? ? ? ?@tmp.seek(0, IO::SEEK_END) # workaround FreeBSD/OSX + MRI 1.8.x bug > ? ? ? ? ? ? return dst > ? ? ? ? ? end > ? ? ? ? rescue EOFError > --- > > Also pushed out to git://git.bogomips.org/unicorn > > Upon further inspection of the Ruby 1.8.7 source, I'm surprised it > worked anywhere, glibc + Linux included :x > > I've managed to open a ticket on the issue for ruby-core: > ?http://redmine.ruby-lang.org/issues/show/2267 > > -- > Eric Wong > _______________________________________________ > mongrel-unicorn mailing list > mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > From normalperson at yhbt.net Tue Oct 27 04:56:13 2009 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 27 Oct 2009 01:56:13 -0700 Subject: [ANN] unicorn 0.93.4 - *BSD stdio compatibility Message-ID: <20091027085612.GA29122@dcvr.yhbt.net> Unicorn is a HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: This release mainly works around BSD stdio compatibility issues that affect at least FreeBSD and OS X. While this issues was documented and fixed in [ruby-core:26300][1], no production release of MRI 1.8 has it, and users typically upgrade MRI more slowly than gems. This issue does NOT affect 1.9 users. Thanks to Vadim Spivak for reporting and testing this issue and Andrey Stikheev for the fix. Additionally there are small documentation bits, one error handling improvement, and one minor change that should improve reliability of signal delivery. Andrey Stikheev (1): workaround FreeBSD/OSX IO bug for large uploads Eric Wong (7): DESIGN: address concerns about on-demand and thundering herd README: alter reply conventions for the mailing list configurator: stop testing for non-portable listens KNOWN_ISSUES: document Rack gem issue w/Rails 2.3.2 stop continually resends signals during shutdowns add news bodies to site NEWS.atom.xml configurator: fix broken example in RDoc Suraj N. Kurapati (1): show configuration file path in errors instead of '(eval)' [1] http://redmine.ruby-lang.org/issues/show/2267 -- Eric Wong From normalperson at yhbt.net Thu Oct 29 17:58:52 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 29 Oct 2009 14:58:52 -0700 Subject: [ANN] unicorn 0.93.5 - fix timeout kill regression Message-ID: <20091029215852.GA18820@dcvr.yhbt.net> Unicorn is a HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: This release fixes a regression introduced in 0.93.3 where timed-out worker processes run a chance of not being killed off at all if they're hung. While it's not ever advisable to have requests take a long time, we realize it's not easy to fix everything :) -- Eric Wong From normalperson at yhbt.net Thu Oct 29 18:44:26 2009 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 29 Oct 2009 15:44:26 -0700 Subject: Unicorn/Rainbows! mailing list migration Message-ID: <20091029224426.GA12314@dcvr.yhbt.net> As I'm sure you've all heard by now, RubyForge will be moving to a read-only state and we'll have to migrate our mailing lists somewhere. == librelist I'm leaning strongly towards librelist.com. I've had good, but limited experiences with it. I like that librelist has no logins or accounts to deal with; just email the list you want to join and reply to the automated message to subscribe. It also has downloadable archives via rsync. librelist is run by Zed Shaw, the same guy who started Mongrel (which eventually led us to Unicorn). While I don't believe in giving any single entity too much power and influence, Zed's done a lot of good for Mongrel and handed it off gracefully when he didn't want to work on it anymore. For these reason, I think librelist will be in good hands for years to come. I probably would've picked librelist if it had been around when I started this RubyForge list in May 2009. I started Rainbows! more recently, but continued using RubyForge because I wanted everything Rainbows!-related on RubyForge.org (oops!). == Other options Google Groups is out. Spam has been a big problem in the few groups I've joined and I'd rather trust Zed than Google for mailing lists. Also I don't know of a way to get archives from Google without scraping. SourceForge - I've had spam problems in the past, mailman... Savannah - no idea about the quality, but uses mailman... I've also considered hosting the list ourselves, but that might be too much work and spreading ourselves too thin at the moment. Any other list hosting options out there we should consider? It must be a mailing list that does not mangle plain-text patches. Or any votes in favor of migrating to librelist sooner rather than later? I'll work on getting the gmane NNTP mirrors moved over as well. == Other stuff related to RubyForge going read-only... Gem hosting will (obviously) be taken care of by our new Gemcut^WRubygems.org overlords. The tarballs for non-RubyGems users will probably just be moved to a files/ section of the websites; it'll be even be easier to find and download tarballs in automated scripts that way, too. Since we're using git, our repositories are mirrored everywhere already and was never centrally-hosted on RubyForge in the first place. http://rainbows.rubyforge.org/ will probably be redirected to http://rainbows.bogomips.org/ (same box as the Unicorn website). Should you ever want to access either of the sites offline, just make sure you have RDoc 2.4.x ("gem install rdoc" if you're on 1.8) and run `make doc' or `gmake doc' in the latest source tree. -- Eric Wong From normalperson at yhbt.net Fri Oct 30 18:17:18 2009 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 30 Oct 2009 15:17:18 -0700 Subject: Unicorn/Rainbows! mailing list migration In-Reply-To: <20091029224426.GA12314@dcvr.yhbt.net> References: <20091029224426.GA12314@dcvr.yhbt.net> Message-ID: <20091030221718.GA18576@dcvr.yhbt.net> Eric Wong wrote: > As I'm sure you've all heard by now, RubyForge will be moving to a > read-only state and we'll have to migrate our mailing lists somewhere. OK, it looks like we'll be able to keep our mailing lists on RubyForge. http://tomcopeland.blogs.com/juniordeveloper/2009/10/things-to-keep-from-rubyforge.html On the other hand, if anybody just *wants* to move to librelist, let us know and we can consider it together, too. Thanks. -- Eric Wong