From mark.titorenko at alces-software.com Fri Jan 11 11:57:12 2013 From: mark.titorenko at alces-software.com (Mark J. Titorenko) Date: Fri, 11 Jan 2013 11:57:12 +0000 Subject: [PATCH] event_machine: join reactor_thread if it is already running rather than falling through worker_loop Message-ID: Prior to the application of this patch, if an EventMachine reactor_thread has already been started elsewhere before the worker_loop is entered, the worker_loop exits as a second call to EM.run does not block the current thread. This patch causes the worker_loop thread to join the reactor_thread if it is running. Cheers, Mark. --- lib/rainbows/event_machine.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/rainbows/event_machine.rb b/lib/rainbows/event_machine.rb index 3ecdb4f..b143b39 100644 --- a/lib/rainbows/event_machine.rb +++ b/lib/rainbows/event_machine.rb @@ -99,6 +99,7 @@ module Rainbows::EventMachine end end } + EM.reactor_thread.join if EM.reactor_running? end end # :enddoc: -- 1.7.9.6 (Apple Git-31.1) From normalperson at yhbt.net Fri Jan 11 20:18:46 2013 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 11 Jan 2013 20:18:46 +0000 Subject: [PATCH] event_machine: join reactor_thread if it is already running rather than falling through worker_loop In-Reply-To: References: Message-ID: <20130111201846.GA14566@dcvr.yhbt.net> "Mark J. Titorenko" wrote: > Prior to the application of this patch, if an EventMachine > reactor_thread has already been started elsewhere before the > worker_loop is entered, the worker_loop exits as a second call to > EM.run does not block the current thread. > > This patch causes the worker_loop thread to join the reactor_thread if > it is running. Thanks Mark! Pushed out as a12d2e64e04b5098f31e63a72ea8330b6c8f3a19 (line-wrapped commit message) to git://bogomips.org/rainbows.git From normalperson at yhbt.net Fri Jan 18 11:31:17 2013 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 18 Jan 2013 11:31:17 +0000 Subject: [ANN] Rainbows! 4.4.3 - bugfixes for EventMachine users Message-ID: <20130118113117.GA12148@dcvr.yhbt.net> Changes: This release contains bugfixes from Lin Jen-Shin and Mark J. Titorenko for EventMachine. There are also some minor cleanups. Lin Jen-Shin (1): event_machine: avoid close on deferred response Mark J. Titorenko (1): event_machine: join reactor_thread if it is already running Eric Wong (2): event_machine: cleanup confusing assignment t/GNUmakefile: cleanup test dependencies * http://rainbows.rubyforge.org/ * rainbows-talk at rubyforge.org * git://bogomips.org/rainbows.git * http://rainbows.rubyforge.org/NEWS.atom.xml -- Eric Wong From andrew at andrewloe.com Fri Jan 18 18:33:56 2013 From: andrew at andrewloe.com (W. Andrew Loe III) Date: Fri, 18 Jan 2013 10:33:56 -0800 Subject: Alternatives to EventMachine for async.callback Message-ID: The documentation (http://rainbows.rubyforge.org/Rainbows/EventMachine.html) specifically lists EventMachine as compatible with this model. Are there alternative ways to get this type interface without EventMachine? My current application is using the EventMachine loop to answer requests and spawn Celluloid Actors (which are in their own threads) which write back to the main thread. Seems like maybe I should just use a Thread-per-client model? I'm currently using Sinatra to do the HTTP acrobatics. -- Andrew From normalperson at yhbt.net Fri Jan 18 20:28:48 2013 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 18 Jan 2013 20:28:48 +0000 Subject: Alternatives to EventMachine for async.callback In-Reply-To: References: Message-ID: <20130118202848.GA30420@dcvr.yhbt.net> "W. Andrew Loe III" wrote: > The documentation > (http://rainbows.rubyforge.org/Rainbows/EventMachine.html) > specifically lists EventMachine as compatible with this model. Are > there alternative ways to get this type interface without > EventMachine? My current application is using the EventMachine loop to > answer requests and spawn Celluloid Actors (which are in their own > threads) which write back to the main thread. Seems like maybe I > should just use a Thread-per-client model? I'm currently using Sinatra > to do the HTTP acrobatics. Thread-per-client is a perfectly good model, especially on 64-bit. async.callback was only added because Thin supports it; and I don't think anybody's tried to make it work without EM. IMHO it's ugly. From andrew at andrewloe.com Fri Jan 18 22:04:48 2013 From: andrew at andrewloe.com (W. Andrew Loe III) Date: Fri, 18 Jan 2013 14:04:48 -0800 Subject: Alternatives to EventMachine for async.callback In-Reply-To: <20130118202848.GA30420@dcvr.yhbt.net> References: <20130118202848.GA30420@dcvr.yhbt.net> Message-ID: On Fri, Jan 18, 2013 at 12:28 PM, Eric Wong wrote: > Thread-per-client is a perfectly good model, especially on 64-bit. I'd like to get this working using the Celluloid threading, then I don't have to manage them myself, seems harder to have sinatra do thread-per client with is streaming code, its tied to async.callback. From andrew at andrewloe.com Sat Jan 19 00:21:14 2013 From: andrew at andrewloe.com (W. Andrew Loe III) Date: Fri, 18 Jan 2013 16:21:14 -0800 Subject: Sinatra stream(:keep_open) not holding connections open In-Reply-To: References: <20121205014454.GA24401@dcvr.yhbt.net> <20121205031848.GA32757@dcvr.yhbt.net> Message-ID: This appears to have been broken in my application in Rainbows 4.4.3. On Tue, Dec 4, 2012 at 10:09 PM, W. Andrew Loe III wrote: > Works great! Thank you very much for the quick response and patch! > > Feature Request: https://github.com/celluloid/celluloid-io > Celluloid::IO backend, the spiritual successor to cool.io. I'm going > to take a stab at this soon, but wanted to get something out. > > On Tue, Dec 4, 2012 at 7:18 PM, Eric Wong wrote: >> "W. Andrew Loe III" wrote: >>> I've pushed a very very stripped down example that just echos the time. >>> https://github.com/loe/sinatra-sse >> >> Thanks for the test case, I've just pushed out the following fix >> to "master" of git://bogomips.org/rainbows.git >> >> I've pushed a rainbows 4.4.1.1.gd5c8c prerelease to RubyGems.org >> It should be installable with: gem install --pre rainbows >> >> I expect this to work for you, but be sure to let me know if it doesn't :x >> >> Since this is a pretty small fix, I think I'll push 4.4.2 final >> very soon. >> >> >From d5c8cc8b51619f0d33f75036c53e3936ad2749b2 Mon Sep 17 00:00:00 2001 >> From: Eric Wong >> Date: Wed, 5 Dec 2012 03:08:19 +0000 >> Subject: [PATCH] event_machine: properly defer body.close for async >> >> Calling body.close in the normal write_response() code path >> is incorrect, and only worked out of sheer luck with >> Cramp and async_sinata. >> >> This change allows stream(:keep_open) in Sinatra to work >> properly. >> >> Thanks to W. Andrew Loe III for the informative bug report >> and reproducible test case. >> >> ref: http://mid.gmane.org/CA+-9oNd1EFqsniPkkPTwu5opTCinbM7-2KHoXov7+y3LE4s4Tg at mail.gmail.com >> --- >> lib/rainbows/event_machine/client.rb | 3 +++ >> 1 file changed, 3 insertions(+) >> >> diff --git a/lib/rainbows/event_machine/client.rb b/lib/rainbows/event_machine/client.rb >> index e56931f..fc0dfe3 100644 >> --- a/lib/rainbows/event_machine/client.rb >> +++ b/lib/rainbows/event_machine/client.rb >> @@ -64,8 +64,11 @@ class Rainbows::EventMachine::Client < EM::Connection >> @state = :headers if alive >> if body.respond_to?(:errback) && body.respond_to?(:callback) >> @deferred = body >> + write_headers(status, headers, alive) >> + write_body_each(body) >> deferred_errback(body) >> deferred_callback(body, alive) >> + return >> elsif body.respond_to?(:to_path) >> st = File.stat(path = body.to_path) >> >> -- >> Eric Wong >> _______________________________________________ >> Rainbows! mailing list - rainbows-talk at rubyforge.org >> http://rubyforge.org/mailman/listinfo/rainbows-talk >> Do not quote signatures (like this one) or top post when replying From andrew at andrewloe.com Sat Jan 19 01:01:58 2013 From: andrew at andrewloe.com (W. Andrew Loe III) Date: Fri, 18 Jan 2013 17:01:58 -0800 Subject: Sinatra stream(:keep_open) not holding connections open In-Reply-To: References: <20121205014454.GA24401@dcvr.yhbt.net> <20121205031848.GA32757@dcvr.yhbt.net> Message-ID: I was too quick to the gun, the change appears to be between Rack 1.4.1 and 1.4.2. Pinning my application to rack 1.4.1 and it works well with kgio 2.8.0 and rainbows 4.4.3. Frustrating. Broken here: https://github.com/rack/rack/commit/18ff5d4a70c93a49de62a359986c1567cb4bf00d On Fri, Jan 18, 2013 at 4:21 PM, W. Andrew Loe III wrote: > This appears to have been broken in my application in Rainbows 4.4.3. > > On Tue, Dec 4, 2012 at 10:09 PM, W. Andrew Loe III wrote: >> Works great! Thank you very much for the quick response and patch! >> >> Feature Request: https://github.com/celluloid/celluloid-io >> Celluloid::IO backend, the spiritual successor to cool.io. I'm going >> to take a stab at this soon, but wanted to get something out. >> >> On Tue, Dec 4, 2012 at 7:18 PM, Eric Wong wrote: >>> "W. Andrew Loe III" wrote: >>>> I've pushed a very very stripped down example that just echos the time. >>>> https://github.com/loe/sinatra-sse >>> >>> Thanks for the test case, I've just pushed out the following fix >>> to "master" of git://bogomips.org/rainbows.git >>> >>> I've pushed a rainbows 4.4.1.1.gd5c8c prerelease to RubyGems.org >>> It should be installable with: gem install --pre rainbows >>> >>> I expect this to work for you, but be sure to let me know if it doesn't :x >>> >>> Since this is a pretty small fix, I think I'll push 4.4.2 final >>> very soon. >>> >>> >From d5c8cc8b51619f0d33f75036c53e3936ad2749b2 Mon Sep 17 00:00:00 2001 >>> From: Eric Wong >>> Date: Wed, 5 Dec 2012 03:08:19 +0000 >>> Subject: [PATCH] event_machine: properly defer body.close for async >>> >>> Calling body.close in the normal write_response() code path >>> is incorrect, and only worked out of sheer luck with >>> Cramp and async_sinata. >>> >>> This change allows stream(:keep_open) in Sinatra to work >>> properly. >>> >>> Thanks to W. Andrew Loe III for the informative bug report >>> and reproducible test case. >>> >>> ref: http://mid.gmane.org/CA+-9oNd1EFqsniPkkPTwu5opTCinbM7-2KHoXov7+y3LE4s4Tg at mail.gmail.com >>> --- >>> lib/rainbows/event_machine/client.rb | 3 +++ >>> 1 file changed, 3 insertions(+) >>> >>> diff --git a/lib/rainbows/event_machine/client.rb b/lib/rainbows/event_machine/client.rb >>> index e56931f..fc0dfe3 100644 >>> --- a/lib/rainbows/event_machine/client.rb >>> +++ b/lib/rainbows/event_machine/client.rb >>> @@ -64,8 +64,11 @@ class Rainbows::EventMachine::Client < EM::Connection >>> @state = :headers if alive >>> if body.respond_to?(:errback) && body.respond_to?(:callback) >>> @deferred = body >>> + write_headers(status, headers, alive) >>> + write_body_each(body) >>> deferred_errback(body) >>> deferred_callback(body, alive) >>> + return >>> elsif body.respond_to?(:to_path) >>> st = File.stat(path = body.to_path) >>> >>> -- >>> Eric Wong >>> _______________________________________________ >>> Rainbows! mailing list - rainbows-talk at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/rainbows-talk >>> Do not quote signatures (like this one) or top post when replying From normalperson at yhbt.net Sat Jan 19 01:09:34 2013 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 19 Jan 2013 01:09:34 +0000 Subject: Sinatra stream(:keep_open) not holding connections open In-Reply-To: References: <20121205014454.GA24401@dcvr.yhbt.net> <20121205031848.GA32757@dcvr.yhbt.net> Message-ID: <20130119010934.GA1771@dcvr.yhbt.net> "W. Andrew Loe III" wrote: > I was too quick to the gun, the change appears to be between Rack > 1.4.1 and 1.4.2. Pinning my application to rack 1.4.1 and it works > well with kgio 2.8.0 and rainbows 4.4.3. Phew! You had me worried. It seems fixed in Sinatra: https://groups.google.com/forum/?fromgroups=#!topic/rack-devel/PGaU2qD9oS8 From normalperson at yhbt.net Thu Jan 24 19:56:38 2013 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 24 Jan 2013 19:56:38 +0000 Subject: [PATCH] epoll: ensure closing of pipelined clients if required Message-ID: <20130124195638.GA21628@dcvr.yhbt.net> on_read normally relies on the close checking in on_readable, but on_deferred_write_complete may be called from on_writable instead (bypassing the close check of on_readable). This affects both Epoll and XEpoll users, but only if they were sending responses via body#to_path and triggering on_deferred_write_complete. While we're at it, favor non-recursive want_more instead of calling on_readable recursively in ev_write_responses to prevent stack growth. --- commit 49f70632e2f4a9b84fd4fced7439d0844fed1bbd in "master" of git://bogomips.org/rainbows.git lib/rainbows/epoll/client.rb | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/lib/rainbows/epoll/client.rb b/lib/rainbows/epoll/client.rb index f690d85..d72696b 100644 --- a/lib/rainbows/epoll/client.rb +++ b/lib/rainbows/epoll/client.rb @@ -102,7 +102,19 @@ module Rainbows::Epoll::Client else write_response(status, headers, body, alive) end - on_read(Z) if alive && 0 == @wr_queue.size && 0 != @buf.size + # try to read more if we didn't have to buffer writes + next_request if alive && 0 == @wr_queue.size + end + + def next_request + if 0 == @buf.size + want_more + else + # pipelined request (already in buffer) + on_read(Z) + return if @wr_queue[0] || closed? + close if :close == @state + end end def epoll_run @@ -120,7 +132,7 @@ module Rainbows::Epoll::Client def on_deferred_write_complete :close == @state and return close - 0 == @buf.size ? on_readable : on_read(Z) + next_request end def handle_error(e) -- Eric Wong From godfat at godfat.org Tue Jan 29 03:44:06 2013 From: godfat at godfat.org (Lin Jen-Shin) Date: Tue, 29 Jan 2013 11:44:06 +0800 Subject: [PATCH] Add -N or --no-default-middleware option. Message-ID: <1359431046-22344-1-git-send-email-godfat@godfat.org> This would prevent Unicorn (Rainbows) from adding default middleware, as if RACK_ENV were always none. (not development nor deployment) This is implemented in Unicorn, so we only need to update the option parser here. Discussion thread on Unicorn mailing list: http://rubyforge.org/pipermail/mongrel-unicorn/2013-January/001675.html --- bin/rainbows | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/bin/rainbows b/bin/rainbows index 878b3f2..4d6a5d6 100644 --- a/bin/rainbows +++ b/bin/rainbows @@ -59,6 +59,11 @@ op = OptionParser.new("", 24, ' ') do |opts| ENV["RACK_ENV"] = e end + opts.on("-N", "--no-default-middleware", + "no default middleware even if RACK_ENV is development") do |e| + rackup_opts[:no_default_middleware] = true + end + opts.on("-D", "--daemonize", "run daemonized in the background") do |d| rackup_opts[:daemonize] = !!d end -- 1.8.1.1 From godfat at godfat.org Tue Jan 29 04:06:00 2013 From: godfat at godfat.org (Lin Jen-Shin) Date: Tue, 29 Jan 2013 12:06:00 +0800 Subject: [PATCH] Add -N or --no-default-middleware option. Message-ID: <1359432360-23122-1-git-send-email-godfat@godfat.org> This would prevent Unicorn (Zbatery) from adding default middleware, as if RACK_ENV were always none. (not development nor deployment) This is implemented in Unicorn, so we only need to update the option parser here. Discussion thread on Unicorn mailing list: http://rubyforge.org/pipermail/mongrel-unicorn/2013-January/001675.html --- bin/zbatery | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/bin/zbatery b/bin/zbatery index 99ca46e..392e5c3 100755 --- a/bin/zbatery +++ b/bin/zbatery @@ -59,6 +59,11 @@ opts = OptionParser.new("", 24, ' ') do |opts| ENV["RACK_ENV"] = e end + opts.on("-N", "--no-default-middleware", + "do not load middleware implied by RACK_ENV") do |e| + rackup_opts[:no_default_middleware] = true + end + opts.on("-D", "--daemonize", "run daemonized in the background") do |d| rackup_opts[:daemonize] = !!d end -- 1.8.1.1 From normalperson at yhbt.net Tue Jan 29 04:49:43 2013 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 29 Jan 2013 04:49:43 +0000 Subject: [PATCH] Add -N or --no-default-middleware option. In-Reply-To: <1359432360-23122-1-git-send-email-godfat@godfat.org> References: <1359432360-23122-1-git-send-email-godfat@godfat.org> Message-ID: <20130129044943.GA2093@dcvr.yhbt.net> Thanks, both applied and pushed (they're no-ops until unicorn 4.6.0, though).