negative timeout in Rainbows::Fiber::Base

Eric Wong normalperson at yhbt.net
Wed Aug 29 21:17:07 UTC 2012


"Lin Jen-Shin (godfat)" <godfat at godfat.org> wrote:
> Sorry that I didn't subscribe the mailing list (now subscribed),
> so I didn't realize there's already a response. The quoted
> text is copied from archive.

Oops :x  I encourage unsubscribed senders to remind repliers
to Cc: them for this reason.

Unfortunately ruby-talk has conditioned the Ruby community to favor
subscriber-only lists and the Mailman config for rainbows-talk avoids
Cc-by-default (even though we'll still accept unsubscribed senders).

> Eric Wong <normalperson at yhbt.net> wrote:
> > "Lin Jen-Shin (godfat)" <godfat at godfat.org> wrote:
> > Yes.  (Though it should've be easy to just pipe that part of my
> > message to "git apply" or even "patch -p1")
> 
> I did that in the beginning, but it looks like your patch was not
> created from 12281e4cee86496588814cd1851e8764caea024c,
> so I cannot apply it? Anyway, not important :)

Anyways, I've pushed my patch to master as
commit e794a40049959a23ba311c572e518bb7c2861812

Will probably release v4.4.1 with that tonight unless there's something
else...

> > All the Thread-based concurrency models should already just work
> > with Celluloid-based apps.
> > http://news.gmane.org/find-root.php?message_id=%3cBANLkTimnNQYpk7TRkiCvhDikzGHVy%3dkOdw%40mail.gmail.com%3e
> 
> And it's not worth the effort to wrap around celluloid-io for
> buffering requests? Since it's using nio4r[0] underneath
> and it's using libev for I/O, I thought that would be a replacement
> for eventmachine?
> 
> [0]: https://github.com/tarcieri/nio4r

I haven't followed celluloid* closely since it was originally announced.
Maybe it's worth it to offer celluloid-io as an option and I will accept
patches for it.

> > It's much easier to unwatch IO objects with Cool.io than with EM, so I
> > haven't done much with EM + Fibers.  There's also rack-fiber_pool...
> 
> True, working with Cool.io is much more pleasant. As for rack-fiber_pool,
> I am not sure if I am correct or not, but in my humble option, that's a
> totally wrong approach. I think fibers should be handled in server level,
> not application level. That way, we don't need the hack for throwing :async,
> and the middlewares do not need to be aware of fibers and :async,
> and we won't need async-rack, and the server doesn't need to be
> aware of :async, so on so forth.
> 
> I am not sure if this approach would have any bad influence, but at least
> it works fine in our applications. That is, just wrap fibers around the calling
> the app. (i.e. Fiber.new{ app_call(env) }.resume)

Rainbows! tries not to favor/disfavor any approaches to concurrency.
And rack-fiber_pool was easy-to-support, so I added support for it.

> > Rainbows::EventMachine::TryDefer already uses the EM-internal thread
> > pool, so I think EventMachineThreadPool would be redundant.
> 
> I didn't know this before, I guess it should work for me, though
> it might not be as direct as:
> 
> https://github.com/godfat/ruby-server-exp/blob/9d44f8387739f5395bf97aff1689f17615cc4c7e/config/rainbows-em-thread-pool.rb#L21-25
>   class REMTPoolClient < Rainbows::EventMachine::Client
>     def app_call input
>       EM.defer{ super }
>     end
>   end

I seem to recall this didn't work well with some corner cases
(large static files/socket-proxying in responses).

> > > https://github.com/cardinalblue/rest-more/blob/master/example/rainbows.rb#L15-19
> > > class REMFClient < Rainbows::EventMachine::Client
> > >   def app_call input
> > >     Fiber.new{ super }.resume
> > >   end
> > > end
> > Does it actually show benefits compared to regular EM?
> > I suppose the Rack application still needs to be made Fiber-aware
> > to reap benefits of Fibers
> 
> The Rack application doesn't need to be fiber-aware, but if
> the libraries are fiber-aware, then it would be beneficial.
> The rest of program doesn't have to know the existence of
> fibers, because we don't throw :async.

OK.  But are clients served concurrently by Rainbows! in this case?

I'm not sure if I'm following this correctly (haven't thought about
Fibers in a long time), but control never goes back to Rainbows! itself
in your above case, does it?

> > Yes, threads are far more compatible with existing gems/libraries and I
> > think that's the better way to go.  Of course, just processes (with
> > regular unicorn) should be most compatible, too.
> 
> At first I thought fibers are better for I/O, but after implementing
> a thread based approach for the same mechanism, I started to
> doubt that. I don't have a conclusion yet though.
> (I am a bit nervous to switch to thread based approach on
> production to find out, since it's not yet tested intensively)
> 
> On the other hand, all our apps are running on Heroku, and
> we need to reduce memory usage because they have a hard
> limit. That's why we're running Zbatery at the moment.

Interesting.  Are you on 32 or 64-bit and are you constrained by VMSize
or RSS?

If it's VMSize and you're not dealing with deep data structures _and_
your code doesn't use recursion much; lowering pthreads stack size to
64K in thread_pthread.c should help with memory usage.  Not sure if you
can supply your own Ruby builds for that hosting service, though.

If you have time, you could also try making something like GNU pth or
npth work with Ruby 1.9.  I suspect Fibers will become unusable
with *pth, though...

> > > Sorry that I might be asking too many questions in a single thread.
> > No problem!  I'm just happy that folks are showing interest
> 
> I really don't understand why Unicorn family didn't get much
> attention in Ruby community. And I always feel wrong when
> people are comparing Unicorn with Thin or even Passenger.
> (sigh)

I'm actually surprised unicorn got the attention it has.

As project leader, my preference to maintain a low public profile and my
refusal to endorse/associate with for-profit interests certainly hurts
adoption.

If somebody else (like you) wants to market these projects, then more
power to them :)  I'm certainly never going to speak at any conference.

Rainbows! just overwhelms potential users with too many choices,
so it's unlikely to ever be widely-adopted.

> Many thanks for your work and help!

No problem :)


More information about the rainbows-talk mailing list