From doasyouasaan7ana at msn.com Fri Jun 3 18:59:49 2011 From: doasyouasaan7ana at msn.com (Louis) Date: Sat, 4 Jun 2011 06:59:49 +0800 Subject: =?GB2312?B?QW5pbWF0ZWQgTW92aWUvVFYgU2VyaWVzIA==?= =?GB2312?B?LSBDYXJ0b29uIE1vdmllL1RWIFNlcmllcw==?= =?GB2312?B?IC0gM0QvMkQgQW5pbWF0aW9uIFNlcnZpYw==?= =?GB2312?B?ZXM=?= Message-ID: You are receiving this email because we wish you to use our 3D/2D Animated Movie/TV Series Services. We are a China based Animated Movie/TV Series Studio. with the technical, our studio is a animation studio with the technical, creative and production capabilities to create a new generation of animated feature films, merchandise and other related products. Our objective is to combine proprietary technology and world-class creative talent to develop computer-animated feature films with memorable characters and heartwarming stories that appeal to audiences of all ages. We utilize the finest equipment available in the industry, offer efficient data delivery and unrivaled quality and work until the client is fully satisfied with the end product. Core Offerings 2D Animation 3D Animation 4D Animation Cell Animation Cartoon Animation Animated Movies Animated TV Series Architecture Visualization Character Animation Animated Commercial Animated Presentation Walkthrough Animation Flash Presentation Logo Animation Animated Wallpapers Animated Graphics We are also looking for agents who can represent us in USA and Europe in order to gain more Animated Movie orders. Best regards, Louis Tanmansoniu Animation Services Contact: ibanicontact at yeah.net Pls send address to koremoveideit at yeah.net for remove From alexandre at objectdata.com.br Wed Jun 8 23:30:43 2011 From: alexandre at objectdata.com.br (Alexandre Riveira) Date: Thu, 09 Jun 2011 00:30:43 -0300 Subject: algoritim schedule from linux nptl Message-ID: <4DF03E63.2030707@objectdata.com.br> I thank all the initiatives proposed by Rainbows she is really extraordinary! My company works with ERP (Enterprise Resource Planning) and this leads to generate large reports. As a matter of contract a customer of ours can have only one worker / multi threaded server. In such cases, when a large report is generated using 100% of processing other requests are waiting in cases of multiple simultaneous requests however small the simultaneity is normal Got a decrease in time using a kernel with ck patch (https: / / wiki.archlinux.org/index.php/Kernel26-ck) in the order of 12 seconds down to 3 seconds but this is not ideal. Does anyone know how the schedule works in linux and that you can adjust the schedule of linux? Alexandre Riveira Object Data From normalperson at yhbt.net Thu Jun 9 02:52:11 2011 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 9 Jun 2011 06:52:11 +0000 Subject: algoritim schedule from linux nptl In-Reply-To: <4DF03E63.2030707@objectdata.com.br> References: <4DF03E63.2030707@objectdata.com.br> Message-ID: <20110609065211.GA3853@dcvr.yhbt.net> Alexandre Riveira wrote: > As a matter of contract a customer of ours can have only one worker > / multi threaded server. That sucks. Rainbows! works best with multiple worker processes for CPU/memory-bound concurrency (as long as you have enough memory/cores). > In such cases, when a large report is generated using 100% of > processing other requests are waiting in cases of multiple > simultaneous requests however small the simultaneity is normal > > Got a decrease in time using a kernel with ck patch (https: / / > wiki.archlinux.org/index.php/Kernel26-ck) in the order of 12 seconds > down to 3 seconds but this is not ideal. How fast is the large report generation if there's only one simultaneous request? That's the fastest it'll ever be unless you optimize the report generation code itself (and not the scheduling/concurrency). > Does anyone know how the schedule works in linux and that you can > adjust the schedule of linux? Matz Ruby 1.9 is entirely at the mercy of Linux scheduler (a good thing IMHO). Unless your report generator is written to release the GVL (in C), you won't get any CPU-bound concurrency with Ruby 1.9 threads. Fortunately the API for releasing the GVL is pretty good (even if most of the documentation means reading thread.c comments in the Ruby source. -- Eric Wong From alexandre at objectdata.com.br Thu Jun 9 10:42:25 2011 From: alexandre at objectdata.com.br (Alexandre Riveira) Date: Thu, 09 Jun 2011 11:42:25 -0300 Subject: algoritim schedule from linux nptl Message-ID: <4DF0DBD1.5040000@objectdata.com.br> Hi Eric ! When I referred to has decreased from 12 to 3 seconds there was the report itself, but the other requests made ??at the same time the report is generated Tanks Alexandre Riveira In such cases, when a large report is generated using 100% of processing other requests are waiting in cases of multiple simultaneous requests however small the simultaneity is normal Got a decrease in time using a kernel with ck patch (https: / / wiki.archlinux.org/index.php/Kernel26-ck) in the order of 12 seconds down to 3 seconds but this is not ideal. ---------------------------------------- How fast is the large report generation if there's only one simultaneous request? That's the fastest it'll ever be unless you optimize the report generation code itself (and not the scheduling/concurrency). From normalperson at yhbt.net Sun Jun 12 18:26:38 2011 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 12 Jun 2011 22:26:38 +0000 Subject: client_max_header_size config directive Message-ID: <20110612222638.GA27267@dcvr.yhbt.net> This new directive limits the maximum size of a request header for all requests. Default: 112 kilobytes (114688 bytes) Lowering this will lower worst-case memory usage and mitigate some denial-of-service attacks. This should be larger than client_header_buffer_size. The default value is carried over from Mongrel and Unicorn. This requires unicorn 3.7.0+ This directive is NOT available in nginx, but is named similarly to client_max_body_size (which Rainbows! also supports) git clone git://bogomips.org/rainbows.git ref: commit 281ef555c40cf292809ad10d623d0571fafc790e -- Eric Wong From kyledrake at gmail.com Mon Jun 20 16:51:23 2011 From: kyledrake at gmail.com (Kyle Drake) Date: Mon, 20 Jun 2011 13:51:23 -0700 Subject: Rainbows and 2.0.0pre - thread safety Message-ID: Hi guys, I've setup a load test for Rainbows with Rubinius 2.0.0pre: https://github.com/kyledrake/rubinius-sinatra-loadtesting/tree/master/rainbows When I load test it on 2.0.0pre (via rvm and with apache bench), it has a lot of weird, random errors. My current hypothesis is this is a bug pertaining to thread safety in Rubinius, but someone suggested I post it here so you could take a look. Tried both ThreadSpawn and ThreadPool. I also made a test for Mongrel (which has similar errors), and Thin (which works fine, but doesn't really utilize threads so it doesn't mean anything). It would also be nice to know which deploy strategy would work well for Rubinius' threading. I wasn't sure if ThreadSpawn or ThreadPool made more sense so I tried both. Cheers! -Kyle From normalperson at yhbt.net Mon Jun 20 17:29:39 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 20 Jun 2011 14:29:39 -0700 Subject: Rainbows and 2.0.0pre - thread safety In-Reply-To: References: Message-ID: <20110620212939.GA18538@dcvr.yhbt.net> Kyle Drake wrote: > Hi guys, > > I've setup a load test for Rainbows with Rubinius 2.0.0pre: > > https://github.com/kyledrake/rubinius-sinatra-loadtesting/tree/master/rainbows > > When I load test it on 2.0.0pre (via rvm and with apache bench), it > has a lot of weird, random errors. My current hypothesis is this is a > bug pertaining to thread safety in Rubinius, but someone suggested I > post it here so you could take a look. Tried both ThreadSpawn and > ThreadPool. I also made a test for Mongrel (which has similar errors), > and Thin (which works fine, but doesn't really utilize threads so it > doesn't mean anything). Sounds likely Rubinius isn't 100% thread-safe. WEBrick should be worth a try, too, since it's part of the standard library. I would start there, actually. Some questions: Does calling C extensions in Rubinius 2.x take a global lock of any sort? I seem to recall reading it did, but am not sure... I also semi-recall reading that JRuby set the precedent for individual Ruby methods on an object being thread-safe even with a free-threading runtime, so stuff like: thread 1 | thread 2 ----------------------------------------------- hash[:t1] = true | hash[:t2] = false ...wouldn't need locking for a given hash. It would be nice to confirm if Rubinius follows that. I don't think the ThreadPool/ThreadSpawn models rely on either of those behaviors, though. > It would also be nice to know which deploy strategy would work well > for Rubinius' threading. I wasn't sure if ThreadSpawn or ThreadPool > made more sense so I tried both. It probably depend more on your OS threading implementation and hardware. Both are fully-supported and worth trying, though. I use ThreadSpawn on my cheap VPS (bogomips.org/rainbows.git) since it's memory-constrained, receives almost no traffic, and spawning threads with NTPL is cheap vs my peak request rate. ThreadPool might be better if you have a very high rate of requests and/or spawning OS threads is expensive given your request rate. > Cheers! Thanks for the interest in Rainbows! -- Eric Wong From chris at altbit.org Mon Jun 27 00:56:49 2011 From: chris at altbit.org (Christopher Hobbs) Date: Sun, 26 Jun 2011 23:56:49 -0500 Subject: Rainbows! Github mirror Message-ID: I just wanted to drop a message to the list to say that I'm mirroring the Rainbows! project on Github. Here's the repo: https://github.com/hooobs/rainbows It updates nightly at roughly 2300 EST. -- Christopher M. Hobbs http://altbit.org From normalperson at yhbt.net Mon Jun 27 01:17:58 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 27 Jun 2011 05:17:58 +0000 Subject: Rainbows! Github mirror In-Reply-To: References: Message-ID: <20110627051758.GA19697@dcvr.yhbt.net> Christopher Hobbs wrote: > I just wanted to drop a message to the list to say that I'm mirroring > the Rainbows! project on Github. Here's the repo: > https://github.com/hooobs/rainbows > > It updates nightly at roughly 2300 EST. Cool! I encourage mirrors anywhere, the more the merrier! -- Eric Wong From normalperson at yhbt.net Mon Jun 27 01:20:47 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 27 Jun 2011 05:20:47 +0000 Subject: Rainbows! Github mirror In-Reply-To: References:

Message-ID: <20110627052047.GB19697@dcvr.yhbt.net> (private emailer) wrote: > Is there any chance you'd consider switching to Github? People seem to > prefer it these days. No, never. I will never endorse the centralization of git repos anywhere. I'll never endorse any for-profit entities/ventures in my work in Free Software, either. -- Eric Wong From normalperson at yhbt.net Mon Jun 27 05:45:17 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 27 Jun 2011 09:45:17 +0000 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS Message-ID: <20110627094517.GA27156@dcvr.yhbt.net> Changes: Rainbows! now scales to more than 1024 worker processes without special privileges. To enable this, Rainbows! now depends on Unicorn 4.x and thus raindrops[1]. client_max_header_size directive is added to limit per-client memory usage in headers. An experimental StreamResponseEpoll concurrency option now exists to buffer outgoing responses without any thread-safe dependencies. Unlike the rest of Rainbows! which works fine without nginx, this concurrency option is /only/ supported behind nginx, even more strongly so than Unicorn itself. non-nginx LAN clients are NOT supported for this. This relies on the sleepy_penguin[2] RubyGem (and Linux). There are some minor bug fixes and cleanups all around. See "git log v3.4.0.." for details. [1] http://raindrops.bogomips.org/ [2] http://bogomips.org/sleepy_penguin/ * http://rainbows.rubyforge.org/ * rainbows-talk at rubyforge.org * git://bogomips.org/rainbows.git -- Eric Wong From normalperson at yhbt.net Mon Jun 27 06:04:23 2011 From: normalperson at yhbt.net (Eric Wong) Date: Mon, 27 Jun 2011 10:04:23 +0000 Subject: [ANN] Zbatery 4.0.0 - another Rainbows! resync Message-ID: <20110627100423.GA28304@dcvr.yhbt.net> Zbatery is an HTTP server for Rack applications on systems that either do not support fork(), or have no memory (nor need) to run the master/worker model. It is based on Rainbows! (which is based on Unicorn (which is based on Mongrel)) and inherits parts of each. Zbatery supports your choice of all the thread/fiber/event/actor-based concurrency models and Rack middleware that Rainbows! supports (or will ever support) in a single process. * http://zbatery.bogomip.org/ * rainbows-talk at rubyforge.org * git://bogomips.org/zbatery.git Changes: This gets most of the improvements Rainbows! 4.0.0 got: * client_max_header_size directive is added to limit per-client memory usage in headers. * An experimental StreamResponseEpoll concurrency option * minor bugfixes, minor stack depth reduction Since Zbatery doesn't fork workers, the ability of Unicorn 4.x to scale to a large amount of worker processes doesn't matter to us. -- Eric Wong From ghazel at gmail.com Wed Jun 29 02:36:22 2011 From: ghazel at gmail.com (ghazel at gmail.com) Date: Tue, 28 Jun 2011 23:36:22 -0700 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS In-Reply-To: <20110627094517.GA27156@dcvr.yhbt.net> References: <20110627094517.GA27156@dcvr.yhbt.net> Message-ID: On Mon, Jun 27, 2011 at 2:45 AM, Eric Wong wrote: > Changes: > > Rainbows! now scales to more than 1024 worker processes without > special privileges. ?To enable this, Rainbows! now depends on > Unicorn 4.x and thus raindrops[1]. I can't seem to upgrade from rainbows-3.4.0 to rainbows-4.0.0. I see: # a bunch in the rainbows.stderr.log. -Greg From normalperson at yhbt.net Wed Jun 29 03:11:01 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 29 Jun 2011 07:11:01 +0000 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS In-Reply-To: References: <20110627094517.GA27156@dcvr.yhbt.net> Message-ID: <20110629071101.GA8110@dcvr.yhbt.net> ghazel at gmail.com wrote: > On Mon, Jun 27, 2011 at 2:45 AM, Eric Wong wrote: > > Changes: > > > > Rainbows! now scales to more than 1024 worker processes without > > special privileges. ?To enable this, Rainbows! now depends on > > Unicorn 4.x and thus raindrops[1]. > > I can't seem to upgrade from rainbows-3.4.0 to rainbows-4.0.0. I see: > > # That's odd, what configuration options and Ruby version are you using? Are you using a per-worker listen socket? -- Eric Wong From ghazel at gmail.com Wed Jun 29 03:16:26 2011 From: ghazel at gmail.com (ghazel at gmail.com) Date: Wed, 29 Jun 2011 00:16:26 -0700 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS In-Reply-To: <20110629071101.GA8110@dcvr.yhbt.net> References: <20110627094517.GA27156@dcvr.yhbt.net> <20110629071101.GA8110@dcvr.yhbt.net> Message-ID: On Wed, Jun 29, 2011 at 12:11 AM, Eric Wong wrote: > ghazel at gmail.com wrote: >> On Mon, Jun 27, 2011 at 2:45 AM, Eric Wong wrote: >> > Changes: >> > >> > Rainbows! now scales to more than 1024 worker processes without >> > special privileges. ?To enable this, Rainbows! now depends on >> > Unicorn 4.x and thus raindrops[1]. >> >> I can't seem to upgrade from rainbows-3.4.0 to rainbows-4.0.0. I see: >> >> # > > That's odd, what configuration options and Ruby version are you > using? ?Are you using a per-worker listen socket? REE 1.8.7 Hopefully relevant config: use :ThreadSpawn ... listen "/tmp/rainbows.sock", :backlog => 2048 listen 8998, :tcp_nopush => true ... after_fork do |server, worker| # per-process listener ports for debugging/admin/migrations addr = "127.0.0.1:#{9100 + worker.nr}" server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true) -Greg From normalperson at yhbt.net Wed Jun 29 03:19:05 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 29 Jun 2011 07:19:05 +0000 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS In-Reply-To: <20110629071101.GA8110@dcvr.yhbt.net> References: <20110627094517.GA27156@dcvr.yhbt.net> <20110629071101.GA8110@dcvr.yhbt.net> Message-ID: <20110629071905.GA6838@dcvr.yhbt.net> Eric Wong wrote: > ghazel at gmail.com wrote: > > I can't seem to upgrade from rainbows-3.4.0 to rainbows-4.0.0. I see: > > > > # > > That's odd, what configuration options and Ruby version are you > using? Are you using a per-worker listen socket? This should fix it, the test case for this was broken, too :< diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb index 02ba965..78d80b4 100644 --- a/lib/unicorn/http_server.rb +++ b/lib/unicorn/http_server.rb @@ -453,7 +453,7 @@ class Unicorn::HttpServer def after_fork_internal @ready_pipe.close if @ready_pipe Unicorn::Configurator::RACKUP.clear - @ready_pipe = @init_listeners = @config = @before_exec = @before_fork = nil + @ready_pipe = @init_listeners = @before_exec = @before_fork = nil srand # http://redmine.ruby-lang.org/issues/4338 @@ -545,6 +545,7 @@ class Unicorn::HttpServer after_fork.call(self, worker) # can drop perms worker.user(*user) if user.kind_of?(Array) && ! worker.switched self.timeout /= 2.0 # halve it for select() + @config = nil build_app! unless preload_app end diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb index 0f6b083..8c33457 100644 --- a/test/exec/test_exec.rb +++ b/test/exec/test_exec.rb @@ -516,7 +516,7 @@ EOF File.unlink(tmp.path) ucfg = Tempfile.new('unicorn_test_config') ucfg.syswrite("listen '#@addr:#@port'\n") - ucfg.syswrite("before_fork { |s,w|\n") + ucfg.syswrite("after_fork { |s,w|\n") ucfg.syswrite(" s.listen('#{tmp.path}', :backlog => 5, :sndbuf => 8192)\n") ucfg.syswrite(" s.listen('#@addr:#{port2}', :rcvbuf => 8192)\n") ucfg.syswrite("\n}\n") -- Eric Wong From normalperson at yhbt.net Wed Jun 29 03:29:36 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 29 Jun 2011 07:29:36 +0000 Subject: [ANN] unicorn 4.0.0.2.g19f7 prerelease Message-ID: <20110629072936.GA10100@dcvr.yhbt.net> I just pushed a prerelease gem to RubyGems.org with the following changes. Please let us know if there were any other regressions in 4.0.0. Expect a final 4.0.1 release soonish... Full git changelog (pushed to git://bogomips.org/unicorn.git): commit 19f798301ac1884f423640efafb277b071bb5439 Author: Eric Wong Date: Wed Jun 29 07:19:32 2011 +0000 fix per-worker listen directive in after_fork hook The testcase for this was broken, too, so we didn't notice this :< Reported-by: ghazel at gmail.com on the Rainbows! mailing list, http://mid.gmane.org/BANLkTi=oQXK5Casq9SuGD3edeUrDPvRm3A at mail.gmail.com commit 38672501206c9e64d241e3d8571f70b198f0c1e5 Author: Eric Wong Date: Mon Jun 27 20:51:16 2011 +0000 configurator: truncate timeouts to 32-bit LONG_MAX IO.select in Ruby can't wait longer than this. This means Unicorn can't support applications that take longer than 68 years to respond :( -- Eric Wong From normalperson at yhbt.net Wed Jun 29 03:31:03 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 29 Jun 2011 07:31:03 +0000 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS In-Reply-To: References: <20110627094517.GA27156@dcvr.yhbt.net> <20110629071101.GA8110@dcvr.yhbt.net> Message-ID: <20110629073103.GB10100@dcvr.yhbt.net> ghazel at gmail.com wrote: > after_fork do |server, worker| > # per-process listener ports for debugging/admin/migrations > addr = "127.0.0.1:#{9100 + worker.nr}" > server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true) Yup, as I suspected. The patch I posted (and the unicorn prerelease Rubygem) should fix it. -- Eric Wong From ghazel at gmail.com Wed Jun 29 03:47:10 2011 From: ghazel at gmail.com (ghazel at gmail.com) Date: Wed, 29 Jun 2011 00:47:10 -0700 Subject: [ANN] Rainbows! 4.0.0 - MOAR concurrency for MOAR COARS In-Reply-To: <20110629073103.GB10100@dcvr.yhbt.net> References: <20110627094517.GA27156@dcvr.yhbt.net> <20110629071101.GA8110@dcvr.yhbt.net> <20110629073103.GB10100@dcvr.yhbt.net> Message-ID: On Wed, Jun 29, 2011 at 12:31 AM, Eric Wong wrote: > ghazel at gmail.com wrote: >> after_fork do |server, worker| >> ? # per-process listener ports for debugging/admin/migrations >> ? addr = "127.0.0.1:#{9100 + worker.nr}" >> ? server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true) > > Yup, as I suspected. ?The patch I posted (and the unicorn prerelease > Rubygem) should fix it. Yup. Unicorn and Rainbows! upgraded fine now. Thanks! -Greg From normalperson at yhbt.net Wed Jun 29 15:10:32 2011 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 29 Jun 2011 19:10:32 +0000 Subject: [ANN] unicorn 4.0.1 - regression bugfixes Message-ID: <20110629191032.GA3828@dcvr.yhbt.net> Changes: This release fixes things for users of per-worker "listen" directives in the after_fork hook. Thanks to ghazel at gmail.com for reporting the bug. The "timeout" configurator directive is now truncated to 0x7ffffffe seconds to prevent overflow when calling IO.select. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://bogomips.org/unicorn.git -- Eric Wong