From tamtomojnewv at mail.ru Sat Jul 7 10:04:49 2012 From: tamtomojnewv at mail.ru (=?Windows-1251?B?yOfs5e1l7ej/IOIg8uDsb+Zl7e3u7CDwZePz6+jw7uLg7ejo?=) Date: Sat, 07 Jul 2012 14:04:49 +0400 Subject: =?Windows-1251?B?0mHsb+bl7e376SDqb+Tl6mM=?= Message-ID: <940845215.20120707660642@ns.zdrastee.com> ???o?????? ???e?? - 2012 A???a????e ?o?po?? ?p??e????? ?o?o?e??? ?o???o ?a????????? ???e??? ??p????. Ho??? ?????o? ?a????e??? ?p?????? ???o?????? ????ap????, ????a????? ?a???e???? c????o??? ?e???a? ? 14 ???? ? 2012 ?. ?. K?e?, ??. Ca?c?????????, 6, "?o? K??o" ?PO?P???? ???e????? ? ?a???????? ???p?????? ???ap??: ?o??? ?op??o? ?a?o??e??? ?p?????? ???o?????? ????a?????. ???o?????? ?po?e???? ???o?e????? ???p??????. ??e??po???? ????a????????? ?o?ap??. Oco???????? ?p??e?e??? ?a???????? ?e?????, ?o??e ?o?????o??? ??? ?o??a???. ?o??? ? ?a?o?????? c???????? ?o?????: ?o??? ?o???o? ?a????e??? ????ap???? ?a???e???? ?????o???. ???e?e??e ?e?o?????? o??e??????? ?a???????? ??o?????? ? ?o?o? pe?????? ?????e????? ?o???c? ???a???. ?o???e???, ?eo?x?????? ??? ?????e??????? ?????e???? ??o??????. ???e????? ?p?? ? ????a??o???? ?????e???? ?p????? ? ????ap????? p??????????????? ???o? p??a????? ???o?e????? ?o????? ???a???. ???c??? ???a?o????? ?e?e??? ???o?e???? op?a???. ??p???? oc??????????? ????po?? ?p??????o??? ??p???????? ?a???e???? c?o??o???. ?p?????e ?a????????? o??a???? ?e?e??? ?o??????????? ?a?o?e???? c?o??????. ?a?o?????? ??a???? ? ??op?. ???e?e??? ? ?a??????? ???o?o?????????? ? c???? c ?p?????e? Ha?o?????? ?o?e?c? ??pa???: ?o??? ???o????? ???o?? ? ?op???? ?x ????e?????, ???? ???a?o? ?a ?????e??? ?a?o?o???? ?????o?????????? ? ?x ????e??, ?a????????????? ????p?? ? ???x ?pe?????? ? ????c?????? o? ???pa????? ?????e????? ?e????, ?op??o? ????e????? ??e?????????? ?p?????? ?o ?o?????????? ?a???o??? o???a?????????. ?o??? ?e?????? ?a?o?e????? ?o???c? ???a???: ?????e??? ? ?a??? ?a????????????? ?????e??o?????????? o?ep????, ???? ??pa???? ? ?????o? ?x ?p????e???, ??a?a ?a ??????e??e ?a???e???? ?op?a???????? ??e ??c? ?ac????????? ?a???e???? ???a??? ??? ?o ?????o???? ??e??, ?c???o??????? ??? ?????e???? o??a???, ?o???o? ?????o????????? ?o?????, ?o?op?? ???e???????? ??????c???? ???a??. ???o? ?a ?o????e???? c?o??o??? ? a??????? ?a?o? ?op??o? ???a??, ?o?????o?e??? ?a?o????? ????a???????, ?a?? ???o?o?????????, ???p????, ?o?o??? ?e ??????c? ??????o? ???o???????????, oco???????? ???a?? ????????o c?op? c ?ap?????????? ?o?ap??. E????? ??o? ? ??o?????????? ?a?o?. ?????o? ?o?????? ???o?????? ??????e? c ????o? ?op? Ha?o?????? ? ???o??????? ?o?e???? ???a???. ??p???????? ? ?po??c? ?o??p??, ?p?????????? ?p????? ???e? ?a?o?e???? ?pa???? ???a???: ?c????????? ?a????e??? ?p???a????????? ?a???e???? ?e????????. ??p???? e? ?p????e???; ??o???????? ?p??e?e??? ?a?o?e????? p????a ???????a; ?p???a????????? ??e????e??? ?a?o?????? ???a??? o ?a?ep???? ??e??? ?o?ap? ?a ???o?e???? ??p??????? ??p????. ??p??, ???o??? ? c?o???? ??e???????? ????a??????? ??pe? ?a?o?e????? ?p?a????: ???a??o??? ?ap?????; ?p???? c?????? ??e???????? ???a?? ?a???e???? ??a?????. ??? o?e???????? ?p???????? ?o???? ?a?o?e????? ?o???c? ??p???? ?a????????????? ? ???e???a???? ??????ep?????? ? ???o??????? ??p???? p?? ?op???????? ?o???e??o? ? ?o?o?????. Ha ???e?? ??o?e????? c??????? ???e? ??e??? ????p ??o?? ?p?????x ?????e??o?. ?E??O?? ?e????e c?e??a?????, p??????????? c???? ? ?????o? ?o????????????? ?????e???? c????? ??p???? ??O??OC?? 1250.00 ?p?. ? ?a o??o?o ??ac????a. ?p? o????? ?a ???x ??a?????o? ?????a ? 7% B c?o??oc?? ?xo???: ??????a??????-???????????????? ??c?????a??? ?a ce??????, c?op??? ???ep?????, ?o??-?????, ??e?, o?c??????? ????a??? ? ???e? ??e????? c ????op???. PE????E?? ?e????????? c 9:30 ?o 10:00 ?a?a?? ? 10:00 O?o?????? ? 18:00 PE??C??A??? ?e?.: (044) 331-64-1-?e???e, 592-75-6-?e??p? Ka???? ??ac???? ?o???ae? ??x?a???p???? ?o???e?? ?o???e??o?. { WORDS[words]>30<300} ?????? ???????? ???? ????????? ???????????? ?????? ??????????? ???????? ???????? ????????? ?????? ??????? ?????? ????????? From somers.ben at gmail.com Tue Jul 10 19:29:51 2012 From: somers.ben at gmail.com (Ben Somers) Date: Tue, 10 Jul 2012 12:29:51 -0700 Subject: [ANN] alicorn 0.3.1 Message-ID: Hi all, I wrote to this list waaaay back in January discussing auto-scaling for Unicorn, in situations where dedicated servers aren't practical. I've now written a separate autoscaler, under the name of alicorn (https://github.com/bensomers/alicorn). My current team's been running it in production for several weeks now with pretty good results, including correctly identifying and responding to a major load spike (and correctly spinning back down afterwards). I would love comments, suggestions, (constructive) criticism, etc. Hopefully there are some other folks out there who will find it useful. Slightly related blog post at http://somanyrobots.com/2012/07/11/switching-to-unicorn, but most info is just in the github repository itself. -ben From normalperson at yhbt.net Tue Jul 10 20:23:31 2012 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 10 Jul 2012 20:23:31 +0000 Subject: [ANN] alicorn 0.3.1 In-Reply-To: References: Message-ID: <20120710202331.GA8632@dcvr.yhbt.net> Ben Somers wrote: > Hi all, > > I wrote to this list waaaay back in January discussing auto-scaling > for Unicorn, in situations where dedicated servers aren't practical. > I've now written a separate autoscaler, under the name of alicorn > (https://github.com/bensomers/alicorn). My current team's been running > it in production for several weeks now with pretty good results, > including correctly identifying and responding to a major load spike > (and correctly spinning back down afterwards). I would love comments, > suggestions, (constructive) criticism, etc. Hopefully there are some > other folks out there who will find it useful. Cool. Feel free to use this mailing list for discussion if you want. Some folks (a minority, I know) just aren't comfortable with /using/ websites. (Reading is fine, logins/cookies/JS is not) I also don't like the wording "Eric Wong's Unicorn" since many people have contributed to this project (and Mongrel). I've never had much of an identity online and I'm trying to minimize that as much as possible[1] > Slightly related blog post at > http://somanyrobots.com/2012/07/11/switching-to-unicorn, but most info That's 404-ing on me [1] The only reason I ever use my name in public is for GPL enforcement (and as a deterrent to would-be GPL violators). Thankfully it hasn't come up in a few years (and never for this project). From somers.ben at gmail.com Tue Jul 10 20:28:31 2012 From: somers.ben at gmail.com (Ben Somers) Date: Tue, 10 Jul 2012 13:28:31 -0700 Subject: [ANN] alicorn 0.3.1 In-Reply-To: <20120710202331.GA8632@dcvr.yhbt.net> References: <20120710202331.GA8632@dcvr.yhbt.net> Message-ID: > I also don't like the wording "Eric Wong's Unicorn" since many people > have contributed to this project (and Mongrel). I've never had much of > an identity online and I'm trying to minimize that as much as > possible[1] Already changed, in that case. > > Slightly related blog post at > > http://somanyrobots.com/2012/07/11/switching-to-unicorn, but most info > > That's 404-ing on me Gah, foolish me. Corrected link: http://somanyrobots.com/2012/07/09/switching-to-unicorn From hongli at phusion.nl Tue Jul 10 21:20:18 2012 From: hongli at phusion.nl (Hongli Lai) Date: Tue, 10 Jul 2012 23:20:18 +0200 Subject: [ANN] alicorn 0.3.1 In-Reply-To: References: <20120710202331.GA8632@dcvr.yhbt.net> Message-ID: On Tue, Jul 10, 2012 at 10:28 PM, Ben Somers wrote: > Gah, foolish me. Corrected link: > http://somanyrobots.com/2012/07/09/switching-to-unicorn Hey Ben. I'd like to point out that Phusion Passenger 3.2's prerelease code is actually already live. You can find it in the 'experimental' branch in our Github repository. -- Phusion | Ruby & Rails deployment, scaling and tuning solutions Web: http://www.phusion.nl/ E-mail: info at phusion.nl Chamber of commerce no: 08173483 (The Netherlands) From Mark.Mccraw at sas.com Tue Jul 17 00:33:58 2012 From: Mark.Mccraw at sas.com (Mark Mccraw) Date: Tue, 17 Jul 2012 00:33:58 +0000 Subject: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD Message-ID: <8902CCD4-F71D-4EC4-AC20-01F2C0B66599@sas.com> Hi There! I'm having a devil of a time figuring out a weird issue I'm running into. I have unicorn configured to start 4 worker processes, and that works great. However, when it's time to cycle the app, everything goes haywire. By trial and error, I have narrowed it down to this: sending any signal to the master process other than SIGKILL fails miserably. No new master process is created, as described in the documentation, nothing happens to the existing workers, nothing gets written to any log, and if I run top -u, I can see that very quickly the master ramps up to 100% CPU utilization. This happens if I run 'kill -HUP ', 'kill -USR2 ', even 'kill -QUIT '. Here's what I'm running on: uname -a FreeBSD bb20web04.unx.sas.com 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan 3 07:46:30 UTC 2012 root at farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 ruby -v ruby 1.9.3p0 (2011-10-30 revision 33570) [amd64-freebsd9] gem list | grep unicorn unicorn (4.3.1) My unicorn.rb file is pasted at the bottom. It should be noted that I have tried every permutation of this I can think of to narrow out the problematic part (set preload_app to false, comment out preload_app), comment out before_exec, before_fork, after_fork, comment out the START_CTX[0] bit, etc), but things always fail the same way, so I'm guessing it's not the config, but I'm open to anything. Any suggestions at all are greatly appreciated. I'd love to know how to interrupt the master process when it's slamming the CPU and get a stack trace, but I have no idea how in ruby. Any thoughts? Thanks! Mark APP_ROOT="/usr/local/rails/partsdb/current" working_directory APP_ROOT pid "#{APP_ROOT}/tmp/pids/unicorn.pid" stderr_path "#{APP_ROOT}/log/unicorn.log" stdout_path "#{APP_ROOT}/log/unicorn.log" Unicorn::HttpServer::START_CTX[0] = "#{APP_ROOT}/bin/unicorn" rails_env = ENV['RAILS_ENV'] || 'production' worker_processes 4 timeout 120 # Speed up worker spawn times preload_app true listen "/tmp/unicorn.sock", :backlog => 10 listen "bb20web04:8080", :backlog => 1024 before_exec do |server| ENV["BUNDLE_GEMFILE"] = "#{APP_ROOT}/Gemfile" end before_fork do |server, worker| ## # When sent a USR2, Unicorn will suffix its pidfile with .oldbin and # immediately start loading up a new version of itself (loaded with a new # version of our app). When this new Unicorn is completely loaded # it will begin spawning workers. The first worker spawned will check to # see if an .oldbin pidfile exists. If so, this means we've just booted up # a new Unicorn and need to tell the old one that it can now die. To do so # we send it a QUIT. # # Using this method we get 0 downtime deploys. if defined?(ActiveRecord::Base) ActiveRecord::Base.connection.disconnect! end old_pid = APP_ROOT + '/tmp/pids/unicorn.pid.oldbin' if File.exists?(old_pid) && server.pid != old_pid begin Process.kill("QUIT", File.read(old_pid).to_i) rescue Errno::ENOENT, Errno::ESRCH # someone else did our job for us end end end after_fork do |server, worker| ## # Unicorn master loads the app then forks off workers - because of the way # Unix forking works, we need to make sure we aren't using any of the parent's # sockets, e.g. db connection if defined?(ActiveRecord::Base) ActiveRecord::Base.establish_connection end # Redis and Memcached would go here but their connections are established # on demand, so the master never opens a socket end From normalperson at yhbt.net Tue Jul 17 02:05:41 2012 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 17 Jul 2012 02:05:41 +0000 Subject: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD In-Reply-To: <8902CCD4-F71D-4EC4-AC20-01F2C0B66599@sas.com> References: <8902CCD4-F71D-4EC4-AC20-01F2C0B66599@sas.com> Message-ID: <20120717020541.GA3431@dcvr.yhbt.net> Mark Mccraw wrote: > Hi There! > > I'm having a devil of a time figuring out a weird issue I'm running > into. I have unicorn configured to start 4 worker processes, and that > works great. However, when it's time to cycle the app, everything > goes haywire. By trial and error, I have narrowed it down to this: > sending any signal to the master process other than SIGKILL fails > miserably. No new master process is created, as described in the > documentation, nothing happens to the existing workers, nothing gets > written to any log, and if I run top -u, I can see that very quickly > the master ramps up to 100% CPU utilization. This happens if I run > 'kill -HUP ', 'kill -USR2 ', even 'kill -QUIT > '. This sounds like a Ruby/FreeBSD bug we've seen before. My script in http://mid.gmane.org/20120201181445.GA31624 at dcvr.yhbt.net should reproduce the issue w/o unicorn. > ruby 1.9.3p0 (2011-10-30 revision 33570) [amd64-freebsd9] I think this is a Ruby bug that was fixed in 1.9.3-p30 according to naruse: http://mid.gmane.org/CAK6HhsppWVPijWLyZMwcKueYDT5sZroGv6ADXkgreht3aLfR9A at mail.gmail.com Since 1.9.3 p194 is the latest, can you try that out and confirm the fix? I don't remember the other bug reported confirmed this issue was fixed by upgrading Ruby. Thanks. From Mark.Mccraw at sas.com Tue Jul 17 11:56:36 2012 From: Mark.Mccraw at sas.com (Mark Mccraw) Date: Tue, 17 Jul 2012 11:56:36 +0000 Subject: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD In-Reply-To: <20120717020541.GA3431@dcvr.yhbt.net> References: <8902CCD4-F71D-4EC4-AC20-01F2C0B66599@sas.com> <20120717020541.GA3431@dcvr.yhbt.net> Message-ID: <1A81B701-FB8C-47EA-9FA1-BC6F79438366@sas.com> On Jul 16, 2012, at 10:05 PM, Eric Wong wrote: > Mark Mccraw wrote: >> Hi There! >> >> I'm having a devil of a time figuring out a weird issue I'm running >> into. I have unicorn configured to start 4 worker processes, and that >> works great. However, when it's time to cycle the app, everything >> goes haywire. By trial and error, I have narrowed it down to this: >> sending any signal to the master process other than SIGKILL fails >> miserably. No new master process is created, as described in the >> documentation, nothing happens to the existing workers, nothing gets >> written to any log, and if I run top -u, I can see that very quickly >> the master ramps up to 100% CPU utilization. This happens if I run >> 'kill -HUP ', 'kill -USR2 ', even 'kill -QUIT >> '. > > This sounds like a Ruby/FreeBSD bug we've seen before. My script > in http://mid.gmane.org/20120201181445.GA31624 at dcvr.yhbt.net should > reproduce the issue w/o unicorn. You are absolutely correct! Your script replicates the problem perfectly. >> ruby 1.9.3p0 (2011-10-30 revision 33570) [amd64-freebsd9] > > I think this is a Ruby bug that was fixed in 1.9.3-p30 according to > naruse: > http://mid.gmane.org/CAK6HhsppWVPijWLyZMwcKueYDT5sZroGv6ADXkgreht3aLfR9A at mail.gmail.com > > Since 1.9.3 p194 is the latest, can you try that out and confirm the > fix? I don't remember the other bug reported confirmed this issue was > fixed by upgrading Ruby. We're upgrading now to see what happens. I'm so glad you knew about this. There's no telling how long it would have taken me to question the ruby interpreter implementation, and since it's FreeBSD, I never would have found it by googling. Thanks for hours (days?) of my life back. Mark From Mark.Mccraw at sas.com Tue Jul 17 21:23:22 2012 From: Mark.Mccraw at sas.com (Mark Mccraw) Date: Tue, 17 Jul 2012 21:23:22 +0000 Subject: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD In-Reply-To: <1A81B701-FB8C-47EA-9FA1-BC6F79438366@sas.com> References: <8902CCD4-F71D-4EC4-AC20-01F2C0B66599@sas.com> <20120717020541.GA3431@dcvr.yhbt.net> <1A81B701-FB8C-47EA-9FA1-BC6F79438366@sas.com> Message-ID: On Jul 17, 2012, at 7:56 AM, Mark McCraw wrote: > > On Jul 16, 2012, at 10:05 PM, Eric Wong wrote: > >> Mark Mccraw wrote: >>> Hi There! >>> >>> I'm having a devil of a time figuring out a weird issue I'm running >>> into. I have unicorn configured to start 4 worker processes, and that >>> works great. However, when it's time to cycle the app, everything >>> goes haywire. By trial and error, I have narrowed it down to this: >>> sending any signal to the master process other than SIGKILL fails >>> miserably. No new master process is created, as described in the >>> documentation, nothing happens to the existing workers, nothing gets >>> written to any log, and if I run top -u, I can see that very quickly >>> the master ramps up to 100% CPU utilization. This happens if I run >>> 'kill -HUP ', 'kill -USR2 ', even 'kill -QUIT >>> '. >> >> This sounds like a Ruby/FreeBSD bug we've seen before. My script >> in http://mid.gmane.org/20120201181445.GA31624 at dcvr.yhbt.net should >> reproduce the issue w/o unicorn. > > You are absolutely correct! Your script replicates the problem perfectly. > >>> ruby 1.9.3p0 (2011-10-30 revision 33570) [amd64-freebsd9] >> >> I think this is a Ruby bug that was fixed in 1.9.3-p30 according to >> naruse: >> http://mid.gmane.org/CAK6HhsppWVPijWLyZMwcKueYDT5sZroGv6ADXkgreht3aLfR9A at mail.gmail.com >> >> Since 1.9.3 p194 is the latest, can you try that out and confirm the >> fix? I don't remember the other bug reported confirmed this issue was >> fixed by upgrading Ruby. > > We're upgrading now to see what happens. I'm so glad you knew about this. > There's no telling how long it would have taken me to question the ruby interpreter implementation, and > since it's FreeBSD, I never would have found it by googling. > Thanks for hours (days?) of my life back. > > Mark > Just to follow up and close out the thread - Eric's recollection was spot on. We upgraded ruby on our FreeBSD server to the latest thing, and the problem completely disappeared. Thanks again! From normalperson at yhbt.net Tue Jul 17 22:17:41 2012 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 17 Jul 2012 15:17:41 -0700 Subject: Any signal other than -9 causes full CPU utilization by master unicorn process on FreeBSD In-Reply-To: References: <8902CCD4-F71D-4EC4-AC20-01F2C0B66599@sas.com> <20120717020541.GA3431@dcvr.yhbt.net> <1A81B701-FB8C-47EA-9FA1-BC6F79438366@sas.com> Message-ID: <20120717221741.GA8208@dcvr.yhbt.net> Mark Mccraw wrote: > On Jul 17, 2012, at 7:56 AM, Mark McCraw wrote: > > We're upgrading now to see what happens. I'm so glad you knew about > > this. There's no telling how long it would have taken me to > > question the ruby interpreter implementation, and since it's > > FreeBSD, I never would have found it by googling. Thanks for hours > > (days?) of my life back. > > Just to follow up and close out the thread - Eric's recollection was > spot on. We upgraded ruby on our FreeBSD server to the latest thing, > and the problem completely disappeared. Thanks again! Thanks for confirming this fix! Fwiw, the Ruby core team probably uses/tests on GNU/Linux more than any other platform. Bugs on less common development platforms (especially w.r.t tricky thread/fork/signal handling issues) may go unnoticed elsewhere. If you're focused on using Ruby + *BSD in a production system, I suggest testing/fixing/reporting issues against the Ruby development branches as much as possible before they hit production :) From matt at modal.org Thu Jul 19 22:43:58 2012 From: matt at modal.org (Matt Sanders) Date: Thu, 19 Jul 2012 17:43:58 -0500 Subject: Detecting unicorn / defining after_fork after master startup Message-ID: Hey everyone, Working on an engine for rails that needs specialized behavior with forking and I had a couple questions. 1. What is the best way to determine whether the app is indeed running inside a unicorn server? Most of the attempts I can find to detect check to see if the main modules for Unicorn are defined, but this really only checks to see that Unicorn is present, not that you are using it. I know a lot of shops may use Unicorn in production but something else for local development. Is there something equivalent to the IN_PHUSION_PASSENGER constant that passenger defines? 2. Is there an established way for adding an after_fork hook after Unicorn has already started up? I'm aware of the ability to do this via the config file but I don't want my users to have to add something to their unicorn config file. Is there an equivalent to passenger's PhusionPassenger.on_event(:starting_worker_process) method? Thanks! -Matt From somers.ben at gmail.com Thu Jul 19 22:55:54 2012 From: somers.ben at gmail.com (Ben Somers) Date: Thu, 19 Jul 2012 15:55:54 -0700 Subject: Detecting unicorn / defining after_fork after master startup In-Reply-To: References: Message-ID: > 1. What is the best way to determine whether the app is indeed running > inside a unicorn server? I actually wound up just setting an environment variable for this. I manage unicorn through an init.d, and just export UNICORN=true before it starts up the server. We have the exact scenario you describe, Unicorn in production and Passenger on our dev machines. > 2. Is there an established way for adding an after_fork hook after > Unicorn has already started up? I just recently had to solve this problem for my team. In my case, I've got multiple applications getting their unicorn configs managed through chef, and so didn't want to fill the chef with app-specific switches. Since the unicorn config file is just ruby code, I just have my config files check the app directory for a unicorn directory containing before_fork.rb and after_fork.rb files and load the files if they exist. It doesn't register the hooks after Unicorn starts, but it does let you define them outside of the config file. From matt at modal.org Thu Jul 19 22:59:30 2012 From: matt at modal.org (Matt Sanders) Date: Thu, 19 Jul 2012 17:59:30 -0500 Subject: Detecting unicorn / defining after_fork after master startup In-Reply-To: References: Message-ID: Ben, Thanks for your thoughtful comments. The engine I'm writing will be used by users outside of our organization so unfortunately I won't have any control over their environment or config files - I'm looking to get as close to zero-config as I can through app server detection. Thanks, -Matt From normalperson at yhbt.net Thu Jul 19 23:13:34 2012 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 19 Jul 2012 16:13:34 -0700 Subject: Detecting unicorn / defining after_fork after master startup In-Reply-To: References: Message-ID: <20120719231334.GA19342@dcvr.yhbt.net> Matt Sanders wrote: > 1. What is the best way to determine whether the app is indeed running > inside a unicorn server? Unicorn.listener_names will return a non-empty array of listener addresses. > 2. Is there an established way for adding an after_fork hook after > Unicorn has already started up? Nothing that I know of. > I'm aware of the ability to do this via the config file but I don't > want my users to have to add something to their unicorn config file. > Is there an equivalent to passenger's > PhusionPassenger.on_event(:starting_worker_process) method? Lately, I've been favoring the following pattern instead: def initialize @init_pid = $$ ... end def initialize_child_fork @init_pid = $$ ... end def call(env) initialize_child_fork if @init_pid != $$ ... end The overhead is negligible and works regardless of server (even in non-HTTP servers somebody may write). I really don't like code that would need to special case for all sorts of servers: foo_for_passenger if defined?(PhusionPassenger) foo_for_unicorn if defined?(Unicorn) foo_for_something_fastcgi if defined?(SomethingFastCGI) foo_for_something_scgi if defined?(SomethingSCGI) foo_for_something_else_that_forks if defined?(YetAnotherForkingRackServer) foo_for_something_else if defined?(SomethingElse) ... From matt at modal.org Sat Jul 21 19:42:57 2012 From: matt at modal.org (Matt Sanders) Date: Sat, 21 Jul 2012 14:42:57 -0500 Subject: Detecting unicorn / defining after_fork after master startup In-Reply-To: <20120719231334.GA19342@dcvr.yhbt.net> References: <20120719231334.GA19342@dcvr.yhbt.net> Message-ID: Eric, Thanks! I definitely agree in general with that pattern. In this case I'd ideally like some activity on the forked child before the next request cycle - that is, if the forked child doesn't serve any requests for a little bit, I'd still like my behavior to be initialized. Do you know of a way to get this behavior without a direct after fork hook? Thanks for your help! -Matt On Thu, Jul 19, 2012 at 6:13 PM, Eric Wong wrote: > Matt Sanders wrote: >> 1. What is the best way to determine whether the app is indeed running >> inside a unicorn server? > > Unicorn.listener_names will return a non-empty array of listener > addresses. > >> 2. Is there an established way for adding an after_fork hook after >> Unicorn has already started up? > > Nothing that I know of. > >> I'm aware of the ability to do this via the config file but I don't >> want my users to have to add something to their unicorn config file. >> Is there an equivalent to passenger's >> PhusionPassenger.on_event(:starting_worker_process) method? > > Lately, I've been favoring the following pattern instead: > > def initialize > @init_pid = $$ > ... > end > > def initialize_child_fork > @init_pid = $$ > ... > end > > def call(env) > initialize_child_fork if @init_pid != $$ > ... > end > > The overhead is negligible and works regardless of server (even in > non-HTTP servers somebody may write). I really don't like code > that would need to special case for all sorts of servers: > > foo_for_passenger if defined?(PhusionPassenger) > foo_for_unicorn if defined?(Unicorn) > foo_for_something_fastcgi if defined?(SomethingFastCGI) > foo_for_something_scgi if defined?(SomethingSCGI) > foo_for_something_else_that_forks if defined?(YetAnotherForkingRackServer) > foo_for_something_else if defined?(SomethingElse) > ... > _______________________________________________ > Unicorn mailing list - mongrel-unicorn at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-unicorn > Do not quote signatures (like this one) or top post when replying From normalperson at yhbt.net Sun Jul 22 21:20:26 2012 From: normalperson at yhbt.net (Eric Wong) Date: Sun, 22 Jul 2012 14:20:26 -0700 Subject: Detecting unicorn / defining after_fork after master startup In-Reply-To: References: <20120719231334.GA19342@dcvr.yhbt.net> Message-ID: <20120722212026.GA17948@dcvr.yhbt.net> Matt Sanders wrote: > On Thu, Jul 19, 2012 at 6:13 PM, Eric Wong wrote: > > Matt Sanders wrote: > >> 2. Is there an established way for adding an after_fork hook after > >> Unicorn has already started up? > > > > Nothing that I know of. > > > >> I'm aware of the ability to do this via the config file but I don't > >> want my users to have to add something to their unicorn config file. > >> Is there an equivalent to passenger's > >> PhusionPassenger.on_event(:starting_worker_process) method? > > > > Lately, I've been favoring the following pattern instead: (top-posting corrected) > Thanks! I definitely agree in general with that pattern. In this case > I'd ideally like some activity on the forked child before the next > request cycle - that is, if the forked child doesn't serve any > requests for a little bit, I'd still like my behavior to be > initialized. Yes, that sucks. I'll often just throw some requests at the child just to start something up if I need it, or initiate a separate daemon. > Do you know of a way to get this behavior without a direct after fork hook? No, unfortunately not. I have considered petitioning for something along the lines of pthread_atfork() in Ruby, but pthread_atfork() itself also has unfortunate drawbacks (it makes fork() async-signal un-safe, and thus _Fork() will be introduced in the next version of POSIX). From normalperson at yhbt.net Sat Jul 28 02:22:36 2012 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 28 Jul 2012 02:22:36 +0000 Subject: [RFC/PATCH] remove Rails-oriented integration tests Message-ID: <20120728022236.GA6129@dcvr.yhbt.net> (This is a large patch which deletes a lot of files, so it's pointless to post in its entirety here. Full diffstat and --diff-filter=M output below. I've pushed this to the "pu" branch of git://bogomips.org/unicorn.git commit 91a3cde091d4ae6ff436681f155b3907daae1c04) It's too much overhead to keep Rails-specific tests working, especially when it's hauling in an ancient version of SQLite3. Since Rails 3 has settled down with Rack and unicorn_rails is unlikely to need changing in the future, we can drop these tests. --- GNUmakefile | 31 +-- script/isolate_for_tests | 18 -- Summary of full diffstat: 94 files changed, 2 insertions(+), 1598 deletions(-) diff --git a/GNUmakefile b/GNUmakefile index 61fb739..ea13486 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -32,16 +32,13 @@ MYLIBS = $(RUBYLIB):$(ISOLATE_LIBS) # dunno how to implement this as concisely in Ruby, and hell, I love awk awk_slow := awk '/def test_/{print FILENAME"--"$$2".n"}' 2>/dev/null -rails_vers := $(subst test/rails/app-,,$(wildcard test/rails/app-*)) slow_tests := test/unit/test_server.rb test/exec/test_exec.rb \ test/unit/test_signals.rb test/unit/test_upload.rb log_suffix = .$(RUBY_ENGINE).$(RUBY_VERSION).log -T_r := $(wildcard test/rails/test*.rb) -T := $(filter-out $(slow_tests) $(T_r), $(wildcard test/*/test*.rb)) +T := $(filter-out $(slow_tests), $(wildcard test/*/test*.rb)) T_n := $(shell $(awk_slow) $(slow_tests)) T_log := $(subst .rb,$(log_suffix),$(T)) T_n_log := $(subst .n,$(log_suffix),$(T_n)) -T_r_log := $(subst .r,$(log_suffix),$(T_r)) test_prefix = $(CURDIR)/test/$(RUBY_ENGINE)-$(RUBY_VERSION) ext := ext/unicorn_http @@ -93,7 +90,7 @@ $(slow_tests): $(test_prefix)/.stamp test-integration: $(test_prefix)/.stamp $(MAKE) -C t -test-all: test test-rails test-integration +test-all: test test-integration TEST_OPTS = -v check_test = grep '0 failures, 0 errors' $(t) >/dev/null @@ -199,30 +196,6 @@ doc_gz: for i in $(docs); do \ gzip --rsyncable -9 < $$i > $$i.gz; touch -r $$i $$i.gz; done -rails_git_url = git://github.com/rails/rails.git -rails_git := vendor/rails.git -$(rails_git)/info/cloned-stamp: - git clone --mirror -q $(rails_git_url) $(rails_git) - > $@ - -$(rails_git)/info/v2.2.3-stamp: $(rails_git)/info/cloned-stamp - cd $(rails_git) && git fetch - cd $(rails_git) && git rev-parse --verify refs/tags/v2.2.3 - > $@ - -rails_tests := $(addsuffix .r,$(addprefix $(T_r).,$(rails_vers))) -test-rails: $(rails_tests) -$(T_r).%.r: t = $(addsuffix $(log_suffix),$@) -$(T_r).%.r: rv = $(subst .r,,$(subst $(T_r).,,$@)) -$(T_r).%.r: extra = ' 'v$(rv) -$(T_r).%.r: arg = $(T_r) -$(T_r).%.r: export PATH := $(test_prefix)/bin:$(PATH) -$(T_r).%.r: export RUBYLIB := $(test_prefix):$(test_prefix)/lib:$(MYLIBS) -$(T_r).%.r: export UNICORN_RAILS_TEST_VERSION = $(rv) -$(T_r).%.r: export RAILS_GIT_REPO = $(CURDIR)/$(rails_git) -$(T_r).%.r: $(test_prefix)/.stamp $(rails_git)/info/v2.2.3-stamp - $(run_test) - ifneq ($(VERSION),) rfproject := mongrel rfpackage := unicorn diff --git a/script/isolate_for_tests b/script/isolate_for_tests index 5e6139c..00a3f2b 100755 --- a/script/isolate_for_tests +++ b/script/isolate_for_tests @@ -16,9 +16,6 @@ opts = { pid = fork do Isolate.now!(opts) do - if RUBY_VERSION.to_f < 2.0 - gem 'sqlite3-ruby', '1.2.5' - end gem 'raindrops', '0.8.0' gem 'kgio-monkey', '0.4.0' gem 'kgio', '2.7.4' @@ -33,18 +30,3 @@ File.open("#{dst}.#$$", "w") do |fp| fp.puts "ISOLATE_LIBS=#{lib_paths.join(':')}" end File.rename("#{dst}.#$$", dst) - -# pure Ruby gems can be shared across all Rubies -%w(3.0.0).each do |rails_ver| - opts[:path] = "tmp/isolate/rails-#{rails_ver}" - pid = fork do - Isolate.now!(opts) do - gem 'rake', '0.8.7' - gem 'rails', rails_ver - end - end - _, status = Process.waitpid2(pid) - status.success? or abort status.inspect - more = Dir["#{opts[:path]}/gems/*-*/lib"].map { |x| File.expand_path(x) } - lib_paths.concat(more) -end