From normalperson at yhbt.net Tue May 4 21:09:44 2010 From: normalperson at yhbt.net (Eric Wong) Date: Tue, 4 May 2010 18:09:44 -0700 Subject: [ANN] unicorn 0.98.0 - Rack HTTP server for fast clients and Unix Message-ID: <20100505010944.GA5329@dcvr.yhbt.net> Unicorn is an HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: Deployments that suspend or hibernate servers should no longer have workers killed off (and restarted) upon resuming. For Linux users of {raindrops}[http://raindrops.bogomips.org/] (v0.2.0+) configuration is easier as raindrops can now automatically detect the active listeners on the server via the new Unicorn.listener_names singleton method. For the pedantic, chunked request bodies without trailers are no longer allowed to omit the final CRLF. This shouldn't affect any real and RFC-compliant clients out there. Chunked requests with trailers have always worked and continue to work the same way. The rest are mostly small internal cleanups and documentation fixes. See the commit logs for full details. -- Eric Wong From kathyjones at mailhost.oualid.fr Wed May 5 16:17:40 2010 From: kathyjones at mailhost.oualid.fr (Levi phosphorylate) Date: Wed, 5 May 2010 22:17:40 +0200 Subject: new business mailing list Message-ID: <201005052017.o45KHeY7031291@NPA4120001.local.dom> Here's a list package on sale this week only: Consumer List Package Credit Inquiries Database - 1 million Full Data Records all with emails American Consumer Database - 1,300,000 records all with emails American Homeowners - 1 million Full Data Records all with emails All complete lists above: $295 There are more packages and not just for consumers. we also have lists for healthcare, business and more. Contact me here for more info or to get samples: Shauna.Nguyen at perfectlist.co.cc to expel yourself from the list go here please email disappear at perfectlist.co.cc From normalperson at yhbt.net Thu May 6 15:37:40 2010 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 6 May 2010 19:37:40 +0000 Subject: [ANN] unicorn 0.99.0 - simplicity wins Message-ID: <20100506193740.GB10280@dcvr.yhbt.net> Unicorn is an HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the the request and response in between Unicorn and slow clients. * http://unicorn.bogomips.org/ * mongrel-unicorn at rubyforge.org * git://git.bogomips.org/unicorn.git Changes: Starting with this release, we'll always load Rack up front at startup. Previously we had complicated ways to avoid loading Rack until after the application was loaded to allow the application to load an alternate version of Rack. However this has proven too error-prone to be worth supporting even though Unicorn does not have strict requirements on currently released Rack versions. If an app requires a different version of Rack than what Unicorn would load by default, it is recommended they only install that version of Rack (and no others) since Unicorn does not have any strict requirements on currently released Rack versions. Rails 2.3.x users should be aware of this as those versions are not compatible with Rack 1.1.0. If it is not possible to only have one Rack version installed "globally", then they should either use Isolate or Bundler and install a private version of Unicorn along with their preferred version of Rack. Users who install in this way are recommended to execute the isolated/bundled version of Unicorn, instead of what would normally be in $PATH. Feedback/tips to mailto:mongrel-unicorn at rubyforge.org from Isolate and Bundler users would be greatly appreciated. Thanks to Aaron Eisenberger for making us aware of this issue. -- Eric Wong From luke at lukemelia.com Thu May 6 16:29:40 2010 From: luke at lukemelia.com (Luke Melia) Date: Thu, 6 May 2010 16:29:40 -0400 Subject: Garbage collection outside of request cycle? Message-ID: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> I've been analyzing our Unicorn-powered Rails app's performance, and have found that garbage collection is a big factor in slow requests. In the interest of avoiding those performance hits while handling requests, would it be possible to have a unicorn worker run garbage collection after handling a request and before waiting for the next one? Would this be a good idea? Cheers, Luke From jamie at internetfamo.us Thu May 6 16:49:12 2010 From: jamie at internetfamo.us (Jamie Wilkinson) Date: Thu, 6 May 2010 13:49:12 -0700 Subject: [ANN] unicorn 0.99.0 - simplicity wins In-Reply-To: <20100506193740.GB10280@dcvr.yhbt.net> References: <20100506193740.GB10280@dcvr.yhbt.net> Message-ID: <9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us> Re: bundling/isolating: > If it is not possible to only have one Rack version installed > "globally", then they should either use Isolate or Bundler and > install a private version of Unicorn along with their preferred > version of Rack. Users who install in this way are recommended > to execute the isolated/bundled version of Unicorn, instead of > what would normally be in $PATH. A caveat about using a bundled unicorn binary (via `bundle exec unicorn`): if you use capistrano-based deployment, make sure to use a shared vendor/bundler_gems! Obviously this is much faster than re-bundling each release, but it also prevents the unicorn binary from going missing -- if unicorn is originally executed as /app/releases/XXY/vendor/bundler_gems/bin/unicorn and that release gets rolled back or pruned by :keep_releases, the original master's binary is now gone & it will no longer be able to fork new unicorn masters (is this expected behavior/a known issue?) -jamie From normalperson at yhbt.net Thu May 6 16:57:54 2010 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 6 May 2010 13:57:54 -0700 Subject: Garbage collection outside of request cycle? In-Reply-To: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> References: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> Message-ID: <20100506205754.GA13412@dcvr.yhbt.net> Luke Melia wrote: > I've been analyzing our Unicorn-powered Rails app's performance, and > have found that garbage collection is a big factor in slow requests. > > In the interest of avoiding those performance hits while handling > requests, would it be possible to have a unicorn worker run garbage > collection after handling a request and before waiting for the next > one? Would this be a good idea? Hi Luke, I made this for one heavyweight app a while back. I guess I should throw this into the examples section, but it won't be the default since it hurts simpler applications that don't generate much garbage. ==> big_app_gc.rb <== # This shouldn't hurt overall performance as long as the server cluster # is at <=50% CPU capacity, and improves the performance of most memory # intensive requests. This serves to improve _client-visible_ # performance (possibly at the cost of overall performance). # # We'll call GC after each request is been written out to the socket, so # the client never sees the extra GC hit it. It's ideal to call the GC # inside the HTTP server (vs middleware or hooks) since the stack is # smaller at this point, so the GC will both be faster and more # effective at releasing unused memory. # # This monkey patch is _only_ effective for applications that use a lot # of memory, and will hurt simpler apps/endpoints that can process # multiple requests before incurring GC. class Unicorn::HttpServer REQ = Unicorn::HttpRequest::REQ alias _process_client process_client undef_method :process_client def process_client(client) _process_client(client) REQ.clear GC.start end end if defined?(Unicorn) -- Eric Wong From normalperson at yhbt.net Thu May 6 17:01:51 2010 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 6 May 2010 14:01:51 -0700 Subject: [ANN] unicorn 0.99.0 - simplicity wins In-Reply-To: <9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us> References: <20100506193740.GB10280@dcvr.yhbt.net> <9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us> Message-ID: <20100506210151.GB13412@dcvr.yhbt.net> Jamie Wilkinson wrote: > Re: bundling/isolating: > > > If it is not possible to only have one Rack version installed > > "globally", then they should either use Isolate or Bundler and > > install a private version of Unicorn along with their preferred > > version of Rack. Users who install in this way are recommended to > > execute the isolated/bundled version of Unicorn, instead of what > > would normally be in $PATH. > > A caveat about using a bundled unicorn binary (via `bundle exec > unicorn`): if you use capistrano-based deployment, make sure to use a > shared vendor/bundler_gems! Thanks for the tip! > Obviously this is much faster than re-bundling each release, but it > also prevents the unicorn binary from going missing -- if unicorn is > originally executed as > /app/releases/XXY/vendor/bundler_gems/bin/unicorn and that release > gets rolled back or pruned by :keep_releases, the original master's > binary is now gone & it will no longer be able to fork new unicorn > masters (is this expected behavior/a known issue?) Yeah, Unicorn will stash the original command-line it was executed with. If you need to override this at runtime, I would do something like this in the Unicorn config file: Unicorn::HttpServer::START_CTX[0] = "/some/path/to/bin/unicorn" -- Eric Wong From normalperson at yhbt.net Thu May 6 17:12:29 2010 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 6 May 2010 14:12:29 -0700 Subject: Garbage collection outside of request cycle? In-Reply-To: <20100506205754.GA13412@dcvr.yhbt.net> References: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> <20100506205754.GA13412@dcvr.yhbt.net> Message-ID: <20100506211229.GA14300@dcvr.yhbt.net> Eric Wong wrote: > I guess I should throw this into the examples section Added a few more comments, but the code is still the same: http://unicorn.bogomips.org/examples/big_app_gc.rb http://git.bogomips.org/cgit/unicorn.git/commit/?id=510a48dafc5f7e2cb618d785885395c79570821c -- Eric Wong From normalperson at yhbt.net Thu May 6 17:19:03 2010 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 6 May 2010 14:19:03 -0700 Subject: [PATCH] Add worker interrogation via INFO signals In-Reply-To: References: Message-ID: <20100506211903.GA15038@dcvr.yhbt.net> Hey Joel, did you get my original reply? http://mid.gmane.org/20100427085922.GA8080 at dcvr.yhbt.net (putting you in the To: header in case your mail server doesn't like Cc:) -- Eric Wong From luke at lukemelia.com Thu May 6 19:17:19 2010 From: luke at lukemelia.com (Luke Melia) Date: Thu, 6 May 2010 19:17:19 -0400 Subject: Garbage collection outside of request cycle? In-Reply-To: <20100506205754.GA13412@dcvr.yhbt.net> References: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> <20100506205754.GA13412@dcvr.yhbt.net> Message-ID: <2B9DF63C-F43F-48B0-A380-2270EDFA53F4@lukemelia.com> On May 6, 2010, at 4:57 PM, Eric Wong wrote: > I made this for one heavyweight app a while back. > > ==> big_app_gc.rb <== > # This shouldn't hurt overall performance as long as the server cluster > # is at <=50% CPU capacity, and improves the performance of most memory > # intensive requests. This serves to improve _client-visible_ > # performance (possibly at the cost of overall performance). This is exactly the tradeoff I'm looking for. Our app has a large footprint. Thanks for the quick response. I'll report back when I have some data. Cheers, Luke From joel at watsonian.net Fri May 7 02:26:52 2010 From: joel at watsonian.net (Joel Watson) Date: Thu, 6 May 2010 23:26:52 -0700 Subject: [PATCH] Add worker interrogation via INFO signals In-Reply-To: <20100506211903.GA15038@dcvr.yhbt.net> References: <20100506211903.GA15038@dcvr.yhbt.net> Message-ID: <0CB4842E-773D-4ADB-857F-02A69FB6DE43@watsonian.net> Yep, I got your original reply. I've been really busy since my initial post and haven't had much time to take a close look at your proposed middleware. I agree with you on it being unnecessary overhead for every request considering it won't be used all that often, so being able to include it as a middleware when needed seems like a great solution. Once I get some free time I'll take a closer look at your reply and give a more in-depth response. Thanks for taking the time to look over my patch and write up your proposed change as well! -Joel On May 6, 2010, at 2:19 PM, Eric Wong wrote: > Hey Joel, did you get my original reply? > > http://mid.gmane.org/20100427085922.GA8080 at dcvr.yhbt.net > > (putting you in the To: header in case your mail server doesn't like Cc:) > > -- > Eric Wong From ibc at aliax.net Fri May 7 09:51:18 2010 From: ibc at aliax.net (=?UTF-8?Q?I=C3=B1aki_Baz_Castillo?=) Date: Fri, 7 May 2010 15:51:18 +0200 Subject: Some issues in the HTTP parser (and suggestions) Message-ID: Hi, by inspecting the Ragel grammar of the HTTP parser (coming from Mongrel) I've realized of some possible issues and bugs: hostname = (alnum | "-" | "." | "_")+; - It doesn't allow IPv6. This is important IMHO. - It allows "_" which is an invalid symbol (not valid for a domain). I suggest: hostname = (alnum | "-" | "." | "[" | "]" | ":")+; ------------------ host_with_port = (hostname (":" digit*)?) >mark %host; - It allows something ugly as "mydomain.org:" I suggest: host_with_port = (hostname (":" digit{1,5})?) >mark %host; ------------------ message_header = ((field_name ":" " "* field_value)|value_cont) :> CRLF; - It doesn't allow valid spaces before ":" as: Host : mydomain.org - Tabulators are also allowed. I suggest: message_header = ((field_name [ \t]* ":" [ \t]* field_value)|value_cont) :> CRLF; Regards. -- I?aki Baz Castillo From normalperson at yhbt.net Fri May 7 14:33:28 2010 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 7 May 2010 11:33:28 -0700 Subject: Some issues in the HTTP parser (and suggestions) In-Reply-To: References: Message-ID: <20100507183328.GA31883@dcvr.yhbt.net> I?aki Baz Castillo wrote: > Hi, by inspecting the Ragel grammar of the HTTP parser (coming from > Mongrel) I've realized of some possible issues and bugs: > > > hostname = (alnum | "-" | "." | "_")+; > > - It doesn't allow IPv6. This is important IMHO. > - It allows "_" which is an invalid symbol (not valid for a domain). > > I suggest: > hostname = (alnum | "-" | "." | "[" | "]" | ":")+; Hi I?aki, Underscore isn't valid for hostnames, but it is allowed in domain names and most DNS servers will resolve them. I've personally seen websites with underscores in their domain names in the wild[1]. We'll have to test the IPv6 addresses and probably split that out into a separate regexp since ":" would raise issues with the port number in existing cases. This is probably something for post-1.0. > ------------------ > > host_with_port = (hostname (":" digit*)?) >mark %host; > > - It allows something ugly as "mydomain.org:" > > I suggest: > host_with_port = (hostname (":" digit{1,5})?) >mark %host; It's ugly, but section 3.2.2 of RFC 2396 appears to allows it. > ------------------ > > message_header = ((field_name ":" " "* field_value)|value_cont) :> CRLF; > > - It doesn't allow valid spaces before ":" as: > Host : mydomain.org Spaces before the ":" aren't allowed in rfc2616, and I have yet to see evidence of clients sending headers like this in ~4 years of using this parser. > - Tabulators are also allowed. > > I suggest: > message_header = ((field_name [ \t]* ":" [ \t]* > field_value)|value_cont) :> CRLF; I just pushed this out to unicorn.git to allow horizontal tabs: >From 935912a422cabfd323f9b4ff268ded09a2ebe7a6 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Fri, 7 May 2010 18:20:49 +0000 Subject: [PATCH] http: allow horizontal tab as leading whitespace in header values MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This is allowed by RFC 2616, section 2.2, where spaces and horizontal tabs are counted as linear white space and linear white space (not just regular spaces) may prefix field-values (section 4.2). This has _not_ been a real issue in ~4 years of using this parser (starting with Mongrel) with clients in the wild. Thanks to I?aki Baz Castillo for pointing this out. --- ext/unicorn_http/unicorn_http_common.rl | 2 +- test/unit/test_http_parser.rb | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletions(-) diff --git a/ext/unicorn_http/unicorn_http_common.rl b/ext/unicorn_http/unicorn_http_common.rl index 6fca604..f165e3f 100644 --- a/ext/unicorn_http/unicorn_http_common.rl +++ b/ext/unicorn_http/unicorn_http_common.rl @@ -54,7 +54,7 @@ value_cont = lws+ any* >start_value %write_cont_value; - message_header = ((field_name ":" " "* field_value)|value_cont) :> CRLF; + message_header = ((field_name ":" lws* field_value)|value_cont) :> CRLF; chunk_ext_val = token*; chunk_ext_name = token*; chunk_extension = ( ";" " "* chunk_ext_name ("=" chunk_ext_val)? )*; diff --git a/test/unit/test_http_parser.rb b/test/unit/test_http_parser.rb index 5b0ca9f..b7c8a1c 100644 --- a/test/unit/test_http_parser.rb +++ b/test/unit/test_http_parser.rb @@ -51,6 +51,14 @@ class HttpParserTest < Test::Unit::TestCase assert parser.keepalive? end + def test_tab_lws + parser = HttpParser.new + req = {} + tmp = "GET / HTTP/1.1\r\nHost:\tfoo.bar\r\n\r\n" + assert_equal req.object_id, parser.headers(req, tmp).object_id + assert_equal "foo.bar", req['HTTP_HOST'] + end + def test_connection_close_no_ka parser = HttpParser.new req = {} -- [1] - and those were wild NSFW sites, of course :> -- Eric Wong From normalperson at yhbt.net Fri May 7 15:17:09 2010 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 7 May 2010 12:17:09 -0700 Subject: Sandbox (bundler+isolate) tips RDoc added Message-ID: <20100507191709.GA10784@dcvr.yhbt.net> http://unicorn.bogomips.org/Sandbox.html Please send patches/pull requests for any more tips you'd like to share, including for similar tools. Thanks to Jamie Wilkinson for the Bundler tips! -- Eric Wong From ibc at aliax.net Sat May 8 09:11:05 2010 From: ibc at aliax.net (=?UTF-8?Q?I=C3=B1aki_Baz_Castillo?=) Date: Sat, 8 May 2010 15:11:05 +0200 Subject: Some issues in the HTTP parser (and suggestions) In-Reply-To: <20100507183328.GA31883@dcvr.yhbt.net> References: <20100507183328.GA31883@dcvr.yhbt.net> Message-ID: 2010/5/7 Eric Wong : > Underscore isn't valid for hostnames, but it is allowed in domain names > and most DNS servers will resolve them. ?I've personally seen websites > with underscores in their domain names in the wild[1]. Hi Eric, could you point me to the spec stating that underscore is allowed for a domain? In the past I've done a SIP parser [*] with Ragel, being 100% strict at BNF grammar, and note that SIP reuses 80% of the grammar of HTTP. I'm pretty sure that "_" is not valid in a domain (host, hostname or whatever). Anyhow it's better just to allow it at parsing level :) > We'll have to test the IPv6 addresses and probably split that out into a > separate regexp since ":" would raise issues with the port number in > existing cases. ?This is probably something for post-1.0. There is a IETF draft to improve and *fix* the existing BNF grammar for IPv6. It also improves the grammar for IPv4 (by dissallowing values greater than 255): http://tools.ietf.org/html/draft-ietf-sip-ipv6-abnf-fix I've already implemented it in Ragel and I can sure that it's 100% valid and strict (I've done lots of tests): alphanum = ALPHA / DIGIT domainlabel = alphanum | ( alphanum ( alphanum | "-" )* alphanum ); toplabel = ALPHA | ( ALPHA ( alphanum | "-" )* alphanum ); hostname = ( domainlabel "." )* toplabel "."?; dec_octet = DIGIT | ( 0x31..0x39 DIGIT ) | ( "1" DIGIT{2} ) | ( "2" 0x30..0x34 DIGIT ) | ( "25" 0x30..0x35 ); IPv4address = dec_octet "." dec_octet "." dec_octet "." dec_octet; h16 = HEXDIG{1,4}; ls32 = ( h16 ":" h16 ) | IPv4address; IPv6address = ( ( h16 ":" ){6} ls32 ) | ( "::" ( h16 ":" ){5} ls32 ) | ( h16? "::" ( h16 ":" ){4} ls32 ) | ( ( ( h16 ":" )? h16 )? "::" ( h16 ":" ){3} ls32 ) | ( ( ( h16 ":" ){,2} h16 )? "::" ( h16 ":" ){2} ls32 ) | ( ( ( h16 ":" ){,3} h16 )? "::" h16 ":" ls32 ) | ( ( ( h16 ":" ){,4} h16 )? "::" ls32 ) | ( ( ( h16 ":" ){,5} h16 )? "::" h16 ) | ( ( ( h16 ":" ){,6} h16 )? "::" ); IPv6reference = "[" IPv6address "]"; host = hostname | IPv4address | IPv6reference; port = DIGIT{1,5}; hostport = host ( ":" port )?; This is much better than the deprecated and bogus grammar in RFC 2396 ;) >> ------------------ >> >> host_with_port = (hostname (":" digit*)?) >mark %host; >> >> - It allows something ugly as "mydomain.org:" >> >> I suggest: >> ? host_with_port = (hostname (":" digit{1,5})?) >mark %host; > > It's ugly, but section 3.2.2 of RFC 2396 appears to allows it. Sometimes there are bugs in the RFC's related to parsing and BNF grammars. I know several cases. Unfortunatelly RFC's cannot be fixed, instead the errors are reported and a new draft or RFC "xxx-fix" appears some years later. >> message_header = ((field_name ":" " "* field_value)|value_cont) :> CRLF; >> >> - It doesn't allow valid spaces before ":" as: >> ? ? ?Host : mydomain.org > > Spaces before the ":" aren't allowed in rfc2616, and I have yet to see > evidence of clients sending headers like this in ~4 years of using this > parser. In SIP protocol spaces and tabulators before ":" are allowed, I really expected that in HTTP the same occurs as SIP grammar is based on HTTP grammar. But it could be different in some aspects, of course. >> - Tabulators are also allowed. >> >> I suggest: >> ? message_header = ((field_name [ \t]* ":" [ \t]* >> field_value)|value_cont) :> CRLF; > > I just pushed this out to unicorn.git to allow horizontal tabs: Thanks. [*] http://dev.sipdoc.net/projects/ragel-sip-parser/wiki/Phase1 -- I?aki Baz Castillo From normalperson at yhbt.net Sat May 8 12:17:06 2010 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 8 May 2010 09:17:06 -0700 Subject: Some issues in the HTTP parser (and suggestions) In-Reply-To: References: <20100507183328.GA31883@dcvr.yhbt.net> Message-ID: <20100508161706.GA31295@dcvr.yhbt.net> I?aki Baz Castillo wrote: > 2010/5/7 Eric Wong : > > Underscore isn't valid for hostnames, but it is allowed in domain names > > and most DNS servers will resolve them. ?I've personally seen websites > > with underscores in their domain names in the wild[1]. > > Hi Eric, could you point me to the spec stating that underscore is > allowed for a domain? In the past I've done a SIP parser [*] with > Ragel, being 100% strict at BNF grammar, and note that SIP reuses 80% > of the grammar of HTTP. I'm pretty sure that "_" is not valid in a > domain (host, hostname or whatever). Anyhow it's better just to allow > it at parsing level :) http://www.ietf.org/rfc/rfc2782.txt Even if it's not part of the RFC, our parser will match reality and accommodate broken things we see in the wild, as it has done in the past: http://mid.gmane.org/20080327215027.GA14531 at untitled > > We'll have to test the IPv6 addresses and probably split that out into a > > separate regexp since ":" would raise issues with the port number in > > existing cases. ?This is probably something for post-1.0. > > There is a IETF draft to improve and *fix* the existing BNF grammar for IPv6. > It also improves the grammar for IPv4 (by dissallowing values greater than 255): > > http://tools.ietf.org/html/draft-ietf-sip-ipv6-abnf-fix > > > I've already implemented it in Ragel and I can sure that it's 100% > valid and strict (I've done lots of tests): > > alphanum = ALPHA / DIGIT > domainlabel = alphanum | ( alphanum ( alphanum | "-" )* alphanum ); > toplabel = ALPHA | ( ALPHA ( alphanum | "-" )* alphanum ); > hostname = ( domainlabel "." )* toplabel "."?; > dec_octet = DIGIT | ( 0x31..0x39 DIGIT ) | ( "1" DIGIT{2} ) | ( "2" > 0x30..0x34 DIGIT ) | ( "25" 0x30..0x35 ); > IPv4address = dec_octet "." dec_octet "." dec_octet "." dec_octet; > h16 = HEXDIG{1,4}; > ls32 = ( h16 ":" h16 ) | IPv4address; > IPv6address = ( ( h16 ":" ){6} ls32 ) | ( "::" ( h16 ":" ){5} ls32 ) | > ( h16? "::" ( h16 ":" ){4} ls32 ) | ( ( ( h16 ":" )? h16 )? "::" ( h16 > ":" ){3} ls32 ) | ( ( ( h16 ":" ){,2} h16 )? "::" ( h16 ":" ){2} ls32 > ) | ( ( ( h16 ":" ){,3} h16 )? "::" h16 ":" ls32 ) | ( ( ( h16 ":" > ){,4} h16 )? "::" ls32 ) | ( ( ( h16 ":" ){,5} h16 )? "::" h16 ) | ( ( > ( h16 ":" ){,6} h16 )? "::" ); > IPv6reference = "[" IPv6address "]"; > host = hostname | IPv4address | IPv6reference; > port = DIGIT{1,5}; > hostport = host ( ":" port )?; > > > This is much better than the deprecated and bogus grammar in RFC 2396 ;) Thanks, it might be worth simplifying a bit for readability, simplicity (and possibly performance) at the expense of 100% conformance. -- Eric Wong From normalperson at yhbt.net Sat May 8 12:21:16 2010 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 8 May 2010 09:21:16 -0700 Subject: testing, ignore Message-ID: <20100508162116.GB31295@dcvr.yhbt.net> (somebody had a stupid autoreply on, I subscribed them) From normalperson at yhbt.net Sat May 8 12:35:50 2010 From: normalperson at yhbt.net (Eric Wong) Date: Sat, 8 May 2010 09:35:50 -0700 Subject: bizarre autoresponder [was: Re: testing, ignore] In-Reply-To: <20100508162116.GB31295@dcvr.yhbt.net> References: <20100508162116.GB31295@dcvr.yhbt.net> Message-ID: <20100508163550.GD31295@dcvr.yhbt.net> Somebody used a support address that automatically entered mailing list posts into a ticketing system. I don't believe it was spam (it wasn't obviously trying to anything, there were no embedded images, it was in correct English). Nevertheless, they've been unsubscribed now. For the record, there's no chance in hell I'd use anything other than email for bug-tracking/support here. -- Eric Wong From ibc at aliax.net Sat May 8 12:50:54 2010 From: ibc at aliax.net (=?UTF-8?Q?I=C3=B1aki_Baz_Castillo?=) Date: Sat, 8 May 2010 18:50:54 +0200 Subject: Some issues in the HTTP parser (and suggestions) In-Reply-To: <20100508161706.GA31295@dcvr.yhbt.net> References: <20100507183328.GA31883@dcvr.yhbt.net> <20100508161706.GA31295@dcvr.yhbt.net> Message-ID: 2010/5/8 Eric Wong : > I?aki Baz Castillo wrote: >> 2010/5/7 Eric Wong : >> > Underscore isn't valid for hostnames, but it is allowed in domain names >> > and most DNS servers will resolve them. ?I've personally seen websites >> > with underscores in their domain names in the wild[1]. >> >> Hi Eric, could you point me to the spec stating that underscore is >> allowed for a domain? In the past I've done a SIP parser [*] with >> Ragel, being 100% strict at BNF grammar, and note that SIP reuses 80% >> of the grammar of HTTP. I'm pretty sure that "_" is not valid in a >> domain (host, hostname or whatever). Anyhow it's better just to allow >> it at parsing level :) > > http://www.ietf.org/rfc/rfc2782.txt Hi Eric. DNS SRV are not domain names, but DNS queries. For example, in SIP (and also in XMPP) if a phone is configured to use "myproxy.org" as proxy then it must perform a DNS SRV query for these entries: _sip._udp.myproxy.org _sip._tcp.myproxy.org Then the DNS query recevives some DNS A records for which the client also retrieves the associated IP's. But after it, when the client generates a SIP request it uses "myproxy.org" rather than "_sip._udp.myproxy.org". This is, "_sip._udp.myproxy.org" is not a domain/hostname, but a format for querying DNS SRV records. > Even if it's not part of the RFC, our parser will match reality and > accommodate broken things we see in the wild, as it has done in the > past: > > ?http://mid.gmane.org/20080327215027.GA14531 at untitled Yes, I agree. >> alphanum ?= ?ALPHA / DIGIT >> domainlabel = alphanum | ( alphanum ( alphanum | "-" )* alphanum ); >> toplabel = ALPHA | ( ALPHA ( alphanum | "-" )* alphanum ); >> hostname = ( domainlabel "." )* toplabel "."?; >> dec_octet = DIGIT | ( 0x31..0x39 DIGIT ) | ( "1" DIGIT{2} ) | ( "2" >> 0x30..0x34 DIGIT ) | ( "25" 0x30..0x35 ); >> IPv4address = dec_octet "." dec_octet "." dec_octet "." dec_octet; >> h16 = HEXDIG{1,4}; >> ls32 = ( h16 ":" h16 ) | IPv4address; >> IPv6address = ( ( h16 ":" ){6} ls32 ) | ( "::" ( h16 ":" ){5} ls32 ) | >> ( h16? "::" ( h16 ":" ){4} ls32 ) | ( ( ( h16 ":" )? h16 )? "::" ( h16 >> ":" ){3} ls32 ) | ( ( ( h16 ":" ){,2} h16 )? "::" ( h16 ":" ){2} ls32 >> ) | ( ( ( h16 ":" ){,3} h16 )? "::" h16 ":" ls32 ) | ( ( ( h16 ":" >> ){,4} h16 )? "::" ls32 ) | ( ( ( h16 ":" ){,5} h16 )? "::" h16 ) | ( ( >> ( h16 ":" ){,6} h16 )? "::" ); >> IPv6reference = "[" IPv6address "]"; >> host = hostname | IPv4address | IPv6reference; >> port = DIGIT{1,5}; >> hostport = host ( ":" port )?; >> >> >> This is much better than the deprecated and bogus grammar in RFC 2396 ;) > > Thanks, it might be worth simplifying a bit for readability, simplicity > (and possibly performance) at the expense of 100% conformance. You can try to simplicity it, but note that some previous IPv6 BNF grammars failed to cover all the valid cases and they are bogus. For example, the original IPv6 BNF grammar appearing in RFC 3261 (SIP protocol) is buggy (even if it seems simpler) and the specification has been fixed with the draft I linked in my previous mail. Regards. -- I?aki Baz Castillo From mrjomiles at rediffmail.com Sat May 8 12:29:10 2010 From: mrjomiles at rediffmail.com (Mr.Miles) Date: Sat, 08 May 2010 12:29:10 -0400 Subject: Scond Notice. Message-ID: Good Day, Be informed that my previous mail was not responded and I am not sure if it did get to you since I have not heard from you. I wish to notify you again that you were listed as a beneficiary to the total sum of 6,000,000.00 GBP (Six Million British Pounds) in the codicil and last testament of the deceased. (Name now withheld since this is our second letter to you). I contacted you because you bear the surname identity and therefore can present you as the beneficiary to the inheritance. I therefore reckoned that you could receive these funds as you are qualified by your name identity. All the legal papers will be processed upon your acceptance. Upon your acceptance of this deal, we request that you kindly forward to us your letter of acceptance, your current telephone and fax numbers and a forwarding address to enable us file necessary LEGAL documents in your name at our high court probate division for the release of the fund in question. Contact me immediately so that we can get this done. Kind regards, John Miles. From paul.p.carey at gmail.com Thu May 13 13:10:10 2010 From: paul.p.carey at gmail.com (Paul Carey) Date: Thu, 13 May 2010 18:10:10 +0100 Subject: Prematurely closed connection Message-ID: Hi I've come across a puzzling issue. The code segment below works as expected, but if I replace :error => true with :error => invite.errors, nginx serves a 502 as per the log below. invite.errors is normal hash. I'm using a bundled merb 1.1 with Unicorn 0.99. Unicorn is a system gem. I'm running nginx 0.7.65. I've only experienced this issue with nginx / unicorn. I'd be grateful for any thoughts or pointers. Many thanks Paul == controller action, serving json if invite.save display :ok => true else Merb.logger.info "invite.errors #{invite.errors}" display :error => true end == nginx.log 2010/05/13 16:34:03 [error] 11448#0: *226 upstream prematurely closed connection while reading response header from upstream, client: 78.147.1.152, server: _, request: "PUT /invites/726a50c3400d2fa16c7ec9e29512ac2cb69b52c8 HTTP/1.1", upstream: "http://unix:/var/apps/sd/tmp/unicorn.sock:/invites/726a50c3400d2fa16c7ec9e29512ac2cb69b52c8", host: "ec2-174-129-71-76.compute-1.amazonaws.com", referrer: "http://ec2-174-129-71-76.compute-1.amazonaws.com/events/djk5qipe87bas51z0qrgeukrr" == unicorn.stderr.log I, [2010-05-13T16:34:03.869423 #15332] INFO -- : reaped # worker=15I, [2010-05-13T16:34:03.890454 #15429] INFO -- : worker=15 spawned pid=15429 I, [2010-05-13T16:34:03.890669 #15429] INFO -- : worker=15 readyI, [2010-05-13T16:34:55.650233 #15332] INFO -- : reloading config_file=config/unicorn.rb I, [2010-05-13T16:34:58.583076 #15332] INFO -- : Refreshing Gem list From normalperson at yhbt.net Thu May 13 16:13:17 2010 From: normalperson at yhbt.net (Eric Wong) Date: Thu, 13 May 2010 20:13:17 +0000 Subject: Prematurely closed connection In-Reply-To: References: Message-ID: <20100513201317.GA9848@dcvr.yhbt.net> Paul Carey wrote: > Hi > > I've come across a puzzling issue. The code segment below works as > expected, but if I replace :error => true with :error => > invite.errors, nginx serves a 502 as per the log below. invite.errors > is normal hash. > > I'm using a bundled merb 1.1 with Unicorn 0.99. Unicorn is a system > gem. I'm running nginx 0.7.65. I've only experienced this issue with > nginx / unicorn. > I'd be grateful for any thoughts or pointers. > > Many thanks > Paul > > == controller action, serving json > if invite.save > display :ok => true > else > Merb.logger.info "invite.errors #{invite.errors}" > display :error => true > end Hi Paul, > == unicorn.stderr.log > I, [2010-05-13T16:34:03.869423 #15332] INFO -- : reaped > # worker=15I, ^^^^^^^ Uh oh, something is causing the Unicorn process to segfault in the response phase. Since Unicorn is only running Ruby code at that point, I suspect there's something in your JSON enoder that's not liking the serialization of invite.errors... I'm not familiar with Merb at all, but I'd be curious to see how your JSON encoder is handling the "invite.errors" hash. Which version of Ruby, JSON/Yajl library, etc are you using? Can you get us a backtrace? Can you run with RACK_ENV=development to get Rack::Lint onto your middleware stack? Perhaps Merb isn't returning a 100% valid Rack response and Rack::Lint will notice. Thanks for any info you can provide. -- Eric Wong From luke at lukemelia.com Thu May 13 16:29:04 2010 From: luke at lukemelia.com (Luke Melia) Date: Thu, 13 May 2010 16:29:04 -0400 Subject: Garbage collection outside of request cycle? In-Reply-To: <2B9DF63C-F43F-48B0-A380-2270EDFA53F4@lukemelia.com> References: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> <20100506205754.GA13412@dcvr.yhbt.net> <2B9DF63C-F43F-48B0-A380-2270EDFA53F4@lukemelia.com> Message-ID: > On May 6, 2010, at 4:57 PM, Eric Wong wrote: > >> ==> big_app_gc.rb <== >> # This shouldn't hurt overall performance as long as the server cluster >> # is at <=50% CPU capacity, and improves the performance of most memory >> # intensive requests. This serves to improve _client-visible_ >> # performance (possibly at the cost of overall performance). I thought the list might be interested in how this worked for us. I applied the patch to execute GC between each request. I'm using NewRelic to measure the app. Prior to the patch our, we spent about 25% of our aggregate time serving a request in GC and our application was running at around 20-30% CPU load. Our running app shows up as using ~330MB of memory. Applying the patch cut the time spent in GC time to nearly zero and as predicted CPU spiked. Client-perceived responsiveness increased as well. Unfortunately, during our busiest time of the day, CPU load got so high that nginx locked up, so we rolled back the patch. I made a simple change to execute GC once every 5 requests and applied it again. Aggregate time spent in GC reduced to out 10% of total request time. This resulted in a bout a 25% overall improvement in client response time. Big win! CPU maxes out at about 80% with this configuration.. One other thing I did was force GC to execute before_fork, on the theory that with COW, we would want to fork in the tidiest state possible. I have not measured this on it's own to evaluate it's impact. Thanks again for the help and code on this, Eric. Considering how useful this is, perhaps unicorn should have an after_request hook, to avoid the need to monkey-patch? Cheers, Luke -- http://www.lukemelia.com/ From paul.p.carey at gmail.com Fri May 14 07:00:22 2010 From: paul.p.carey at gmail.com (Paul Carey) Date: Fri, 14 May 2010 12:00:22 +0100 Subject: Prematurely closed connection In-Reply-To: <20100513201317.GA9848@dcvr.yhbt.net> References: <20100513201317.GA9848@dcvr.yhbt.net> Message-ID: >> == unicorn.stderr.log >> I, [2010-05-13T16:34:03.869423 #15332] ?INFO -- : reaped >> # worker=15I, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ^^^^^^^ > > Uh oh, something is causing the Unicorn process to segfault in the > response phase. ?Since Unicorn is only running Ruby code at that point, > I suspect there's something in your JSON enoder that's not liking > the serialization of invite.errors... Hi Eric You're absolutely right. The problem had nothing to do with Unicorn. JSON 1.4.3 was segfaulting on that particular hash - I don't know why just yet - but all worked as expected when I reverted to JSON 1.4.2. Many thanks Paul From normalperson at yhbt.net Fri May 14 15:02:31 2010 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 14 May 2010 19:02:31 +0000 Subject: Garbage collection outside of request cycle? In-Reply-To: References: <0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com> <20100506205754.GA13412@dcvr.yhbt.net> <2B9DF63C-F43F-48B0-A380-2270EDFA53F4@lukemelia.com> Message-ID: <20100514190231.GA21711@dcvr.yhbt.net> Luke Melia wrote: > > On May 6, 2010, at 4:57 PM, Eric Wong wrote: > > > >> ==> big_app_gc.rb <== # This shouldn't hurt overall performance as > >> long as the server cluster # is at <=50% CPU capacity, and improves > >> the performance of most memory # intensive requests. This serves > >> to improve _client-visible_ # performance (possibly at the cost of > >> overall performance). > > I thought the list might be interested in how this worked for us. I > applied the patch to execute GC between each request. I'm using > NewRelic to measure the app. Prior to the patch our, we spent about > 25% of our aggregate time serving a request in GC and our application > was running at around 20-30% CPU load. Our running app shows up as > using ~330MB of memory. Thanks for the feedback, Luke. Was the original 30% CPU load during peak traffic or normal traffic? > Applying the patch cut the time spent in GC time to nearly zero and as > predicted CPU spiked. Client-perceived responsiveness increased as > well. Unfortunately, during our busiest time of the day, CPU load got > so high that nginx locked up, so we rolled back the patch. Yikes, nginx locking up is rare. It's worth investigating and fixing that from the nginx side if you can reproduce it. > I made a simple change to execute GC once every 5 requests and applied > it again. Aggregate time spent in GC reduced to out 10% of total > request time. This resulted in a bout a 25% overall improvement in > client response time. Big win! CPU maxes out at about 80% with this > configuration.. How many Unicorn workers do you have per-core? I forgot to mention that you might want to run more workers to "hide" GC costs, something like a poor man's concurrent GC. > One other thing I did was force GC to execute before_fork, on the > theory that with COW, we would want to fork in the tidiest state > possible. I have not measured this on it's own to evaluate it's > impact. I doubt it'd help for anything other than the first <=5 requests that hit the worker process. > Thanks again for the help and code on this, Eric. Thank _you_ for actually being willing to run and report back on crazy experiments I come up with :) > Considering how useful this is, perhaps unicorn should have an > after_request hook, to avoid the need to monkey-patch? Given configurability requirements and the ability of such a feature to penalize some apps, I think making the monkey patch into middleware is a nice compromise. The GC.start in the below middleware runs at a slightly deeper stack level, meaning GC will scan more and reap less than the original monkey patch, but I doubt there'll be real difference if your app is already spending enough time in GC to be a problem. I've also pushed the following up to git://git.bogomips.org/unicorn.git Let me know if you get a chance to test it on your app in place of the monkey patch. >From 95b75a5043b34f39ece4f52befb4b3f884dfdd20 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Fri, 14 May 2010 18:27:35 +0000 Subject: [PATCH] add Unicorn::OobGC middleware This middleware allows configurable out-of-band garbage collection outside of the normal request/response cycle. It offers configurable paths (to only GC on expensive actions) and intervals to limit GC frequency. It is only expected to work well with Unicorn, as it would hurt performance on single-threaded servers if they have keepalive enabled. Obviously this does not work well for multi-threaded or evented servers that serve multiple clients at once. --- lib/unicorn/oob_gc.rb | 58 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 58 insertions(+), 0 deletions(-) create mode 100644 lib/unicorn/oob_gc.rb diff --git a/lib/unicorn/oob_gc.rb b/lib/unicorn/oob_gc.rb new file mode 100644 index 0000000..8dc4dcf --- /dev/null +++ b/lib/unicorn/oob_gc.rb @@ -0,0 +1,58 @@ +# -*- encoding: binary -*- +module Unicorn + + # Run GC after every request, after closing the client socket and + # before attempting to accept more connections. + # + # This shouldn't hurt overall performance as long as the server cluster + # is at <50% CPU capacity, and improves the performance of most memory + # intensive requests. This serves to improve _client-visible_ + # performance (possibly at the cost of overall performance). + # + # We'll call GC after each request is been written out to the socket, so + # the client never sees the extra GC hit it. + # + # This middleware is _only_ effective for applications that use a lot + # of memory, and will hurt simpler apps/endpoints that can process + # multiple requests before incurring GC. + # + # This middleware is only designed to work with Unicorn, as it harms + # keepalive performance. + # + # Example (in config.ru): + # + # require 'unicorn/oob_gc' + # + # # GC ever two requests that hit /expensive/foo or /more_expensive/foo + # # in your app. By default, this will GC once every 5 requests + # # for all endpoints in your app + # use Unicorn::OobGC, 2, %r{\A/(?:expensive/foo|more_expensive/foo)} + class OobGC < Struct.new(:app, :interval, :path, :nr, :env, :body) + + def initialize(app, interval = 5, path = %r{\A/}) + super(app, interval, path, interval) + end + + def call(env) + status, headers, self.body = app.call(self.env = env) + [ status, headers, self ] + end + + def each(&block) + body.each(&block) + end + + # in Unicorn, this is closed _after_ the client socket + def close + body.close if body.respond_to?(:close) + + if path =~ env['PATH_INFO'] && ((self.nr -= 1) <= 0) + self.nr = interval + self.body = nil + env.clear + GC.start + end + end + + end +end -- Eric Wong From alex at simonov.me Mon May 24 13:59:20 2010 From: alex at simonov.me (Alexander Simonov) Date: Mon, 24 May 2010 20:59:20 +0300 Subject: [ANN] golden_brindle Message-ID: <60ED089E-ADEF-4A92-9F9B-3E6E0C5D55A2@simonov.me> Golden Brindle - Unicorn HTTP server clustering tool like mongrel_cluster for Mongrel http://rubygems.org/gems/golden_brindle -- WBR, Alexander Simonov Web developer & High-load application deployer. Web Site: http://simonov.me E-Mail: alex at simonov.me From win at wincent.com Mon May 24 14:10:05 2010 From: win at wincent.com (Wincent Colaiuta) Date: Mon, 24 May 2010 20:10:05 +0200 Subject: [ANN] golden_brindle In-Reply-To: <60ED089E-ADEF-4A92-9F9B-3E6E0C5D55A2@simonov.me> References: <60ED089E-ADEF-4A92-9F9B-3E6E0C5D55A2@simonov.me> Message-ID: El 24/05/2010, a las 19:59, Alexander Simonov escribi?: > Golden Brindle - Unicorn HTTP server clustering tool like mongrel_cluster for Mongrel > http://rubygems.org/gems/golden_brindle What's the point of running a "cluster" of Unicorn processes? Isn't the whole idea of Unicorn to run a single master process with requests efficiently distributed among a group of worker processes which Unicorn itself manages? Cheers, Wincent From alex at simonov.me Mon May 24 14:44:57 2010 From: alex at simonov.me (Alexander Simonov) Date: Mon, 24 May 2010 21:44:57 +0300 Subject: [ANN] golden_brindle In-Reply-To: References: <60ED089E-ADEF-4A92-9F9B-3E6E0C5D55A2@simonov.me> Message-ID: The main idea give automatization tool for manage(start/stop/restart) a lot of unicorn instances. On May 24, 2010, at 9:10 PM, Wincent Colaiuta wrote: > El 24/05/2010, a las 19:59, Alexander Simonov escribi?: > >> Golden Brindle - Unicorn HTTP server clustering tool like mongrel_cluster for Mongrel >> http://rubygems.org/gems/golden_brindle > > What's the point of running a "cluster" of Unicorn processes? Isn't the whole idea of Unicorn to run a single master process with requests efficiently distributed among a group of worker processes which Unicorn itself manages? > > Cheers, > Wincent > From chewbranca at gmail.com Tue May 25 14:53:02 2010 From: chewbranca at gmail.com (Russell Branca) Date: Tue, 25 May 2010 11:53:02 -0700 Subject: Forking off the unicorn master process to create a background worker Message-ID: Hello, I'm trying to find an efficient way to create a new instance of a rails application to perform some background tasks without having to load up the entire rails stack every time, so I figured forking off the master process would be a good way to go. Now I can easily just increment the worker count and then send a web request in, but the new worker would be part of the main worker pool, so in the time between spawning a new worker and sending the request, another request could have come in and snagged that worker. Is it possible to create a new worker and not have it enter the main worker pool so I could access it directly? I know this is not your typical use case for unicorn, and you're probably thinking there is a lot better ways to do this, however, I currently have a rails framework that powers a handful of standalone applications on a server with limited resources, and I'm trying to make a centralized queue that all the applications use, so the queue needs to be able to spawn a new worker for each of the applications efficiently, and incrementing/decrementing worker counts in unicorn is the most efficient way I've found to spawn a new rails instance. Any help, suggestions or insight into this would be greatly appreciated. -Russell From ghazel at gmail.com Wed May 26 04:31:13 2010 From: ghazel at gmail.com (ghazel at gmail.com) Date: Wed, 26 May 2010 01:31:13 -0700 Subject: Issue with unicorn not starting via bluepill? In-Reply-To: <20100319085759.GA3323@dcvr.yhbt.net> References: <20100319082631.GA5672@dcvr.yhbt.net> <20100319085759.GA3323@dcvr.yhbt.net> Message-ID: On Fri, Mar 19, 2010 at 1:57 AM, Eric Wong wrote: > Eric Wong wrote: >> James Cox wrote: >> > trying to start my unicorn via bluepill, and running into this : > >> > master failed to start, check stderr log for details For reference, this is not the error I get. I'm running "upgrade", which is sig USR2, and getting two masters (one marked (old)) and two sets of children. > Both of you: > > Was there anything useful in stderr_path? ?The daemomized > process can never output to the terminal, so it had to > log somewhere... There's also strace, but looking at the > Unicorn code I haven't been able to find much (I'm very > sleep deprived right now, though). The only thing out of the ordinary I see in the log is: retrying in 5 seconds (infinite tries left) adding listener failed addr=127.0.0.1:9011 (in use) as each of the admin interface TCP socket listeners fails on the workers because there are still the old workers running. > @Greg: I got through to /v6lUsuzD finally, can you put some > debug statements around your before_fork hook and print out > the pid path name and whether it exists or not? server.config[:pid]: "/data/myapp/shared/pids/unicorn.pid" exists?: true old_pid: "/data/myapp/shared/pids/unicorn.pid.oldbin" exists?: true server.pid: "/data/myapp/shared/pids/unicorn.pid" old_pid: "/data/myapp/shared/pids/unicorn.pid.oldbin" server.pid != old_pid: true It looks like trying to send a SIGQUIT to the old master does nothing. Sending it a SIGTTOU does seem to reduce the number of workers, but even when there are 0 left it does not die after a SIGQUIT. Looking at the Unicorn source and my config: http://codepad.org/v6lUsuzD I'm not sure what from the 0.96.1 was supposed to be sending SIGTTOU. There are not any being sent, from what I can tell. In the 0.96.1 to 0.96.1 upgrade case, I see the old master get a bunch of SIGQUITs and reap its own children. In the 0.99.0 case I see no SIGQUITs arrive at the old master... In fact, just running 0.99.0 and trying to send it a SIGQUIT does nothing. SIGINT works fine, so do TTOU, TTIN, USR1 etc. I added a log message to the trap handler registered in trap_deferred, and it is never being called... Turns out, I had query_trace installed, and the change in to trap handler initialization registered it before query_trace initialized. Later, query_trace registered a QUIT handler and did not call any existing handlers. I have since uninstalled that broken junk. -Greg From normalperson at yhbt.net Wed May 26 17:05:42 2010 From: normalperson at yhbt.net (Eric Wong) Date: Wed, 26 May 2010 14:05:42 -0700 Subject: Forking off the unicorn master process to create a background worker In-Reply-To: References: Message-ID: <20100526210542.GC24211@dcvr.yhbt.net> Russell Branca wrote: > Hello, > > I'm trying to find an efficient way to create a new instance of a > rails application to perform some background tasks without having to > load up the entire rails stack every time, so I figured forking off > the master process would be a good way to go. Now I can easily just > increment the worker count and then send a web request in, but the new > worker would be part of the main worker pool, so in the time between > spawning a new worker and sending the request, another request could > have come in and snagged that worker. Is it possible to create a new > worker and not have it enter the main worker pool so I could access it > directly? Hi Russell, You could try having an endpoint in your webapp (with authentication, or have it reject env['REMOTE_ADDR'] != '127.0.0.1') that runs the background task for you. Since it's a background app, you should probably fork + Process.setsid + fork (or Process.daemon in 1.9), and return an HTTP response immediately so your app can serve other requests. The following example should be enough to get you started (totally untested) ------------ config.ru ------------- require 'rack/lobster' map "/.seekrit_endpoint" do use Rack::ContentLength use Rack::ContentType, 'text/plain' run(lambda { |env| return [ 403, {}, [] ] if env['REMOTE_ADDR'] != '127.0.0.1' pid = fork if pid Process.waitpid(pid) # cheap way to avoid unintentional fd sharing with our children, # this causes the current Unicorn worker to exit after sending # the response: # Otherwise you'd have to be careful to disconnect+reconnect # databases/memcached/redis/whatever (in both the parent and # child) to avoid unintentional sharing that'll lead to headaches Process.kill(:QUIT, $$) [ 200, {}, [ "started background process\n" ] ] else # child, daemonize it so the unicorn master won't need to # reap it (that's the job of init) Process.setsid exit if fork begin # run your background code here instead of sleeping sleep 5 env["rack.logger"].info "done sleeping" rescue => e env["rack.logger"].error(e.inspect) end # make sure we don't enter the normal response cycle back in the # worker... exit!(0) end }) end map "/" do run Rack::Lobster.new end > I know this is not your typical use case for unicorn, and you're > probably thinking there is a lot better ways to do this, however, I > currently have a rails framework that powers a handful of standalone > applications on a server with limited resources, and I'm trying to > make a centralized queue that all the applications use, so the queue > needs to be able to spawn a new worker for each of the applications > efficiently, and incrementing/decrementing worker counts in unicorn is > the most efficient way I've found to spawn a new rails instance. Yeah, it's definitely an odd case and there are ways to shoot yourself in the foot with it (especially with unintentional fd sharing), but Ruby exposes all the Unix process management goodies better than most languages (probably better than anything else I've used). > Any help, suggestions or insight into this would be greatly appreciated. Let us know how it goes :) -- Eric Wong From stefanmaier at gmail.com Fri May 28 04:55:21 2010 From: stefanmaier at gmail.com (Stefan Maier) Date: Fri, 28 May 2010 10:55:21 +0200 Subject: unicorn failing to start Message-ID: <4BFF84F9.2000906@gmail.com> Hi, i'm trying to start up unicorn_rails with a rails 3 beta3 project. All I can get out of it is this: I, [2010-05-28T08:54:45.770957 #17852] INFO -- : reaped # worker=0 I, [2010-05-28T08:54:45.771200 #17852] INFO -- : worker=0 spawning... I, [2010-05-28T08:54:45.774049 #17858] INFO -- : worker=0 spawned pid=17858 I, [2010-05-28T08:54:45.774350 #17858] INFO -- : Refreshing Gem list /usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.0.beta3/lib/active_support/deprecation/proxy_wrappers.rb:17:in `new': wrong number of arguments (1 for 2) (ArgumentError) from /usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.0.beta3/lib/active_support/deprecation/proxy_wrappers.rb:17:in `method_missing' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:54:in `block in use' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:73:in `call' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:73:in `block in to_app' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:73:in `each' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:73:in `inject' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:73:in `to_app' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:63:in `map' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:66:in `map' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/bin/unicorn_rails:173:in `block (2 levels) in rails_builder' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:46:in `instance_eval' from /usr/local/lib/ruby/gems/1.9.1/gems/rack-1.1.0/lib/rack/builder.rb:46:in `initialize' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/bin/unicorn_rails:156:in `new' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/bin/unicorn_rails:156:in `block in rails_builder' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:816:in `call' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:816:in `build_app!' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:673:in `init_worker_process' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:688:in `worker_loop' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:605:in `block (2 levels) in spawn_missing_workers' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:602:in `fork' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:602:in `block in spawn_missing_workers' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:598:in `each' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:598:in `spawn_missing_workers' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:612:in `maintain_worker_count' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:408:in `block in join' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:396:in `loop' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:396:in `join' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/lib/unicorn.rb:28:in `run' from /usr/local/lib/ruby/gems/1.9.1/gems/unicorn-0.99.0/bin/unicorn_rails:203:in `' from /usr/local/bin/unicorn_rails:19:in `load' from /usr/local/bin/unicorn_rails:19:in `
' I, [2010-05-28T08:54:49.301901 #17852] INFO -- : reaped # worker=0 I, [2010-05-28T08:54:49.302066 #17852] INFO -- : master complete Any ideas what's wrong? Thanks, Stefan Maier From normalperson at yhbt.net Fri May 28 14:49:41 2010 From: normalperson at yhbt.net (Eric Wong) Date: Fri, 28 May 2010 11:49:41 -0700 Subject: unicorn failing to start In-Reply-To: <4BFF84F9.2000906@gmail.com> References: <4BFF84F9.2000906@gmail.com> Message-ID: <20100528184941.GA27195@dcvr.yhbt.net> Stefan Maier wrote: > Hi, > > i'm trying to start up unicorn_rails with a rails 3 beta3 project. All I > can get out of it is this: > I, [2010-05-28T08:54:45.770957 #17852] INFO -- : reaped > # worker=0 > I, [2010-05-28T08:54:45.771200 #17852] INFO -- : worker=0 spawning... > I, [2010-05-28T08:54:45.774049 #17858] INFO -- : worker=0 spawned pid=17858 > I, [2010-05-28T08:54:45.774350 #17858] INFO -- : Refreshing Gem list > > /usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.0.beta3/lib/active_support/deprecation/proxy_wrappers.rb:17:in > `new': wrong number of arguments (1 for 2) (ArgumentError) > from > /usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.0.beta3/lib/active_support/deprecation/proxy_wrappers.rb:17:in > `method_missing' > Any ideas what's wrong? Hi Stephan, I've heard (but not confirmed myself) Rails 3 doesn't work well with Ruby 1.9.1, but does with 1.9.2dev (trunk), and 1.8.7. Can you give either of those versions of Ruby a try? Another thing that's probably not the issue here, but since config.ru is present, give "unicorn" a shot instead of "unicorn_rails" as the latter hasn't been tested heavily with Rails 3. "unicorn_rails" was designed with older Rails in mind. Let us know what you find, thanks! -- Eric Wong From stefanmaier at gmail.com Fri May 28 15:19:14 2010 From: stefanmaier at gmail.com (Stefan Maier) Date: Fri, 28 May 2010 21:19:14 +0200 Subject: unicorn failing to start In-Reply-To: <20100528184941.GA27195@dcvr.yhbt.net> References: <4BFF84F9.2000906@gmail.com> <20100528184941.GA27195@dcvr.yhbt.net> Message-ID: <4C001732.8030202@gmail.com> Am 28.05.10 20:49, schrieb Eric Wong: > Stefan Maier wrote: >> Hi, >> >> i'm trying to start up unicorn_rails with a rails 3 beta3 project. All I >> can get out of it is this: >> I, [2010-05-28T08:54:45.770957 #17852] INFO -- : reaped >> # worker=0 >> I, [2010-05-28T08:54:45.771200 #17852] INFO -- : worker=0 spawning... >> I, [2010-05-28T08:54:45.774049 #17858] INFO -- : worker=0 spawned pid=17858 >> I, [2010-05-28T08:54:45.774350 #17858] INFO -- : Refreshing Gem list >> >> /usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.0.beta3/lib/active_support/deprecation/proxy_wrappers.rb:17:in >> `new': wrong number of arguments (1 for 2) (ArgumentError) >> from >> /usr/local/lib/ruby/gems/1.9.1/gems/activesupport-3.0.0.beta3/lib/active_support/deprecation/proxy_wrappers.rb:17:in >> `method_missing' > > > >> Any ideas what's wrong? > > Hi Stephan, > > I've heard (but not confirmed myself) Rails 3 doesn't work well with > Ruby 1.9.1, but does with 1.9.2dev (trunk), and 1.8.7. Can you give > either of those versions of Ruby a try? > > Another thing that's probably not the issue here, but since config.ru is > present, give "unicorn" a shot instead of "unicorn_rails" as the latter > hasn't been tested heavily with Rails 3. "unicorn_rails" was designed > with older Rails in mind. > > Let us know what you find, thanks! > Hi Eric, Using the config.ru with "unicorn" does indeed work. If I have the time I'll investigate other ruby versions tomorrow. Thanks, Stefan Maier