From fingermark at gmail.com  Mon Oct  1 15:31:07 2012
From: fingermark at gmail.com (bradford)
Date: Mon, 1 Oct 2012 11:31:07 -0400
Subject: after_fork and redis
Message-ID: <CAEbKVFSAgG=-EhWeVMEGrG9_gLGPg1yS-KXoCpaB0rtJ7E9pPw@mail.gmail.com>

I'm using unicorn w/ a rails app.  I have the following in my
environment.rb $redis = MyApplication::Application.config.redis and in
production.rb I have config.redis = Redis.new(host: "localhost").

I've read I'm supposed to $redis = Redis.new(host: "localhost") in
after_fork when preload_app is true.

When I don't do this, each worker/pid seems to have their own redis
instance.  So, why is this needed?  Here's the logs of me printing out
$redis.client.inspect when both $redis = Redis.new in the after_fork
and just $redis = Redis.new in the environment.rb.

No after fork
https://gist.github.com/bc2c2a3bda01c35730e2

After fork
https://gist.github.com/0c96550660d3926ffe16

The only thing different I notice is Connection::TCPSocket:fd is
always 13 w/ the after fork.

From normalperson at yhbt.net  Mon Oct  1 19:31:15 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 1 Oct 2012 19:31:15 +0000
Subject: after_fork and redis
In-Reply-To: <CAEbKVFSAgG=-EhWeVMEGrG9_gLGPg1yS-KXoCpaB0rtJ7E9pPw@mail.gmail.com>
References: <CAEbKVFSAgG=-EhWeVMEGrG9_gLGPg1yS-KXoCpaB0rtJ7E9pPw@mail.gmail.com>
Message-ID: <20121001193115.GA23907@dcvr.yhbt.net>

bradford <fingermark at gmail.com> wrote:
> I'm using unicorn w/ a rails app.  I have the following in my
> environment.rb $redis = MyApplication::Application.config.redis and in
> production.rb I have config.redis = Redis.new(host: "localhost").
> 
> I've read I'm supposed to $redis = Redis.new(host: "localhost") in
> after_fork when preload_app is true.
> 
> When I don't do this, each worker/pid seems to have their own redis
> instance.  So, why is this needed?  Here's the logs of me printing out
> $redis.client.inspect when both $redis = Redis.new in the after_fork
> and just $redis = Redis.new in the environment.rb.

(Disclaimer: I still haven't used Redis, but similar knowledge applies
to every TCP-based service)

If a Redis client opens a TCP (or any stream) socket connection before
forking, all the children that are forked will share that _same_ local
TCP socket.  Sharing stream sockets across processes/threads is nearly
always a bad idea: data streams become interleaved and impossible to
separate in either direction.

> No after fork
> https://gist.github.com/bc2c2a3bda01c35730e2
> 
> After fork
> https://gist.github.com/0c96550660d3926ffe16
> 
> The only thing different I notice is Connection::TCPSocket:fd is
> always 13 w/ the after fork.

FD shouldn't really matter.  FDs get recycled ASAP once they're closed
(and GC can automatically close them).  What you want is a different
local port for the TCPSocket on every worker process, and the after_fork
hook will give you that.

Btw, you can demonstrate FD recycling with:

  require 'socket'
  loop do
    c = TCPSocket.new("example.com", 80)
    p [ :fileno, c.fileno ]
    p [ :addr, c.addr ] # (local address)
    c.close
    sleep 1 # be nice to the remote server
  end

You'll see the same FD recycled over and over, but c.addr will
have a different local port.

You can see if after_fork is working correctly by checking the output of
tools like `ss' or `netstat':
(something like: ss -tan | grep -F :$PORT_OF_REDIS)

You can also check connections on the Redis server/process itself
using "lsof -p $PID_OF_REDIS_DAEMON"

From somers.ben at gmail.com  Mon Oct  1 20:28:19 2012
From: somers.ben at gmail.com (Ben Somers)
Date: Mon, 1 Oct 2012 13:28:19 -0700
Subject: after_fork and redis
In-Reply-To: <20121001193115.GA23907@dcvr.yhbt.net>
References: <CAEbKVFSAgG=-EhWeVMEGrG9_gLGPg1yS-KXoCpaB0rtJ7E9pPw@mail.gmail.com>
	<20121001193115.GA23907@dcvr.yhbt.net>
Message-ID: <CAO1NZAqKgGwitq3b+UFnxUr-xnuobgOT04qQXJX0thVNkUqsCg@mail.gmail.com>

>> When I don't do this, each worker/pid seems to have their own redis
>> instance.  So, why is this needed?  Here's the logs of me printing out
>> $redis.client.inspect when both $redis = Redis.new in the after_fork
>> and just $redis = Redis.new in the environment.rb.

Checked through the redis.rb source a little bit, and I believe it's
got code specifically to avoid opening the connection until it's
needed, to prevent some shared connection problems
(https://github.com/redis/redis-rb/commit/d1bc88f85bc9bfbb7aabdfc2700bf4b30c0b0115).
You probably aren't making actual use of the connection until after
app startup, so that's why things are working so far for you. It's still
a good idea to initialize Redis in the after_fork, though, because if
something in your app's startup or initializer code does wind up using
that redis connection, you'd still get the shared connection problem.

> Sharing stream sockets across processes/threads is nearly
> always a bad idea: data streams become interleaved and impossible to
> separate in either direction.

"Nearly always a bad idea" is a bit of an understatement; the
interleaved data streams tend to create catastrophic errors, things
like users unable to log in or (god forbid) logging into other
people's sessions. And they usually manifest as race conditions, so
they're hard to debug if you don't know what to look for.

From joel at expectedbehavior.com  Tue Oct  2 00:59:08 2012
From: joel at expectedbehavior.com (Joel Meador)
Date: Mon, 01 Oct 2012 20:59:08 -0400
Subject: [PATCH] chowning /dev/null should be guarded against
Message-ID: <506A3C5C.4020308@expectedbehavior.com>

We've run into this problem several times and it's not really expected 
that someone is going to monkey with /dev/null. So here's a simple patch.
I am not on the mailing list, so please Cc: me.

---
  lib/unicorn/util.rb | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
index cde2563..6b6cca2 100644
--- a/lib/unicorn/util.rb
+++ b/lib/unicorn/util.rb
@@ -15,7  +15,7  @@ module Unicorn::Util

    def self.chown_logs(uid, gid)
      ObjectSpace.each_object(File) do |fp|
-      fp.chown(uid, gid) if is_log?(fp)
+      fp.chown(uid, gid) if is_log?(fp)&&  fp.path != "/dev/null"
      end
    end
  # :startdoc:
-- 1.7.12.1


From normalperson at yhbt.net  Tue Oct  2 01:27:31 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 1 Oct 2012 18:27:31 -0700
Subject: [PATCH] chowning /dev/null should be guarded against
In-Reply-To: <506A3C5C.4020308@expectedbehavior.com>
References: <506A3C5C.4020308@expectedbehavior.com>
Message-ID: <20121002012731.GA12583@dcvr.yhbt.net>

Joel Meador <joel at expectedbehavior.com> wrote:
> We've run into this problem several times and it's not really
> expected that someone is going to monkey with /dev/null. So here's a
> simple patch.
> I am not on the mailing list, so please Cc: me.

Interesting nobody found this bug earlier (I think pointing logs
to /dev/null is bad :)

> diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
> index cde2563..6b6cca2 100644
> --- a/lib/unicorn/util.rb
> +++ b/lib/unicorn/util.rb
> @@ -15,7  +15,7  @@ module Unicorn::Util
> 
>    def self.chown_logs(uid, gid)
>      ObjectSpace.each_object(File) do |fp|
> -      fp.chown(uid, gid) if is_log?(fp)
> +      fp.chown(uid, gid) if is_log?(fp)&&  fp.path != "/dev/null"
>      end
>    end
>  # :startdoc:

How about this, instead?  This will work in case some weirdo setups
use a non-standard path for /dev/null, and also allows logging to
FIFOs (e.g. cronolog users).

diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
index cde2563..f84241c 100644
--- a/lib/unicorn/util.rb
+++ b/lib/unicorn/util.rb
@@ -7,6 +7,7 @@ def self.is_log?(fp)
     append_flags = File::WRONLY | File::APPEND
 
     ! fp.closed? &&
+      fp.stat.file? &&
       fp.sync &&
       (fp.fcntl(Fcntl::F_GETFL) & append_flags) == append_flags
     rescue IOError, Errno::EBADF

From joel at expectedbehavior.com  Tue Oct  2 01:48:05 2012
From: joel at expectedbehavior.com (Joel Meador)
Date: Mon, 01 Oct 2012 21:48:05 -0400
Subject: [PATCH] chowning /dev/null should be guarded against
In-Reply-To: <20121002012731.GA12583@dcvr.yhbt.net>
References: <506A3C5C.4020308@expectedbehavior.com>
	<20121002012731.GA12583@dcvr.yhbt.net>
Message-ID: <506A47D5.5020709@expectedbehavior.com>

Eric Wong wrote:
> Interesting nobody found this bug earlier (I think pointing logs
> to /dev/null is bad :)
Generally agree. Sometimes it happens. :|
> How about this, instead?  This will work in case some weirdo setups
> use a non-standard path for /dev/null, and also allows logging to
> FIFOs (e.g. cronolog users).
>
> diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
> index cde2563..f84241c 100644
> --- a/lib/unicorn/util.rb
> +++ b/lib/unicorn/util.rb
> @@ -7,6 +7,7 @@ def self.is_log?(fp)
>       append_flags = File::WRONLY | File::APPEND
>
>       ! fp.closed?&&
> +      fp.stat.file?&&
>         fp.sync&&
>         (fp.fcntl(Fcntl::F_GETFL)&  append_flags) == append_flags
>       rescue IOError, Errno::EBADF
This looks much better as a general solution to the problem. +1 Thanks, 
Eric!

From normalperson at yhbt.net  Tue Oct  2 04:25:08 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 1 Oct 2012 21:25:08 -0700
Subject: [PATCH] chowning /dev/null should be guarded against
In-Reply-To: <506A47D5.5020709@expectedbehavior.com>
References: <506A3C5C.4020308@expectedbehavior.com>
	<20121002012731.GA12583@dcvr.yhbt.net>
	<506A47D5.5020709@expectedbehavior.com>
Message-ID: <20121002042508.GA24004@dcvr.yhbt.net>

Joel Meador <joel at expectedbehavior.com> wrote:
> Eric Wong wrote:
> >--- a/lib/unicorn/util.rb
> >+++ b/lib/unicorn/util.rb
> >@@ -7,6 +7,7 @@ def self.is_log?(fp)
> >      append_flags = File::WRONLY | File::APPEND
> >
> >      ! fp.closed?&&
> >+      fp.stat.file?&&
> >        fp.sync&&
> >        (fp.fcntl(Fcntl::F_GETFL)&  append_flags) == append_flags
> >      rescue IOError, Errno::EBADF
> This looks much better as a general solution to the problem. +1
> Thanks, Eric!

Thanks for reporting!  Pushed to "master" of git://bogomips.org/unicorn
as commit 032791b9a367f67febbe7534f6ea4cac127e7897

Will probably tag + release a new version in day or two.

From naresh.sharma at triburg.co.in  Mon Oct  8 10:11:20 2012
From: naresh.sharma at triburg.co.in (=?windows-1251?Q?=C0=EB=FC=E1=E8=ED=EE=F7=EA=E0_=CD=E0=E7=E0=F0=EE=E2=E0?=)
Date: Mon, 8 Oct 2012 11:11:20 +0100
Subject: =?windows-1251?Q?=E7=E4=E0=F0=F0=F0=EE=E2=E0=2C_=E4=F3=EC=E0=E5=F8_=EE=E1_=EC=ED=E5=3F.._?=
Message-ID: <728897164.20121008111120@triburg.co.in>

http://ruki.nam.by/tmp/mk.php

From jimmy.soho at gmail.com  Tue Oct  9 00:39:45 2012
From: jimmy.soho at gmail.com (Jimmy Soho)
Date: Tue, 9 Oct 2012 11:39:45 +1100
Subject: Is a client uploading a file a slow client from unicorn's point of
	view?
Message-ID: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>

Hi All,

I was wondering what would happen when large files were uploaded to
our system in parallel to endpoints that don't process file uploads.
In particular I was wondering if we're vulnerable to a simple DoS
attack.

The setup I tested with was nginx v1.2.4 with upload module (v2.2.0)
configured only for location /uploads with 2 unicorn (v4.3.1) workers
with timeout 30 secs, all running on 1 small unix box.

In a few terminals I started this command 3 times in parallel:

   $ curl -i -F importer_input=@/Users/admin/largefile.tar.gz
https://mywebserver.com/doesnotexist

In a browser I then tried to go a page that would be served by a unicorn worker.

My expectation was that I would not get to see the web page as all
unicorn workers would be busy with receiving / saving the upload. As
discussed in for example this article:
http://stackoverflow.com/questions/9592664/unicorn-rails-large-uploads.
Or as https://github.com/dwilkie/carrierwave_direct describes it:

  "Processing and saving file uploads are typically long running tasks
and should be done in a background process."

But I don't see this. The page is served just fine in my setup. The
requests for the file uploads appear in the nginx access log at the
same time the curl upload command eventually finishes minutes later
client side, and then it's handed off to a unicorn/rack worker
process, which quickly returns a 404 page not found. Response times of
less than 50ms.

What am I missing here? I'm starting to wonder what's the use of the
nginx upload module? My understanding was that it's use was to keep
unicorn workers available as long as a file upload was in progress,
but it seems that without that module it does the same thing.

Another question (more an nginx question though I guess): is there a
way to kill an upload request as early as possible if the request is
not made against known / accepted URI locations, instead of waiting
for it to be completely uploaded to our system and/or waiting for it
to reach the unicorn workers?


Cheers,
Jim

From normalperson at yhbt.net  Tue Oct  9 01:58:00 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 9 Oct 2012 01:58:00 +0000
Subject: Is a client uploading a file a slow client from unicorn's point
	of view?
In-Reply-To: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
References: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
Message-ID: <20121009015800.GA1772@dcvr.yhbt.net>

Jimmy Soho <jimmy.soho at gmail.com> wrote:
> Hi All,
> 
> I was wondering what would happen when large files were uploaded to
> our system in parallel to endpoints that don't process file uploads.
> In particular I was wondering if we're vulnerable to a simple DoS
> attack.

nginx will protect you by buffering large requests to disk, so slow
requests are taken care of (of course you may still run out of disk
space)

> The setup I tested with was nginx v1.2.4 with upload module (v2.2.0)
> configured only for location /uploads with 2 unicorn (v4.3.1) workers
> with timeout 30 secs, all running on 1 small unix box.
> 
> In a few terminals I started this command 3 times in parallel:
> 
>    $ curl -i -F importer_input=@/Users/admin/largefile.tar.gz
> https://mywebserver.com/doesnotexist
> 
> In a browser I then tried to go a page that would be served by a unicorn worker.
> 
> My expectation was that I would not get to see the web page as all
> unicorn workers would be busy with receiving / saving the upload. As
> discussed in for example this article:
> http://stackoverflow.com/questions/9592664/unicorn-rails-large-uploads.
> Or as https://github.com/dwilkie/carrierwave_direct describes it:
> 
>   "Processing and saving file uploads are typically long running tasks
> and should be done in a background process."

That is true.  It's good to move slow jobs to background processes if
possible if the bottleneck is either:

a) your application processing
b) the storage destination of your app (e.g. cloud storage)

However, if your only bottleneck is client <-> your app, then nginx
will take care of that part for you.

> But I don't see this. The page is served just fine in my setup. The
> requests for the file uploads appear in the nginx access log at the
> same time the curl upload command eventually finishes minutes later
> client side, and then it's handed off to a unicorn/rack worker
> process, which quickly returns a 404 page not found. Response times of
> less than 50ms.
> 
> What am I missing here? I'm starting to wonder what's the use of the
> nginx upload module? My understanding was that it's use was to keep
> unicorn workers available as long as a file upload was in progress,
> but it seems that without that module it does the same thing.

I'm not familiar with the nginx upload module, but stock nginx will
already do full request buffering for you.  It looks like the nginx
upload module[1] is mostly meant for standalone apps written for
nginx, and not when nginx is used as a proxy for Rails app...

[1] http://www.grid.net.ru/nginx/upload.en.html

> Another question (more an nginx question though I guess): is there a
> way to kill an upload request as early as possible if the request is
> not made against known / accepted URI locations, instead of waiting
> for it to be completely uploaded to our system and/or waiting for it
> to reach the unicorn workers?

I'm not sure if nginx has this functionality, but unicorn lazily buffers
uploads.  So your upload will be fully read by nginx, but unicorn
will only read the uploaded request body if your application wants to
read it.

Unfortunately, I think most application frameworks (e.g. Rails) will
attempt to do all the multipart parsing up front.  To get around this,
you'll probably want some middleware along the following lines (and
placed in front of whichever part of your stack calls
Rack::Multipart.parse_multipart)

class BadUploadStopper
  def initialize(app)
    @app = app
  end

  def call(env)
    case env["REQUEST_METHOD"]
    when "POST", "PUT"
      case env["PATH_INFO"]
      when "/upload_allowed"
        @app.call(env) # forward to the app
      else
        # bad path, don't waste time with @app.call
        [ 403, {}, [ "Go away\n" ] ]
      end
    else
      @app.call(env)  # forward to the app
    end
  end
end

------------------- config.ru ---------------------
use BadUploadStopper
run YourApp.new

From laas at toom.ee  Tue Oct  9 06:31:36 2012
From: laas at toom.ee (Laas Toom)
Date: Tue, 9 Oct 2012 09:31:36 +0300
Subject: Is a client uploading a file a slow client from unicorn's point
	of view?
In-Reply-To: <20121009015800.GA1772@dcvr.yhbt.net>
References: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
	<20121009015800.GA1772@dcvr.yhbt.net>
Message-ID: <CFC93863-1061-4710-A195-58CBCCBED7B8@toom.ee>


On 09.10.2012, at 4:58, Eric Wong <normalperson at yhbt.net> wrote:

> I'm not familiar with the nginx upload module, but stock nginx will
> already do full request buffering for you.  It looks like the nginx
> upload module[1] is mostly meant for standalone apps written for
> nginx, and not when nginx is used as a proxy for Rails app...

AFAIK the upload module will give you two things:

1) handle the whole body parsing up to the point of storing the file to disk in correct place. Then it strips the file from POST request and replaces with reference to the location on disk.

2) make the upload progress available, so e.g. AJAX-powered upload forms can show progressbar, which is really neat. No need for Flash-based uploaders.


I have a Rails app that accepts media uploads. All the processing happens in background, front-end handles only the actual upload and stores it to disk.
But with uploads as large as 1.4 GB, I've seen Rails response times > 200 secs. This starts to give timeouts in weird places.

Eric, correct me if I'm wrong, but doesn't Nginx-Unicorn-Rails stack write the whole file up to 3 times to disk:
1) Nginx buffers the body (in encoded state)
2) Unicorn parses the body and writes to TMP folder (as requested by Rails)
3) if Rails accepts the file, it moves it to actual storage. But as /tmp is often different device from storage, this is actually a full copy.

In such a situation the upload module would help out, because instead of simply buffering the body on disk, it actually parses the body. And it is implemented in C, which should make it faster.
Afterwards it will only handle out the file location and Rails can complete it's work a lot faster, freeing up workers.

Unicorn won't even see the file and Rails has the responsibility to delete the file if it's invalid.


Best,
Laas

From normalperson at yhbt.net  Tue Oct  9 20:03:06 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 9 Oct 2012 20:03:06 +0000
Subject: Is a client uploading a file a slow client from unicorn's point
	of view?
In-Reply-To: <CFC93863-1061-4710-A195-58CBCCBED7B8@toom.ee>
References: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
	<20121009015800.GA1772@dcvr.yhbt.net>
	<CFC93863-1061-4710-A195-58CBCCBED7B8@toom.ee>
Message-ID: <20121009200306.GA17437@dcvr.yhbt.net>

Laas Toom <laas at toom.ee> wrote:
> On 09.10.2012, at 4:58, Eric Wong <normalperson at yhbt.net> wrote:
> > I'm not familiar with the nginx upload module, but stock nginx will
> > already do full request buffering for you.  It looks like the nginx
> > upload module[1] is mostly meant for standalone apps written for
> > nginx, and not when nginx is used as a proxy for Rails app...
> 
> AFAIK the upload module will give you two things:
> 
> 1) handle the whole body parsing up to the point of storing the file
> to disk in correct place. Then it strips the file from POST request
> and replaces with reference to the location on disk.

That sounds awesome performance-wise.

> 2) make the upload progress available, so e.g. AJAX-powered upload forms can show progressbar, which is really neat. No need for Flash-based uploaders.

It does?  I'm not seeing it in the documentation, but I know there's
a separate upload progress module, though:
http://wiki.nginx.org/HttpUploadProgressModule

A side note on upload progress: I wrote upr[1] back in the day since I
wanted to share upload progress state via memcached for multi-machine
configurations:  http://upr.bogomips.org/

> I have a Rails app that accepts media uploads. All the processing happens in background, front-end handles only the actual upload and stores it to disk.
> But with uploads as large as 1.4 GB, I've seen Rails response times > 200 secs. This starts to give timeouts in weird places.

Yikes.  I assume you're constrained by disk I/O there?

For some of the large file situations under Linux, I find it beneficial
to lower the dirty_*ratio/*bytes drastically to avoid large, sudden
bursts of disk activity and instead favor smaller writes.  I get lower
throughput, but more consistent performance.

> Eric, correct me if I'm wrong, but doesn't Nginx-Unicorn-Rails stack
> write the whole file up to 3 times to disk:
> 
> 1) Nginx buffers the body (in encoded state)

Correct.

> 2) Unicorn parses the body and writes to TMP folder (as requested by Rails)

Rack does multipart parsing.  Unicorn itself doesn't do body parsing
other than handling Transfer-Encoding:chunked (which hardly anybody
sends).

> 3) if Rails accepts the file, it moves it to actual storage. But as /tmp is often different device from storage, this is actually a full copy.

Depends on the Rack/Rails app, but usually this is the case.

For my use, all uploads are PUT requests with "curl -T", so there's no
multipart parsing involved and much faster :)

> In such a situation the upload module would help out, because instead
> of simply buffering the body on disk, it actually parses the body. And
> it is implemented in C, which should make it faster.

Yep.

> Afterwards it will only handle out the file location and Rails can
> complete it's work a lot faster, freeing up workers.
> 
> Unicorn won't even see the file and Rails has the responsibility to
> delete the file if it's invalid.

I think the only problem with this approach is it won't work well on
setups where nginx is on separate machines from unicorn.  Shared
storage would be required, but that ends up adding to network I/O,
too...

From laas at toom.ee  Tue Oct  9 23:06:55 2012
From: laas at toom.ee (Laas Toom)
Date: Wed, 10 Oct 2012 02:06:55 +0300
Subject: Is a client uploading a file a slow client from unicorn's point
	of view?
In-Reply-To: <20121009200306.GA17437@dcvr.yhbt.net>
References: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
	<20121009015800.GA1772@dcvr.yhbt.net>
	<CFC93863-1061-4710-A195-58CBCCBED7B8@toom.ee>
	<20121009200306.GA17437@dcvr.yhbt.net>
Message-ID: <3EDF9F45-B5FB-4C3B-B0E5-37C28ECFE1D1@toom.ee>


On 09.10.2012, at 23:03, Eric Wong <normalperson at yhbt.net> wrote:

>> 2) make the upload progress available, so e.g. AJAX-powered upload forms can show progressbar, which is really neat. No need for Flash-based uploaders.
> 
> It does?  I'm not seeing it in the documentation, but I know there's
> a separate upload progress module, though:
> http://wiki.nginx.org/HttpUploadProgressModule

I must have mixed them up then. :-)

>> I have a Rails app that accepts media uploads. All the processing happens in background, front-end handles only the actual upload and stores it to disk.
>> But with uploads as large as 1.4 GB, I've seen Rails response times > 200 secs. This starts to give timeouts in weird places.
> 
> Yikes.  I assume you're constrained by disk I/O there?

Might be. Additionally, the disk is SAN, so network activity there too.

> For some of the large file situations under Linux, I find it beneficial
> to lower the dirty_*ratio/*bytes drastically to avoid large, sudden
> bursts of disk activity and instead favor smaller writes.  I get lower
> throughput, but more consistent performance.

I shall look into it when I get to fixing this issue.


>> Afterwards it will only handle out the file location and Rails can
>> complete it's work a lot faster, freeing up workers.
>> 
>> Unicorn won't even see the file and Rails has the responsibility to
>> delete the file if it's invalid.
> 
> I think the only problem with this approach is it won't work well on
> setups where nginx is on separate machines from unicorn.  Shared
> storage would be required, but that ends up adding to network I/O,
> too...

But won't (almost) the same network I/O be evident anyway, because of nginx transferring the data to Unicorn over network (as they are on different machines)?

Thanks for clarifying,
Laas


From normalperson at yhbt.net  Tue Oct  9 23:54:35 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 9 Oct 2012 16:54:35 -0700
Subject: Is a client uploading a file a slow client from unicorn's point
	of view?
In-Reply-To: <3EDF9F45-B5FB-4C3B-B0E5-37C28ECFE1D1@toom.ee>
References: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
	<20121009015800.GA1772@dcvr.yhbt.net>
	<CFC93863-1061-4710-A195-58CBCCBED7B8@toom.ee>
	<20121009200306.GA17437@dcvr.yhbt.net>
	<3EDF9F45-B5FB-4C3B-B0E5-37C28ECFE1D1@toom.ee>
Message-ID: <20121009235435.GA29908@dcvr.yhbt.net>

Laas Toom <laas at toom.ee> wrote:
> On 09.10.2012, at 23:03, Eric Wong <normalperson at yhbt.net> wrote:
> > Laas Toom <laas at toom.ee> wrote:
> >> Afterwards it will only handle out the file location and Rails can
> >> complete it's work a lot faster, freeing up workers.
> >> 
> >> Unicorn won't even see the file and Rails has the responsibility to
> >> delete the file if it's invalid.
> > 
> > I think the only problem with this approach is it won't work well on
> > setups where nginx is on separate machines from unicorn.  Shared
> > storage would be required, but that ends up adding to network I/O,
> > too...
> 
> But won't (almost) the same network I/O be evident anyway, because of
> nginx transferring the data to Unicorn over network (as they are on
> different machines)?

It depends on your shared storage implementation.

It'll likely be a win if the shared storage is on the same server as
nginx (but that might mean you can only have one nginx server).  But I
think it'll be a loss if there needs to be multiple nginx servers (and
multiple unicorn servers)...

* With nginx_upload_module + shared storage:

nginx server  ------ shared storage -------- unicorn server
------------------------------------------------------------------
1. sequential write to shared storage

2. file could remain cached              do processing on file parts
   on nginx server, even if              remotely, network latency
   we'll never need to read              from reads (and possible cache
   it again                              coherency checks on rereads)

3. unlink on error                       unlink/rename/copy on success


* Without nginx_upload_module:

   nginx server -------------------------- unicorn server
------------------------------------------------------------------

1. sequential write of tempfile
2. sequential read of tempfile ----------> sequential write by Rack
3. unlink (able to free up cache)          do processing on file locally
                                           (no remote cache coherency
                                           checks)

The benefit of this approach is there's only 2 components interacting at
any one time, and the network costs are paid in full up front

Basically, it's the message passing concurrency model vs shared
memory+locking.  There's no clear winner, it just depends on the
situation.  99% of the time I get away with keeping everything on
one machine :)

From laas at toom.ee  Wed Oct 10 06:59:16 2012
From: laas at toom.ee (Laas Toom)
Date: Wed, 10 Oct 2012 09:59:16 +0300
Subject: Is a client uploading a file a slow client from unicorn's point
	of view?
In-Reply-To: <20121009235435.GA29908@dcvr.yhbt.net>
References: <CAHStS5iqBqJxoPZGEUaSHL2qSVf_tfZfbdedc_bRcjcTbcKwAQ@mail.gmail.com>
	<20121009015800.GA1772@dcvr.yhbt.net>
	<CFC93863-1061-4710-A195-58CBCCBED7B8@toom.ee>
	<20121009200306.GA17437@dcvr.yhbt.net>
	<3EDF9F45-B5FB-4C3B-B0E5-37C28ECFE1D1@toom.ee>
	<20121009235435.GA29908@dcvr.yhbt.net>
Message-ID: <35DC89B8-AA73-49EF-845A-50D13A4540E6@toom.ee>


On 10.10.2012, at 2:54, Eric Wong <normalperson at yhbt.net> wrote:

> Basically, it's the message passing concurrency model vs shared
> memory+locking.  There's no clear winner, it just depends on the
> situation.  99% of the time I get away with keeping everything on
> one machine :)

Exactly.
For now we have nginx and unicorn on the same machine, both interacting w/ the storage  over network, so there is no difference if the initial write is done by Rails or Nginx.

Later, Resque will pick up the background processing task, which again could be on totally separate machine(s).

And in the most complex scenario - there could be multiple Nginx, multiple Unicorn and multiple Resque nodes, each doing it's part. In such a situation, the Unicorn node can skip the network I/O, if it can base it's upload validation solely on filename and attrs, disregarding the file data itself.

Best,
Laas

From 3394374534 at tim.it  Wed Oct 10 22:56:30 2012
From: 3394374534 at tim.it (=?windows-1251?Q?=CF=F0=EE=E4=E0=E6=E0_M=EE=ED=E5=F2?=)
Date: Thu, 11 Oct 2012 5:56:30 +0700
Subject: =?windows-1251?Q?=D0=E5=E4=EA=E8=E5_=ECo=ED=E5=F2=FB?=
Message-ID: <3441779864.20121011055630@tim.it>

????: http://datasystemsworld.com/download/ber.php
E-mail: inkuzru at gmail.com
???.:  7-911-924-47-78

From normalperson at yhbt.net  Thu Oct 11 09:19:51 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 11 Oct 2012 09:19:51 +0000
Subject: [ANN] unicorn 4.4.0 - minor updates
Message-ID: <20121011091951.GA23576@dcvr.yhbt.net>

Changes:

Non-regular files are no longer reopened on SIGUSR1.  This
allows users to specify FIFOs as log destinations.

TCP_NOPUSH/TCP_CORK is no longer set/unset by default.  Use
:tcp_nopush explicitly with the "listen" directive if you wish
to enable TCP_NOPUSH/TCP_CORK.

Listen sockets are now bound _after_ loading the application for
preload_app(true) users.  This prevents load balancers from
sending traffic to an application server while the application
is still loading.

There are also minor test suite cleanups.

* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://bogomips.org/unicorn.git
* http://unicorn.bogomips.org/NEWS.atom.xml

-- 
Eric Wong

From 375gnu at gmail.com  Fri Oct 12 17:55:16 2012
From: 375gnu at gmail.com (Hleb Valoshka)
Date: Fri, 12 Oct 2012 20:55:16 +0300
Subject: [PATCH] explicitly use escaped minus in man pages
Message-ID: <1350064516-32132-1-git-send-email-375GNU@Gmail.COM>

---
 man/man1/unicorn.1       |   32 ++++++++++++++++----------------
 man/man1/unicorn_rails.1 |   34 +++++++++++++++++-----------------
 2 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/man/man1/unicorn.1 b/man/man1/unicorn.1
index 0b496af..749272a 100644
--- a/man/man1/unicorn.1
+++ b/man/man1/unicorn.1
@@ -4,7 +4,7 @@
 unicorn - a rackup-like command to launch the Unicorn HTTP server
 .SH SYNOPSIS
 .PP
-unicorn [-c CONFIG_FILE] [-E RACK_ENV] [-D] [RACKUP_FILE]
+unicorn [\-c CONFIG_FILE] [\-E RACK_ENV] [\-D] [RACKUP_FILE]
 .SH DESCRIPTION
 .PP
 A rackup(1)-like command to launch Rack applications using Unicorn.
@@ -26,7 +26,7 @@ Embedded command-line options are mostly parsed for compatibility
 with rackup(1) but strongly discouraged.
 .SH UNICORN OPTIONS
 .TP
-.B -c, --config-file CONFIG_FILE
+.B \-c, \-\-config\-file CONFIG_FILE
 Path to the Unicorn-specific config file.
 The config file is implemented as a Ruby DSL, so Ruby code may
 executed.
@@ -38,7 +38,7 @@ viewing ps(1) output.
 .RS
 .RE
 .TP
-.B -D, --daemonize
+.B \-D, \-\-daemonize
 Run daemonized in the background.
 The process is detached from the controlling terminal and stdin is
 redirected to \[lq]/dev/null\[rq].
@@ -50,13 +50,13 @@ redirected to \[lq]/dev/null\[rq].
 .RS
 .RE
 .TP
-.B -E, --env RACK_ENV
+.B \-E, \-\-env RACK_ENV
 Run under the given RACK_ENV.
 See the RACK ENVIRONMENT section for more details.
 .RS
 .RE
 .TP
-.B -l, --listen ADDRESS
+.B \-l, \-\-listen ADDRESS
 Listens on a given ADDRESS.
 ADDRESS may be in the form of HOST:PORT or PATH, HOST:PORT is taken
 to mean a TCP socket and PATH is meant to be a path to a UNIX
@@ -69,51 +69,51 @@ socket options.
 .RE
 .SH RACKUP COMPATIBILITY OPTIONS
 .TP
-.B -o, --host HOST
+.B \-o, \-\-host HOST
 Listen on a TCP socket belonging to HOST, default is
 \[lq]0.0.0.0\[rq] (all addresses).
 If specified multiple times on the command-line, only the
 last-specified value takes effect.
 This option only exists for compatibility with the rackup(1)
-command, use of \[lq]-l\[rq]/\[lq]--listen\[rq] switch is
+command, use of \[lq]\-l\[rq]/\[lq]\-\-listen\[rq] switch is
 recommended instead.
 .RS
 .RE
 .TP
-.B -p, --port PORT
+.B \-p, \-\-port PORT
 Listen on the specified TCP PORT, default is 8080.
 If specified multiple times on the command-line, only the
 last-specified value takes effect.
 This option only exists for compatibility with the rackup(1)
-command, use of \[lq]-l\[rq]/\[lq]--listen\[rq] switch is
+command, use of \[lq]\-l\[rq]/\[lq]\-\-listen\[rq] switch is
 recommended instead.
 .RS
 .RE
 .TP
-.B -s, --server SERVER
+.B \-s, \-\-server SERVER
 No-op, this exists only for compatibility with rackup(1).
 .RS
 .RE
 .SH RUBY OPTIONS
 .TP
-.B -e, --eval LINE
+.B \-e, \-\-eval LINE
 Evaluate a LINE of Ruby code.
 This evaluation happens immediately as the command-line is being
 parsed.
 .RS
 .RE
 .TP
-.B -d, --debug
+.B \-d, \-\-debug
 Turn on debug mode, the $DEBUG variable is set to true.
 .RS
 .RE
 .TP
-.B -w, --warn
+.B \-w, \-\-warn
 Turn on verbose warnings, the $VERBOSE variable is set to true.
 .RS
 .RE
 .TP
-.B -I, --include PATH
+.B \-I, \-\-include PATH
 specify $LOAD_PATH.
 PATH will be prepended to $LOAD_PATH.
 The \[aq]:\[aq] character may be used to delimit multiple
@@ -124,7 +124,7 @@ they were specified on the command-line.
 .RS
 .RE
 .TP
-.B -r, --require LIBRARY
+.B \-r, \-\-require LIBRARY
 require a specified LIBRARY before executing the application.
 The "require" statement will be executed immediately and in the
 order they were specified on the command-line.
@@ -185,7 +185,7 @@ If needed, they should be individually specified in the
 RACKUP_FILE, some frameworks do not require them.
 .SH ENVIRONMENT VARIABLES
 .PP
-The RACK_ENV variable is set by the aforementioned -E switch.
+The RACK_ENV variable is set by the aforementioned \-E switch.
 All application or library-specific environment variables (e.g.
 TMPDIR) may always be set in the Unicorn CONFIG_FILE in addition to
 the spawning shell.
diff --git a/man/man1/unicorn_rails.1 b/man/man1/unicorn_rails.1
index a593acb..c644d03 100644
--- a/man/man1/unicorn_rails.1
+++ b/man/man1/unicorn_rails.1
@@ -5,7 +5,7 @@ unicorn_rails - a script/server-like command to launch the Unicorn
 HTTP server
 .SH SYNOPSIS
 .PP
-unicorn_rails [-c CONFIG_FILE] [-E RAILS_ENV] [-D] [RACKUP_FILE]
+unicorn_rails [\-c CONFIG_FILE] [\-E RAILS_ENV] [\-D] [RACKUP_FILE]
 .SH DESCRIPTION
 .PP
 A rackup(1)-like command to launch Rails applications using
@@ -31,7 +31,7 @@ stick to the few command-line options specified in the SYNOPSIS and
 use the CONFIG_FILE as much as possible.
 .SH UNICORN OPTIONS
 .TP
-.B -c, --config-file CONFIG_FILE
+.B \-c, \-\-config-file CONFIG_FILE
 Path to the Unicorn-specific config file.
 The config file is implemented as a Ruby DSL, so Ruby code may
 executed.
@@ -43,7 +43,7 @@ viewing ps(1) output.
 .RS
 .RE
 .TP
-.B -D, --daemonize
+.B \-D, \-\-daemonize
 Run daemonized in the background.
 The process is detached from the controlling terminal and stdin is
 redirected to \[lq]/dev/null\[rq].
@@ -61,7 +61,7 @@ override this Unicorn config file.
 .RS
 .RE
 .TP
-.B -E, --env RAILS_ENV
+.B \-E, \-\-env RAILS_ENV
 Run under the given RAILS_ENV.
 This sets the RAILS_ENV environment variable.
 Acceptable values are exactly those you expect in your Rails
@@ -70,7 +70,7 @@ application, typically \[lq]development\[rq] or
 .RS
 .RE
 .TP
-.B -l, --listen ADDRESS
+.B \-l, \-\-listen ADDRESS
 Listens on a given ADDRESS.
 ADDRESS may be in the form of HOST:PORT or PATH, HOST:PORT is taken
 to mean a TCP socket and PATH is meant to be a path to a UNIX
@@ -84,28 +84,28 @@ socket options.
 .RE
 .SH RACKUP COMPATIBILITY OPTIONS
 .TP
-.B -o, --host HOST
+.B \-o, \-\-host HOST
 Listen on a TCP socket belonging to HOST, default is
 \[lq]0.0.0.0\[rq] (all addresses).
 If specified multiple times on the command-line, only the
 last-specified value takes effect.
 This option only exists for compatibility with the rackup(1)
-command, use of \[lq]-l\[rq]/\[lq]--listen\[rq] switch is
+command, use of \[lq]\-l\[rq]/\[lq]\-\-listen\[rq] switch is
 recommended instead.
 .RS
 .RE
 .TP
-.B -p, --port PORT
+.B \-p, \-\-port PORT
 Listen on the specified TCP PORT, default is 8080.
 If specified multiple times on the command-line, only the
 last-specified value takes effect.
 This option only exists for compatibility with the rackup(1)
-command, use of \[lq]-l\[rq]/\[lq]--listen\[rq] switch is
+command, use of \[lq]\-l\[rq]/\[lq]\-\-listen\[rq] switch is
 recommended instead.
 .RS
 .RE
 .TP
-.B --path PATH
+.B \-\-path PATH
 Mounts the Rails application at the given PATH (instead of
 \[lq]/\[rq]).
 This is equivalent to setting the RAILS_RELATIVE_URL_ROOT
@@ -115,26 +115,26 @@ This is only supported under Rails 2.3 or later at the moment.
 .RE
 .SH RUBY OPTIONS
 .TP
-.B -e, --eval LINE
+.B \-e, \-\-eval LINE
 Evaluate a LINE of Ruby code.
 This evaluation happens immediately as the command-line is being
 parsed.
 .RS
 .RE
 .TP
-.B -d, --debug
+.B \-d, \-\-debug
 Turn on debug mode, the $DEBUG variable is set to true.
 For Rails >= 2.3.x, this loads the \f[I]Rails::Rack::Debugger\f[]
 middleware.
 .RS
 .RE
 .TP
-.B -w, --warn
+.B \-w, \-\-warn
 Turn on verbose warnings, the $VERBOSE variable is set to true.
 .RS
 .RE
 .TP
-.B -I, --include PATH
+.B \-I, \-\-include PATH
 specify $LOAD_PATH.
 PATH will be prepended to $LOAD_PATH.
 The \[aq]:\[aq] character may be used to delimit multiple
@@ -145,7 +145,7 @@ they were specified on the command-line.
 .RS
 .RE
 .TP
-.B -r, --require LIBRARY
+.B \-r, \-\-require LIBRARY
 require a specified LIBRARY before executing the application.
 The "require" statement will be executed immediately and in the
 order they were specified on the command-line.
@@ -164,8 +164,8 @@ Embedded command-line options are mostly parsed for compatibility
 with rackup(1) but strongly discouraged.
 .SH ENVIRONMENT VARIABLES
 .PP
-The RAILS_ENV variable is set by the aforementioned -E switch.
-The RAILS_RELATIVE_URL_ROOT is set by the aforementioned --path
+The RAILS_ENV variable is set by the aforementioned \-E switch.
+The RAILS_RELATIVE_URL_ROOT is set by the aforementioned \-\-path
 switch.
 Either of these variables may also be set in the shell or the
 Unicorn CONFIG_FILE.
-- 
1.7.10.4


From 375gnu at gmail.com  Fri Oct 12 18:03:53 2012
From: 375gnu at gmail.com (Hleb Valoshka)
Date: Fri, 12 Oct 2012 21:03:53 +0300
Subject: about patch for man pages
Message-ID: <CAAB-Kckmf0AKGscbCr0y7w7RxWiPJw5_V700BEc3DTUqNfemYA@mail.gmail.com>

Lintial (tool to check debian packages) says:

====
This manual page seems to contain a hyphen where a minus sign was
intended. By default, "-" chars are interpreted as hyphens (U+2010) by
groff, not as minus signs (U+002D). Since options to programs use minus
signs (U+002D), this means for example in UTF-8 locales that you cannot
cut and paste options, nor search for them easily. The Debian groff
package currently forces "-" to be interpreted as a minus sign due to
the number of manual pages with this problem, but this is a
Debian-specific modification and hopefully eventually can be removed.

"-" must be escaped ("\-") to be interpreted as minus. If you really
intend a hyphen (normally you don't), write it as "\(hy" to emphasise
that fact. See groff(7) and especially groff_char(7) for details, and
also the thread starting with
http://lists.debian.org/debian-devel/2003/debian-devel-200303/msg01481.html
===

From normalperson at yhbt.net  Mon Oct 15 18:50:46 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 15 Oct 2012 18:50:46 +0000
Subject: [PATCH] explicitly use escaped minus in man pages
In-Reply-To: <1350064516-32132-1-git-send-email-375GNU@Gmail.COM>
References: <1350064516-32132-1-git-send-email-375GNU@Gmail.COM>
Message-ID: <20121015185046.GA24214@dcvr.yhbt.net>

Hleb Valoshka <375gnu at gmail.com> wrote:

Hi, the commit message body needs to include the reasoning behind
this change.

Something along the lines of your second email would be good.
ref: mid.gmane.org/CAAB-Kckmf0AKGscbCr0y7w7RxWiPJw5_V700BEc3DTUqNfemYA at mail.gmail.com

> ---
>  man/man1/unicorn.1       |   32 ++++++++++++++++----------------
>  man/man1/unicorn_rails.1 |   34 +++++++++++++++++-----------------

All the manpages are generated with pandoc.  Can you send a patch
for the Markdown files in Documentation/*.1.txt instead?  Thanks
in advance.

I generate manpages using pandoc 1.5.1.1-5+b1 from Debian stable
using: make -C Documentation

However, I think distributing your patch as-is with the Debian source
package is fine to avoid an extra Build-Depends on pandoc+haskell until
the next unicorn release.

From 20680535 at qq.com  Tue Oct 16 00:02:44 2012
From: 20680535 at qq.com (Richard)
Date: Tue, 16 Oct 2012 08:02:44 +0800
Subject: Racing tent
Message-ID: <mailman.0.1350345771.12398.mongrel-unicorn@rubyforge.org>

Tents Company Introduction:

To Whom It May Concern,

     We find your company's information on the internet. We would like to introduce ourselves to you. We have been in the quickup tent manufacturing business for many years and are currently in the process of expanding and our customer base.

     We are specialized in quickup tents and mountain tents. All of our ranges are displayed on our website. You can try to view our webpage http://www.tent.net.cn
     The quickup tents can be easy setup and fold within 60 second. It was widely used in market booths, fairs, racing sports, carport etc.... And they can be imprinting your logos for advertisement. We would be interested in receiving more information from you so we could submit a suitable offer to you.

We are awaiting your favorable response.

Richard Zheng
Marketing Director
XiaMen JiaoXia Trade Company
No.89, CangHong Rd, HaiCang Xia Men China(361026)
Tel: 0086-592-5553842



P.s please  let us know if you don't want to receive the mail! thank you!

From shanon at mail2reggie.com  Wed Oct 17 05:06:07 2012
From: shanon at mail2reggie.com (Jock Preston)
Date: Wed, 17 Oct 2012 09:06:07 +0400
Subject: Don't disapoint your wife at night
Message-ID: <002f01cdac25$642e0f40$45377cbc@home031d24054aibn>

She needs your love at night http://turkuolamuzik.com.tr/sugar.html


From 375gnu at gmail.com  Wed Oct 17 15:33:51 2012
From: 375gnu at gmail.com (Hleb Valoshka)
Date: Wed, 17 Oct 2012 18:33:51 +0300
Subject: [PATCH] explicitly use escaped minus in man pages
In-Reply-To: <20121015185046.GA24214@dcvr.yhbt.net>
References: <1350064516-32132-1-git-send-email-375GNU@Gmail.COM>
	<20121015185046.GA24214@dcvr.yhbt.net>
Message-ID: <CAAB-KcmCDnMQp5E=D29_iqyCB+e-krYB9AacB-iJ6+S8FF+eew@mail.gmail.com>

On 10/15/12, Eric Wong <normalperson at yhbt.net> wrote:
> All the manpages are generated with pandoc.  Can you send a patch
> for the Markdown files in Documentation/*.1.txt instead?  Thanks
> in advance.

I'll try but I'm not sure that it's possible, it seems to be only
groff-specific issue to treat minus as hyphen.

May be it's better to try to patch pandoc.

> I generate manpages using pandoc 1.5.1.1-5+b1 from Debian stable
> using: make -C Documentation

I'll try it with pandoc 1.9.4.2-2 from sid, maybe it knows groff
better than 1.5 :)

> However, I think distributing your patch as-is with the Debian source
> package is fine to avoid an extra Build-Depends on pandoc+haskell until
> the next unicorn release.

I should agree with you.

From betelgeuse at gentoo.org  Thu Oct 18 06:33:45 2012
From: betelgeuse at gentoo.org (=?ISO-8859-1?Q?Petteri_R=E4ty?=)
Date: Thu, 18 Oct 2012 09:33:45 +0300
Subject: Rack env rack.multiprocess true with single worker
Message-ID: <507FA2C9.2070300@gentoo.org>

Hi,
unicorn unconditionally sets rack.multiprocess to true in the Rack
environment. The Rack spec [0] says the following about the variable:

"true if an equivalent application object may be simultaneously invoked
by another process, false otherwise."

When unicorn is running with a single worker this does not hold so what
do you think about setting the variable to false when only a single
worker is configured? I want to use the variable to check if I can do a
HTTP call back to the application (long story) but currently with
unicorn and single worker this is not possible.

Regards,
Petteri

[0] http://rack.rubyforge.org/doc/files/SPEC.html

From normalperson at yhbt.net  Thu Oct 18 07:53:02 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 18 Oct 2012 07:53:02 +0000
Subject: Rack env rack.multiprocess true with single worker
In-Reply-To: <507FA2C9.2070300@gentoo.org>
References: <507FA2C9.2070300@gentoo.org>
Message-ID: <20121018075302.GA961@dcvr.yhbt.net>

Petteri R?ty <betelgeuse at gentoo.org> wrote:
> Hi,
> unicorn unconditionally sets rack.multiprocess to true in the Rack
> environment. The Rack spec [0] says the following about the variable:
> 
> "true if an equivalent application object may be simultaneously invoked
> by another process, false otherwise."
> 
> When unicorn is running with a single worker this does not hold so what
> do you think about setting the variable to false when only a single
> worker is configured? I want to use the variable to check if I can do a
> HTTP call back to the application (long story) but currently with
> unicorn and single worker this is not possible.

We cannot safely set rack.multiprocess=false.

Even if unicorn is started with a single worker, it is possible
to start more workers via SIGTTIN, and the first worker would
never see the change.


However, if you're certain nobody on your server will use SIGTTIN,
you can set the following anywhere in your unicorn config file:

	Unicorn::HttpRequest::DEFAULTS["rack.multiprocess"] = false

Changing the DEFAULTS constant has been and will remain supported
for as long as this project supports Rack 1.x (forever? :)


Since a single process deployment is a corner-case in production
deployments, I don't think it's worth the effort to jump through hoops
and set rack.multiprocess=false automatically.

From normalperson at yhbt.net  Thu Oct 18 08:04:05 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 18 Oct 2012 08:04:05 +0000
Subject: [PATCH] explicitly use escaped minus in man pages
In-Reply-To: <CAAB-KcmCDnMQp5E=D29_iqyCB+e-krYB9AacB-iJ6+S8FF+eew@mail.gmail.com>
References: <1350064516-32132-1-git-send-email-375GNU@Gmail.COM>
	<20121015185046.GA24214@dcvr.yhbt.net>
	<CAAB-KcmCDnMQp5E=D29_iqyCB+e-krYB9AacB-iJ6+S8FF+eew@mail.gmail.com>
Message-ID: <20121018080405.GB961@dcvr.yhbt.net>

Hleb Valoshka <375gnu at gmail.com> wrote:
> On 10/15/12, Eric Wong <normalperson at yhbt.net> wrote:
> > All the manpages are generated with pandoc.  Can you send a patch
> > for the Markdown files in Documentation/*.1.txt instead?  Thanks
> > in advance.
> 
> I'll try but I'm not sure that it's possible, it seems to be only
> groff-specific issue to treat minus as hyphen.
> 
> May be it's better to try to patch pandoc.

Yes, it is probably a bug in pandoc.

Btw, I'm not married to pandoc, either.  I'm open to migrating to ronn
(or similar) if somebody is willing to try it out.  ronn might be an
easier tool for Rubyists on various platforms to install, too, and I
likely would've chosen it if I had known about it back in the day...

> > I generate manpages using pandoc 1.5.1.1-5+b1 from Debian stable
> > using: make -C Documentation
> 
> I'll try it with pandoc 1.9.4.2-2 from sid, maybe it knows groff
> better than 1.5 :)

I have the same version on my wheezy machine, and it doesn't seem
to escape hyphens, either.

From betelgeuse at gentoo.org  Thu Oct 18 08:52:25 2012
From: betelgeuse at gentoo.org (=?UTF-8?B?UGV0dGVyaSBSw6R0eQ==?=)
Date: Thu, 18 Oct 2012 11:52:25 +0300
Subject: Rack env rack.multiprocess true with single worker
In-Reply-To: <20121018075302.GA961@dcvr.yhbt.net>
References: <507FA2C9.2070300@gentoo.org> <20121018075302.GA961@dcvr.yhbt.net>
Message-ID: <507FC349.80300@gentoo.org>

On 18.10.2012 10.53, Eric Wong wrote:
> 
> Since a single process deployment is a corner-case in production
> deployments, I don't think it's worth the effort to jump through hoops
> and set rack.multiprocess=false automatically.
> 

Do the workers currently know how many others there are? I am trying to
understand if you are saying that writing a patch would not be trivial.
If it's relative straightforward I might take a stab.

Regards,
Petteri

From datastudio41 at gmail.com  Thu Oct 18 15:59:55 2012
From: datastudio41 at gmail.com (Data Studio)
Date: Thu, 18 Oct 2012 12:59:55 -0300
Subject: =?ISO-8859-1?Q?DATA_STUDIO_-_EST=DADIO_DE_GRAVA=C7=C3O_DE_BANDAS,
	_JINGLES_E_SPOTS_PUBLICIT=C1RIOS.?=
Message-ID: <10181259.YCIDAWWH@gmail.com>

OL?, VENHO ATRAV?S DESTE E-MAIL APRESENTAR O DATA STUDIO:
PARA CONSULTAS DE OR?AMENTOS DE JINGLES, SPOTS, VINHETAS, LOCU??ES E TAMB?M CONVERS?O DE VINIL E CASSETE PARA CD E VHS PARA DVD, BASTA ENTRAR EM CONTATO POR ESTE E-MAIL OU PELO SITE.
QUEM TIVER INTERESSE TENHO UM PORTIF?LIO DE AUDIO NO ENDERE?O DO SITE, BASTA ADICIONAR O FACEBOOK DO EST?DIO.
FACEBOOK  http://www.facebook.com/datastudio.estudiodegravacao
GRATO PELA ATEN??O.



From normalperson at yhbt.net  Thu Oct 18 17:38:56 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 18 Oct 2012 17:38:56 +0000
Subject: Rack env rack.multiprocess true with single worker
In-Reply-To: <507FC349.80300@gentoo.org>
References: <507FA2C9.2070300@gentoo.org> <20121018075302.GA961@dcvr.yhbt.net>
	<507FC349.80300@gentoo.org>
Message-ID: <20121018173856.GA22944@dcvr.yhbt.net>

Petteri R?ty <betelgeuse at gentoo.org> wrote:
> On 18.10.2012 10.53, Eric Wong wrote:
> > 
> > Since a single process deployment is a corner-case in production
> > deployments, I don't think it's worth the effort to jump through hoops
> > and set rack.multiprocess=false automatically.
> > 
> 
> Do the workers currently know how many others there are? I am trying to
> understand if you are saying that writing a patch would not be trivial.
> If it's relative straightforward I might take a stab.

No, workers don't keep track of other workers and it's not easy to
support this.

The workaround with the DEFAULTS hash is your best option.

From dbenhur at whitepages.com  Thu Oct 18 18:11:38 2012
From: dbenhur at whitepages.com (Devin Ben-Hur)
Date: Thu, 18 Oct 2012 11:11:38 -0700
Subject: Rack env rack.multiprocess true with single worker
In-Reply-To: <20121018173856.GA22944@dcvr.yhbt.net>
References: <507FA2C9.2070300@gentoo.org> <20121018075302.GA961@dcvr.yhbt.net>
	<507FC349.80300@gentoo.org> <20121018173856.GA22944@dcvr.yhbt.net>
Message-ID: <5080465A.7080106@whitepages.com>

On 10/18/2012 10:38 AM, Eric Wong wrote:
> Petteri R?ty <betelgeuse at gentoo.org> wrote:
>> On 18.10.2012 10.53, Eric Wong wrote:
>>> Since a single process deployment is a corner-case in production
>>> deployments, I don't think it's worth the effort to jump through hoops
>>> and set rack.multiprocess=false automatically.
>>
>> Do the workers currently know how many others there are? I am trying to
>> understand if you are saying that writing a patch would not be trivial.
>> If it's relative straightforward I might take a stab.
>
> No, workers don't keep track of other workers and it's not easy to
> support this.
>
> The workaround with the DEFAULTS hash is your best option.

I think's Eric's suggested workaround is right, it is a bunch of work to 
make workers track other workers to little benefit; but one could cheat 
and use ps to find sibling processes (children of the worker's parent 
which should be the unicorn master):

def num_workers
   `ps -ef`.
   split("\n").
   map{ |line| line.split(' ',4)[2].to_i }.
   select{ |ppid| ppid == Process.ppid }.size
end

From aditya.sanghi at risingsuntech.net  Wed Oct 24 10:12:35 2012
From: aditya.sanghi at risingsuntech.net (Aditya Sanghi)
Date: Wed, 24 Oct 2012 15:42:35 +0530
Subject: Handling timeouts?
Message-ID: <F277EBC4-61EB-46CA-9E51-C5A92DA9CA2B@risingsuntech.net>

Hi Guys,

Thanks for Unicorn, it makes my world rock. You guy have done a wonderful job and also huge thanks to the predecessors too.

The default timeout on my Rails app is 30 seconds. The app works fine most of the time but there are some parts of the app which are slow and occasionally totter over the 30 sec response time and timeout. Is there a clean way for me to hook into timeouts and notify myself when timeouts occur? Something like a chance to email myself or send an airbrake notification?

Cheers,
Aditya

From mcupples at cloudspace.com  Wed Oct 24 13:25:20 2012
From: mcupples at cloudspace.com (Matthew Cupples)
Date: Wed, 24 Oct 2012 09:25:20 -0400
Subject: Handling timeouts?
In-Reply-To: <F277EBC4-61EB-46CA-9E51-C5A92DA9CA2B@risingsuntech.net>
References: <F277EBC4-61EB-46CA-9E51-C5A92DA9CA2B@risingsuntech.net>
Message-ID: <CA+EW9occp1BuDcAmxHOw8UUFZehiqMUKfbG4E250Z8QpP81gHQ@mail.gmail.com>

I just handle it by having NRPE grep through the Nginx error log;
that's where you'll see the timeout come in.

Example timeout:
2012/10/22 19:52:10 [error] 2071#0: *7269 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client: 70.127.166.167, server: www.domain.com, request: "GET
/follow_outlink?url=http%3A%2F%2Fblogs.orlandosentinel.com%2Fnews_politics%2F2012%2F10%2Frace-for-hd-49-gets-nastier.html&name=Sentinel%3A%20Race%20for%20HD%2049%20gets%20nastier
HTTP/1.1", upstream:
"http://unix:/etc/unicorn/domain.sock:/follow_outlink?url=http%3A%2F%2Fblogs.orlandosentinel.com%2Fnews_politics%2F2012%2F10%2Frace-for-hd-49-gets-nastier.html&name=Sentinel%3A%20Race%20for%20HD%2049%20gets%20nastier",
host: "www.domain.com", referrer: "http://www.domain.com/"

On 10/24/12, Aditya Sanghi <aditya.sanghi at risingsuntech.net> wrote:
> Hi Guys,
>
> Thanks for Unicorn, it makes my world rock. You guy have done a wonderful
> job and also huge thanks to the predecessors too.
>
> The default timeout on my Rails app is 30 seconds. The app works fine most
> of the time but there are some parts of the app which are slow and
> occasionally totter over the 30 sec response time and timeout. Is there a
> clean way for me to hook into timeouts and notify myself when timeouts
> occur? Something like a chance to email myself or send an airbrake
> notification?
>
> Cheers,
> Aditya
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

From lawrence.pit at gmail.com  Mon Oct 29 00:07:54 2012
From: lawrence.pit at gmail.com (Lawrence Pit)
Date: Mon, 29 Oct 2012 11:07:54 +1100
Subject: Handling timeouts?
In-Reply-To: <F277EBC4-61EB-46CA-9E51-C5A92DA9CA2B@risingsuntech.net>
References: <F277EBC4-61EB-46CA-9E51-C5A92DA9CA2B@risingsuntech.net>
Message-ID: <2BD3D349-1E06-4E12-A292-10ED1AE24ECC@gmail.com>

Hi Aditya

> The default timeout on my Rails app is 30 seconds. The app works fine 
> most of the time but there are some parts of the app which are slow 
> and occasionally totter over the 30 sec response time and timeout. Is 
> there a clean way for me to hook into timeouts and notify myself when 
> timeouts occur? Something like a chance to email myself or send an 
> airbrake notification?

In your unicorn config.rb file, this example would email a snapshot of 
top output when a worker times out:

before_fork do |server, worker|
   `tail -n 5 "#{log_path}/unicorn.log"`.split("\n").each do |line|
     next unless line =~ /SIGKILL.*worker=(\d+)$/
     worker_nr = $1
     next unless worker_nr = worker.nr
     `top -n1 -b | mail -s "#{line}" sysadmin at example.com`
     break
   end

   # ?

end


Cheers,
Lawrence

From tom.burns at jadedpixel.com  Mon Oct 29 17:44:27 2012
From: tom.burns at jadedpixel.com (Tom Burns)
Date: Mon, 29 Oct 2012 13:44:27 -0400
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
Message-ID: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>

Hi,

We're dealing with an issue with our large-scale deployment of unicorn
& nginx.  The problem occurs during "flash" scenarios where we receive
an order magnitude more traffic for up to an hour.  Much of this
traffic cannot be cached and it's typical for some of our rails
responses to take a few seconds to generate.  Our preference is to
queue incoming requests instead of returning 502 if we can respond
within a reasonable amount of time.

On each of our servers the stack is nginx -> unicorn.

The main connection queue in front of Rails is the unicorn connection
queue.  Our problem is that when the traffic hits, the unicorn queue
grows.  When users begin hitting refresh, their abandoned requests in
the unicorn queue are still passed to the rails application and
rendered.  In this case we see a 200 HTTP response in our Rails log
and a 499 in the nginx access log.  Once this starts happening the
problem can compound: app workers are busy rendering pages for clients
who have already detached so response time grows and more users hit
refresh, etc.

Our nginx config:
6 nginx workers, 1024 worker connections.

Our unicorn config:
70 unicorn workers, 2048 connection backlog.

Our goal is to not have our app process requests for clients that have
already disconnected while their connection is still queued in
unicorn.  We also would prefer not to shrink our queue such that we
begin to return 502 when our queue is a few seconds deep.

We're looking at potential solutions to this problem, including:
- modifying unicorn to select() on the client connection after reading
the request, to see if it's been closed upstream, and avoid calling
the app.
- Replacing nginx with haproxy and queuing connections there.  This
goes against the nginx recommendation at
http://unicorn.bogomips.org/PHILOSOPHY.html

Any input would be appreciated.

Thanks,
Tom
Developer @ Shopify

From normalperson at yhbt.net  Mon Oct 29 18:45:49 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 29 Oct 2012 18:45:49 +0000
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
Message-ID: <20121029184549.GA9690@dcvr.yhbt.net>

Tom Burns <tom.burns at jadedpixel.com> wrote:
> We're looking at potential solutions to this problem, including:
> - modifying unicorn to select() on the client connection after reading
> the request, to see if it's been closed upstream, and avoid calling
> the app.

You should be able to get the socket out of
Unicorn::TeeInput/Unicorn::StreamInput object via:

  env["rack.input"].instance_variable_get(:@socket)

Make sure you don't have other middleware which wraps rack.input
such as Rack::Lint.  I highly doubt the ivar name will change in
the future.

If you're accepting uploads, you (or your app framework) will need to
drain the socket of upload data, first (Rails does this for you, I
think).

> - Replacing nginx with haproxy and queuing connections there.  This
> goes against the nginx recommendation at
> http://unicorn.bogomips.org/PHILOSOPHY.html

Haproxy is fine as long as you have nginx /somewhere/ in between unicorn
and clients.  You have some extra overhead in data copying, but it could
save you cycles...

I'm unsure about the ordering, however:

a) nginx -> haproxy -> unicorn
b) haproxy -> nginx -> unicorn

Though, I suspect a) will be better.

> Any input would be appreciated.

I'd love to hear back on how you eventually solve this, too :>

From hongli at phusion.nl  Mon Oct 29 19:27:09 2012
From: hongli at phusion.nl (Hongli Lai)
Date: Mon, 29 Oct 2012 20:27:09 +0100
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <20121029184549.GA9690@dcvr.yhbt.net>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
Message-ID: <CAM3ce8F4j4_h4BmzgAUAJaEU-jOw6VJZczs=Zq3SjSgQbm0xGw@mail.gmail.com>

On Mon, Oct 29, 2012 at 7:45 PM, Eric Wong <normalperson at yhbt.net> wrote:
>> Any input would be appreciated.
>
> I'd love to hear back on how you eventually solve this, too :>

Does haproxy support removing clients from the queue like this?

-- 
Phusion | Ruby & Rails deployment, scaling and tuning solutions

Web: http://www.phusion.nl/
E-mail: info at phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

From normalperson at yhbt.net  Mon Oct 29 19:41:01 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 29 Oct 2012 19:41:01 +0000
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <CAM3ce8F4j4_h4BmzgAUAJaEU-jOw6VJZczs=Zq3SjSgQbm0xGw@mail.gmail.com>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
	<CAM3ce8F4j4_h4BmzgAUAJaEU-jOw6VJZczs=Zq3SjSgQbm0xGw@mail.gmail.com>
Message-ID: <20121029194101.GA10907@dcvr.yhbt.net>

Hongli Lai <hongli at phusion.nl> wrote:
> On Mon, Oct 29, 2012 at 7:45 PM, Eric Wong <normalperson at yhbt.net> wrote:
> >> Any input would be appreciated.
> >
> > I'd love to hear back on how you eventually solve this, too :>
> 
> Does haproxy support removing clients from the queue like this?

I'm not sure, Tom can investigate further.

I do recall haproxy having more intelligent load distribution than
nginx, so it can send requests to less-busy machines, at least.

From hongli at phusion.nl  Mon Oct 29 21:06:43 2012
From: hongli at phusion.nl (Hongli Lai)
Date: Mon, 29 Oct 2012 22:06:43 +0100
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <20121029194101.GA10907@dcvr.yhbt.net>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
	<CAM3ce8F4j4_h4BmzgAUAJaEU-jOw6VJZczs=Zq3SjSgQbm0xGw@mail.gmail.com>
	<20121029194101.GA10907@dcvr.yhbt.net>
Message-ID: <CAM3ce8HT3su-VP9R=bPGxUiq4XahJdgx8uCNkhR0uyb_vLyeBw@mail.gmail.com>

On Mon, Oct 29, 2012 at 8:41 PM, Eric Wong <normalperson at yhbt.net> wrote:
> I'm not sure, Tom can investigate further.
>
> I do recall haproxy having more intelligent load distribution than
> nginx, so it can send requests to less-busy machines, at least.

It is correct that HAProxy can do more intelligent load distribution.
The "maxconn 1" feature in HAProxy is a bit like global queuing in
Phusion Passenger, or the kernel socket queuing that Unicorn relies
on. Phusion Passenger sends the request to the first worker that
becomes available, and the kernel passes the connection to the first
Unicorn instance that accept()s the socket. Nginx's proxy_module
passes requests immediately and only does round robin.

So to solve this problem we need to add 2 features to the layer that
manages the queue:
1. It must be able to detect an early disconnect.
2. It must be able to remove a client from the queue.

The kernel obviously can't do this, but I'm very curious as to whether
HAProxy supports these two things. I also think it shouldn't be too
hard to implement this in Phusion Passenger 4's new architecture.

-- 
Phusion | Ruby & Rails deployment, scaling and tuning solutions

Web: http://www.phusion.nl/
E-mail: info at phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

From normalperson at yhbt.net  Mon Oct 29 21:53:12 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 29 Oct 2012 21:53:12 +0000
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <20121029184549.GA9690@dcvr.yhbt.net>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
Message-ID: <20121029215312.GA29353@dcvr.yhbt.net>

Eric Wong <normalperson at yhbt.net> wrote:
> Tom Burns <tom.burns at jadedpixel.com> wrote:
> > We're looking at potential solutions to this problem, including:
> > - modifying unicorn to select() on the client connection after reading
> > the request, to see if it's been closed upstream, and avoid calling
> > the app.
> 
> You should be able to get the socket out of
> Unicorn::TeeInput/Unicorn::StreamInput object via:
> 
>   env["rack.input"].instance_variable_get(:@socket)
> 
> Make sure you don't have other middleware which wraps rack.input
> such as Rack::Lint.  I highly doubt the ivar name will change in
> the future.
> 
> If you're accepting uploads, you (or your app framework) will need to
> drain the socket of upload data, first (Rails does this for you, I
> think).

Maybe this gross hack can work for you guys.  It writes the first
chunk of the HTTP response header out immediately after reading
the request headers, and sends the rest once it gets the status...

It probably needs :tcp_nodelay => false for it to be reliable, though.
(I don't think I need each_byte below, two separate write()s should
be enough Nagle)

It screws up error/exception reporting in some cases, though:

diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index b3d8d71..818fac8 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
@@ -33,7 +33,7 @@ module Unicorn::Const
   ERROR_414_RESPONSE = "HTTP/1.1 414 Request-URI Too Long\r\n\r\n"
   ERROR_413_RESPONSE = "HTTP/1.1 413 Request Entity Too Large\r\n\r\n"
   ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n"
-  EXPECT_100_RESPONSE = "HTTP/1.1 100 Continue\r\n\r\n"
+  EXPECT_100_RESPONSE = "100 Continue\r\n\r\n HTTP/1.1"
 
   HTTP_EXPECT = "HTTP_EXPECT"
   # :startdoc:
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index a0435d6..73fbd41 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -70,6 +70,12 @@ class Unicorn::HttpParser
       # an Exception thrown from the parser will throw us out of the loop
       false until add_parse(socket.kgio_read!(16384))
     end
+
+    # detect if the socket is valid by writing a partial response:
+    if headers?
+      "HTTP/1.1 ".each_char { |c| socket.write(c) }
+    end
+
     e[RACK_INPUT] = 0 == content_length ?
                     NULL_IO : @@input_class.new(socket, self)
     e.merge!(DEFAULTS)
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index b781e20..c2e9d1d 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -22,7 +22,7 @@ module Unicorn::HttpResponse
     status = CODES[status.to_i] || status
 
     if headers
-      buf = "HTTP/1.1 #{status}\r\n" \
+      buf = "#{status}\r\n" \
             "Date: #{httpdate}\r\n" \
             "Status: #{status}\r\n" \
             "Connection: close\r\n"

From tom.burns at jadedpixel.com  Mon Oct 29 22:21:23 2012
From: tom.burns at jadedpixel.com (Tom Burns)
Date: Mon, 29 Oct 2012 18:21:23 -0400
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <20121029215312.GA29353@dcvr.yhbt.net>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
	<20121029215312.GA29353@dcvr.yhbt.net>
Message-ID: <CAK4qKG2Znt946ZknCr9jyChRwJYRZq=H_Mw2dGxeOJ_rh1-W0w@mail.gmail.com>

On Mon, Oct 29, 2012 at 5:53 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Maybe this gross hack can work for you guys.  It writes the first
> chunk of the HTTP response header out immediately after reading
> the request headers, and sends the rest once it gets the status...

Eric, thank you very much for your replies.  We'd debated this as an
alternate solution along with the two I mentioned in my original
email, and to be honest your tentative patch is cleaner than I'd had
expected this solution to look like :)

I will test this and respond back.

One of our goals in solving this problem would be to get any changes
merged back into unicorn master, and this looks like it would actually
lead to a cleaner result than having to select() on the socket.

Another side effect of the "select() in a middleware" solution was
going to be removing the NULL_IO optimization that sets rack.input to
a StringIO.

Cheers,
Tom

From tom.burns at jadedpixel.com  Tue Oct 30 20:40:23 2012
From: tom.burns at jadedpixel.com (Tom Burns)
Date: Tue, 30 Oct 2012 16:40:23 -0400
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <20121029215312.GA29353@dcvr.yhbt.net>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
	<20121029215312.GA29353@dcvr.yhbt.net>
Message-ID: <CAK4qKG0UPqkbM8HYccx0moNJ7WsCf-e4uu6Eh6DGNvcn58BLYg@mail.gmail.com>

On Mon, Oct 29, 2012 at 5:53 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Maybe this gross hack can work for you guys.  It writes the first
> chunk of the HTTP response header out immediately after reading
> the request headers, and sends the rest once it gets the status...

I tested the patch today and it does what we want, dropping
connections before passing them to the rails app when the client has
already disconnected.

I also benchmarked the patch to see if it had a negligible performance
hit and it did not.  The cost was absorbed by the variation in speed
of the other components in the stack (nginx & rails).

I noticed on my computer applying the patch breaks
test_rack_lint_big_put in the unicorn test suite.  This may be just my
issue as the test suite does not run cleanly anyways if I checkout
origin/master.

We'd prefer to not have to fork unicorn for this change.  How do you
feel about merging this or a derivative thereof?  I can develop this
further if you can send me what you'd want.

Cheers,
Tom

From normalperson at yhbt.net  Tue Oct 30 21:37:19 2012
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 30 Oct 2012 14:37:19 -0700
Subject: Combating nginx 499 HTTP responses during flash traffic scenario
In-Reply-To: <CAK4qKG0UPqkbM8HYccx0moNJ7WsCf-e4uu6Eh6DGNvcn58BLYg@mail.gmail.com>
References: <CAK4qKG32dGgNbzTzCb6NoH5hu_DrHMOFaFhk-6Xvy-T86++ukA@mail.gmail.com>
	<20121029184549.GA9690@dcvr.yhbt.net>
	<20121029215312.GA29353@dcvr.yhbt.net>
	<CAK4qKG0UPqkbM8HYccx0moNJ7WsCf-e4uu6Eh6DGNvcn58BLYg@mail.gmail.com>
Message-ID: <20121030213719.GA6701@dcvr.yhbt.net>

Tom Burns <tom.burns at jadedpixel.com> wrote:
> On Mon, Oct 29, 2012 at 5:53 PM, Eric Wong <normalperson at yhbt.net> wrote:
> > Maybe this gross hack can work for you guys.  It writes the first
> > chunk of the HTTP response header out immediately after reading
> > the request headers, and sends the rest once it gets the status...
> 
> I tested the patch today and it does what we want, dropping
> connections before passing them to the rails app when the client has
> already disconnected.
> 
> I also benchmarked the patch to see if it had a negligible performance
> hit and it did not.  The cost was absorbed by the variation in speed
> of the other components in the stack (nginx & rails).

Good to know.  Thanks for reporting back.

> I noticed on my computer applying the patch breaks
> test_rack_lint_big_put in the unicorn test suite.  This may be just my
> issue as the test suite does not run cleanly anyways if I checkout
> origin/master.

The test suite in master should be passing cleanly, at least on a
GNU/Linux machine...

Yes, this hacky patch breaks some tests/internals and screws up
exception error/reporting badly.

> We'd prefer to not have to fork unicorn for this change.  How do you
> feel about merging this or a derivative thereof?  I can develop this
> further if you can send me what you'd want.

Sure thing!

I strongly prefer this to be optional behavior and off-by-default.

Also, I'm nearly certain two write()s is all that's needed and the
each_char is unnecessary syscall/packet overhead.

This will trigger bugs in badly-written HTTP clients/parsers (probably
some test suites :x) which assume a header comes in a single read().

For TCP users, I believe this requires both tcp_nodelay:false and
tcp_nopush:false to be completely reliable, so we need to enforce that
those if this option is in effect.  The current each_char usage is
probably masking the tcp_nodelay:false requirement.

Thanks again.