From noreply at www.eyari.com  Wed Aug  4 07:04:24 2010
From: noreply at www.eyari.com (eYari)
Date: Wed, 4 Aug 2010 11:04:24 +0000
Subject: [eYari] New friendship request from Sonia Gupta
Message-ID: <7bb27ecdc364228667e4e4b131b50cb1@www.eyari.com>

Sonia Gupta wants to add you as a friend.

To view all of your pending friendship requests: http://www.eyari.com/mongrel-unicorn/friends/requests/

To view Sonia Gupta's profile: http://www.eyari.com/soniagupta/

---------------------
To disable these notifications please log in and go to: http://www.eyari.com/mongrel-unicorn/settings/notifications


From noreply at www.eyari.com  Wed Aug  4 07:04:24 2010
From: noreply at www.eyari.com (eYari)
Date: Wed, 4 Aug 2010 11:04:24 +0000
Subject: [eYari] New message from Sonia Gupta
Message-ID: <d4073562497b5c3b405db689ca1d0397@www.eyari.com>

Sonia Gupta sent you a new message:

Subject: Hello

"Hello, I would like to be your friend. Please tell me about yourself. What kind of things do you like to do?  - Sonia"

To view and read your messages please log in and visit: http://www.eyari.com/mongrel-unicorn/messages/

---------------------


From noreply at www.eyari.com  Wed Aug  4 08:53:24 2010
From: noreply at www.eyari.com (eYari)
Date: Wed, 4 Aug 2010 12:53:24 +0000
Subject: [eYari] New friendship request from susanna jaber
Message-ID: <87fe701e38d262ee09103a7c42156deb@www.eyari.com>

susanna jaber wants to add you as a friend.

To view all of your pending friendship requests: http://www.eyari.com/mongrel-unicorn/friends/requests/

To view susanna jaber's profile: http://www.eyari.com/susanna8227/

---------------------
To disable these notifications please log in and go to: http://www.eyari.com/mongrel-unicorn/settings/notifications


From oct at fotopedia.com  Fri Aug  6 10:11:42 2010
From: oct at fotopedia.com (Pierre Baillet)
Date: Fri, 6 Aug 2010 16:11:42 +0200
Subject: Number of simultaneous clients
Message-ID: <AANLkTik4YPRjHBiL_x0tbDuh+NzLYf5HAkd4RfQYG4er@mail.gmail.com>

Hello,

We've working on some improvement on our caching system and while
testing it, I encountered errors between Varnish and Unicorn. Namely,
Varnish claims that there is no backend connection and will not send
any data from Unicorn.

After some investigations (tcpdump), I found out that sometimes, I got
no reply from my backend server. See for example this complete
conversation between my Varnish server and my Unicorn server:

GET /items/422qq2nh931gk-aqXuxXfs/esi/card/sitemap/nil.xml/http HTTP/1.1
Accept: */*
Host: www.testing.ftnz.net
X-Forwarded-For: 127.0.0.1
Accept-Encoding: identity
X-Ftn-Is-Logged: no
X-Varnish: 1063393845


Yes, that's all. The last packet sent by the backend is an ACK to the
data packet sent by Varnish.

This happens when I really overload the Varnish with incoming
requests. I suspect it opens a lot of connection to our backend. But I
fail to see why Unicorn would not reply (even at a slow rate). Of
course, the issue can be reproduced more or less easily. Trying to
reproduce the issue by directly hitting the Unicorn has not succeeded.

Any thoughts ? Cheers,
-- 
Pierre Baillet <oct at fotopedia.com>
http://www.fotopedia.com/

From normalperson at yhbt.net  Fri Aug  6 17:05:48 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Fri, 6 Aug 2010 14:05:48 -0700
Subject: Number of simultaneous clients
In-Reply-To: <AANLkTik4YPRjHBiL_x0tbDuh+NzLYf5HAkd4RfQYG4er@mail.gmail.com>
References: <AANLkTik4YPRjHBiL_x0tbDuh+NzLYf5HAkd4RfQYG4er@mail.gmail.com>
Message-ID: <20100806210548.GA21336@dcvr.yhbt.net>

Pierre Baillet <oct at fotopedia.com> wrote:
> Hello,
> 
> We've working on some improvement on our caching system and while
> testing it, I encountered errors between Varnish and Unicorn. Namely,
> Varnish claims that there is no backend connection and will not send
> any data from Unicorn.
> 
> After some investigations (tcpdump), I found out that sometimes, I got
> no reply from my backend server. See for example this complete
> conversation between my Varnish server and my Unicorn server:
> 
> GET /items/422qq2nh931gk-aqXuxXfs/esi/card/sitemap/nil.xml/http HTTP/1.1
> Accept: */*
> Host: www.testing.ftnz.net
> X-Forwarded-For: 127.0.0.1
> Accept-Encoding: identity
> X-Ftn-Is-Logged: no
> X-Varnish: 1063393845
> 
> 
> Yes, that's all. The last packet sent by the backend is an ACK to the
> data packet sent by Varnish.

Hi Pierre,

Do you have any timeouts configured Varnish that would cause it to not
wait if Unicorn is overloaded?

Can you strace/truss the Unicorn process and see if it's actually
accepting that particular request and doing something with it?

Also, enable access logs for Unicorn (Rack::CommonLogger or Clogger[1])
and see if the requests are actually going through it.

To make it easier to track Varnish requests, you can easily customize
the Clogger format to include the X-Varnish header by adding
$http_x_varnish (or any other header) to the :format parameter:

  use Clogger,
        :format => Clogger::Format::Combined + ' $http_x_varnish',
        :logger => ::File.open("/path/to/log", "ab")
  run MyApp.new

[1] - http://clogger.rubyforge.org/

> This happens when I really overload the Varnish with incoming
> requests. I suspect it opens a lot of connection to our backend. But I
> fail to see why Unicorn would not reply (even at a slow rate). Of
> course, the issue can be reproduced more or less easily. Trying to
> reproduce the issue by directly hitting the Unicorn has not succeeded.
> 
> Any thoughts ? Cheers,

What's your listen :backlog parameter set to?

If the backlog fills up (the actual value inside the kernel is usually
higher than what we set it to), then the kernel will start rejecting
connections.  You got an ACK for one particular request, but perhaps
others are not getting it...

We default the listen backlog to 1024 which is pretty high, but for
benchmarking some apps you can increase that.  In Linux, you may need
to increase the net.core.somaxconn sysctl, too.

Keep in mind that if you through enough load at *anything*, it'll get
overloaded and stop working well.

For nginx users, I recommend using fail_timeout=0 so nginx will always
try requests anyways (it's cheap) and never mark it as down for any
period.

-- 
Eric Wong

From normalperson at yhbt.net  Mon Aug  9 18:54:29 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 9 Aug 2010 15:54:29 -0700
Subject: [ANN] 1.9 users: socket_dontwait - MSG_DONTWAIT socket methods
Message-ID: <20100809225429.GB12996@dcvr.yhbt.net>

Hi all,

I initially announced[1] this on the Rainbows! mailing list since
Rainbows! is more experimental in nature, but I've been running this for
a few days with real traffic (with Rainbows!) and it seems safe enough
for less crazy people to use :)

If you're using Ruby 1.9.1, then this library includes a fix for
errno getting zeroed and hitting rb_bug() during rb_sys_fail()
This is fixed in Ruby trunk r27401, but 1.9.1 doesn't have this.

This library is a drop-in replacement that reimplements several IO
methods with replacements using MSG_DONTWAIT for BasicSocket.  This
allows us to avoid unnecessary system calls and GVL bouncing.

[1] - http://mid.gmane.org/20100803091847.GC3255 at dcvr.yhbt.net

The rest of the README below:

We've reimplemented the +readpartial+, +read_nonblock+,
+write_nonblock+, +read+ and +write+ instance methods normally inherited
from the IO class directly into BasicSocket with socket-specific system
calls and flags.

This library is only intended for Ruby 1.9 and will not build with other
versions of Ruby.  This only supports operating systems with the
non-POSIX MSG_DONTWAIT flag for send(2) and recv(2) syscalls.

This library is considered EXPERIMENTAL.  If successful, we'll see about
getting them integrated into the standard Ruby socket library.

== Features

* Avoid use of fcntl(2) to set O_NONBLOCK in favor of MSG_DONTWAIT when
  using non-blocking I/O.  We _unset_ O_NONBLOCK if we need to block
  and release the GVL instead of relying on select(2).

* Avoids select(2) entirely in favor of blocking I/O when the
  GVL is released.  This allows using file descriptor numbers higher
  than 1023 without overflowing select(2) buffers.

* BasicSocket#read uses recv(2) with MSG_WAITALL to avoid extra system
  calls for larger reads.

* Thread and signal-safe, releases the GVL for all blocking operations
  and retries if system calls are interrupted.

* Includes a 1.9.1-specific workaround to preserve errno after reacquiring
  the GVL.  This is
  {fixed}[http://redmine.ruby-lang.org/repositories/diff/ruby-19?rev=27401]
  in newer versions of Ruby.

* Falls back to line-buffered IO if needed (not recommended).

== Bugs/Caveats

* We ignore taint/$SAFE checks, we'll support it if there's demand,
  but we doubt there is...

* Does not support 1.9 encoding filters.  1.9 defaults all sockets to
  Encoding::BINARY anyways, so this should not be noticeable to code
  that leaves socket encodings untouched.

* Does not support write buffering in userspace.  Ruby defaults all
  sockets to "IO#sync = true", anyways so this does not affect code
  that leaves the default setting untouched.

* Avoid using line-buffered IO on sockets (IO#gets, IO#each_line),
  nearly all of the features of this library are cancelled out when
  the line-buffering fallback is used.

== Install

If you're using a packaged Ruby distribution, make sure you have a C
compiler and the matching Ruby development libraries and headers.
You need Ruby 1.9 to install socket_dontwait.  Previous versions of
Ruby will NOT be supported.

If you use RubyGems:

    gem install socket_dontwait

Otherwise grab the latest tarball from:

http://bogomips.org/socket_dontwait/files/

Unpack it, and run "ruby setup.rb"

== Development

You can get the latest source via git from the following locations:

  git://git.bogomips.org/socket_dontwait.git
  git://repo.or.cz/socket_dontwait.git (mirror)

You may browse the code from the web and download the latest snapshot
tarballs here:

* http://git.bogomips.org/cgit/socket_dontwait.git (cgit)
* http://repo.or.cz/w/socket_dontwait.git (gitweb)

Inline patches (from "git format-patch") to the mailing list are
preferred because they allow code review and comments in the reply to
the patch.

We will adhere to mostly the same conventions for patch submissions as
git itself.  See the Documentation/SubmittingPatches document
distributed with git on on patch submission guidelines to follow.  Just
don't email the git mailing list or maintainer with socket_dontwait
patches.

== Contact/Bug Reports/Feedback/Patches/Pull-Requests

This was originally created for the Rainbows! project (but may be used by
others), so we'll reuse their mailing list at
{rainbows-talk at rubyforge.org}[mailto:rainbows-talk at rubyforge.org].

-- 
Eric Wong

From normalperson at yhbt.net  Thu Aug 12 14:52:49 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 12 Aug 2010 18:52:49 +0000
Subject: How to configure Unicorn on Ubuntu 10.04
In-Reply-To: <AANLkTikoc15s+3v4ROAfWRY8NdDq1izgN8AWtea9Gogt@mail.gmail.com>
References: <AANLkTimarMAxOsRVUi_wjqmATQCYUz43+w8KJwSSzgOK@mail.gmail.com>
	<AANLkTikoc15s+3v4ROAfWRY8NdDq1izgN8AWtea9Gogt@mail.gmail.com>
Message-ID: <20100812185249.GA559@dcvr.yhbt.net>

Ranjith Kumar <ranjithtenz at gmail.com> wrote:
> Hi All,
>      I`m using ubuntu 10.04 I have installed Unicorn I need to know how to
> configure the unicorn from "validator.w3.org". I can`t add JVM parameters
> with unicorn point. tell me how to add Unicorn with JVM parameter.

Huh?  Wrong mailing list, I think. Perhaps this is what you're looking for?
  http://code.w3.org/unicorn/#Feedback

This mailing list is for the Unicorn HTTP application server[1], and is
completely unrelated to the validator.  I didn't even know about the
Unicorn validator until now :x

[1] http://unicorn.bogomips.org/

-- 
Eric Wong

From singhi.surendra at gmail.com  Sun Aug 15 11:50:33 2010
From: singhi.surendra at gmail.com (Surendra Singhi)
Date: Sun, 15 Aug 2010 21:20:33 +0530
Subject: unicorn behind apache with file uploads
Message-ID: <AANLkTinqP0P4HQLdRL+tLjAs4M=dXY=_ysd-z+n3qrdy@mail.gmail.com>

Hi,

For one of our apps we switched to unicorn from mongrel.
The unicorn server use to run behind an apache with timeout of 300sec,
as sometime the customer may do a large file upload.

After we made the switch we started getting lots of
Unicorn::ClientShutdown occurred in photos#flash_upload
bytes_read=2762663 (pr some other number)
We were not able to recreate it in our tests, but support was getting
customer emails, that the files upload is not working.
It finally forced us to switch back to mongrel.

I know a fully buffering reverse proxy like nginx is advised as
reverse proxy for unicorn, but apache has been serving us well, and we
don't want to change it.
Any idea what can be going wrong, or why was unicorn throwing these errors?

Thanks!
-- 
Surendra

From normalperson at yhbt.net  Sun Aug 15 18:56:12 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sun, 15 Aug 2010 22:56:12 +0000
Subject: unicorn behind apache with file uploads
In-Reply-To: <AANLkTinqP0P4HQLdRL+tLjAs4M=dXY=_ysd-z+n3qrdy@mail.gmail.com>
References: <AANLkTinqP0P4HQLdRL+tLjAs4M=dXY=_ysd-z+n3qrdy@mail.gmail.com>
Message-ID: <20100815225612.GA23536@dcvr.yhbt.net>

Surendra Singhi <singhi.surendra at gmail.com> wrote:
> Hi,
> 
> For one of our apps we switched to unicorn from mongrel.
> The unicorn server use to run behind an apache with timeout of 300sec,
> as sometime the customer may do a large file upload.
> 
> After we made the switch we started getting lots of
> Unicorn::ClientShutdown occurred in photos#flash_upload
> bytes_read=2762663 (pr some other number)
> We were not able to recreate it in our tests, but support was getting
> customer emails, that the files upload is not working.
> It finally forced us to switch back to mongrel.

Hi Surendra,

Could you be hitting a timeout in your Apache config?  Can you try
simulating a slow connection to reproduce it?  Apache defaults to 300s,
too, so double check your config and make sure it's not different
anymore.   The other thing could be your kernel listen queue used by
Unicorn is completely full and Apache is just giving up on certain
requests...

> I know a fully buffering reverse proxy like nginx is advised as
> reverse proxy for unicorn, but apache has been serving us well, and we
> don't want to change it.
> Any idea what can be going wrong, or why was unicorn throwing these errors?

While it's generally bad for Unicorn to run behind anything other than
nginx, "generally bad" does not apply to the case of file uploads.

Honestly, when dealing with file uploads from slow clients,
Unicorn is the WORST SERVER YOU COULD EVER PICK.  EVER :)

Really, trust me on this one :)


Use nginx, but if you insist on using Apache, then you're far better off
sticking with Mongrel or trying out Rainbows![1] for uploads.

Rainbows! has the advantage of using Unicorn's prefork and shared
listeners along with your choice of threads/fibers/events/fibers/actors.

[1] - http://rainbows.rubyforge.org/

-- 
Eric Wong

From superjoe at gmail.com  Mon Aug 16 21:29:34 2010
From: superjoe at gmail.com (Joseph McDonald)
Date: Mon, 16 Aug 2010 18:29:34 -0700
Subject: running unicorn under rvm
Message-ID: <AANLkTimQRE-sAyd0X9xpPUmjj6QGJdYPcu3SKkaLwCqM@mail.gmail.com>

unicorn 1.1.2 runs fine under my system ruby, but when trying to run
it using rvm, I get:


% rvm use 1.8.7
info: Using ruby 1.8.7 p299

% unicorn
/home/joe/.rvm/gems/ruby-1.8.7-p299/gems/unicorn-1.1.2/lib/unicorn/configurator.rb:494:in
`eval': undefined local variable or method `host' for main:Object
(NameError)

line 494 looks like:

 # XXX ugly as hell, WILL FIX in 2.x (along with Rainbows!/Zbatery)
    host, port, set_listener, options, daemonize =
                    eval("[ host, port, set_listener, options, daemonize ]",
                         TOPLEVEL_BINDING)

any ideas why it would have a problem when running under rvm?

thanks,
-joe

From normalperson at yhbt.net  Mon Aug 16 23:52:02 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 16 Aug 2010 20:52:02 -0700
Subject: running unicorn under rvm
In-Reply-To: <AANLkTimQRE-sAyd0X9xpPUmjj6QGJdYPcu3SKkaLwCqM@mail.gmail.com>
References: <AANLkTimQRE-sAyd0X9xpPUmjj6QGJdYPcu3SKkaLwCqM@mail.gmail.com>
Message-ID: <20100817035202.GA2570@dcvr.yhbt.net>

Joseph McDonald <superjoe at gmail.com> wrote:
> unicorn 1.1.2 runs fine under my system ruby, but when trying to run
> it using rvm, I get:
> 
> 
> % rvm use 1.8.7
> info: Using ruby 1.8.7 p299
> 
> % unicorn
> /home/joe/.rvm/gems/ruby-1.8.7-p299/gems/unicorn-1.1.2/lib/unicorn/configurator.rb:494:in
> `eval': undefined local variable or method `host' for main:Object
> (NameError)

Hi Joseph,

Is there any chance you have an older "unicorn" executable script
somewhere that gets exposed rvm switches the environment on you?  That's
the only thing I can think of right now.

> line 494 looks like:
> 
>  # XXX ugly as hell, WILL FIX in 2.x (along with Rainbows!/Zbatery)

Yes, I still need to work on 2.x and clean up a lot of the
ugly internal API bits :)


-- 
Eric Wong

P.S.: At least nowadays I'm (finally) getting some experience
developing/supporting applications using Unicorn and Rainbows! (or
Zbatery).  Things like e4d0b226391948ef433f1d0135814315e4c48535 in
unicorn.git are a direct result of that.

From superjoe at gmail.com  Tue Aug 17 18:37:18 2010
From: superjoe at gmail.com (Joseph McDonald)
Date: Tue, 17 Aug 2010 15:37:18 -0700
Subject: running unicorn under rvm
In-Reply-To: <20100817035202.GA2570@dcvr.yhbt.net>
References: <AANLkTimQRE-sAyd0X9xpPUmjj6QGJdYPcu3SKkaLwCqM@mail.gmail.com>
	<20100817035202.GA2570@dcvr.yhbt.net>
Message-ID: <AANLkTim3YJOG4fnsE6nLfygEo_7u4h6Haoo118ck-9JQ@mail.gmail.com>

Thanks Eric, see below:

On Mon, Aug 16, 2010 at 8:52 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Joseph McDonald <superjoe at gmail.com> wrote:
>> unicorn 1.1.2 runs fine under my system ruby, but when trying to run
>> it using rvm, I get:
>>
>>
>> % rvm use 1.8.7
>> info: Using ruby 1.8.7 p299
>>
>> % unicorn
>> /home/joe/.rvm/gems/ruby-1.8.7-p299/gems/unicorn-1.1.2/lib/unicorn/configurator.rb:494:in
>> `eval': undefined local variable or method `host' for main:Object
>> (NameError)
>
> Hi Joseph,
>
> Is there any chance you have an older "unicorn" executable script
> somewhere that gets exposed rvm switches the environment on you? ?That's
> the only thing I can think of right now.

I'm sure I'm running the right version of unicorn.
what's weird is if I remove the reference to TOPLEVEL_BINDING, it works, i.e.:

 host, port, set_listener, options, daemonize =
                    eval("[ host, port, set_listener, options, daemonize ]")

works, it gets the right port from the config.ru file.

my config.ru file looks like"
#\ -w -p 4452
require './mystuff'
run Sinatra::Application

another strange thing:  I need to add the "./"  in front of mystuff,
but when i'm not using rvm (rvm use system), I don't need to do that.

I'm curious if anyone else is using unicorn with rvm and not having
this problem.  I'm guessing it's something weird with my system, but
not sure.

thanks,
-joe





>
>> line 494 looks like:
>>
>> ?# XXX ugly as hell, WILL FIX in 2.x (along with Rainbows!/Zbatery)
>
> Yes, I still need to work on 2.x and clean up a lot of the
> ugly internal API bits :)
>
>
> --
> Eric Wong
>
> P.S.: At least nowadays I'm (finally) getting some experience
> developing/supporting applications using Unicorn and Rainbows! (or
> Zbatery). ?Things like e4d0b226391948ef433f1d0135814315e4c48535 in
> unicorn.git are a direct result of that.
>

From jimmy.soho at gmail.com  Tue Aug 17 19:38:08 2010
From: jimmy.soho at gmail.com (Jimmy Soho)
Date: Wed, 18 Aug 2010 09:38:08 +1000
Subject: javascript disappears
Message-ID: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>

Hi All,

When I deploy new code I always go through the USR2 - QUIT process.
However, after a few deployments it seems our base javascript file is
missing. I've experienced this a few times now. We use rails 2.3.8,
and have something like this in our layouts file:

<%= javascript_include_tag 'webtoolkit', 'jquery', 'jquery.metadata',
'jquery.url_utils', 'jquery.form', 'jquery.hotkeys',
'jquery.livequery', 'application', :cache => 'base'  %>

When it's missing it doesn't matter how often I kill with USR2 / QUIT,
it won't appear. If I stop the unicorn master process completely and
then start again, then it does create the base.js file again.

Anyone experienced this as well?  Anyone know what could be the reason for this?



Jimmy

From jimmy.soho at gmail.com  Tue Aug 17 20:02:33 2010
From: jimmy.soho at gmail.com (Jimmy Soho)
Date: Wed, 18 Aug 2010 10:02:33 +1000
Subject: javascript disappears
In-Reply-To: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
Message-ID: <AANLkTi=Yk68Ju=UzP6hfYStooiJXDFk7GdKBGh86Z4R-@mail.gmail.com>

Btw, we also had this weird thing last week where after a new
deployment of code almost all field labels had the same value. We use
I18n v0.4.1 with a cache using memory_store, like so:

I18n::Backend::Simple.send(:include, I18n::Backend::Cache)
I18n.cache_store = ActiveSupport::Cache.lookup_store(:memory_store)
I18n.cache_store.silence!

I get the feeling that memory from the previous master/workers somehow
isn't released and is used by the new unicorn master and/or worker
processes.


Jimmy

From normalperson at yhbt.net  Tue Aug 17 21:08:50 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 17 Aug 2010 18:08:50 -0700
Subject: running unicorn under rvm
In-Reply-To: <AANLkTim3YJOG4fnsE6nLfygEo_7u4h6Haoo118ck-9JQ@mail.gmail.com>
References: <AANLkTimQRE-sAyd0X9xpPUmjj6QGJdYPcu3SKkaLwCqM@mail.gmail.com>
	<20100817035202.GA2570@dcvr.yhbt.net>
	<AANLkTim3YJOG4fnsE6nLfygEo_7u4h6Haoo118ck-9JQ@mail.gmail.com>
Message-ID: <20100818010850.GA14094@dcvr.yhbt.net>

Joseph McDonald <superjoe at gmail.com> wrote:
> I'm sure I'm running the right version of unicorn.
> what's weird is if I remove the reference to TOPLEVEL_BINDING, it works, i.e.:
> 
>  host, port, set_listener, options, daemonize =
>                     eval("[ host, port, set_listener, options, daemonize ]")
> 
> works, it gets the right port from the config.ru file.

Hm... I'm stumped.  However, you're really encouraging me to pause other
projects, work on 2.0 and release it this week/weekend :)  I suppose
some of the crazier 2.0 stuff I proposed can wait until 3.x.

> my config.ru file looks like"
> #\ -w -p 4452
> require './mystuff'
> run Sinatra::Application
> 
> another strange thing:  I need to add the "./"  in front of mystuff,
> but when i'm not using rvm (rvm use system), I don't need to do that.

Odd... needing the './' feels like it's using Ruby 1.9.2 or later...
Can you print out RUBY_VERSION in there to be certain?

I just tried rvm for the first time with a fresh user/home directory on my
system and everything works as expected with 1.8.7 (it just picked p302, not
p299, but that shouldn't make a difference...).

> I'm curious if anyone else is using unicorn with rvm and not having
> this problem.  I'm guessing it's something weird with my system, but
> not sure.

If you have root on your machine, just try creating a new user and start
with a blank $HOME.  I often do that to narrow things down when
debugging/testing code and don't feel like going all the way with chroot
or VM.

-- 
Eric Wong

From normalperson at yhbt.net  Tue Aug 17 21:14:44 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 17 Aug 2010 18:14:44 -0700
Subject: javascript disappears
In-Reply-To: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
Message-ID: <20100818011444.GA14052@dcvr.yhbt.net>

Jimmy Soho <jimmy.soho at gmail.com> wrote:
> Hi All,
> 
> When I deploy new code I always go through the USR2 - QUIT process.
> However, after a few deployments it seems our base javascript file is
> missing. I've experienced this a few times now. We use rails 2.3.8,
> and have something like this in our layouts file:
> 
> <%= javascript_include_tag 'webtoolkit', 'jquery', 'jquery.metadata',
> 'jquery.url_utils', 'jquery.form', 'jquery.hotkeys',
> 'jquery.livequery', 'application', :cache => 'base'  %>
> 
> When it's missing it doesn't matter how often I kill with USR2 / QUIT,
> it won't appear. If I stop the unicorn master process completely and
> then start again, then it does create the base.js file again.
> 
> Anyone experienced this as well?  Anyone know what could be the reason
> for this?

Hi Jimmy,   Are you deploying with Capistrano and you originally started
Unicorn in a working_directory that no longer exists on the filesystem?

Under Linux, you can check the working directory of any process
with `ls -l /proc/$PID/cwd`

That's the only thing I can think of at the moment.

In your other email:
> I get the feeling that memory from the previous master/workers somehow
> isn't released and is used by the new unicorn master and/or worker
> processes.

Shouldn't be possible unless your OS is broken.  A more likely
explanation is that Unicorn process is pinned to the working
directory of an old deploy.

-- 
Eric Wong

From superjoe at gmail.com  Tue Aug 17 21:22:24 2010
From: superjoe at gmail.com (Joseph McDonald)
Date: Tue, 17 Aug 2010 18:22:24 -0700
Subject: running unicorn under rvm
In-Reply-To: <20100817035202.GA2570@dcvr.yhbt.net>
References: <AANLkTimQRE-sAyd0X9xpPUmjj6QGJdYPcu3SKkaLwCqM@mail.gmail.com>
	<20100817035202.GA2570@dcvr.yhbt.net>
Message-ID: <AANLkTinAmcqR8nzE9Wn=xPykFZ3KZmz-J2LoNLYmeOuJ@mail.gmail.com>

I added
TOPLEVEL_BINDING = binding
to gems/unicorn-1.1.2/bin/unicorn
and it seemed to fix it:

#!/this/will/be/overwritten/or/wrapped/anyways/do/not/worry/ruby
# -*- encoding: binary -*-
require 'unicorn/launcher'
require 'optparse'
# !!!!!!!!!  added the line below:
TOPLEVEL_BINDING = binding
ENV["RACK_ENV"] ||= "development"

I still have no idea why I need to do that.

thanks,
-joe


On Mon, Aug 16, 2010 at 8:52 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Joseph McDonald <superjoe at gmail.com> wrote:
>> unicorn 1.1.2 runs fine under my system ruby, but when trying to run
>> it using rvm, I get:
>>
>>
>> % rvm use 1.8.7
>> info: Using ruby 1.8.7 p299
>>
>> % unicorn
>> /home/joe/.rvm/gems/ruby-1.8.7-p299/gems/unicorn-1.1.2/lib/unicorn/configurator.rb:494:in
>> `eval': undefined local variable or method `host' for main:Object
>> (NameError)
>
> Hi Joseph,
>
> Is there any chance you have an older "unicorn" executable script
> somewhere that gets exposed rvm switches the environment on you? ?That's
> the only thing I can think of right now.
>
>> line 494 looks like:
>>
>> ?# XXX ugly as hell, WILL FIX in 2.x (along with Rainbows!/Zbatery)
>
> Yes, I still need to work on 2.x and clean up a lot of the
> ugly internal API bits :)
>
>
> --
> Eric Wong
>
> P.S.: At least nowadays I'm (finally) getting some experience
> developing/supporting applications using Unicorn and Rainbows! (or
> Zbatery). ?Things like e4d0b226391948ef433f1d0135814315e4c48535 in
> unicorn.git are a direct result of that.
>

From jimmy.soho at gmail.com  Tue Aug 17 22:23:59 2010
From: jimmy.soho at gmail.com (Jimmy Soho)
Date: Wed, 18 Aug 2010 12:23:59 +1000
Subject: javascript disappears
In-Reply-To: <20100818011444.GA14052@dcvr.yhbt.net>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
	<20100818011444.GA14052@dcvr.yhbt.net>
Message-ID: <AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>

> Hi Jimmy, ? Are you deploying with Capistrano and you originally started
> Unicorn in a working_directory that no longer exists on the filesystem?
>
> Under Linux, you can check the working directory of any process
> with `ls -l /proc/$PID/cwd`

Ah! Yes. I use Capistrano, and it pruned releases. But definitely
/after/ a new version was deployed.

What I don't understand then though is that I have this in my unicorn
config file:

working_directory "/srv/app1/current"

How could the new master / workers still be pointing to the old
directory? Isn't what this working_directory directive is for?



Jimmy

From normalperson at yhbt.net  Tue Aug 17 22:45:40 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 17 Aug 2010 19:45:40 -0700
Subject: javascript disappears
In-Reply-To: <AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
	<20100818011444.GA14052@dcvr.yhbt.net>
	<AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>
Message-ID: <20100818024540.GA29950@dcvr.yhbt.net>

Jimmy Soho <jimmy.soho at gmail.com> wrote:
> > Hi Jimmy, ? Are you deploying with Capistrano and you originally started
> > Unicorn in a working_directory that no longer exists on the filesystem?
> >
> > Under Linux, you can check the working directory of any process
> > with `ls -l /proc/$PID/cwd`
> 
> Ah! Yes. I use Capistrano, and it pruned releases. But definitely
> /after/ a new version was deployed.
> 
> What I don't understand then though is that I have this in my unicorn
> config file:
> 
> working_directory "/srv/app1/current"
> 
> How could the new master / workers still be pointing to the old
> directory? Isn't what this working_directory directive is for?

In your Capistrano deploy tasks, do you create/update the current
symlink before you send USR2?  You should have the symlink updated
before sending USR2..

Other folks here may have more experience with Capistrano+Unicorn,
maybe they have portable recipes they can share.

-- 
Eric Wong

From jimmy.soho at gmail.com  Tue Aug 17 23:39:44 2010
From: jimmy.soho at gmail.com (Jimmy Soho)
Date: Wed, 18 Aug 2010 13:39:44 +1000
Subject: javascript disappears
In-Reply-To: <20100818024540.GA29950@dcvr.yhbt.net>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
	<20100818011444.GA14052@dcvr.yhbt.net>
	<AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>
	<20100818024540.GA29950@dcvr.yhbt.net>
Message-ID: <AANLkTikvpbqX9UZjM1tMbZYNqBZGHZGUL9P-JDak49of@mail.gmail.com>

> In your Capistrano deploy tasks, do you create/update the current
> symlink before you send USR2? ?You should have the symlink updated
> before sending USR2..

Yes I do.

Meanwhile I've been able to indeed reproduce the issue I was seeing if
I prune releases. This is what I do:

- ls /srv/app/releases shows only:

  /srv/app/releases/20100818022900

- stop unicorn
- start unicorn
- deploy new release (same code actually)
- unicorn now has a different PID, /proc/31761/cwd does point to the
latest release directory:

  /proc/31761/cwd -> /srv/app/releases/20100818024514

- refresh browser, ok, no issue.
- rm -Rf /srv/app/releases/20100818022900
- deploy another new release
- refresh browser: javascripts/base.js is missing

nothing in unicorn.stderr.log, but looking in unicorn.stdout.log it mentions:

  /srv/app/releases/20100818022900/Gemfile not found

The unicorn process PID hasn't changed, nor has proc/31761/cwd

My /srv/app/current dir now points to the latest release. Which I
guess is wrong as well, as unicorn failed to start with the latest
release; so it (= capistrano) needs to rollback somehow (which is a
different issue).

If I now peek in /proc/31761/environ then I notice these:

  GEM_HOME=/srv/app/releases/20100818022900/vendor/bundler_gems
  PATH=/srv/app/releases/20100818022900/vendor/bundler_gems/bin:[... etc.]
  GEM_PATH=/srv/app/releases/20100818022900/vendor/bundler_gems
  BUNDLE_GEMFILE=/srv/app/releases/20100818022900/Gemfile

which are all pointing to the working directory I pruned. Hence the
message about Gemfile not found I guess.

Any suggestions how / when to update those environment variables with
correct values?


> Other folks here may have more experience with Capistrano+Unicorn,
> maybe they have portable recipes they can share.

Maybe it's more a gem and/or bundler thing?  Though it seems like a
general issue to me that environment variables can have references to
old working directories, which need to be updated if you restart
Unicorn with USR2 all the time.  Is this something Unicorn could
possibly take care off?


Jimmy

From normalperson at yhbt.net  Wed Aug 18 03:14:10 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 18 Aug 2010 00:14:10 -0700
Subject: javascript disappears
In-Reply-To: <AANLkTikvpbqX9UZjM1tMbZYNqBZGHZGUL9P-JDak49of@mail.gmail.com>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
	<20100818011444.GA14052@dcvr.yhbt.net>
	<AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>
	<20100818024540.GA29950@dcvr.yhbt.net>
	<AANLkTikvpbqX9UZjM1tMbZYNqBZGHZGUL9P-JDak49of@mail.gmail.com>
Message-ID: <20100818071410.GB29950@dcvr.yhbt.net>

Jimmy Soho <jimmy.soho at gmail.com> wrote:
> If I now peek in /proc/31761/environ then I notice these:
> 
>   GEM_HOME=/srv/app/releases/20100818022900/vendor/bundler_gems
>   PATH=/srv/app/releases/20100818022900/vendor/bundler_gems/bin:[... etc.]
>   GEM_PATH=/srv/app/releases/20100818022900/vendor/bundler_gems
>   BUNDLE_GEMFILE=/srv/app/releases/20100818022900/Gemfile
> 
> which are all pointing to the working directory I pruned. Hence the
> message about Gemfile not found I guess.
> 
> Any suggestions how / when to update those environment variables with
> correct values?

Aha!  I believe there's an option for Bundler to use the /srv/app/shared
directory with Capistrano.  That's probably your best option.

> > Other folks here may have more experience with Capistrano+Unicorn,
> > maybe they have portable recipes they can share.
> 
> Maybe it's more a gem and/or bundler thing?  Though it seems like a
> general issue to me that environment variables can have references to
> old working directories, which need to be updated if you restart
> Unicorn with USR2 all the time.  Is this something Unicorn could
> possibly take care off?

You can also try force the environment variables with the
before_exec hook in your config file:

  before_exec do |server|
    ENV['GEM_HOME'] = '/srv/app/current/vendor/bundler_gems'
  end

But I think the best option is to use a shared directory for Bundler
since it should speed up deploys, too.  On the other hand, maybe Bundler
should be taught to respect symlink directories...

-- 
Eric Wong

From jimmy.soho at gmail.com  Wed Aug 18 09:20:49 2010
From: jimmy.soho at gmail.com (Jimmy Soho)
Date: Wed, 18 Aug 2010 23:20:49 +1000
Subject: javascript disappears
In-Reply-To: <20100818071410.GB29950@dcvr.yhbt.net>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
	<20100818011444.GA14052@dcvr.yhbt.net>
	<AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>
	<20100818024540.GA29950@dcvr.yhbt.net>
	<AANLkTikvpbqX9UZjM1tMbZYNqBZGHZGUL9P-JDak49of@mail.gmail.com>
	<20100818071410.GB29950@dcvr.yhbt.net>
Message-ID: <AANLkTinP7=NJY8X2zZiyyh=nhJohdnCjRyNVf=vKqOfO@mail.gmail.com>

> Aha!  I believe there's an option for Bundler to use the /srv/app/shared
> directory with Capistrano.  That's probably your best option.

I am using the shared bundler gems option, I use the example floating
around the internets:

run("cd #{release_path} && bundle install vendor/bundler_gems
--without development test --disable-shared-gems")

> You can also try force the environment variables with the
> before_exec hook in your config file:
>
>  before_exec do |server|
>    ENV['GEM_HOME'] = '/srv/app/current/vendor/bundler_gems'
>  end

After a few hours hacking I've come to the conclusion that's the only
way it can work, for now.

Having delved into bundler I think Bundler.setup needs to change. I'll
file an issue for that with them.


PS. I: for those interested, this is now the unicorn-conf.rb file I'm
using, so I can now happily prune all old releases:

http://gist.github.com/534668


PS. II: While I trying out various things I found that if I modified
the before_exec hook, then deployed that code, send USR2, then it
executes the before_exec hook of the old master, not the new master.
So I then had to send a USR2 a second time to see the new code in
before_exec taking effect. Is that intentional behaviour?


PS. III: Haven't tested it yet, but in case a USR2 fails, and unicorn
reverts back to the old master, does the old master then have the new
environment variable values? Or does it run the before_exec hook again
for the old master?



Jimmy

From normalperson at yhbt.net  Wed Aug 18 19:34:00 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 18 Aug 2010 16:34:00 -0700
Subject: javascript disappears
In-Reply-To: <AANLkTinP7=NJY8X2zZiyyh=nhJohdnCjRyNVf=vKqOfO@mail.gmail.com>
References: <AANLkTimi_P9G9Gp0M+rHPjsUQfG8FPDAXoMoxcqMx8Lv@mail.gmail.com>
	<20100818011444.GA14052@dcvr.yhbt.net>
	<AANLkTinWs+WM9sd1796S2YpEsbLTQa_1V6xbBQG_LL6=@mail.gmail.com>
	<20100818024540.GA29950@dcvr.yhbt.net>
	<AANLkTikvpbqX9UZjM1tMbZYNqBZGHZGUL9P-JDak49of@mail.gmail.com>
	<20100818071410.GB29950@dcvr.yhbt.net>
	<AANLkTinP7=NJY8X2zZiyyh=nhJohdnCjRyNVf=vKqOfO@mail.gmail.com>
Message-ID: <20100818233400.GA11128@dcvr.yhbt.net>

Jimmy Soho <jimmy.soho at gmail.com> wrote:
> PS. II: While I trying out various things I found that if I modified
> the before_exec hook, then deployed that code, send USR2, then it
> executes the before_exec hook of the old master, not the new master.
> So I then had to send a USR2 a second time to see the new code in
> before_exec taking effect. Is that intentional behaviour?

Yes.  USR2 doesn't reload the config file before by default.  Send a HUP
before USR2 if you want it to reload the config file.  Just keep in mind
if you add any new dependencies on your new deploy and HUP, things could
break, too, especially if you're using "preload_app true".

> PS. III: Haven't tested it yet, but in case a USR2 fails, and unicorn
> reverts back to the old master, does the old master then have the new
> environment variable values? Or does it run the before_exec hook again
> for the old master?

The old master doesn't get affected by environment (or any other local)
changes in before_exec.  before_exec only runs in the new master
process, so it can only affect the new master process (unless it sends
signals to the old master).

-- 
Eric Wong

From normalperson at yhbt.net  Tue Aug 24 15:58:13 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Tue, 24 Aug 2010 12:58:13 -0700
Subject: Unicorn and home_run
In-Reply-To: <AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
References: <mailman.96.1282670516.23553.mongrel-unicorn@rubyforge.org>
	<AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
Message-ID: <20100824195813.GA10622@dcvr.yhbt.net>

Clifton King <cliftonk at gmail.com> wrote:
> I got a "The message's content type was not explicitly allowed" when trying
> to send an email to the group.

Hi, we only allow plain text and patches on the mailing list, no HTML.

> > From: Clifton King <cliftonk at gmail.com>
> > To: mongrel-unicorn at rubyforge.org
> > Date: Tue, 24 Aug 2010 12:21:49 -0500
> > Subject: Unicorn and home_run
> > Hi All,
> >
> > I was trying to get home_run (a C-based Date/DateTime implementation)
> > working with Unicorn http://github.com/jeremyevans/home_run, but using
> > home_run --install then running unicorn_rails would crash with the following
> > stack trace on a Ruby 1.8.7 CentOS staging server:
> > http://pastie.org/1111144
> >
> > I believe this has to do with the order in which ruby's default libraries
> > are loaded when running unicorn_rails. Jeremy Evans (home_run's author) is
> > available to answer any questions specific to his libraries.

Are you starting it with "home_run unicorn_rails" or did you install
it system-wide?

Jeremy: would it be easier for home_run to just require the core-Ruby
classes first and then monkey patch the methods individually?  I haven't
gotten a chance to look into home_run myself much.

-- 
Eric Wong

From cliftonk at gmail.com  Tue Aug 24 16:39:25 2010
From: cliftonk at gmail.com (Clifton King)
Date: Tue, 24 Aug 2010 15:39:25 -0500
Subject: Unicorn and home_run
In-Reply-To: <20100824195813.GA10622@dcvr.yhbt.net>
References: <mailman.96.1282670516.23553.mongrel-unicorn@rubyforge.org>
	<AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
	<20100824195813.GA10622@dcvr.yhbt.net>
Message-ID: <AANLkTinxaP5qj_Z5SSYbWh=BF2skcw7SPWn9cKi2iYDY@mail.gmail.com>

Eric,

Starting it with "home_run unicorn_rails" does work.

Using home_run --install and then unicorn_rails does not.

We use preload_app in production and I doubt that anything except the
home_run --install method would work for us. Using irb, script/server
(mongrel), etc work fine using home_run --install.

Thanks,
Clifton

On Tue, Aug 24, 2010 at 2:58 PM, Eric Wong <normalperson at yhbt.net> wrote:
>
> Clifton King <cliftonk at gmail.com> wrote:
> > I got a "The message's content type was not explicitly allowed" when trying
> > to send an email to the group.
>
> Hi, we only allow plain text and patches on the mailing list, no HTML.
>
> > > From: Clifton King <cliftonk at gmail.com>
> > > To: mongrel-unicorn at rubyforge.org
> > > Date: Tue, 24 Aug 2010 12:21:49 -0500
> > > Subject: Unicorn and home_run
> > > Hi All,
> > >
> > > I was trying to get home_run (a C-based Date/DateTime implementation)
> > > working with Unicorn http://github.com/jeremyevans/home_run, but using
> > > home_run --install then running unicorn_rails would crash with the following
> > > stack trace on a Ruby 1.8.7 CentOS staging server:
> > > http://pastie.org/1111144
> > >
> > > I believe this has to do with the order in which ruby's default libraries
> > > are loaded when running unicorn_rails. Jeremy Evans (home_run's author) is
> > > available to answer any questions specific to his libraries.
>
> Are you starting it with "home_run unicorn_rails" or did you install
> it system-wide?
>
> Jeremy: would it be easier for home_run to just require the core-Ruby
> classes first and then monkey patch the methods individually? ?I haven't
> gotten a chance to look into home_run myself much.
>
> --
> Eric Wong

From jeremyevans0 at gmail.com  Tue Aug 24 17:38:17 2010
From: jeremyevans0 at gmail.com (Jeremy Evans)
Date: Tue, 24 Aug 2010 14:38:17 -0700
Subject: Unicorn and home_run
In-Reply-To: <20100824195813.GA10622@dcvr.yhbt.net>
References: <mailman.96.1282670516.23553.mongrel-unicorn@rubyforge.org>
	<AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
	<20100824195813.GA10622@dcvr.yhbt.net>
Message-ID: <AANLkTin5V8NOOTecaj2FNAw9H89Rr7Ab3-VtjNx+PZkH@mail.gmail.com>

On Tue, Aug 24, 2010 at 12:58 PM, Eric Wong <normalperson at yhbt.net> wrote:
> Clifton King <cliftonk at gmail.com> wrote:
>> I got a "The message's content type was not explicitly allowed" when trying
>> to send an email to the group.
>
> Hi, we only allow plain text and patches on the mailing list, no HTML.
>
>> > From: Clifton King <cliftonk at gmail.com>
>> > To: mongrel-unicorn at rubyforge.org
>> > Date: Tue, 24 Aug 2010 12:21:49 -0500
>> > Subject: Unicorn and home_run
>> > Hi All,
>> >
>> > I was trying to get home_run (a C-based Date/DateTime implementation)
>> > working with Unicorn http://github.com/jeremyevans/home_run, but using
>> > home_run --install then running unicorn_rails would crash with the following
>> > stack trace on a Ruby 1.8.7 CentOS staging server:
>> > http://pastie.org/1111144
>> >
>> > I believe this has to do with the order in which ruby's default libraries
>> > are loaded when running unicorn_rails. Jeremy Evans (home_run's author) is
>> > available to answer any questions specific to his libraries.
>
> Are you starting it with "home_run unicorn_rails" or did you install
> it system-wide?

Clifton told me that was working for him.  One of the reasons it may
work where the other way didn't is that in addition to setting up the
load path, it also requires the library up front.

I think this problem is solved in the master branch.  The problem is
that some libraries require date/format without requiring date.  In
the standard library date/format.rb does not require date, where it
does in home_run.  I believe the reason for this error is that
previously, in date/format.rb, I was doing:

  require 'date' unless defined?(Date)

But if something opens up the Date class to monkey-patch, it never
requires 'date', since the Date class then exists.  It appears that
json, syck, and rubygems all do this.  home_run now does:

  require 'date' unless defined?(Date::ZONES)

Which should fix the issue, as only home_run should be defining that.

> Jeremy: would it be easier for home_run to just require the core-Ruby
> classes first and then monkey patch the methods individually? ?I haven't
> gotten a chance to look into home_run myself much.

home_run is designed as a complete replacement for the standard date
library.  It's not a monkey patching solution like Date::Performance,
so it wouldn't be easier to just require and then overwrite.

Jeremy

From normalperson at yhbt.net  Wed Aug 25 03:50:56 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 25 Aug 2010 00:50:56 -0700
Subject: Unicorn and home_run
In-Reply-To: <AANLkTin5V8NOOTecaj2FNAw9H89Rr7Ab3-VtjNx+PZkH@mail.gmail.com>
References: <mailman.96.1282670516.23553.mongrel-unicorn@rubyforge.org>
	<AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
	<20100824195813.GA10622@dcvr.yhbt.net>
	<AANLkTin5V8NOOTecaj2FNAw9H89Rr7Ab3-VtjNx+PZkH@mail.gmail.com>
Message-ID: <20100825075056.GA14658@dcvr.yhbt.net>

Jeremy Evans <jeremyevans0 at gmail.com> wrote:
> > Are you starting it with "home_run unicorn_rails" or did you install
> > it system-wide?
> 
> Clifton told me that was working for him.  One of the reasons it may
> work where the other way didn't is that in addition to setting up the
> load path, it also requires the library up front.

Yeah, I figured using the "home_run" wrapper would be less than
ideal since it would get lost during exec upgrades and newer
versions of home_run wouldn't get picked up the same way it'd get
picked up in a system install.

> I think this problem is solved in the master branch.

Thanks.  I'd be eager to hear how it works out since this could be a
popular library very soon :) Is there a home_run mailing list I could
follow?

-- 
Eric Wong

From jeremyevans0 at gmail.com  Wed Aug 25 10:46:08 2010
From: jeremyevans0 at gmail.com (Jeremy Evans)
Date: Wed, 25 Aug 2010 07:46:08 -0700
Subject: Unicorn and home_run
In-Reply-To: <20100825075056.GA14658@dcvr.yhbt.net>
References: <mailman.96.1282670516.23553.mongrel-unicorn@rubyforge.org>
	<AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
	<20100824195813.GA10622@dcvr.yhbt.net>
	<AANLkTin5V8NOOTecaj2FNAw9H89Rr7Ab3-VtjNx+PZkH@mail.gmail.com>
	<20100825075056.GA14658@dcvr.yhbt.net>
Message-ID: <AANLkTimNQDZK5AvTbMb85wXur4GyOgXtiW5G=-htc4ZG@mail.gmail.com>

On Wed, Aug 25, 2010 at 12:50 AM, Eric Wong <normalperson at yhbt.net> wrote:
> Jeremy Evans <jeremyevans0 at gmail.com> wrote:
>> > Are you starting it with "home_run unicorn_rails" or did you install
>> > it system-wide?
>>
>> Clifton told me that was working for him. ?One of the reasons it may
>> work where the other way didn't is that in addition to setting up the
>> load path, it also requires the library up front.
>
> Yeah, I figured using the "home_run" wrapper would be less than
> ideal since it would get lost during exec upgrades and newer
> versions of home_run wouldn't get picked up the same way it'd get
> picked up in a system install.

The home_run wrapper works by modifying the environment (RUBYLIB and
RUBYOPT). Does the environment get reset during exec upgrades?

You are correct that newer versions would not get picked up, as it's
going to hard code a specific gem version.

>> I think this problem is solved in the master branch.
>
> Thanks. ?I'd be eager to hear how it works out since this could be a
> popular library very soon :) Is there a home_run mailing list I could
> follow?

I haven't set up a mailing list for it (you are the first person to
ask about one), but I've been hanging out by myself in #home_run on
freenode (IRC).

Jeremy

From normalperson at yhbt.net  Thu Aug 26 00:33:39 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Wed, 25 Aug 2010 21:33:39 -0700
Subject: Unicorn and home_run
In-Reply-To: <AANLkTimNQDZK5AvTbMb85wXur4GyOgXtiW5G=-htc4ZG@mail.gmail.com>
References: <mailman.96.1282670516.23553.mongrel-unicorn@rubyforge.org>
	<AANLkTingOCVVTyCDbA8RC=Vf7LV=HyCMhtqcJ4RLg9Xy@mail.gmail.com>
	<20100824195813.GA10622@dcvr.yhbt.net>
	<AANLkTin5V8NOOTecaj2FNAw9H89Rr7Ab3-VtjNx+PZkH@mail.gmail.com>
	<20100825075056.GA14658@dcvr.yhbt.net>
	<AANLkTimNQDZK5AvTbMb85wXur4GyOgXtiW5G=-htc4ZG@mail.gmail.com>
Message-ID: <20100826043339.GB14658@dcvr.yhbt.net>

Jeremy Evans <jeremyevans0 at gmail.com> wrote:
> On Wed, Aug 25, 2010 at 12:50 AM, Eric Wong <normalperson at yhbt.net> wrote:
> > Yeah, I figured using the "home_run" wrapper would be less than
> > ideal since it would get lost during exec upgrades and newer
> > versions of home_run wouldn't get picked up the same way it'd get
> > picked up in a system install.
> 
> The home_run wrapper works by modifying the environment (RUBYLIB and
> RUBYOPT). Does the environment get reset during exec upgrades?

The environment is untouched by default.  I meant Unicorn can only see
$0 as 'unicorn' or 'unicorn_rails', and never 'home_run', so the
version changes can't be picked up.

> You are correct that newer versions would not get picked up, as it's
> going to hard code a specific gem version.

> > Thanks. ?I'd be eager to hear how it works out since this could be a
> > popular library very soon :) Is there a home_run mailing list I could
> > follow?
> 
> I haven't set up a mailing list for it (you are the first person to
> ask about one), but I've been hanging out by myself in #home_run on
> freenode (IRC).

Ah, I've mostly given up on IRC since I get distracted easily or
everybody pops in at different times and lack of threading makes it hard
to follow.  I much prefer the asynchronous nature of email along with
its ease of archival, search, and patch exchange.  For intense
emergencies where money is lost for every second something is awry, yes,
IRC is great, but I do Free Software development at a much more relaxed
pace :)

-- 
Eric Wong

From normalperson at yhbt.net  Sat Aug 28 15:57:58 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Sat, 28 Aug 2010 19:57:58 +0000
Subject: [ANN] unicorn 1.1.3 - small fixes
Message-ID: <20100828195758.GA17686@dcvr.yhbt.net>

Changes:

This release fixes race conditions during SIGUSR1 log cycling.
This bug mainly affects Rainbows! users serving static files,
but some Rack apps use threads internally even under Unicorn.

Other small fixes:
* SIGTTIN works as documented after SIGWINCH
* --help output from `unicorn` and `unicorn_rails` is more consistent


Unless people scream, there will be no further releases from the 1.0.x
line and all users are encouraged to upgrade to 1.1.x.  I'm still
thinking about how to approach some internal cleanups for 2.x, but 2.x
could be out as early as next week  (but 1.x will continue to be
maintained).

* http://unicorn.bogomips.org/
* mongrel-unicorn at rubyforge.org
* git://git.bogomips.org/unicorn.git

-- 
Eric Wong

From curtis.schofield at gmail.com  Mon Aug 30 23:57:48 2010
From: curtis.schofield at gmail.com (Curtis)
Date: Mon, 30 Aug 2010 20:57:48 -0700
Subject: Hello excellent friends
Message-ID: <2D00600E-51CB-49E4-ACD0-238FE9DB783A@gmail.com>


I have read your tun in article and I have tried to make sysctl changes - the benchmarks are not very significant. I love how well the signals are laid out and how excellently unicorn responds to them - I am hoping to get back to mongrel like performance . We were using nginx to haproxy to 32 mongrels
No I have set up a unix domain socket with 2mb send and receive buffer and a backlog of 2048 - I didn't see a change with the send receive buffer tuning and I have yet to try a smaller backlog to see. If that helps. Any info?

I am tuning for osx in dev and centos5 in production. Thank you for your great work cleaning mongrel into a first class unix friend.

--


From normalperson at yhbt.net  Tue Aug 31 02:17:08 2010
From: normalperson at yhbt.net (Eric Wong)
Date: Mon, 30 Aug 2010 23:17:08 -0700
Subject: Hello excellent friends
In-Reply-To: <2D00600E-51CB-49E4-ACD0-238FE9DB783A@gmail.com>
References: <2D00600E-51CB-49E4-ACD0-238FE9DB783A@gmail.com>
Message-ID: <20100831061708.GA26170@dcvr.yhbt.net>

Curtis <curtis.schofield at gmail.com> wrote:
> I have read your tun in article and I have tried to make sysctl
> changes - the benchmarks are not very significant. I love how well the
> signals are laid out and how excellently unicorn responds to them - I
> am hoping to get back to mongrel like performance . We were using
> nginx to haproxy to 32 mongrels
> 
> No I have set up a unix domain socket with 2mb send and receive buffer
> and a backlog of 2048 - I didn't see a change with the send receive
> buffer tuning and I have yet to try a smaller backlog to see. If that
> helps. Any info?

Hi, I'm not sure what size of requests and responses you get, but 2mb
seems like a lot, especially for requests (unless you handle a lot of
big uploads).

Even with large requests/responses, huge buffers means more pressure on
the kernel allocator and you're also hurting CPU cache locality.  So
larger buffers can hurt performance as well as help, I'll make a note of
that in the documentation.

The backlog shouldn't change performance much.  A larger one can let you
withstand short traffic bursts and increase reliability while
benchmarking.

> I am tuning for osx in dev and centos5 in production. Thank you for
> your great work cleaning mongrel into a first class unix friend.

You're welcome!

-- 
Eric Wong

From jamie at tramchase.com  Tue Aug 31 14:30:19 2010
From: jamie at tramchase.com (Jamie Wilkinson)
Date: Tue, 31 Aug 2010 11:30:19 -0700
Subject: synchronous restart
Message-ID: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>

Has anyone tried to synchronously restart their unicorns, to ensure that things restart OK?

I imagine I could write a script that sent USR2 and then watched the log for a successful before exiting, but I dream there is something more MAGICAL


Yours in mythical web servers,

-jamie


From cliftonk at gmail.com  Tue Aug 31 15:08:39 2010
From: cliftonk at gmail.com (Clifton King)
Date: Tue, 31 Aug 2010 14:08:39 -0500
Subject: synchronous restart
In-Reply-To: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>
References: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>
Message-ID: <AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>

Jamie,

Here is a sample unicorn config file that automatically sends the old
master process a QUIT signal if the new master process successfully
boots and starts forking (which is what I assume you are asking
about). We get 0 downtime deploys using this method (and changes
typically propagate live in 5-10 seconds). If the new master process
fails at booting, you can tail the unicorn.stdout/err.log files to see
why.

http://pastie.org/1129610

Most of that configuration file was from a github blog post about
unicorn (you can try googling for it).

I also added a section that writes out PID files for workers so you
can monitor their memory usage (and send them quit signals when they
exceed the limit). We do that via god.

Hope that helps,
Clifton

On Tue, Aug 31, 2010 at 1:30 PM, Jamie Wilkinson <jamie at tramchase.com> wrote:
> Has anyone tried to synchronously restart their unicorns, to ensure that things restart OK?
>
> I imagine I could write a script that sent USR2 and then watched the log for a successful before exiting, but I dream there is something more MAGICAL
>
>
> Yours in mythical web servers,
>
> -jamie
>
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

From jamie at tramchase.com  Tue Aug 31 16:08:52 2010
From: jamie at tramchase.com (Jamie Wilkinson)
Date: Tue, 31 Aug 2010 13:08:52 -0700
Subject: synchronous restart
In-Reply-To: <AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>
References: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>
	<AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>
Message-ID: <CC111CB2-3EBE-459D-81F7-71B273A4784F@tramchase.com>

On Aug 31, 2010, at 12:08 PM, Clifton King wrote:

> If the new master process
> fails at booting, you can tail the unicorn.stdout/err.log files to see
> why.


I should clarify... the above is exactly what I'm trying to avoid. i.e. how do you know if your new master failed to boot unless you are actively tailing the logs? 

It is extremely infrequent that our unicorns fail to start, but when it does we sometimes don't notice for quite some time. Our unicorns also restart so quickly that it is not an issue to do the restarts synchronously during deployment and trade speed for peace of mind

I will probably just replace our basic `kill -USR2` with a small script that sends the signal and doesn't exit until the pidfile handover is complete. I'll be sure to share my results in case anyone else might find this useful.


-jamie

http://jamiedubs.com | http://fffff.at

> On Tue, Aug 31, 2010 at 1:30 PM, Jamie Wilkinson <jamie at tramchase.com> wrote:
>> Has anyone tried to synchronously restart their unicorns, to ensure that things restart OK?
>> 
>> I imagine I could write a script that sent USR2 and then watched the log for a successful before exiting, but I dream there is something more MAGICAL
>> 
>> 
>> Yours in mythical web servers,
>> 
>> -jamie
>> 
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>> 
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying


From cliftonk at gmail.com  Tue Aug 31 16:15:04 2010
From: cliftonk at gmail.com (Clifton King)
Date: Tue, 31 Aug 2010 15:15:04 -0500
Subject: synchronous restart
In-Reply-To: <CC111CB2-3EBE-459D-81F7-71B273A4784F@tramchase.com>
References: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>
	<AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>
	<CC111CB2-3EBE-459D-81F7-71B273A4784F@tramchase.com>
Message-ID: <AANLkTikorRi-m0B-NT_4mzTm6M_ii3fA17TM9qe45G=T@mail.gmail.com>

Jamie,

Check to see that the new unicorn master process has a different PID
than the old one. You could have a script that sleeps for X seconds
and checks the PID after the deploy and if it matches the old one
alerts you with a tail of the stdout/stderr log files. I personally do
"ps aux | grep unicorn" a few times during the process if there are
any changes being deployed I'm wary of.

Clifton

On Tue, Aug 31, 2010 at 3:08 PM, Jamie Wilkinson <jamie at tramchase.com> wrote:
> On Aug 31, 2010, at 12:08 PM, Clifton King wrote:
>
>> If the new master process
>> fails at booting, you can tail the unicorn.stdout/err.log files to see
>> why.
>
>
> I should clarify... the above is exactly what I'm trying to avoid. i.e. how do you know if your new master failed to boot unless you are actively tailing the logs?
>
> It is extremely infrequent that our unicorns fail to start, but when it does we sometimes don't notice for quite some time. Our unicorns also restart so quickly that it is not an issue to do the restarts synchronously during deployment and trade speed for peace of mind
>
> I will probably just replace our basic `kill -USR2` with a small script that sends the signal and doesn't exit until the pidfile handover is complete. I'll be sure to share my results in case anyone else might find this useful.
>
>
> -jamie
>
> http://jamiedubs.com | http://fffff.at
>
>> On Tue, Aug 31, 2010 at 1:30 PM, Jamie Wilkinson <jamie at tramchase.com> wrote:
>>> Has anyone tried to synchronously restart their unicorns, to ensure that things restart OK?
>>>
>>> I imagine I could write a script that sent USR2 and then watched the log for a successful before exiting, but I dream there is something more MAGICAL
>>>
>>>
>>> Yours in mythical web servers,
>>>
>>> -jamie
>>>
>>> _______________________________________________
>>> Unicorn mailing list - mongrel-unicorn at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>>> Do not quote signatures (like this one) or top post when replying
>>>
>> _______________________________________________
>> Unicorn mailing list - mongrel-unicorn at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
>> Do not quote signatures (like this one) or top post when replying
>
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
>

From sfeley at gmail.com  Tue Aug 31 16:29:56 2010
From: sfeley at gmail.com (Stephen Eley)
Date: Tue, 31 Aug 2010 16:29:56 -0400
Subject: synchronous restart
In-Reply-To: <CC111CB2-3EBE-459D-81F7-71B273A4784F@tramchase.com>
References: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>
	<AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>
	<CC111CB2-3EBE-459D-81F7-71B273A4784F@tramchase.com>
Message-ID: <AANLkTikOiY9Dscz+tr1t1Powa8zPbHonawMzt2oFr_1F@mail.gmail.com>

On Tue, Aug 31, 2010 at 4:08 PM, Jamie Wilkinson <jamie at tramchase.com> wrote:
>
> I should clarify... the above is exactly what I'm trying to avoid. i.e. how do you know if your new master failed to boot unless you are actively tailing the logs?

Well, first, you can specify a .pid file in your Unicorn configuration
file, then look at it before and a few seconds after the USR2 signal
to see if the process IDs are different.  Unicorn's pretty smart about
maintaining that file, and copying the old one out until you kill the
old process.

Second...  I'm confused as to why you think tailing the logs is
something to be avoided.  Isn't finding out status what logs are
*for?*  You could certainly automate the inspection and have some
process that emails you with an alarm if things don't look right.
Regardless of how you do it, if you're hoping to get away with
frequent automated deploys without *some* verification that requests
are working, you're taking big risks.  Even if your unicorn process
runs, you could still have problems at the application level throwing
exceptions at every user.




-- 
Have Fun,
?? Steve Eley (sfeley at gmail.com)
?? ESCAPE POD - The Science Fiction Podcast Magazine
?? http://www.escapepod.org

From sfeley at gmail.com  Tue Aug 31 16:58:40 2010
From: sfeley at gmail.com (Stephen Eley)
Date: Tue, 31 Aug 2010 16:58:40 -0400
Subject: synchronous restart
In-Reply-To: <AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>
References: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>
	<AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>
Message-ID: <AANLkTinc+L2+ivKRvVFXXtOoy2WEjpxSUex+u2imt9wU@mail.gmail.com>

On Tue, Aug 31, 2010 at 3:08 PM, Clifton King <cliftonk at gmail.com> wrote:
>
> http://pastie.org/1129610
>
> Most of that configuration file was from a github blog post about
> unicorn (you can try googling for it).

Cool.  In that spirit, here's the config file I use with Vlad the
Deployer in all of my Rails apps:
http://gist.github.com/559710

Four Unicorn-related tasks are included:

* start_unicorn: Calls the unicorn server with the path to the
appropriate config/unicorn.rb file
* reload_unicorn: Sends HUP for a graceful reload (and falls back to
start_unicorn if any status errors come back)
* kick_unicorn: Sends USR2 for synchronous restart.  If successful,
sends WINCH to the old process and then QUIT after one minute (to give
time for testing and fallback if necessary).
* rescue_unicorn: Recovers from a failed kick_unicorn if necessary.
Does a HUP to revive the old master and QUITs the new one.

I've been meaning to bundle this into a Vlad plugin gem, but haven't
gotten around to it yet.  In the meantime, here it is in case anyone
would find it useful.  (I also have Git automation in there that reads
the branch name for the environment.  I.e., I have branches for
'staging', 'production', etc. -- and named users and server subdomains
for them, too.)

-- 
Have Fun,
?? Steve Eley (sfeley at gmail.com)
?? ESCAPE POD - The Science Fiction Podcast Magazine
?? http://www.escapepod.org

From lawrence.pit at gmail.com  Tue Aug 31 19:59:15 2010
From: lawrence.pit at gmail.com (Lawrence Pit)
Date: Wed, 01 Sep 2010 09:59:15 +1000
Subject: synchronous restart
In-Reply-To: <AANLkTikOiY9Dscz+tr1t1Powa8zPbHonawMzt2oFr_1F@mail.gmail.com>
References: <9ADECA6F-ED48-467D-8826-2C94BF42B18A@tramchase.com>	<AANLkTik5v5mCRecaWo0R-4m+ns-=pwJhZJf-XFzUdzGp@mail.gmail.com>	<CC111CB2-3EBE-459D-81F7-71B273A4784F@tramchase.com>
	<AANLkTikOiY9Dscz+tr1t1Powa8zPbHonawMzt2oFr_1F@mail.gmail.com>
Message-ID: <4C7D9753.9030507@gmail.com>


I totally get what Jamie is asking for, I'm in the same boat. 

I think Jamie was saying he wants to avoid tailing the log while 
deploying new code. The only thing you want to know is 0 for success or 
-1 for failure, only in the case of -1 would you make the effort to have 
your brain waste time parsing log data.

Jamie, I was thinking of modifying my unicorn init.d script somewhat 
similar to this from a suggested nginx init.d script:

http://wiki.nginx.org/Nginx-init-ubuntu

see the quietupgrade() function.

As Unicorn follows the nginx patterns it'll be mostly a copy-paste I think.



Cheers,
Lawrence

> On Tue, Aug 31, 2010 at 4:08 PM, Jamie Wilkinson <jamie at tramchase.com> wrote:
>   
>> I should clarify... the above is exactly what I'm trying to avoid. i.e. how do you know if your new master failed to boot unless you are actively tailing the logs?
>>     
>
> Well, first, you can specify a .pid file in your Unicorn configuration
> file, then look at it before and a few seconds after the USR2 signal
> to see if the process IDs are different.  Unicorn's pretty smart about
> maintaining that file, and copying the old one out until you kill the
> old process.
>
> Second...  I'm confused as to why you think tailing the logs is
> something to be avoided.  Isn't finding out status what logs are
> *for?*  You could certainly automate the inspection and have some
> process that emails you with an alarm if things don't look right.
> Regardless of how you do it, if you're hoping to get away with
> frequent automated deploys without *some* verification that requests
> are working, you're taking big risks.  Even if your unicorn process
> runs, you could still have problems at the application level throwing
> exceptions at every user.
>
>
>
>
>