From gethemant at gmail.com  Sat Mar  1 21:17:43 2008
From: gethemant at gmail.com (hemant)
Date: Sun, 2 Mar 2008 07:47:43 +0530
Subject: [Mongrel] Offloading Background Tasks
In-Reply-To: <2d8ffda38541bfdc3425ca8511bfcd00@ruby-forum.com>
References: <2d8ffda38541bfdc3425ca8511bfcd00@ruby-forum.com>
Message-ID: <fb283d5e0803011817t764d1b12w7fdafda64ea7ebc5@mail.gmail.com>

On Sat, Mar 1, 2008 at 7:10 AM, Nathan Esquenazi <lists at ruby-forum.com> wrote:
> Hello everyone,
>
>  I am very interested in off-loading tasks from the request/response
>  cycle in order to speed up my application. My application is heavily
>  connected to using web services both external and from other servers we
>  host.
>
>  My application does the following long-running tasks:
>
>  - accesses / pushes to the twitter api's (with possible need for status
>  responses)
>  - accesses / pushes the facebook api's  (with need for status responses)
>  - requesting data from another server we run (rails needs access data)
>  - pushes information to aim using ruburple
>
>  I am just curious and grateful for any help people can give for what
>  would suit me best. I am familar with backgroundrb and bj along with
>  starling, etc. I don't know which one will suit me best based on the
>  tasks I need to queue.
>
>  My main concern is how to properly request data and then show it to the
>  user. For instance, when the user logs into the rails application, I
>  need to show them certain information about them which comes from
>  complex analyzing of data done by another server we run.
>
>  When a user logs in, I need to show them the current results of this
>  analysis. To do this, I need to perform a GET request to another server
>  and get the results and then show them to the user. What is the "best
>  practice" way to achieve this in which I can offload this task to a
>  background worker and then get the results and display them to the user.
>
>  For instance, say the user logs in and then the background worker kicks
>  off to retrieve their current analyzed data from the server. How do I
>  then wait for the data to be retrieved by the worker and when it's ready
>  display it to the user?
>
>  I feel like I am missing something because I don't see how the worker
>  can "tell" the main rails application that it now has the data and then
>  somehow show it when its "ready". Do I need to constantly poll the
>  server or how is this usually done.
>
>  Also, I need to allow the user to make certain changes to data fields
>  and then I need to push those changes to twitter and facebook via a
>  background process.
>
>  What solution is best suited based on what I described and could someone
>  please help me understand the pattern for how to do this. I will be
>  eternally grateful.
>  --

For what you are describing, I think BackgrounDRb will be best fit.
Since, its the only solution ( that I know of ) provides interfacing
with rails so as you can actually fetch results retrieved by a worker
in rails.

Bj is a worker queue to offload tasks. AFAIK, you can't really
interact seamlessly from rails. I don't know much about startling. But
this is coming from guy who maintains bdrb to take it with a grain of
salt.

From public at misuse.org  Sat Mar  1 21:51:35 2008
From: public at misuse.org (Steve Midgley)
Date: Sat, 01 Mar 2008 18:51:35 -0800
Subject: [Mongrel] Offloading Background Tasks
In-Reply-To: <mailman.311.1204424265.7196.mongrel-users@rubyforge.org>
References: <mailman.311.1204424265.7196.mongrel-users@rubyforge.org>
Message-ID: <20080302025200.5B44F1858660@rubyforge.org>

At 06:17 PM 3/1/2008, mongrel-users-request at rubyforge.org wrote:
>Date: Sun, 2 Mar 2008 07:47:43 +0530
>From: hemant <gethemant at gmail.com>
>Subject: Re: [Mongrel] Offloading Background Tasks
>To: mongrel-users at rubyforge.org
>Message-ID: 
><fb283d5e0803011817t764d1b12w7fdafda64ea7ebc5 at mail.gmail.com>
>Content-Type: text/plain; charset=ISO-8859-1
>
>On Sat, Mar 1, 2008 at 7:10 AM, Nathan Esquenazi 
><lists at ruby-forum.com> wrote:
> > Hello everyone,
> >
> >  I am very interested in off-loading tasks from the 
> request/response
> >  cycle in order to speed up my application. My application is 
> heavily
> >  connected to using web services both external and from other 
> servers we
> >  host.
>[snip]
> >
> >  I feel like I am missing something because I don't see how the 
> worker
> >  can "tell" the main rails application that it now has the data and 
> then
> >  somehow show it when its "ready". Do I need to constantly poll the
> >  server or how is this usually done.
> >
> >  Also, I need to allow the user to make certain changes to data 
> fields
> >  and then I need to push those changes to twitter and facebook via 
> a
> >  background process.
> >
> >  What solution is best suited based on what I described and could 
> someone
> >  please help me understand the pattern for how to do this. I will 
> be
> >  eternally grateful.
> >  --
>
>For what you are describing, I think BackgrounDRb will be best fit.
>Since, its the only solution ( that I know of ) provides interfacing
>with rails so as you can actually fetch results retrieved by a worker
>in rails.
>
>Bj is a worker queue to offload tasks. AFAIK, you can't really
>interact seamlessly from rails. I don't know much about startling. But
>this is coming from guy who maintains bdrb to take it with a grain of
>salt.

Hi,

Firstly - apologies to Hemant for misinformation from me a while ago 
about the current status of BackgrounDrb. It's clear now of course to 
me that this an active and healthy project.

I have been messing around with Ara Howards BackgroundJob (bj) and it's 
got a way to spin out a bunch of jobs and then wait until they all 
return, which is similar to the way threads work:

>       jobs = Bj.submit list_of_jobs, :tag => 'important'
>       ...
>
>       jobs.each do |job|
>         if job.finished?
>           p job.exit_status
>           p job.stdout
>           p job.stderr
>         end
>       end

You can see that if by the time you get down to the "if jobs.finished?" 
section, you could choose to ignore whatever jobs haven't returned yet 
(or possibly even TERM them if that's appropriate).

I wonder also if you've considered adding some architecture to your 
system so that you can collect whatever you can from your external 
services from within Rails but then whatever is left over you could 
pull down via Ajax after the page had mostly loaded (so some parts of 
the page might be left with placeholders and then get filled in with 
the correct info later as the Ajax calls retrieve it from Rails..

That'd be my thought on this: build something (with BackgrounDRb, Bj or 
whatever) that lets you spawn a bunch of a async queries. Give those 
queries some kind of unique handle. When you're done building the whole 
page, have your view fill in data from whichever queries have returned. 
Then send the incomplete page to the browser and have ajax queries pull 
down the remaining query results (using a webservice interface passing 
in those unique handles you set up above to ID which job you want).

Hope this helps!

Steve


From mail at flydown.org  Sun Mar  2 05:10:54 2008
From: mail at flydown.org (Michele)
Date: Sun, 2 Mar 2008 11:10:54 +0100
Subject: [Mongrel] Append Mongrel port after HTML
Message-ID: <6D1EA332-9FD4-4ADA-953B-50D056916AB1@flydown.org>

Hi,

I'm trying to debug an issue I'm having with an app where certain  
requests fail but work after a refresh which led me to think it might  
be a problem with only one Mongrel instance out of the bunch.

So I was wondering if there's an easy way (i.e. without having to  
manually patch Mongrel) to append a comment at the end of the pages'  
HTML with something like the PID or port so I can track which instance  
served the request.

Thanks for your help!

- Michele

From mike.engelhart at gmail.com  Tue Mar  4 09:36:33 2008
From: mike.engelhart at gmail.com (Michael Engelhart)
Date: Tue, 4 Mar 2008 09:36:33 -0500
Subject: [Mongrel] Offloading Background Tasks
In-Reply-To: <20080302025200.5B44F1858660@rubyforge.org>
References: <mailman.311.1204424265.7196.mongrel-users@rubyforge.org>
	<20080302025200.5B44F1858660@rubyforge.org>
Message-ID: <42642c790803040636x5830a20x5f20ce43aa058784@mail.gmail.com>

We've been testing using Spawn.    Has anyone had any success in a
production environment with the spawn plugin?   It's very simple but
seems to work well although there doesn't seem to be any recent
activity around it.

Eventually I'd like to use Amazons SQS for our application but we
still have other architecture issues to sort out before going that
route.

Mike

On Sat, Mar 1, 2008 at 9:51 PM, Steve Midgley <public at misuse.org> wrote:
> At 06:17 PM 3/1/2008, mongrel-users-request at rubyforge.org wrote:
>  >Date: Sun, 2 Mar 2008 07:47:43 +0530
>  >From: hemant <gethemant at gmail.com>
>  >Subject: Re: [Mongrel] Offloading Background Tasks
>  >To: mongrel-users at rubyforge.org
>  >Message-ID:
>  ><fb283d5e0803011817t764d1b12w7fdafda64ea7ebc5 at mail.gmail.com>
>  >Content-Type: text/plain; charset=ISO-8859-1
>
> >
>  >On Sat, Mar 1, 2008 at 7:10 AM, Nathan Esquenazi
>  ><lists at ruby-forum.com> wrote:
>  > > Hello everyone,
>  > >
>  > >  I am very interested in off-loading tasks from the
>  > request/response
>  > >  cycle in order to speed up my application. My application is
>  > heavily
>  > >  connected to using web services both external and from other
>  > servers we
>  > >  host.
>  >[snip]
>
> > >
>  > >  I feel like I am missing something because I don't see how the
>  > worker
>  > >  can "tell" the main rails application that it now has the data and
>  > then
>  > >  somehow show it when its "ready". Do I need to constantly poll the
>  > >  server or how is this usually done.
>  > >
>  > >  Also, I need to allow the user to make certain changes to data
>  > fields
>  > >  and then I need to push those changes to twitter and facebook via
>  > a
>  > >  background process.
>  > >
>  > >  What solution is best suited based on what I described and could
>  > someone
>  > >  please help me understand the pattern for how to do this. I will
>  > be
>  > >  eternally grateful.
>  > >  --
>  >
>  >For what you are describing, I think BackgrounDRb will be best fit.
>  >Since, its the only solution ( that I know of ) provides interfacing
>  >with rails so as you can actually fetch results retrieved by a worker
>  >in rails.
>  >
>  >Bj is a worker queue to offload tasks. AFAIK, you can't really
>  >interact seamlessly from rails. I don't know much about startling. But
>  >this is coming from guy who maintains bdrb to take it with a grain of
>  >salt.
>
>  Hi,
>
>  Firstly - apologies to Hemant for misinformation from me a while ago
>  about the current status of BackgrounDrb. It's clear now of course to
>  me that this an active and healthy project.
>
>  I have been messing around with Ara Howards BackgroundJob (bj) and it's
>  got a way to spin out a bunch of jobs and then wait until they all
>  return, which is similar to the way threads work:
>
>  >       jobs = Bj.submit list_of_jobs, :tag => 'important'
>  >       ...
>  >
>  >       jobs.each do |job|
>  >         if job.finished?
>  >           p job.exit_status
>  >           p job.stdout
>  >           p job.stderr
>  >         end
>  >       end
>
>  You can see that if by the time you get down to the "if jobs.finished?"
>  section, you could choose to ignore whatever jobs haven't returned yet
>  (or possibly even TERM them if that's appropriate).
>
>  I wonder also if you've considered adding some architecture to your
>  system so that you can collect whatever you can from your external
>  services from within Rails but then whatever is left over you could
>  pull down via Ajax after the page had mostly loaded (so some parts of
>  the page might be left with placeholders and then get filled in with
>  the correct info later as the Ajax calls retrieve it from Rails..
>
>  That'd be my thought on this: build something (with BackgrounDRb, Bj or
>  whatever) that lets you spawn a bunch of a async queries. Give those
>  queries some kind of unique handle. When you're done building the whole
>  page, have your view fill in data from whichever queries have returned.
>  Then send the incomplete page to the browser and have ajax queries pull
>  down the remaining query results (using a webservice interface passing
>  in those unique handles you set up above to ID which job you want).
>
>  Hope this helps!
>
>  Steve
>
>
>
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From elw at brandorr.com  Tue Mar  4 21:37:35 2008
From: elw at brandorr.com (Elijah Wright)
Date: Tue, 4 Mar 2008 21:37:35 -0500
Subject: [Mongrel] best practices for monitoring mongrels with monit?
Message-ID: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>

Hello,

We're interested in hearing from folks who have worked out a set of "best
practices" for monitoring mongrels with monit.

My local setup is remarkably similar to the one currently found in the "
Practical Programmers Deploying Rails Apps Beta" PDF, as well as several
other examples that we've seen floating around on the web.

Has someone gone to the trouble to do a pro/con analysis of a couple of
different approaches, so that we can get a better idea of what folks have
tried?

[We'd be sort of miserable if we didn't have monit... it allows us to build
some beautiful things!]

thanks very much in advance,

--elijah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080304/877ff197/attachment.html 

From rochkind at jhu.edu  Wed Mar  5 11:59:49 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 11:59:49 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
Message-ID: <47CED185.3030101@jhu.edu>

So I am using the instructions here to have mongrel_cluster start my 
various mongrel-deployed apps on server boot.
http://mongrel.rubyforge.org/wiki/MongrelCluster

There is a problem mentioned here:
http://www.ruby-forum.com/topic/105849

In summary---if the server (or the mongrel processes) die ungracefully 
leaving their pids behind, then on next boot, mongrel_cluster won't 
succesfully start them up, choking on the leftover pids.

What is the reccommended solution to this?  I'd like to have my mongrels 
automatically start on boot, in a way that forces them to start even if 
there are stale pids. Thanks for any advice!

[ PS: I just discovered this problem the hard way, when my server 
unexpetedly lost power, and my production apps failed to start up on the 
next boot!  Very unpleasant. ]

Jonathan


-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From eden at mojiti.com  Wed Mar  5 12:29:37 2008
From: eden at mojiti.com (Eden Li)
Date: Wed, 5 Mar 2008 09:29:37 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CED185.3030101@jhu.edu>
References: <47CED185.3030101@jhu.edu>
Message-ID: <A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>

Newer versions of mongrel_cluster have a --clean option which will  
remove stale pids if it finds them.  Provide that option to  
mongrel_rails cluster::* in your scripts.

On Mar 5, 2008, at 8:59 AM, Jonathan Rochkind wrote:

> So I am using the instructions here to have mongrel_cluster start my
> various mongrel-deployed apps on server boot.
> http://mongrel.rubyforge.org/wiki/MongrelCluster
>
> There is a problem mentioned here:
> http://www.ruby-forum.com/topic/105849
>
> In summary---if the server (or the mongrel processes) die ungracefully
> leaving their pids behind, then on next boot, mongrel_cluster won't
> succesfully start them up, choking on the leftover pids.
>
> What is the reccommended solution to this?  I'd like to have my  
> mongrels
> automatically start on boot, in a way that forces them to start even  
> if
> there are stale pids. Thanks for any advice!
>
> [ PS: I just discovered this problem the hard way, when my server
> unexpetedly lost power, and my production apps failed to start up on  
> the
> next boot!  Very unpleasant. ]
>
> Jonathan
>
>
> -- 
> Jonathan Rochkind
> Digital Services Software Engineer
> The Sheridan Libraries
> Johns Hopkins University
> 410.516.8886
> rochkind (at) jhu.edu
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From rochkind at jhu.edu  Wed Mar  5 12:40:26 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 12:40:26 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
Message-ID: <47CEDB0A.6070406@jhu.edu>

Thanks that's helpful, I'm investigating that.

Using the instructions at:

http://mongrel.rubyforge.org/wiki/MongrelCluster#OnBootInitializationSetup

I had a little bash script at /etc/init.d/mongrel_cluster that was copied from
/path/to/mongrel_cluster_gem/resources/mongrel_cluster

That bash script calls mongrel_cluster_ctl with various options. I have added a "--clean" option to the "start" condition.  Would it make sense to suggest this be added to resources/mongrel_cluster in the actual gem, to not require this change?  

However, now I'm trying to set up an intentionally created stale pid, to test this feature. I am invoking mongrel_cluster_ctl manually (but in the manner of the resources/mongrel_cluster script) with the --clean option. I get this:

****
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in `unlink': No such file or directory - tmp/pids/mongrel.3000.pid (Errno::ENOENT)
        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in `start'
        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in `each'
        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in `start'
        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:232:in `run'
        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'
        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281
        from /usr/bin/mongrel_rails:18:in `load'
        from /usr/bin/mongrel_rails:18
mongrel_rails cluster::start returned an error.
*****

That was using a mongrel_config.yml file that did have 'cwd' set.  There certainly is a tmp/pids/mongrel.3000.pid off of the 'cwd' location.  But perhaps the 'unlink' isn't being executed off the 'cwd' location as it ought to be?

Is this a bug in mongrel_cluster_ctl --clean, or perhaps just a bug in my attempt to duplicate the problem condition and make sure I've solved it? 

Thanks for any help. 

Jonathan



Eden Li wrote:
> Newer versions of mongrel_cluster have a --clean option which will  
> remove stale pids if it finds them.  Provide that option to  
> mongrel_rails cluster::* in your scripts.
>
> On Mar 5, 2008, at 8:59 AM, Jonathan Rochkind wrote:
>
>   
>> So I am using the instructions here to have mongrel_cluster start my
>> various mongrel-deployed apps on server boot.
>> http://mongrel.rubyforge.org/wiki/MongrelCluster
>>
>> There is a problem mentioned here:
>> http://www.ruby-forum.com/topic/105849
>>
>> In summary---if the server (or the mongrel processes) die ungracefully
>> leaving their pids behind, then on next boot, mongrel_cluster won't
>> succesfully start them up, choking on the leftover pids.
>>
>> What is the reccommended solution to this?  I'd like to have my  
>> mongrels
>> automatically start on boot, in a way that forces them to start even  
>> if
>> there are stale pids. Thanks for any advice!
>>
>> [ PS: I just discovered this problem the hard way, when my server
>> unexpetedly lost power, and my production apps failed to start up on  
>> the
>> next boot!  Very unpleasant. ]
>>
>> Jonathan
>>
>>
>> -- 
>> Jonathan Rochkind
>> Digital Services Software Engineer
>> The Sheridan Libraries
>> Johns Hopkins University
>> 410.516.8886
>> rochkind (at) jhu.edu
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>     
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From rochkind at jhu.edu  Wed Mar  5 12:45:51 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 12:45:51 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CEDB0A.6070406@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu>
Message-ID: <47CEDC4F.5020202@jhu.edu>

Actually, I have investigated further, and am pretty sure I am 
duplicating the problem condition properly.

mongrel_cluster_ctl start --clean doesn't seem to work properly, it's 
throwing that error. Hmm.

I am guessing that the script I find in the mongrel_cluster gem at 
resources/mongrel_cluster, which is reccommended on that wiki page... is 
not actually what you all are using.  And perhaps shouldn't be included 
in the distro anymore?   What are others using to ensure mongrel 
clusters start at boot?

Jonathan

Jonathan Rochkind wrote:
> Thanks that's helpful, I'm investigating that.
>
> Using the instructions at:
>
> http://mongrel.rubyforge.org/wiki/MongrelCluster#OnBootInitializationSetup
>
> I had a little bash script at /etc/init.d/mongrel_cluster that was copied from
> /path/to/mongrel_cluster_gem/resources/mongrel_cluster
>
> That bash script calls mongrel_cluster_ctl with various options. I have added a "--clean" option to the "start" condition.  Would it make sense to suggest this be added to resources/mongrel_cluster in the actual gem, to not require this change?  
>
> However, now I'm trying to set up an intentionally created stale pid, to test this feature. I am invoking mongrel_cluster_ctl manually (but in the manner of the resources/mongrel_cluster script) with the --clean option. I get this:
>
> ****
> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in `unlink': No such file or directory - tmp/pids/mongrel.3000.pid (Errno::ENOENT)
>         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in `start'
>         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in `each'
>         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in `start'
>         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:232:in `run'
>         from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'
>         from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281
>         from /usr/bin/mongrel_rails:18:in `load'
>         from /usr/bin/mongrel_rails:18
> mongrel_rails cluster::start returned an error.
> *****
>
> That was using a mongrel_config.yml file that did have 'cwd' set.  There certainly is a tmp/pids/mongrel.3000.pid off of the 'cwd' location.  But perhaps the 'unlink' isn't being executed off the 'cwd' location as it ought to be?
>
> Is this a bug in mongrel_cluster_ctl --clean, or perhaps just a bug in my attempt to duplicate the problem condition and make sure I've solved it? 
>
> Thanks for any help. 
>
> Jonathan
>
>
>
> Eden Li wrote:
>   
>> Newer versions of mongrel_cluster have a --clean option which will  
>> remove stale pids if it finds them.  Provide that option to  
>> mongrel_rails cluster::* in your scripts.
>>
>> On Mar 5, 2008, at 8:59 AM, Jonathan Rochkind wrote:
>>
>>   
>>     
>>> So I am using the instructions here to have mongrel_cluster start my
>>> various mongrel-deployed apps on server boot.
>>> http://mongrel.rubyforge.org/wiki/MongrelCluster
>>>
>>> There is a problem mentioned here:
>>> http://www.ruby-forum.com/topic/105849
>>>
>>> In summary---if the server (or the mongrel processes) die ungracefully
>>> leaving their pids behind, then on next boot, mongrel_cluster won't
>>> succesfully start them up, choking on the leftover pids.
>>>
>>> What is the reccommended solution to this?  I'd like to have my  
>>> mongrels
>>> automatically start on boot, in a way that forces them to start even  
>>> if
>>> there are stale pids. Thanks for any advice!
>>>
>>> [ PS: I just discovered this problem the hard way, when my server
>>> unexpetedly lost power, and my production apps failed to start up on  
>>> the
>>> next boot!  Very unpleasant. ]
>>>
>>> Jonathan
>>>
>>>
>>> -- 
>>> Jonathan Rochkind
>>> Digital Services Software Engineer
>>> The Sheridan Libraries
>>> Johns Hopkins University
>>> 410.516.8886
>>> rochkind (at) jhu.edu
>>>
>>> _______________________________________________
>>> Mongrel-users mailing list
>>> Mongrel-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>     
>>>       
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>   
>>     
>
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From njvack at wisc.edu  Wed Mar  5 12:57:44 2008
From: njvack at wisc.edu (Nate Vack)
Date: Wed, 5 Mar 2008 11:57:44 -0600
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CEDC4F.5020202@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
Message-ID: <8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>

I'm doing everything (everything!) via either monit (on linux) or SMF
(on Solaris).

Using a process manager is a good idea... and having only one
'production' way to start mongrel is generally a good thing, too :)

-n

On Wed, Mar 5, 2008 at 11:45 AM, Jonathan Rochkind <rochkind at jhu.edu> wrote:
> Actually, I have investigated further, and am pretty sure I am
>  duplicating the problem condition properly.
>
>  mongrel_cluster_ctl start --clean doesn't seem to work properly, it's
>  throwing that error. Hmm.
>
>  I am guessing that the script I find in the mongrel_cluster gem at
>  resources/mongrel_cluster, which is reccommended on that wiki page... is
>  not actually what you all are using.  And perhaps shouldn't be included
>  in the distro anymore?   What are others using to ensure mongrel
>  clusters start at boot?
>
>  Jonathan
>
>
>
>  Jonathan Rochkind wrote:
>  > Thanks that's helpful, I'm investigating that.
>  >
>  > Using the instructions at:
>  >
>  > http://mongrel.rubyforge.org/wiki/MongrelCluster#OnBootInitializationSetup
>  >
>  > I had a little bash script at /etc/init.d/mongrel_cluster that was copied from
>  > /path/to/mongrel_cluster_gem/resources/mongrel_cluster
>  >
>  > That bash script calls mongrel_cluster_ctl with various options. I have added a "--clean" option to the "start" condition.  Would it make sense to suggest this be added to resources/mongrel_cluster in the actual gem, to not require this change?
>  >
>  > However, now I'm trying to set up an intentionally created stale pid, to test this feature. I am invoking mongrel_cluster_ctl manually (but in the manner of the resources/mongrel_cluster script) with the --clean option. I get this:
>  >
>  > ****
>  > /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in `unlink': No such file or directory - tmp/pids/mongrel.3000.pid (Errno::ENOENT)
>  >         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:83:in `start'
>  >         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in `each'
>  >         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:79:in `start'
>  >         from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb:232:in `run'
>  >         from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/mongrel/command.rb:212:in `run'
>  >         from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:281
>  >         from /usr/bin/mongrel_rails:18:in `load'
>  >         from /usr/bin/mongrel_rails:18
>  > mongrel_rails cluster::start returned an error.
>  > *****
>  >
>  > That was using a mongrel_config.yml file that did have 'cwd' set.  There certainly is a tmp/pids/mongrel.3000.pid off of the 'cwd' location.  But perhaps the 'unlink' isn't being executed off the 'cwd' location as it ought to be?
>  >
>  > Is this a bug in mongrel_cluster_ctl --clean, or perhaps just a bug in my attempt to duplicate the problem condition and make sure I've solved it?
>  >
>  > Thanks for any help.
>  >
>  > Jonathan
>  >
>  >
>  >
>  > Eden Li wrote:
>  >
>  >> Newer versions of mongrel_cluster have a --clean option which will
>  >> remove stale pids if it finds them.  Provide that option to
>  >> mongrel_rails cluster::* in your scripts.
>  >>
>  >> On Mar 5, 2008, at 8:59 AM, Jonathan Rochkind wrote:
>  >>
>  >>
>  >>
>  >>> So I am using the instructions here to have mongrel_cluster start my
>  >>> various mongrel-deployed apps on server boot.
>  >>> http://mongrel.rubyforge.org/wiki/MongrelCluster
>  >>>
>  >>> There is a problem mentioned here:
>  >>> http://www.ruby-forum.com/topic/105849
>  >>>
>  >>> In summary---if the server (or the mongrel processes) die ungracefully
>  >>> leaving their pids behind, then on next boot, mongrel_cluster won't
>  >>> succesfully start them up, choking on the leftover pids.
>  >>>
>  >>> What is the reccommended solution to this?  I'd like to have my
>  >>> mongrels
>  >>> automatically start on boot, in a way that forces them to start even
>  >>> if
>  >>> there are stale pids. Thanks for any advice!
>  >>>
>  >>> [ PS: I just discovered this problem the hard way, when my server
>  >>> unexpetedly lost power, and my production apps failed to start up on
>  >>> the
>  >>> next boot!  Very unpleasant. ]
>  >>>
>  >>> Jonathan
>  >>>
>  >>>
>  >>> --
>  >>> Jonathan Rochkind
>  >>> Digital Services Software Engineer
>  >>> The Sheridan Libraries
>  >>> Johns Hopkins University
>  >>> 410.516.8886
>  >>> rochkind (at) jhu.edu
>  >>>
>  >>> _______________________________________________
>  >>> Mongrel-users mailing list
>  >>> Mongrel-users at rubyforge.org
>  >>> http://rubyforge.org/mailman/listinfo/mongrel-users
>  >>>
>  >>>
>  >> _______________________________________________
>  >> Mongrel-users mailing list
>  >> Mongrel-users at rubyforge.org
>  >> http://rubyforge.org/mailman/listinfo/mongrel-users
>  >>
>  >>
>  >
>  >
>
>  --
>  Jonathan Rochkind
>  Digital Services Software Engineer
>  The Sheridan Libraries
>  Johns Hopkins University
>  410.516.8886
>  rochkind (at) jhu.edu
>
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From eden at mojiti.com  Wed Mar  5 13:07:55 2008
From: eden at mojiti.com (Eden Li)
Date: Wed, 5 Mar 2008 10:07:55 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CEDC4F.5020202@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
Message-ID: <AD93986E-974E-40CB-BA90-89902130303B@mojiti.com>

I use mongrel_rails cluster::* instead of mongrel_cluster_ctl, however  
it looks like you're using an older version of mongrel_cluster.  You  
might try upgrading to 1.0.5.

On Mar 5, 2008, at 9:45 AM, Jonathan Rochkind wrote:

> Actually, I have investigated further, and am pretty sure I am
> duplicating the problem condition properly.
>
> mongrel_cluster_ctl start --clean doesn't seem to work properly, it's
> throwing that error. Hmm.
>
> I am guessing that the script I find in the mongrel_cluster gem at
> resources/mongrel_cluster, which is reccommended on that wiki  
> page... is
> not actually what you all are using.  And perhaps shouldn't be  
> included
> in the distro anymore?   What are others using to ensure mongrel
> clusters start at boot?
>
> Jonathan
>
> Jonathan Rochkind wrote:
>> Thanks that's helpful, I'm investigating that.
>>
>> Using the instructions at:
>>
>> http://mongrel.rubyforge.org/wiki/MongrelCluster#OnBootInitializationSetup
>>
>> I had a little bash script at /etc/init.d/mongrel_cluster that was  
>> copied from
>> /path/to/mongrel_cluster_gem/resources/mongrel_cluster
>>
>> That bash script calls mongrel_cluster_ctl with various options. I  
>> have added a "--clean" option to the "start" condition.  Would it  
>> make sense to suggest this be added to resources/mongrel_cluster in  
>> the actual gem, to not require this change?
>>
>> However, now I'm trying to set up an intentionally created stale  
>> pid, to test this feature. I am invoking mongrel_cluster_ctl  
>> manually (but in the manner of the resources/mongrel_cluster  
>> script) with the --clean option. I get this:
>>
>> ****
>> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>> mongrel_cluster/init.rb:83:in `unlink': No such file or directory -  
>> tmp/pids/mongrel.3000.pid (Errno::ENOENT)
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>> mongrel_cluster/init.rb:83:in `start'
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>> mongrel_cluster/init.rb:79:in `each'
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>> mongrel_cluster/init.rb:79:in `start'
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>> mongrel_cluster/init.rb:232:in `run'
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/ 
>> mongrel/command.rb:212:in `run'
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/ 
>> mongrel_rails:281
>>        from /usr/bin/mongrel_rails:18:in `load'
>>        from /usr/bin/mongrel_rails:18
>> mongrel_rails cluster::start returned an error.
>> *****
>>
>> That was using a mongrel_config.yml file that did have 'cwd' set.   
>> There certainly is a tmp/pids/mongrel.3000.pid off of the 'cwd'  
>> location.  But perhaps the 'unlink' isn't being executed off the  
>> 'cwd' location as it ought to be?
>>
>> Is this a bug in mongrel_cluster_ctl --clean, or perhaps just a bug  
>> in my attempt to duplicate the problem condition and make sure I've  
>> solved it?
>>
>> Thanks for any help.
>>
>> Jonathan
>>
>>
>>
>> Eden Li wrote:
>>
>>> Newer versions of mongrel_cluster have a --clean option which will
>>> remove stale pids if it finds them.  Provide that option to
>>> mongrel_rails cluster::* in your scripts.
>>>
>>> On Mar 5, 2008, at 8:59 AM, Jonathan Rochkind wrote:
>>>
>>>
>>>
>>>> So I am using the instructions here to have mongrel_cluster start  
>>>> my
>>>> various mongrel-deployed apps on server boot.
>>>> http://mongrel.rubyforge.org/wiki/MongrelCluster
>>>>
>>>> There is a problem mentioned here:
>>>> http://www.ruby-forum.com/topic/105849
>>>>
>>>> In summary---if the server (or the mongrel processes) die  
>>>> ungracefully
>>>> leaving their pids behind, then on next boot, mongrel_cluster won't
>>>> succesfully start them up, choking on the leftover pids.
>>>>
>>>> What is the reccommended solution to this?  I'd like to have my
>>>> mongrels
>>>> automatically start on boot, in a way that forces them to start  
>>>> even
>>>> if
>>>> there are stale pids. Thanks for any advice!
>>>>
>>>> [ PS: I just discovered this problem the hard way, when my server
>>>> unexpetedly lost power, and my production apps failed to start up  
>>>> on
>>>> the
>>>> next boot!  Very unpleasant. ]
>>>>
>>>> Jonathan
>>>>
>>>>
>>>> -- 
>>>> Jonathan Rochkind
>>>> Digital Services Software Engineer
>>>> The Sheridan Libraries
>>>> Johns Hopkins University
>>>> 410.516.8886
>>>> rochkind (at) jhu.edu
>>>>
>>>> _______________________________________________
>>>> Mongrel-users mailing list
>>>> Mongrel-users at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>>
>>>>
>>> _______________________________________________
>>> Mongrel-users mailing list
>>> Mongrel-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>
>>>
>>
>>
>
> -- 
> Jonathan Rochkind
> Digital Services Software Engineer
> The Sheridan Libraries
> Johns Hopkins University
> 410.516.8886
> rochkind (at) jhu.edu
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From rochkind at jhu.edu  Wed Mar  5 13:15:57 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 13:15:57 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <AD93986E-974E-40CB-BA90-89902130303B@mojiti.com>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<AD93986E-974E-40CB-BA90-89902130303B@mojiti.com>
Message-ID: <47CEE35D.5030809@jhu.edu>

Believe it or not, I have indeed upgraded to 1.0.5 of mongrel_cluster 
and 1.1.4 of mongrel.  At least I think I have. I have installed the 
newer versions of the gems, and I believe it's what my scripts are using.

What makes you think it looks like I am not?  Because maybe I'm not even 
though I intend to be.

I wonder again, if the sample script distributed with the gem at 
resources/mongrel_cluster doesn't work/isn't supported... should someone 
with commit privs remove it? Or add a comment to it warning it might not 
work?

Jonathan

Eden Li wrote:
> I use mongrel_rails cluster::* instead of mongrel_cluster_ctl, however  
> it looks like you're using an older version of mongrel_cluster.  You  
> might try upgrading to 1.0.5.
>
> On Mar 5, 2008, at 9:45 AM, Jonathan Rochkind wrote:
>
>   
>> Actually, I have investigated further, and am pretty sure I am
>> duplicating the problem condition properly.
>>
>> mongrel_cluster_ctl start --clean doesn't seem to work properly, it's
>> throwing that error. Hmm.
>>
>> I am guessing that the script I find in the mongrel_cluster gem at
>> resources/mongrel_cluster, which is reccommended on that wiki  
>> page... is
>> not actually what you all are using.  And perhaps shouldn't be  
>> included
>> in the distro anymore?   What are others using to ensure mongrel
>> clusters start at boot?
>>
>> Jonathan
>>
>> Jonathan Rochkind wrote:
>>     
>>> Thanks that's helpful, I'm investigating that.
>>>
>>> Using the instructions at:
>>>
>>> http://mongrel.rubyforge.org/wiki/MongrelCluster#OnBootInitializationSetup
>>>
>>> I had a little bash script at /etc/init.d/mongrel_cluster that was  
>>> copied from
>>> /path/to/mongrel_cluster_gem/resources/mongrel_cluster
>>>
>>> That bash script calls mongrel_cluster_ctl with various options. I  
>>> have added a "--clean" option to the "start" condition.  Would it  
>>> make sense to suggest this be added to resources/mongrel_cluster in  
>>> the actual gem, to not require this change?
>>>
>>> However, now I'm trying to set up an intentionally created stale  
>>> pid, to test this feature. I am invoking mongrel_cluster_ctl  
>>> manually (but in the manner of the resources/mongrel_cluster  
>>> script) with the --clean option. I get this:
>>>
>>> ****
>>> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>>> mongrel_cluster/init.rb:83:in `unlink': No such file or directory -  
>>> tmp/pids/mongrel.3000.pid (Errno::ENOENT)
>>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>>> mongrel_cluster/init.rb:83:in `start'
>>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>>> mongrel_cluster/init.rb:79:in `each'
>>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>>> mongrel_cluster/init.rb:79:in `start'
>>>        from /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>>> mongrel_cluster/init.rb:232:in `run'
>>>        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/../lib/ 
>>> mongrel/command.rb:212:in `run'
>>>        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/ 
>>> mongrel_rails:281
>>>        from /usr/bin/mongrel_rails:18:in `load'
>>>        from /usr/bin/mongrel_rails:18
>>> mongrel_rails cluster::start returned an error.
>>> *****
>>>
>>> That was using a mongrel_config.yml file that did have 'cwd' set.   
>>> There certainly is a tmp/pids/mongrel.3000.pid off of the 'cwd'  
>>> location.  But perhaps the 'unlink' isn't being executed off the  
>>> 'cwd' location as it ought to be?
>>>
>>> Is this a bug in mongrel_cluster_ctl --clean, or perhaps just a bug  
>>> in my attempt to duplicate the problem condition and make sure I've  
>>> solved it?
>>>
>>> Thanks for any help.
>>>
>>> Jonathan
>>>
>>>
>>>
>>> Eden Li wrote:
>>>
>>>       
>>>> Newer versions of mongrel_cluster have a --clean option which will
>>>> remove stale pids if it finds them.  Provide that option to
>>>> mongrel_rails cluster::* in your scripts.
>>>>
>>>> On Mar 5, 2008, at 8:59 AM, Jonathan Rochkind wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> So I am using the instructions here to have mongrel_cluster start  
>>>>> my
>>>>> various mongrel-deployed apps on server boot.
>>>>> http://mongrel.rubyforge.org/wiki/MongrelCluster
>>>>>
>>>>> There is a problem mentioned here:
>>>>> http://www.ruby-forum.com/topic/105849
>>>>>
>>>>> In summary---if the server (or the mongrel processes) die  
>>>>> ungracefully
>>>>> leaving their pids behind, then on next boot, mongrel_cluster won't
>>>>> succesfully start them up, choking on the leftover pids.
>>>>>
>>>>> What is the reccommended solution to this?  I'd like to have my
>>>>> mongrels
>>>>> automatically start on boot, in a way that forces them to start  
>>>>> even
>>>>> if
>>>>> there are stale pids. Thanks for any advice!
>>>>>
>>>>> [ PS: I just discovered this problem the hard way, when my server
>>>>> unexpetedly lost power, and my production apps failed to start up  
>>>>> on
>>>>> the
>>>>> next boot!  Very unpleasant. ]
>>>>>
>>>>> Jonathan
>>>>>
>>>>>
>>>>> -- 
>>>>> Jonathan Rochkind
>>>>> Digital Services Software Engineer
>>>>> The Sheridan Libraries
>>>>> Johns Hopkins University
>>>>> 410.516.8886
>>>>> rochkind (at) jhu.edu
>>>>>
>>>>> _______________________________________________
>>>>> Mongrel-users mailing list
>>>>> Mongrel-users at rubyforge.org
>>>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>>>
>>>>>
>>>>>           
>>>> _______________________________________________
>>>> Mongrel-users mailing list
>>>> Mongrel-users at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>>
>>>>
>>>>         
>>>       
>> -- 
>> Jonathan Rochkind
>> Digital Services Software Engineer
>> The Sheridan Libraries
>> Johns Hopkins University
>> 410.516.8886
>> rochkind (at) jhu.edu
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>     
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From simon.santoro at gmail.com  Wed Mar  5 14:16:22 2008
From: simon.santoro at gmail.com (Simon Santoro)
Date: Wed, 5 Mar 2008 20:16:22 +0100
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CED185.3030101@jhu.edu>
References: <47CED185.3030101@jhu.edu>
Message-ID: <200803052016.22601.simon.santoro@gmail.com>

On Wednesday 05 March 2008 17:59:49 Jonathan Rochkind wrote:
> What is the reccommended solution to this?

I don't know, put there is a patch here:
http://textsnippets.com/posts/show/931
and a ticket got submitted here:
http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147

but it was not committed because "mongrel_rails now supports a --clean".
my mongrel_rails does not support it, but I may have an old version.

I think it would be right for mongrel to clean up it's stale pid files. Every 
program I know of does that, and I patched my mongrel.


From eden at mojiti.com  Wed Mar  5 14:35:00 2008
From: eden at mojiti.com (Eden Li)
Date: Wed, 5 Mar 2008 11:35:00 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CEE35D.5030809@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<AD93986E-974E-40CB-BA90-89902130303B@mojiti.com>
	<47CEE35D.5030809@jhu.edu>
Message-ID: <CC913341-6AA5-4B5A-9F8E-00E196A92E1C@mojiti.com>

Your backtrace:

>>> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/
>>> mongrel_cluster/init.rb

You might try removing stale gems by running `sudo gem uninstall  
mongrel_cluster` and removing anything that's not mongrel_cluster  
1.0.5.  Also make sure your /usr/bin/mongrel_* are pointing to the  
right version.

On Mar 5, 2008, at 10:15 AM, Jonathan Rochkind wrote:

> What makes you think it looks like I am not?  Because maybe I'm not  
> even
> though I intend to be.


From rochkind at jhu.edu  Wed Mar  5 14:41:31 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 14:41:31 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <200803052016.22601.simon.santoro@gmail.com>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
Message-ID: <47CEF76B.2000003@jhu.edu>

Yeah, I think that closing of that ticket is right. As far as 
mongrel_rails is concerned, it makes sense to use the --clean option, 
rather than put it in there to happen by default. (although actually, 
I'd have --clean be default).

My problem is that I'm using mongrel_cluster_ctl instead of 
mongrel_rails. The nice feature of mongrel_cluster_ctl here is that you 
can give it a -c argument which is a directory containing mongrel 
cluster yml files. And it will do the invoked operation to ALL of those 
config files. I like this. I want to use it.

And mongrel_cluster_ctl's --clean option appears to be broken to me. I 
guess I should figure out how to fix it and submit a patch?

The other problem is that the sample script at 
resources/mongrel_cluster, which invokes mongrel_cluster_ctl, doesn't 
invoke the --clean option (which wouldn't work right even if it did, but 
once it does). So I guess I should fix that and submit a patch too? 

I'm new to submitting patches for this kind of stuff. Can anyone 
reccommend patch etiquette/procedure to do this properly? And/or confirm 
that I'm approaching this sanely?

Thanks for the help, all.

Jonathan

Simon Santoro wrote:
> On Wednesday 05 March 2008 17:59:49 Jonathan Rochkind wrote:
>   
>> What is the reccommended solution to this?
>>     
>
> I don't know, put there is a patch here:
> http://textsnippets.com/posts/show/931
> and a ticket got submitted here:
> http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
>
> but it was not committed because "mongrel_rails now supports a --clean".
> my mongrel_rails does not support it, but I may have an old version.
>
> I think it would be right for mongrel to clean up it's stale pid files. Every 
> program I know of does that, and I patched my mongrel.
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From leccine at gmail.com  Wed Mar  5 14:38:02 2008
From: leccine at gmail.com (Istvan Szukacs)
Date: Wed, 05 Mar 2008 20:38:02 +0100
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CED185.3030101@jhu.edu>
References: <47CED185.3030101@jhu.edu>
Message-ID: <47CEF69A.9040303@gmail.com>


to write a script which checks the running pids and pid files and if 
there is staled pid just remove it?
(and after let the normal starting process run)

mongrel_rails and mongrel_cluster_ctl are written in ruby,
you will not hurt the original library with this modification

if the --clean is not working well...

istvan,
http://weho.st

Jonathan Rochkind wrote:
> So I am using the instructions here to have mongrel_cluster start my 
> various mongrel-deployed apps on server boot.
> http://mongrel.rubyforge.org/wiki/MongrelCluster
>
> There is a problem mentioned here:
> http://www.ruby-forum.com/topic/105849
>
> In summary---if the server (or the mongrel processes) die ungracefully 
> leaving their pids behind, then on next boot, mongrel_cluster won't 
> succesfully start them up, choking on the leftover pids.
>
> What is the reccommended solution to this?  I'd like to have my mongrels 
> automatically start on boot, in a way that forces them to start even if 
> there are stale pids. Thanks for any advice!
>
> [ PS: I just discovered this problem the hard way, when my server 
> unexpetedly lost power, and my production apps failed to start up on the 
> next boot!  Very unpleasant. ]
>
> Jonathan
>
>
>   


From rochkind at jhu.edu  Wed Mar  5 14:52:08 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 14:52:08 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <CC913341-6AA5-4B5A-9F8E-00E196A92E1C@mojiti.com>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<AD93986E-974E-40CB-BA90-89902130303B@mojiti.com>
	<47CEE35D.5030809@jhu.edu>
	<CC913341-6AA5-4B5A-9F8E-00E196A92E1C@mojiti.com>
Message-ID: <47CEF9E8.80108@jhu.edu>

Aha. Thanks. This is curious. I would actually like to have both 
versions of mongrel_cluster installed at the moment, so I can easily 
switch back if needed. gem is supposed to be capable of this of course.

I'm confused as to why my scirpt in /usr/bin/mongrel_cluster_ctl is 
calling the 1.0.2 version, when I have 1.0.5 installed. When I look at 
the very brief code in /usr/bin/mongrel_cluster_ctl, it looks like it 
should use the latest mongrel_cluster stuff installed. Here's what's in 
/usr/bin/mongrel_cluster, anyone know why this would be using 1.0.2 even 
though I have 1.0.5 installed?

#!/usr/bin/env ruby
#
# This file was generated by RubyGems.
#
# The application 'mongrel_cluster' is installed as part of a gem, and
# this file is here to facilitate running it.
#

require 'rubygems'
version = "> 0"
if ARGV.size > 0 && ARGV[0][0]==95 && ARGV[0][-1]==95
  if Gem::Version.correct?(ARGV[0][1..-2])
    version = ARGV[0][1..-2]
    ARGV.shift
  end
end
require_gem 'mongrel_cluster', version
load 'mongrel_cluster_ctl'


Eden Li wrote:
> Your backtrace:
>
>   
>>>> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/
>>>> mongrel_cluster/init.rb
>>>>         
>
> You might try removing stale gems by running `sudo gem uninstall  
> mongrel_cluster` and removing anything that's not mongrel_cluster  
> 1.0.5.  Also make sure your /usr/bin/mongrel_* are pointing to the  
> right version.
>
> On Mar 5, 2008, at 10:15 AM, Jonathan Rochkind wrote:
>
>   
>> What makes you think it looks like I am not?  Because maybe I'm not  
>> even
>> though I intend to be.
>>     
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From rochkind at jhu.edu  Wed Mar  5 15:02:05 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 15:02:05 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CEF76B.2000003@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
	<47CEF76B.2000003@jhu.edu>
Message-ID: <47CEFC3D.4050604@jhu.edu>

Okay, I figured out more. Sorry for spamming the list. This is 
frustrating me.

I deleted mongrel_cluster 1.0.2 entirely. But check out this stack 
trace. Why the heck is mongrel-1.1.4/bin/mongrel_rails requiring 
mongrel_rails 1.0.2 and refusing to use mongrel_rails 1.0.5?

/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in 
`gem_original_require': no such file to load -- 
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/mongrel_cluster/init.rb 
(LoadError)
        from 
/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
        from 
/usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb:134:in `load'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:142:in 
`each'
        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb:142:in 
`each'
        from 
/usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb:112:in `load'
        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/mongrel_rails:278
        from /usr/bin/mongrel_rails:18:in `load'
        from /usr/bin/mongrel_rails:18


Jonathan Rochkind wrote:
> Yeah, I think that closing of that ticket is right. As far as 
> mongrel_rails is concerned, it makes sense to use the --clean option, 
> rather than put it in there to happen by default. (although actually, 
> I'd have --clean be default).
>
> My problem is that I'm using mongrel_cluster_ctl instead of 
> mongrel_rails. The nice feature of mongrel_cluster_ctl here is that you 
> can give it a -c argument which is a directory containing mongrel 
> cluster yml files. And it will do the invoked operation to ALL of those 
> config files. I like this. I want to use it.
>
> And mongrel_cluster_ctl's --clean option appears to be broken to me. I 
> guess I should figure out how to fix it and submit a patch?
>
> The other problem is that the sample script at 
> resources/mongrel_cluster, which invokes mongrel_cluster_ctl, doesn't 
> invoke the --clean option (which wouldn't work right even if it did, but 
> once it does). So I guess I should fix that and submit a patch too? 
>
> I'm new to submitting patches for this kind of stuff. Can anyone 
> reccommend patch etiquette/procedure to do this properly? And/or confirm 
> that I'm approaching this sanely?
>
> Thanks for the help, all.
>
> Jonathan
>
> Simon Santoro wrote:
>   
>> On Wednesday 05 March 2008 17:59:49 Jonathan Rochkind wrote:
>>   
>>     
>>> What is the reccommended solution to this?
>>>     
>>>       
>> I don't know, put there is a patch here:
>> http://textsnippets.com/posts/show/931
>> and a ticket got submitted here:
>> http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
>>
>> but it was not committed because "mongrel_rails now supports a --clean".
>> my mongrel_rails does not support it, but I may have an old version.
>>
>> I think it would be right for mongrel to clean up it's stale pid files. Every 
>> program I know of does that, and I patched my mongrel.
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>   
>>     
>
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From eden at mojiti.com  Wed Mar  5 15:20:53 2008
From: eden at mojiti.com (Eden Li)
Date: Wed, 5 Mar 2008 12:20:53 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CEFC3D.4050604@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
	<47CEF76B.2000003@jhu.edu> <47CEFC3D.4050604@jhu.edu>
Message-ID: <0FABB3FD-3713-4455-9C09-1073808AC4C9@mojiti.com>

try

sudo rm /usr/lib/ruby/gems/1.8/specifications/ 
mongrel_cluster-1.0.2.gemspec

i think your specs are stale

On Mar 5, 2008, at 12:02 PM, Jonathan Rochkind wrote:

> Okay, I figured out more. Sorry for spamming the list. This is
> frustrating me.
>
> I deleted mongrel_cluster 1.0.2 entirely. But check out this stack
> trace. Why the heck is mongrel-1.1.4/bin/mongrel_rails requiring
> mongrel_rails 1.0.2 and refusing to use mongrel_rails 1.0.5?
>
> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
> `gem_original_require': no such file to load --
> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
> mongrel_cluster/init.rb
> (LoadError)
>        from
> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
>        from
> /usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb: 
> 134:in `load'
>        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb: 
> 142:in
> `each'
>        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb: 
> 142:in
> `each'
>        from
> /usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb: 
> 112:in `load'
>        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/ 
> mongrel_rails:278
>        from /usr/bin/mongrel_rails:18:in `load'
>        from /usr/bin/mongrel_rails:18
>
>
> Jonathan Rochkind wrote:
>> Yeah, I think that closing of that ticket is right. As far as
>> mongrel_rails is concerned, it makes sense to use the --clean option,
>> rather than put it in there to happen by default. (although actually,
>> I'd have --clean be default).
>>
>> My problem is that I'm using mongrel_cluster_ctl instead of
>> mongrel_rails. The nice feature of mongrel_cluster_ctl here is that  
>> you
>> can give it a -c argument which is a directory containing mongrel
>> cluster yml files. And it will do the invoked operation to ALL of  
>> those
>> config files. I like this. I want to use it.
>>
>> And mongrel_cluster_ctl's --clean option appears to be broken to  
>> me. I
>> guess I should figure out how to fix it and submit a patch?
>>
>> The other problem is that the sample script at
>> resources/mongrel_cluster, which invokes mongrel_cluster_ctl, doesn't
>> invoke the --clean option (which wouldn't work right even if it  
>> did, but
>> once it does). So I guess I should fix that and submit a patch too?
>>
>> I'm new to submitting patches for this kind of stuff. Can anyone
>> reccommend patch etiquette/procedure to do this properly? And/or  
>> confirm
>> that I'm approaching this sanely?
>>
>> Thanks for the help, all.
>>
>> Jonathan
>>
>> Simon Santoro wrote:
>>
>>> On Wednesday 05 March 2008 17:59:49 Jonathan Rochkind wrote:
>>>
>>>
>>>> What is the reccommended solution to this?
>>>>
>>>>
>>> I don't know, put there is a patch here:
>>> http://textsnippets.com/posts/show/931
>>> and a ticket got submitted here:
>>> http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
>>>
>>> but it was not committed because "mongrel_rails now supports a -- 
>>> clean".
>>> my mongrel_rails does not support it, but I may have an old version.
>>>
>>> I think it would be right for mongrel to clean up it's stale pid  
>>> files. Every
>>> program I know of does that, and I patched my mongrel.
>>>
>>> _______________________________________________
>>> Mongrel-users mailing list
>>> Mongrel-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>
>>>
>>
>>
>
> -- 
> Jonathan Rochkind
> Digital Services Software Engineer
> The Sheridan Libraries
> Johns Hopkins University
> 410.516.8886
> rochkind (at) jhu.edu
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From rochkind at jhu.edu  Wed Mar  5 15:29:23 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 15:29:23 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <0FABB3FD-3713-4455-9C09-1073808AC4C9@mojiti.com>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
	<47CEF76B.2000003@jhu.edu> <47CEFC3D.4050604@jhu.edu>
	<0FABB3FD-3713-4455-9C09-1073808AC4C9@mojiti.com>
Message-ID: <47CF02A3.6020805@jhu.edu>

Okay, you're right. That did work, I didn't do a good job of removing 
mongrel_cluster-1.0.2 before. The question still remains, why the heck 
do I need to actually REMOVE 1.0.2 to get mongrel 1.1.4 to use it, when 
there's a newer mongrel_cluster 1.0.5 installed? This shouldn't actually 
be, should it?  Aren't you supposed to be able to have both installed in 
gems at once?   Anyone have any explanation for why mongrel was 
preferring the older one when both were installed? [It's this kind of 
thing that drives me nuts with ruby/rails, especially with deployment 
issues].

Now that I've gotten it to use mongrel_cluster 1.0.5, 
mongrel_cluster_ctl is indeed properly dealing with the --clean 
argument, that bug is gone.

Very frustrating. Thanks for the help all. Still plan to submit a patch 
for the resources/mongrel_cluster script to use --clean. Does this make 
sense?  That resource/mongrel_cluster script is mentioned in the 
mongrel_cluster README still, so it does seem like it's supposed to be 
still a non-deprecated thing?

Jonathan

Eden Li wrote:
> try
>
> sudo rm /usr/lib/ruby/gems/1.8/specifications/ 
> mongrel_cluster-1.0.2.gemspec
>
> i think your specs are stale
>
> On Mar 5, 2008, at 12:02 PM, Jonathan Rochkind wrote:
>
>   
>> Okay, I figured out more. Sorry for spamming the list. This is
>> frustrating me.
>>
>> I deleted mongrel_cluster 1.0.2 entirely. But check out this stack
>> trace. Why the heck is mongrel-1.1.4/bin/mongrel_rails requiring
>> mongrel_rails 1.0.2 and refusing to use mongrel_rails 1.0.5?
>>
>> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
>> `gem_original_require': no such file to load --
>> /usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/lib/ 
>> mongrel_cluster/init.rb
>> (LoadError)
>>        from
>> /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'
>>        from
>> /usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb: 
>> 134:in `load'
>>        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb: 
>> 142:in
>> `each'
>>        from /usr/lib/ruby/site_ruby/1.8/rubygems/source_index.rb: 
>> 142:in
>> `each'
>>        from
>> /usr/lib/ruby/gems/1.8/gems/gem_plugin-0.2.3/lib/gem_plugin.rb: 
>> 112:in `load'
>>        from /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/bin/ 
>> mongrel_rails:278
>>        from /usr/bin/mongrel_rails:18:in `load'
>>        from /usr/bin/mongrel_rails:18
>>
>>
>> Jonathan Rochkind wrote:
>>     
>>> Yeah, I think that closing of that ticket is right. As far as
>>> mongrel_rails is concerned, it makes sense to use the --clean option,
>>> rather than put it in there to happen by default. (although actually,
>>> I'd have --clean be default).
>>>
>>> My problem is that I'm using mongrel_cluster_ctl instead of
>>> mongrel_rails. The nice feature of mongrel_cluster_ctl here is that  
>>> you
>>> can give it a -c argument which is a directory containing mongrel
>>> cluster yml files. And it will do the invoked operation to ALL of  
>>> those
>>> config files. I like this. I want to use it.
>>>
>>> And mongrel_cluster_ctl's --clean option appears to be broken to  
>>> me. I
>>> guess I should figure out how to fix it and submit a patch?
>>>
>>> The other problem is that the sample script at
>>> resources/mongrel_cluster, which invokes mongrel_cluster_ctl, doesn't
>>> invoke the --clean option (which wouldn't work right even if it  
>>> did, but
>>> once it does). So I guess I should fix that and submit a patch too?
>>>
>>> I'm new to submitting patches for this kind of stuff. Can anyone
>>> reccommend patch etiquette/procedure to do this properly? And/or  
>>> confirm
>>> that I'm approaching this sanely?
>>>
>>> Thanks for the help, all.
>>>
>>> Jonathan
>>>
>>> Simon Santoro wrote:
>>>
>>>       
>>>> On Wednesday 05 March 2008 17:59:49 Jonathan Rochkind wrote:
>>>>
>>>>
>>>>         
>>>>> What is the reccommended solution to this?
>>>>>
>>>>>
>>>>>           
>>>> I don't know, put there is a patch here:
>>>> http://textsnippets.com/posts/show/931
>>>> and a ticket got submitted here:
>>>> http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
>>>>
>>>> but it was not committed because "mongrel_rails now supports a -- 
>>>> clean".
>>>> my mongrel_rails does not support it, but I may have an old version.
>>>>
>>>> I think it would be right for mongrel to clean up it's stale pid  
>>>> files. Every
>>>> program I know of does that, and I patched my mongrel.
>>>>
>>>> _______________________________________________
>>>> Mongrel-users mailing list
>>>> Mongrel-users at rubyforge.org
>>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>>
>>>>
>>>>         
>>>       
>> -- 
>> Jonathan Rochkind
>> Digital Services Software Engineer
>> The Sheridan Libraries
>> Johns Hopkins University
>> 410.516.8886
>> rochkind (at) jhu.edu
>>
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>     
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From eden at mojiti.com  Wed Mar  5 17:02:51 2008
From: eden at mojiti.com (Eden Li)
Date: Wed, 5 Mar 2008 14:02:51 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <47CF02A3.6020805@jhu.edu>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
	<47CEF76B.2000003@jhu.edu> <47CEFC3D.4050604@jhu.edu>
	<0FABB3FD-3713-4455-9C09-1073808AC4C9@mojiti.com>
	<47CF02A3.6020805@jhu.edu>
Message-ID: <67617D18-01BC-485C-ABA1-C1A3937FE3BD@mojiti.com>

I'm not sure why it prefers the oldest version, but the  
mongrel_cluster_ctl script effectively runs:

require_gem "mongrel_cluster", "> 0"

which seems like it would prefer the oldest.  reading the script a tad  
more, it looks like you can specify which version to run by passing in  
_VERSION_ (e.g. _1.0.5_) as the first argument to mongrel_cluster_ctl

On Mar 5, 2008, at 12:29 PM, Jonathan Rochkind wrote:

> Okay, you're right. That did work, I didn't do a good job of removing
> mongrel_cluster-1.0.2 before. The question still remains, why the heck
> do I need to actually REMOVE 1.0.2 to get mongrel 1.1.4 to use it,  
> when
> there's a newer mongrel_cluster 1.0.5 installed? This shouldn't  
> actually
> be, should it?  Aren't you supposed to be able to have both  
> installed in
> gems at once?   Anyone have any explanation for why mongrel was
> preferring the older one when both were installed? [It's this kind of
> thing that drives me nuts with ruby/rails, especially with deployment
> issues].
>
> Now that I've gotten it to use mongrel_cluster 1.0.5,
> mongrel_cluster_ctl is indeed properly dealing with the --clean
> argument, that bug is gone.
>
> Very frustrating. Thanks for the help all. Still plan to submit a  
> patch
> for the resources/mongrel_cluster script to use --clean. Does this  
> make
> sense?  That resource/mongrel_cluster script is mentioned in the
> mongrel_cluster README still, so it does seem like it's supposed to be
> still a non-deprecated thing?


From mohammed.a at aol.in  Wed Mar  5 17:04:21 2008
From: mohammed.a at aol.in (mohammed.a at aol.in)
Date: Wed, 05 Mar 2008 17:04:21 -0500
Subject: [Mongrel] problems with Mongrel + Pound
In-Reply-To: <325148f70802270020g55b091fay5f0ef84bc415eccc@mail.gmail.com>
References: <8CA466852327A10-D0C-3759@web-cen-v02.sim.aol.com>	<24E69670-2EB4-43B1-9897-46314FD06857@cheney.net>	<8CA467BA54ED6F0-D0C-3859@web-cen-v02.sim.aol.com>	<CBC7DB32-2A25-4D0C-BF00-2D22068D4F90@gmail.com>	<E49881E1-19D5-45E6-B2BA-C78A89326F65@airbladesoftware.com>	<8CA472B61107470-D0C-3E7E@web-cen-v02.sim.aol.com>	<00B61B4A-1D5E-4EEA-88F8-613DB9A02EFB@cheney.net>
	<325148f70802270020g55b091fay5f0ef84bc415eccc@mail.gmail.com>
Message-ID: <8CA4D23FFA9B560-7A8-2CB2@web-cen-v06.sim.aol.com>


 Switching to NGINX indeed fixed much of the problems. But mysql gem continues to evade no matter what kind of config I use. 
But SQL has been tuned up, major bugs have been tuned.

Much better now. Will post later with detailed analysis of the problem and solutions

regards

Mohammed. A


 


 

-----Original Message-----
From: Piyush Ranjan <piyush.pr at gmail.com>
To: mongrel-users at rubyforge.org
Sent: Wed, 27 Feb 2008 1:50 pm
Subject: Re: [Mongrel] problems with Mongrel + Pound









To install mysql gem you need to specify the mysql_config path(if it is not installed then into mysql dev or devel package)
To install sudo gem install mysql -- --with-mysql-config=<path-to-mysql_config like on my ubuntu it is /usr/bin/mysql_config>



And your problems are similar(not same) I used to encounter. I had these problems with my mongrel handler when it used to? write to database in asynchronous queries by the users. Log file used to scream connection lost to mysql. After spending a few days trying to boost up my mysql server I did the gem install and since then it has handled 100K + queries (the mongrel handler part of the app)each day without any problem whatsoever. 


Also switch to apache/ngnix if you can. Pound could also be the trouble maker though I am not very sure about this comment.
 

On Wed, Feb 27, 2008 at 1:29 PM, Dave Cheney <dave at cheney.net> wrote:



I hacked our vendor tree to explode on startup if the mysql gem could not be found. IMHO this should be the default?behavior,?the built in mysql adapter is crap and its silent use is a ticking time bomb




Cheers




Dave







On 27/02/2008, at 6:42 PM, mohammed.a at aol.in wrote:



Thank you for the replies.


Here's what I found - mysql gem was not installed and that maybe a core source of problems as suggested by Zed. We've had? too many problems when we tried to install this, and found that none of the solutions that are mentioned really work. We will try to install this gem again.?







_______________________________________________

Mongrel-users mailing list

Mongrel-users at rubyforge.org

http://rubyforge.org/mailman/listinfo/mongrel-users






 





_______________________________________________
Mongrel-users mailing list
Mongrel-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-users



 


________________________________________________________________________
You are invited to Get a Free AOL Email ID. - http://webmail.aol.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080305/b37322c5/attachment-0001.html 

From rochkind at jhu.edu  Wed Mar  5 17:31:51 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Wed, 05 Mar 2008 17:31:51 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <67617D18-01BC-485C-ABA1-C1A3937FE3BD@mojiti.com>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
	<47CEF76B.2000003@jhu.edu> <47CEFC3D.4050604@jhu.edu>
	<0FABB3FD-3713-4455-9C09-1073808AC4C9@mojiti.com>
	<47CF02A3.6020805@jhu.edu>
	<67617D18-01BC-485C-ABA1-C1A3937FE3BD@mojiti.com>
Message-ID: <47CF1F57.5040702@jhu.edu>

Should I try to patch this to prefer the newest version instead?

If anyone's actually been following along, I filed a patch to fix 
resources/mongrel_cluster to pass --clean in.

http://rubyforge.org/tracker/index.php?func=detail&aid=18627&group_id=1306&atid=5147

Jonathan

Eden Li wrote:
> I'm not sure why it prefers the oldest version, but the  
> mongrel_cluster_ctl script effectively runs:
>
> require_gem "mongrel_cluster", "> 0"
>
> which seems like it would prefer the oldest.  reading the script a tad  
> more, it looks like you can specify which version to run by passing in  
> _VERSION_ (e.g. _1.0.5_) as the first argument to mongrel_cluster_ctl
>
> On Mar 5, 2008, at 12:29 PM, Jonathan Rochkind wrote:
>
>   
>> Okay, you're right. That did work, I didn't do a good job of removing
>> mongrel_cluster-1.0.2 before. The question still remains, why the heck
>> do I need to actually REMOVE 1.0.2 to get mongrel 1.1.4 to use it,  
>> when
>> there's a newer mongrel_cluster 1.0.5 installed? This shouldn't  
>> actually
>> be, should it?  Aren't you supposed to be able to have both  
>> installed in
>> gems at once?   Anyone have any explanation for why mongrel was
>> preferring the older one when both were installed? [It's this kind of
>> thing that drives me nuts with ruby/rails, especially with deployment
>> issues].
>>
>> Now that I've gotten it to use mongrel_cluster 1.0.5,
>> mongrel_cluster_ctl is indeed properly dealing with the --clean
>> argument, that bug is gone.
>>
>> Very frustrating. Thanks for the help all. Still plan to submit a  
>> patch
>> for the resources/mongrel_cluster script to use --clean. Does this  
>> make
>> sense?  That resource/mongrel_cluster script is mentioned in the
>> mongrel_cluster README still, so it does seem like it's supposed to be
>> still a non-deprecated thing?
>>     
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From jos at catnook.com  Thu Mar  6 12:29:37 2008
From: jos at catnook.com (Jos Backus)
Date: Thu, 6 Mar 2008 09:29:37 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
Message-ID: <20080306172937.GA24906@lizzy.catnook.local>

On Wed, Mar 05, 2008 at 11:57:44AM -0600, Nate Vack wrote:
> I'm doing everything (everything!) via either monit (on linux) or SMF
> (on Solaris).
> 
> Using a process manager is a good idea... and having only one
> 'production' way to start mongrel is generally a good thing, too :)

If only Mongrel/mongrel_cluster would work with daemontools/runit. Pid files
are evil.

(Maybe Thin will grow support for this;
http://thin.lighthouseapp.com/projects/7212/tickets/40-support-running-thin-under-daemontools-runit)

-- 
Jos Backus
jos at catnook.com

From alexey.verkhovsky at gmail.com  Thu Mar  6 14:41:11 2008
From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky)
Date: Thu, 6 Mar 2008 12:41:11 -0700
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <20080306172937.GA24906@lizzy.catnook.local>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
	<20080306172937.GA24906@lizzy.catnook.local>
Message-ID: <3945c4270803061141s5b2c66d2pbf23c8affb9bd4f9@mail.gmail.com>

On Thu, Mar 6, 2008 at 10:29 AM, Jos Backus <jos at catnook.com> wrote:
>  If only Mongrel/mongrel_cluster would work with daemontools/runit. Pid files
>  are evil.

Mongrel runs perfectly well with runit.

-- 
Alexey Verkhovsky
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
RubyWorks [http://rubyworks.thoughtworks.com]

From jeremy at bitsweat.net  Thu Mar  6 14:51:35 2008
From: jeremy at bitsweat.net (Jeremy Kemper)
Date: Thu, 6 Mar 2008 11:51:35 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <20080306172937.GA24906@lizzy.catnook.local>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
	<20080306172937.GA24906@lizzy.catnook.local>
Message-ID: <69a2885c0803061151v312aaa79h58b584bd0b4de10f@mail.gmail.com>

On Thu, Mar 6, 2008 at 9:29 AM, Jos Backus <jos at catnook.com> wrote:
> On Wed, Mar 05, 2008 at 11:57:44AM -0600, Nate Vack wrote:
>  > I'm doing everything (everything!) via either monit (on linux) or SMF
>  > (on Solaris).
>  >
>  > Using a process manager is a good idea... and having only one
>  > 'production' way to start mongrel is generally a good thing, too :)
>
>  If only Mongrel/mongrel_cluster would work with daemontools/runit. Pid files
>  are evil.
>
>  (Maybe Thin will grow support for this;
>  http://thin.lighthouseapp.com/projects/7212/tickets/40-support-running-thin-under-daemontools-runit)

Mongrel works great with process supervision. We use runit
extensively; it has eliminated all the pidfile & deployment headaches
discussed here.

jeremy

From drbuettner at gmail.com  Thu Mar  6 15:15:15 2008
From: drbuettner at gmail.com (Dan Buettner)
Date: Thu, 6 Mar 2008 20:15:15 +0000
Subject: [Mongrel] mongrel and uploaded files (temp location)
Message-ID: <c97ba7a80803061215g2ef7e854s59d478aa2a478e20@mail.gmail.com>

Asked this about a month ago but no response - would appreciate any thoughts
anyone might have.

I'd like to change where mongrel stores files uploaded via web form from the
local tmp dir to a place on the same filesystem where they will eventually
reside, due to speed considerations (copying from tmp to NFS mount takes a
little time while the user wait for the web form to return)

Is there a way to do this do you think?

Thanks for any info,
Dan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080306/6134391c/attachment.html 

From wyhaines at gmail.com  Thu Mar  6 15:26:23 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Thu, 6 Mar 2008 13:26:23 -0700
Subject: [Mongrel] mongrel and uploaded files (temp location)
In-Reply-To: <c97ba7a80803061215g2ef7e854s59d478aa2a478e20@mail.gmail.com>
References: <c97ba7a80803061215g2ef7e854s59d478aa2a478e20@mail.gmail.com>
Message-ID: <f4cd26df0803061226q3b345e37j81b60e38e28e653e@mail.gmail.com>

On Thu, Mar 6, 2008 at 1:15 PM, Dan Buettner <drbuettner at gmail.com> wrote:

> I'd like to change where mongrel stores files uploaded via web form from the
> local tmp dir to a place on the same filesystem where they will eventually
> reside, due to speed considerations (copying from tmp to NFS mount takes a
> little time while the user wait for the web form to return)
>
> Is there a way to do this do you think?

There's a way to do most anything.

Mongrel just uses Tempfile.  Tempfile just uses Dir::tmpdir, provided
by tmpdir.rb, to determine the temporary directory.

Dir::tmpdir defaults to /tmp, but it checks some environment variables first.

ENV['TMPDIR']
ENV['TMP']
ENV['TEMP']
ENV['USERPROFILE']

Just set one of those to the directory that you want it to use.



Kirk Haines

From johnjosephbachir at gmail.com  Thu Mar  6 18:31:56 2008
From: johnjosephbachir at gmail.com (John Joseph Bachir)
Date: Thu, 6 Mar 2008 18:31:56 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <200803052016.22601.simon.santoro@gmail.com>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
Message-ID: <4a3752180803061531v3f001dceycdd2d1aeeb6e1d53@mail.gmail.com>

On Wed, Mar 5, 2008 at 2:16 PM, Simon Santoro <simon.santoro at gmail.com> wrote:
>  there is a patch here:
>  http://textsnippets.com/posts/show/931
>  and a ticket got submitted here:
>  http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
>
>  but it was not committed because "mongrel_rails now supports a --clean".
>  my mongrel_rails does not support it, but I may have an old version.
>
>  I think it would be right for mongrel to clean up it's stale pid files. Every
>  program I know of does that, and I patched my mongrel.


Furthermore, the behavior of the --clean feature isn't very
elaborately documented (at least not in the output of mongrel_rails
cluster::start -h). I left this comment on the ticket (no way to
reopen it I guess):

----
The patch mentioned checks if the pid files refer to running processes.

Does the --clean feature do this? Or does it just blindly remove the pid files?

An additionally nice and easy feature would be to check if the running
process contains the string "mongrel".


John

-- 
John Joseph Bachir
http://blog.johnjosephbachir.org
http://lyceum.ibiblio.org
http://dissent.cc
http://jjb.cc

From jos at catnook.com  Thu Mar  6 20:39:19 2008
From: jos at catnook.com (Jos Backus)
Date: Thu, 6 Mar 2008 17:39:19 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <69a2885c0803061151v312aaa79h58b584bd0b4de10f@mail.gmail.com>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
	<20080306172937.GA24906@lizzy.catnook.local>
	<69a2885c0803061151v312aaa79h58b584bd0b4de10f@mail.gmail.com>
Message-ID: <20080307013919.GA11168@lizzy.catnook.local>

On Thu, Mar 06, 2008 at 11:51:35AM -0800, Jeremy Kemper wrote:
> Mongrel works great with process supervision. We use runit
> extensively; it has eliminated all the pidfile & deployment headaches
> discussed here.
 
So how does this work? What watches the multiple mongrels running in the
foreground? Running each mongrel process as a separate service is ugly imo, it
would be much nicer to have a supervised multi_mongrel controller (that's the
idea described in the Thin feature request).

-- 
Jos Backus
jos at catnook.com

From alexey.verkhovsky at gmail.com  Fri Mar  7 01:42:04 2008
From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky)
Date: Thu, 6 Mar 2008 23:42:04 -0700
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <20080307013919.GA11168@lizzy.catnook.local>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
	<20080306172937.GA24906@lizzy.catnook.local>
	<69a2885c0803061151v312aaa79h58b584bd0b4de10f@mail.gmail.com>
	<20080307013919.GA11168@lizzy.catnook.local>
Message-ID: <3945c4270803062242l534fcb9dteaa31c2f8cedf641@mail.gmail.com>

On Thu, Mar 6, 2008 at 6:39 PM, Jos Backus <jos at catnook.com> wrote:
>  So how does this work? What watches the multiple mongrels running in the
>  foreground? Running each mongrel process as a separate service is ugly imo, it
>  would be much nicer to have a supervised multi_mongrel controller (that's the
>  idea described in the Thin feature request).

Here is one example of how this can be done:
http://rubyworks.rubyforge.org/manual/runit.html

-- 
Alexey Verkhovsky
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
RubyWorks [http://rubyworks.thoughtworks.com]

From rochkind at jhu.edu  Fri Mar  7 09:56:45 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Fri, 07 Mar 2008 09:56:45 -0500
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <4a3752180803061531v3f001dceycdd2d1aeeb6e1d53@mail.gmail.com>
References: <47CED185.3030101@jhu.edu>
	<200803052016.22601.simon.santoro@gmail.com>
	<4a3752180803061531v3f001dceycdd2d1aeeb6e1d53@mail.gmail.com>
Message-ID: <47D157AD.6010604@jhu.edu>

The --clean feature, in my testing, seems to indeed checks to make sure 
that pid files really -are- stale (ie, the pid listed in there does not 
correspond to any running process) before deleting them. 

I still don't understand why --clean isn't the default behavior though, 
but I guess the developer didn't like that for some reason. Seems to me 
documentation should recommend everyone always use --clean----especially 
in any automated scripts that will be run unattended!

Jonathan

John Joseph Bachir wrote:
> On Wed, Mar 5, 2008 at 2:16 PM, Simon Santoro <simon.santoro at gmail.com> wrote:
>   
>>  there is a patch here:
>>  http://textsnippets.com/posts/show/931
>>  and a ticket got submitted here:
>>  http://rubyforge.org/tracker/index.php?func=detail&aid=11098&group_id=1306&atid=5147
>>
>>  but it was not committed because "mongrel_rails now supports a --clean".
>>  my mongrel_rails does not support it, but I may have an old version.
>>
>>  I think it would be right for mongrel to clean up it's stale pid files. Every
>>  program I know of does that, and I patched my mongrel.
>>     
>
>
> Furthermore, the behavior of the --clean feature isn't very
> elaborately documented (at least not in the output of mongrel_rails
> cluster::start -h). I left this comment on the ticket (no way to
> reopen it I guess):
>
> ----
> The patch mentioned checks if the pid files refer to running processes.
>
> Does the --clean feature do this? Or does it just blindly remove the pid files?
>
> An additionally nice and easy feature would be to check if the running
> process contains the string "mongrel".
>
>
> John
>
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From jos at catnook.com  Fri Mar  7 12:51:18 2008
From: jos at catnook.com (Jos Backus)
Date: Fri, 7 Mar 2008 09:51:18 -0800
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <3945c4270803062242l534fcb9dteaa31c2f8cedf641@mail.gmail.com>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
	<20080306172937.GA24906@lizzy.catnook.local>
	<69a2885c0803061151v312aaa79h58b584bd0b4de10f@mail.gmail.com>
	<20080307013919.GA11168@lizzy.catnook.local>
	<3945c4270803062242l534fcb9dteaa31c2f8cedf641@mail.gmail.com>
Message-ID: <20080307175118.GA72219@lizzy.catnook.local>

On Thu, Mar 06, 2008 at 11:42:04PM -0700, Alexey Verkhovsky wrote:
> On Thu, Mar 6, 2008 at 6:39 PM, Jos Backus <jos at catnook.com> wrote:
> >  So how does this work? What watches the multiple mongrels running in the
> >  foreground? Running each mongrel process as a separate service is ugly imo, it
> >  would be much nicer to have a supervised multi_mongrel controller (that's the
> >  idea described in the Thin feature request).
> 
> Here is one example of how this can be done:
> http://rubyworks.rubyforge.org/manual/runit.html

Looks like a separate service is used for each Mongrel. Of course that can be
done easily. The solution I'm looking for manages multiple Mongrels under a
single service, just like Apache manages its own worker children.

Thanks for the link, Alexey.

-- 
Jos Backus
jos at catnook.com

From njvack at wisc.edu  Fri Mar  7 13:10:23 2008
From: njvack at wisc.edu (Nate Vack)
Date: Fri, 7 Mar 2008 12:10:23 -0600
Subject: [Mongrel] mongrel_cluster, start on boot, stale pids
In-Reply-To: <20080307175118.GA72219@lizzy.catnook.local>
References: <47CED185.3030101@jhu.edu>
	<A26294D7-D427-416D-9F40-E1E9D2409598@mojiti.com>
	<47CEDB0A.6070406@jhu.edu> <47CEDC4F.5020202@jhu.edu>
	<8be813400803050957xa51710cxf7c67e567694db64@mail.gmail.com>
	<20080306172937.GA24906@lizzy.catnook.local>
	<69a2885c0803061151v312aaa79h58b584bd0b4de10f@mail.gmail.com>
	<20080307013919.GA11168@lizzy.catnook.local>
	<3945c4270803062242l534fcb9dteaa31c2f8cedf641@mail.gmail.com>
	<20080307175118.GA72219@lizzy.catnook.local>
Message-ID: <8be813400803071010i4f365bcav69ec6ce1771eb7cb@mail.gmail.com>

On Fri, Mar 7, 2008 at 11:51 AM, Jos Backus <jos at catnook.com> wrote:

>  Looks like a separate service is used for each Mongrel. Of course that can be
>  done easily. The solution I'm looking for manages multiple Mongrels under a
>  single service, just like Apache manages its own worker children.

As far as I know, Mongrel doesn't do that -- there's no parent process
managing worker children at all; every mongrel is, in effect, an
independent peer. I'm not sure of the technical reason for this --
perhaps spawning workers is too expensive (or workers are generally
too large) for this to be effective in practice?

I was also a bit confused by this at first, having come from the
Apache world -- but yeah, I've never seen any solution that didn't
involve treating every mongrel as a separate service.

Cheers,
-Nate

From lists at ruby-forum.com  Fri Mar  7 14:39:06 2008
From: lists at ruby-forum.com (Eduardo Silva)
Date: Fri, 7 Mar 2008 20:39:06 +0100
Subject: [Mongrel] Piping logs to rotatelog
Message-ID: <95f6942716c1728c5ef8cd54d41116ad@ruby-forum.com>

Mongrel clusters lets yuo pick a path to your logs with the "-l "
switch. I tried giving a pathc that contains the rotatelogs like

-l
|/usr/sbin/rotatelogs /var/log/mongrel.log 3600"

And it just gave me error. So I rolledup my sleeves and dig into:

1)
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/lib/mongrel_cluster/init.rb

2) /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.4/lib/mongrel/configurator.rb

I tried to re-built the log file name so on "1)" if passes and ARGV
like:

       ##exec_cmd +=" -l  #{port_log_file(port)}"
       exec_cmd +=" -l |/home/admin/rotatelogs #{port_log_file(port)}
30"

but I get this:
starting port 3001
mongrel_rails start -d -e production -p 3001 -P log/silvex.3001.pid -l
|/home/admin/rotatelogs /var/log/my_mongrel3300.3001.log 30
ERROR RUNNING 'mongrel::start': missing argument: -l
Use help command to get help

Silvex ...
starting port 3002
mongrel_rails start -d -e production -p 3002 -P log/silvex.3002.pid -l
|/home/admin/rotatelogs /var/log/my_mongrel3300.3002.log 30
ERROR RUNNING 'mongrel::start': missing argument: -l
Use help command to get help

I tried a similar stunt on "2)" with

      logfile = ops[:log_file]
        ##if logfile[0].chr != "/"
         #logfile = File.join(ops[:cwd],logfile)
         #if not File.exist?(File.dirname(logfile))
         #   log "!!! Log file directory not found at full path
#{File.dirname(logfile)}.  Update your configuration to use a full
path."
         #   exit 1
         # end
        #end

        lati_logfile = "|/home/admin/rotatelogs " + logfile + " 15"
        rotate_log = File.popen(lati_logfile,"w")

        #Daemonize.daemonize(logfile)
        Daemonize.daemonize(rotate_log)

Is this EVEN feasiable ?
-- 
Posted via http://www.ruby-forum.com/.

From lists at ruby-forum.com  Sat Mar  8 10:41:27 2008
From: lists at ruby-forum.com (Ben Blakley)
Date: Sat, 8 Mar 2008 16:41:27 +0100
Subject: [Mongrel] restarting cluster without killing in progress file
	uploads
Message-ID: <a2ad5e205059dee2ef56362c1d845723@ruby-forum.com>

Hello,

I have a cluster of three Mongrels running behind Pound. All is running
well except when I restart the cluster (mongrel_rails cluster::restart)
any in progress file uploads (handled through mongrel_upload_progress)
stop uploading. That is, if I'm uploading a file (via a web app being
served by the mongrel cluster) during a restart, my browser will
indefinitely pause as if it is still uploading the file but I can see
from my computer's bandwidth usage that the file has indeed stopped
uploading. I tried the rolling restart technique discussed in the
following post but the result is the same:

http://www.ruby-forum.com/topic/130285

Anyone have any ideas how this can be solved? Or please let me know if
you need more information. Thanks!
-- 
Posted via http://www.ruby-forum.com/.

From lists at ruby-forum.com  Sat Mar  8 21:48:29 2008
From: lists at ruby-forum.com (Eduardo Silva)
Date: Sun, 9 Mar 2008 03:48:29 +0100
Subject: [Mongrel] Timestamping in log file?
In-Reply-To: <ef3e5d50711211529l73f0479cn134803005bd943ba@mail.gmail.com>
References: <20071120192112.23C9C18585D2@rubyforge.org>
	<ef3e5d50711201848s3c399fcbsf24f53c763c24cb7@mail.gmail.com>
	<ef3e5d50711201947q63b382feub5a0288d10f2c8ca@mail.gmail.com>
	<327FCCA3-9BEA-4C11-B381-F4DC2E7EC209@ncsu.edu>
	<ef3e5d50711211529l73f0479cn134803005bd943ba@mail.gmail.com>
Message-ID: <a0e27e7fb829835aefa0325d79f90292@ruby-forum.com>

Wayne E. Seguin wrote:
> On Nov 21, 2007 3:08 PM, Jason Young <jason.young at ncsu.edu> wrote:
>> HOORAY!  HOORAY!  HOORAY!
>>
>> Jason
> 
> :)
> 
> Please let me know if you find any issues.
> 
> Thanks,
> 
>   ~Wayne

Any changes of being able to  "pipe" log(s) to "crono/rotatelogs" like a 
apache does ?
-- 
Posted via http://www.ruby-forum.com/.

From lists at ruby-forum.com  Sat Mar  8 23:58:36 2008
From: lists at ruby-forum.com (Nathan Esquenazi)
Date: Sun, 9 Mar 2008 05:58:36 +0100
Subject: [Mongrel] Offloading Background Tasks
In-Reply-To: <42642c790803040636x5830a20x5f20ce43aa058784@mail.gmail.com>
References: <2d8ffda38541bfdc3425ca8511bfcd00@ruby-forum.com>
	<20080302025200.5B44F1858660@rubyforge.org>
	<42642c790803040636x5830a20x5f20ce43aa058784@mail.gmail.com>
Message-ID: <e3eb064df5c4f11ea7736265cf8053b5@ruby-forum.com>

Thanks for your responses. Just wanted to reply back that I installed 
backgroundrb (the new version) and I definitely appreciate how useful it 
is and how it is fairly easy to use.

Thanks for all your hard work Hermant.
-- 
Posted via http://www.ruby-forum.com/.

From lists at ruby-forum.com  Sun Mar  9 00:31:44 2008
From: lists at ruby-forum.com (Nathan Esquenazi)
Date: Sun, 9 Mar 2008 06:31:44 +0100
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
Message-ID: <46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>

Here is a good recipe for a mongrel process, as provided on some site I 
found:

(I believe the forum post was by Ezra)

check process mongrel_cluster_8000
   with pidfile /var/apps/twocb/shared/pids/mongrel.8000.pid
   start program = "/usr/bin/mongrel_rails cluster::start -C 
/var/apps/twocb/current/config/mongrel_cluster.yml --clean --only 8000"
   stop program = "/usr/bin/mongrel_rails cluster::stop -C 
/var/apps/twocb/current/config/mongrel_cluster.yml --clean --only 8000"
   if totalmem is greater than 110.0 MB for 4 cycles then  restart
# eating up memory?
   if cpu is greater than 50% for 2 cycles then  alert
 # send an email to admin
   if cpu is greater than 80% for 3 cycles then  restart
# hung process?
   if loadavg(5min) greater than 10 for 8 cycles then  restart
# bad, bad, bad
   if 20 restarts within 20 cycles then timeout
# something is wrong, call the sys-admin
   group mongrel_cluster
-- 
Posted via http://www.ruby-forum.com/.

From dave at cheney.net  Sun Mar  9 01:16:07 2008
From: dave at cheney.net (Dave Cheney)
Date: Sun, 9 Mar 2008 17:16:07 +1100
Subject: [Mongrel] restarting cluster without killing in progress file
	uploads
In-Reply-To: <a2ad5e205059dee2ef56362c1d845723@ruby-forum.com>
References: <a2ad5e205059dee2ef56362c1d845723@ruby-forum.com>
Message-ID: <0AA04937-7852-43D5-8EA7-73BE0F3C2897@cheney.net>

I can't think of a solution to the problem if you need to use  
mongrel_upload_progress as the mongrel that is handling the upload is  
the one you are restarting.

If you can avoid using an upload monitor then i suggest letting nginx  
handle your uploads, it can be configured to try every backend mongrel  
in sequence and only drop the upload if all of them are out of service.

Cheers

Dave

On 09/03/2008, at 2:41 AM, Ben Blakley wrote:

> Hello,
>
> I have a cluster of three Mongrels running behind Pound. All is  
> running
> well except when I restart the cluster (mongrel_rails  
> cluster::restart)
> any in progress file uploads (handled through mongrel_upload_progress)
> stop uploading. That is, if I'm uploading a file (via a web app being
> served by the mongrel cluster) during a restart, my browser will
> indefinitely pause as if it is still uploading the file but I can see
> from my computer's bandwidth usage that the file has indeed stopped
> uploading. I tried the rolling restart technique discussed in the
> following post but the result is the same:
>
> http://www.ruby-forum.com/topic/130285
>
> Anyone have any ideas how this can be solved? Or please let me know if
> you need more information. Thanks!
> -- 
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From gethemant at gmail.com  Sun Mar  9 03:16:41 2008
From: gethemant at gmail.com (hemant kumar)
Date: Sun, 09 Mar 2008 12:46:41 +0530
Subject: [Mongrel] Offloading Background Tasks
In-Reply-To: <e3eb064df5c4f11ea7736265cf8053b5@ruby-forum.com>
References: <2d8ffda38541bfdc3425ca8511bfcd00@ruby-forum.com>
	<20080302025200.5B44F1858660@rubyforge.org>
	<42642c790803040636x5830a20x5f20ce43aa058784@mail.gmail.com>
	<e3eb064df5c4f11ea7736265cf8053b5@ruby-forum.com>
Message-ID: <1205047001.16817.3.camel@shire>


On Sun, 2008-03-09 at 05:58 +0100, Nathan Esquenazi wrote:
> Thanks for your responses. Just wanted to reply back that I installed 
> backgroundrb (the new version) and I definitely appreciate how useful it 
> is and how it is fairly easy to use.
> 
> Thanks for all your hard work Hermant.

No problems!

If you are having any problems feel free to shoot your questions on
BackgrounDRb mailing list.






From lists at ruby-forum.com  Mon Mar 10 11:50:27 2008
From: lists at ruby-forum.com (Ben Blakley)
Date: Mon, 10 Mar 2008 16:50:27 +0100
Subject: [Mongrel] restarting cluster without killing in progress file
	uplo
In-Reply-To: <0AA04937-7852-43D5-8EA7-73BE0F3C2897@cheney.net>
References: <a2ad5e205059dee2ef56362c1d845723@ruby-forum.com>
	<0AA04937-7852-43D5-8EA7-73BE0F3C2897@cheney.net>
Message-ID: <ee202a1b6bafcedd6fb94074643046c0@ruby-forum.com>

Thanks Dave. I switched from Pound and mongrel_upload_progress to Nginx 
and its upload progress module as described at the following address:

http://nginx.fehot.com/NginxHttpUploadProgressModule

It's working really well. Thanks again.

Ben
-- 
Posted via http://www.ruby-forum.com/.

From john at oxyliquit.de  Tue Mar 11 12:21:19 2008
From: john at oxyliquit.de (Jonathan Buch)
Date: Tue, 11 Mar 2008 17:21:19 +0100
Subject: [Mongrel] Mongrel 1.1.4, possible bug http_response
Message-ID: <op.t7u01tmbxegpx6@jo.local>

Hi,

I found this while digging inside mongrel to provide IO streaming
for mongrel (and thus for Rack and Ramaze):

* In socket_error a local variable done is set, which has no effect.
* done=() always sets to true, where it (for completeness sake) should
   honor the passed value.


--- a/http_response.rb	2008-03-11 16:59:24.000000000 +0100
+++ b/http_response.rb	2008-03-11 17:02:20.000000000 +0100
@@ -129,7 +129,7 @@
      def socket_error(details)
        # ignore these since it means the client closed off early
        @socket.close rescue nil
-      done = true
+      self.done = true
        raise details
      end

@@ -150,9 +150,9 @@
      # Used during error conditions to mark the response as "done" so  
there isn't any more processing
      # sent to the client.
      def done=(val)
-      @status_sent = true
-      @header_sent = true
-      @body_sent = true
+      @status_sent = val
+      @header_sent = val
+      @body_sent = val
      end

      def done

* If done=() is never reverted, it might be more beneficial to replace
   done=(val) just by a done!() and call that in socket_error() instead.

Regards,

Jo

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

From evan at cloudbur.st  Tue Mar 11 15:20:02 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Tue, 11 Mar 2008 15:20:02 -0400
Subject: [Mongrel] Mongrel 1.1.4, possible bug http_response
In-Reply-To: <op.t7u01tmbxegpx6@jo.local>
References: <op.t7u01tmbxegpx6@jo.local>
Message-ID: <b6f68fc60803111220j22fb1c42j5a8985eb36b0edb5@mail.gmail.com>

Thanks. File a ticket please if you think it's definitely bug:

http://mongrel.rubyforge.org/report/1

Evan

On Tue, Mar 11, 2008 at 12:21 PM, Jonathan Buch <john at oxyliquit.de> wrote:
> Hi,
>
>  I found this while digging inside mongrel to provide IO streaming
>  for mongrel (and thus for Rack and Ramaze):
>
>  * In socket_error a local variable done is set, which has no effect.
>  * done=() always sets to true, where it (for completeness sake) should
>    honor the passed value.
>
>
>  --- a/http_response.rb  2008-03-11 16:59:24.000000000 +0100
>  +++ b/http_response.rb  2008-03-11 17:02:20.000000000 +0100
>  @@ -129,7 +129,7 @@
>       def socket_error(details)
>         # ignore these since it means the client closed off early
>         @socket.close rescue nil
>  -      done = true
>  +      self.done = true
>         raise details
>       end
>
>  @@ -150,9 +150,9 @@
>       # Used during error conditions to mark the response as "done" so
>  there isn't any more processing
>       # sent to the client.
>       def done=(val)
>  -      @status_sent = true
>  -      @header_sent = true
>  -      @body_sent = true
>  +      @status_sent = val
>  +      @header_sent = val
>  +      @body_sent = val
>       end
>
>       def done
>
>  * If done=() is never reverted, it might be more beneficial to replace
>    done=(val) just by a done!() and call that in socket_error() instead.
>
>  Regards,
>
>  Jo
>
>  --
>  Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From petite.abeille at gmail.com  Tue Mar 11 15:34:31 2008
From: petite.abeille at gmail.com (Petite Abeille)
Date: Tue, 11 Mar 2008 20:34:31 +0100
Subject: [Mongrel] [OT] HTTP headers status diagram
Message-ID: <AF49391F-5CA2-4801-9370-43D525C0AC29@gmail>

To paraphrase Woody Allen:

"Everything you always wanted to know about HTTP headers status, but  
were afraid to ask"

Alternatively:

"An activity diagram to describe the resolution of HTTP response  
status codes, given various headers."

http://thoughtpad.net/alan-dean/http-headers-status.html

http://thoughtpad.net/alan-dean/http-headers-status.gif

Cheers,

--
PA
http://alt.textdrive.com/nanoki/

From john at oxyliquit.de  Tue Mar 11 15:48:25 2008
From: john at oxyliquit.de (Jonathan Buch)
Date: Tue, 11 Mar 2008 20:48:25 +0100
Subject: [Mongrel] Mongrel 1.1.4, possible bug http_response
In-Reply-To: <b6f68fc60803111220j22fb1c42j5a8985eb36b0edb5@mail.gmail.com>
References: <op.t7u01tmbxegpx6@jo.local>
	<b6f68fc60803111220j22fb1c42j5a8985eb36b0edb5@mail.gmail.com>
Message-ID: <op.t7vamzmhxegpx6@jo.local>

Hi,

> Thanks. File a ticket please if you think it's definitely bug:
>
> http://mongrel.rubyforge.org/report/1

sorry to be a bother, but:

I can't decide if it is really a bug, as my knowledge regarding mongrel
innards is next to non-existant.
The implications of 'done' being set or not set in certain corner cases
is not something which I can easily be sure of.

>>  I found this while digging inside mongrel to provide IO streaming
>>  for mongrel (and thus for Rack and Ramaze):
>>
>>  * In socket_error a local variable done is set, which has no effect.
>>  * done=() always sets to true, where it (for completeness sake) should
>>    honor the passed value.
>>  * If done=() is never reverted, it might be more beneficial to replace
>>    done=(val) just by a done!() and call that in socket_error() instead.

As I see it, done=() is never called at the moment (I might be wrong), and
Mongrel still works.  Making done=() work correctly might have other
implications.  Right now it might process additional handlers (mongrel.rb:
160) which it might not do after the fix.

Jo

-- 
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

From public at misuse.org  Tue Mar 11 16:17:47 2008
From: public at misuse.org (Steve Midgley)
Date: Tue, 11 Mar 2008 13:17:47 -0700
Subject: [Mongrel] [OT] HTTP headers status diagram
In-Reply-To: <mailman.239.1205264081.26975.mongrel-users@rubyforge.org>
References: <mailman.239.1205264081.26975.mongrel-users@rubyforge.org>
Message-ID: <20080311201802.CCFA418585FA@rubyforge.org>

At 12:34 PM 3/11/2008, mongrel-users-request at rubyforge.org wrote:
>Date: Tue, 11 Mar 2008 20:34:31 +0100
>From: Petite Abeille <petite.abeille at gmail.com>
>Subject: [Mongrel] [OT] HTTP headers status diagram
>To: mongrel-users at rubyforge.org
>Message-ID: <AF49391F-5CA2-4801-9370-43D525C0AC29 at gmail>
>
>[snip]
>"An activity diagram to describe the resolution of HTTP response
>status codes, given various headers."
>
>http://thoughtpad.net/alan-dean/http-headers-status.html
>
>http://thoughtpad.net/alan-dean/http-headers-status.gif

Wow - a state diagram for HTTP response codes. I'm impressed you've put 
this together - it ought to help people building new app servers for 
sure (and it seems there's a new one every week). You should send this 
up to W3 too - I've never seen anything like it on their site (but 
haven't looked specifically either).

Nice one,

Steve


From petite.abeille at gmail.com  Tue Mar 11 16:24:24 2008
From: petite.abeille at gmail.com (Petite Abeille)
Date: Tue, 11 Mar 2008 21:24:24 +0100
Subject: [Mongrel] [OT] HTTP headers status diagram
In-Reply-To: <47d6e8fa.1de0220a.41fe.7df1SMTPIN_ADDED@mx.google.com>
References: <mailman.239.1205264081.26975.mongrel-users@rubyforge.org>
	<47d6e8fa.1de0220a.41fe.7df1SMTPIN_ADDED@mx.google.com>
Message-ID: <866EF2EA-A46D-4047-8965-15F8AB91C168@gmail>


On Mar 11, 2008, at 9:17 PM, Steve Midgley wrote:

> Wow - a state diagram for HTTP response codes. I'm impressed you've  
> put this together

Full credit to Alan Dean for this gem.

http://thoughtpad.net/alan-dean/http-headers-status.html

> - it ought to help people building new app servers for sure (and it  
> seems there's a new one every week).

Indeed :)

Here is one in Lua:

http://svr225.stepx.com:3388/nanoki

> You should send this up to W3 too - I've never seen anything like it  
> on their site (but haven't looked specifically either).


Cheers,

PA.


From www at saepia.net  Wed Mar 12 04:32:59 2008
From: www at saepia.net (Marcin Lewandowski)
Date: Wed, 12 Mar 2008 09:32:59 +0100
Subject: [Mongrel] Mongrel has crashed
Message-ID: <47D7953B.4020600@saepia.net>

Hi,

my mongrel has crashed with following errors in the logfile:

** Starting Rails with development environment...
** Rails loaded.
** Loading any Rails specific GemPlugins
** Signals ready.  TERM => stop.  USR2 => restart.  INT => stop (no 
restart).
** Rails signals registered.  HUP => reload (without restart).  It might 
not work well.
** Mongrel 1.1.3 available at 0.0.0.0:3000
** Writing PID file to ./script/../config/../tmp/pids/mongrel.pid
/usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4: 
[BUG] gc_sweep(): unknown data type 0x0(0xa8b2241c)
ruby 1.8.6 (2008-03-03) [i686-linux]


Rails 1.2.6

What's wrong?

My system is too important to hang up in so stupid way and it's too late 
to choose something different than rails..

m.

From wyhaines at gmail.com  Wed Mar 12 12:33:58 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Wed, 12 Mar 2008 10:33:58 -0600
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <47D7953B.4020600@saepia.net>
References: <47D7953B.4020600@saepia.net>
Message-ID: <f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>

On Wed, Mar 12, 2008 at 2:32 AM, Marcin Lewandowski <www at saepia.net> wrote:

> /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
> [BUG] gc_sweep(): unknown data type 0x0(0xa8b2241c)
> ruby 1.8.6 (2008-03-03) [i686-linux]
>
>
> Rails 1.2.6
>
> What's wrong?

That's a fatal Ruby exception.  Something triggered an error that the
Ruby interpreter thinks is a bug in the interpreter.

Did this happen just once, or are you getting it often?  Do you have
anything monitoring the mongrel processes so that if they do go away,
they can be restarted?


Kirk Haines

From njvack at wisc.edu  Wed Mar 12 12:52:13 2008
From: njvack at wisc.edu (Nate Vack)
Date: Wed, 12 Mar 2008 11:52:13 -0500
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
Message-ID: <8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>

On Wed, Mar 12, 2008 at 11:33 AM, Kirk Haines <wyhaines at gmail.com> wrote:
> On Wed, Mar 12, 2008 at 2:32 AM, Marcin Lewandowski <www at saepia.net> wrote:
>
>  > /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
>  > [BUG] gc_sweep(): unknown data type 0x0(0xa8b2241c)
>  > ruby 1.8.6 (2008-03-03) [i686-linux]
>  >
>  >
>  > Rails 1.2.6
>  >
>  > What's wrong?

My off-the-cuff guesses, in order of where I'd look

* A funny gem or plugin is in use with your Rails app, and that's
causing problems

* Bad hardware

* A bad build of ruby

What third-party code (other than Rails) are you using?

Cheers,
-Nate

From www at saepia.net  Wed Mar 12 15:15:51 2008
From: www at saepia.net (Marcin Lewandowski)
Date: Wed, 12 Mar 2008 20:15:51 +0100
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
Message-ID: <47D82BE7.8050803@saepia.net>

Nate Vack pisze:
> On Wed, Mar 12, 2008 at 11:33 AM, Kirk Haines <wyhaines at gmail.com> wrote:
>> On Wed, Mar 12, 2008 at 2:32 AM, Marcin Lewandowski <www at saepia.net> wrote:
>>
>>  > /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
>>  > [BUG] gc_sweep(): unknown data type 0x0(0xa8b2241c)
>>  > ruby 1.8.6 (2008-03-03) [i686-linux]
>>  >
>>  >
>>  > Rails 1.2.6
>>  >
>>  > What's wrong?
> 
> My off-the-cuff guesses, in order of where I'd look
> 
> * A funny gem or plugin is in use with your Rails app, and that's
> causing problems

I realized that there's a new version, so I'll try to upgrade to 1.1.4. 
Maybe also there's new ruby.

It happened from time to time, e.g. once for 3-4 weeks with my previous 
mongrel version (1.0.x). So I've made an upgrade to 1.1.3, but it worked 
only for 2 weeks.

I don't use any plugins in my rails app.

I'll come back if it remained with 1.1.4

Thanks,

-- 
m.

From wyhaines at gmail.com  Wed Mar 12 15:48:24 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Wed, 12 Mar 2008 13:48:24 -0600
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <47D82BE7.8050803@saepia.net>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
	<47D82BE7.8050803@saepia.net>
Message-ID: <f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>

On Wed, Mar 12, 2008 at 1:15 PM, Marcin Lewandowski <www at saepia.net> wrote:

> I realized that there's a new version, so I'll try to upgrade to 1.1.4.
> Maybe also there's new ruby.

> It happened from time to time, e.g. once for 3-4 weeks with my previous
> mongrel version (1.0.x). So I've made an upgrade to 1.1.3, but it worked
> only for 2 weeks.
>
> I don't use any plugins in my rails app.
>
> I'll come back if it remained with 1.1.4

It's unlikely to be an issue with Mongrel, or we would have heard
about it before, especially since you say that you have seen the error
while running under two different versions of Mongrel, both of which
have been out for a long time, and that you have seen this error
multiple times.

/usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
[BUG] gc_sweep(): unknown data type 0x0(0xa8b2241c)
ruby 1.8.6 (2008-03-03) [i686-linux]

If you look at the Ruby source code, that error is called during
obj_free() (defined in gc.c) when the object being freed doesn't have
a known node type.

My suspicion is that something that you are using has an extension in
it that is doing something which interacts poorly with Ruby's GC code.

Which version of Ruby 1.8.6 is it that you are running?  Make sure you
are using the latest patch level of 1.8.6.  This is very important.


Kirk Haines

From www at saepia.net  Wed Mar 12 16:03:04 2008
From: www at saepia.net (Marcin Lewandowski)
Date: Wed, 12 Mar 2008 21:03:04 +0100
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>	<47D82BE7.8050803@saepia.net>
	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
Message-ID: <47D836F8.6070809@saepia.net>

> It's unlikely to be an issue with Mongrel, or we would have heard
> about it before, especially since you say that you have seen the error
> while running under two different versions of Mongrel, both of which
> have been out for a long time, and that you have seen this error
> multiple times.

The error message was a bit different with previous version of mongrel 
but I assume that it was similar.

> My suspicion is that something that you are using has an extension in
> it that is doing something which interacts poorly with Ruby's GC code.
> 
> Which version of Ruby 1.8.6 is it that you are running?  Make sure you
> are using the latest patch level of 1.8.6.  This is very important.

I use latest available in gentoo's portage: patchlevel 1.8.6 p114 and 
AFAIK it's latest stable version.

My gems:

*** LOCAL GEMS ***

actionmailer (1.3.6)
     Service layer for easy email delivery and testing.

actionpack (1.13.6)
     Web-flow and rendering framework putting the VC in MVC.

actionwebservice (1.2.6)
     Web service support for Action Pack.

activerecord (1.15.6)
     Implements the ActiveRecord pattern for ORM.

activesupport (1.4.4)
     Support and utility classes used by the Rails framework.

cgi_multipart_eof_fix (2.5.0)
     Fix an exploitable bug in CGI multipart parsing.

daemons (1.0.5)
     A toolkit to create and control daemons in different ways

fastthread (1.0.1)
     Optimized replacement for thread.rb primitives

gem_plugin (0.2.3)
     A plugin system based on rubygems that uses dependencies only

gettext (1.9.0)
     Ruby-GetText-Package is a libary and tools to localize messages.

mongrel (1.1.3)
     A small fast HTTP library and server that runs Rails, Camping, Nitro
     and Iowa apps.

mongrel_upload_progress (0.2.2)
     The mongrel_upload_progress gemplugin

rails (1.2.6)
     Web-application framework with template engine, control-flow layer,
     and ORM.

rake (0.7.3)
     Ruby based make-like utility.

sources (0.0.1)
     This package provides download sources for remote gem installation


I'll try to upgrade them later.

Thanks!

m.

From mohammed.a at aol.in  Thu Mar 13 16:22:47 2008
From: mohammed.a at aol.in (mohammed.a at aol.in)
Date: Thu, 13 Mar 2008 16:22:47 -0400
Subject: [Mongrel] mongrel throwing error 500
Message-ID: <8CA535F22EFB860-A00-542@web-cen-v10.sim.aol.com>

Dear Users,

Often at times when the code encounters an exception, mongrel starts throwing error 500 (after which most other cluster instances are affected).

My queries are

- why does an exception in one thread have to affect others?
- how can I stop mongrel throwing 500 and start showing me the actual error on the browser (this will make things so much more easier)
- Can anyone direct me to some detailed documents on mongrel configuration?

regards

Mohammed. A

________________________________________________________________________
You are invited to Get a Free AOL Email ID. - http://webmail.aol.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080313/b620498a/attachment.html 

From lists at ruby-forum.com  Fri Mar 14 01:16:39 2008
From: lists at ruby-forum.com (Dan Lamet)
Date: Fri, 14 Mar 2008 06:16:39 +0100
Subject: [Mongrel] SSL doesn't activate in Mongrel
Message-ID: <456271b6c29699fac405e13c930d7e55@ruby-forum.com>

I have an app I've been working on for a while.  I just tried to
activate SSL for it.  I followed the instructions found at
<http://wiki.rubyonrails.com/rails/pages/HowToEnableSSL>.  Everything
seemed to install/configure OK, no errors.  But I now find that nothing
is listening on port 443.  "Connection refused"

So what might be the problem?
-- 
Posted via http://www.ruby-forum.com/.

From nathen at visualcv.com  Fri Mar 14 06:03:32 2008
From: nathen at visualcv.com (Nathen Harvey)
Date: Fri, 14 Mar 2008 06:03:32 -0400
Subject: [Mongrel] SSL doesn't activate in Mongrel
Message-ID: <47DA4D74.3080401@visualcv.com>

Have you set the X-Forwarded-Proto request header?  Here's a snip from 
the apache config:

    # Set header to indentify https requests for Mongrel
    RequestHeader set X-Forwarded-Proto "https"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080314/d622b01c/attachment.html 

From lists at ruby-forum.com  Fri Mar 14 13:12:23 2008
From: lists at ruby-forum.com (Dan Lamet)
Date: Fri, 14 Mar 2008 18:12:23 +0100
Subject: [Mongrel] SSL doesn't activate in Mongrel
In-Reply-To: <47DA4D74.3080401@visualcv.com>
References: <47DA4D74.3080401@visualcv.com>
Message-ID: <78a0e730cd98432bbddc6c0babeac4ef@ruby-forum.com>

I haven't actually set up Apache.  Is it possible to enable SSL with 
Mongrel by itself?
-- 
Posted via http://www.ruby-forum.com/.

From technoweenie at gmail.com  Fri Mar 14 13:23:38 2008
From: technoweenie at gmail.com (Rick Olson)
Date: Fri, 14 Mar 2008 10:23:38 -0700
Subject: [Mongrel] SSL doesn't activate in Mongrel
In-Reply-To: <78a0e730cd98432bbddc6c0babeac4ef@ruby-forum.com>
References: <47DA4D74.3080401@visualcv.com>
	<78a0e730cd98432bbddc6c0babeac4ef@ruby-forum.com>
Message-ID: <48fe25b0803141023t2c5cb64ctff2999535aad908d@mail.gmail.com>

On Fri, Mar 14, 2008 at 10:12 AM, Dan Lamet <lists at ruby-forum.com> wrote:
> I haven't actually set up Apache.  Is it possible to enable SSL with
>  Mongrel by itself?

Nope.

http://blog.innerewut.de/2006/06/21/mongrel-and-rails-behind-apache-2-2-and-ssl

-- 
Rick Olson
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com

From lists at ruby-forum.com  Tue Mar 18 07:47:45 2008
From: lists at ruby-forum.com (James George)
Date: Tue, 18 Mar 2008 12:47:45 +0100
Subject: [Mongrel] Mongrel Crashes in Production
Message-ID: <8b1b96220823aa131e1d804604795907@ruby-forum.com>

I have been getting this error on the production server after rails was
updated to 2.0.2 and moreover this worked perfectly in the development
mode. This crash occurs when no one is using the app for an hour or so.
This is the error that I get in the logs. (nohup.out file). The
production log doesnt show anything unusual. Please help me out guys.
 /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/rails.rb:98:in
`reload!': undefined method `reset_application!' for
ActionController::Dispatcher:Class (NoMethodError)
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/rails.rb:95:in
`synchronize'
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/rails.rb:95:in
`reload!'
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/rails.rb:165:in
`reload!'
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/rails.rb:178:in
`setup_rails_signals'
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/configurator.rb:303:in
`call'
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/configurator.rb:303:in
`join'
       from
/usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel


Thanks in advance.
-- 
Posted via http://www.ruby-forum.com/.

From luislavena at gmail.com  Tue Mar 18 10:45:25 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Tue, 18 Mar 2008 11:45:25 -0300
Subject: [Mongrel] Mongrel Crashes in Production
In-Reply-To: <8b1b96220823aa131e1d804604795907@ruby-forum.com>
References: <8b1b96220823aa131e1d804604795907@ruby-forum.com>
Message-ID: <71166b3b0803180745r2a10fe0ax1c00904c772327de@mail.gmail.com>

On Tue, Mar 18, 2008 at 8:47 AM, James George <lists at ruby-forum.com> wrote:
> I have been getting this error on the production server after rails was
>  updated to 2.0.2 and moreover this worked perfectly in the development
>  mode. This crash occurs when no one is using the app for an hour or so.
>  This is the error that I get in the logs. (nohup.out file). The
>  production log doesnt show anything unusual. Please help me out guys.
>   /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.1.1/bin/../lib/mongrel/rails.rb:98:in
>  `reload!': undefined method `reset_application!' for
>  ActionController::Dispatcher:Class (NoMethodError)
>        from

It seems you're getting a restart/reload signal to the mongrel
process, and it is trying to reload the Rails environment calling the
reset_application! method of ActionController::Dispatcher

There is no ActionController::Dispatcher, but there is a Dispatcher in
the global namespace (::Dispatcher)?

I cannot test it since Windows don't generate the HUP signal required
for this, but I think I'm in the good path.

HTH,
-- 
Luis Lavena
Multimedia systems
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

From yves.dastous at intertrade.com  Tue Mar 18 17:10:02 2008
From: yves.dastous at intertrade.com (Yves D'Astous)
Date: Tue, 18 Mar 2008 17:10:02 -0400
Subject: [Mongrel] Running multiple mongrel to "serve" the same rail
 application with active scaffold
Message-ID: <47E02FAA.30304@intertrade.com>

Hi,

I want to run more then one mongrel for the same rail application on the
same machine, using the same rails files.
This application use "active scaffold"  when I start one mongrel some
files are writes in the  public/plugin_assets/  folder.

If I start more then one mongrel it is possible these files become
scrambles ?, there is also some files in tmp/{cache,pids,sessions
,sockets, webfile}
who can i make sure the 2 mongrels will not interfere each other ?, Do i
really need to use a separated copy of the same rail application for
each mongrel ?

Yves




From david at vrensk.com  Wed Mar 19 19:25:52 2008
From: david at vrensk.com (David Vrensk)
Date: Thu, 20 Mar 2008 00:25:52 +0100
Subject: [Mongrel] mongrel throwing error 500
In-Reply-To: <8CA535F22EFB860-A00-542@web-cen-v10.sim.aol.com>
References: <8CA535F22EFB860-A00-542@web-cen-v10.sim.aol.com>
Message-ID: <81b453920803191625q2995701k9785863693fadbbc@mail.gmail.com>

On 3/13/08, mohammed.a at aol.in <mohammed.a at aol.in> wrote:
>
> Dear Users,
>
> Often at times when the code encounters an exception, mongrel starts
> throwing error 500 (after which most other cluster instances are affected).
>
> My queries are
>
> - why does an exception in one thread have to affect others?


I think you need to give us a bit more detail.  Like, are you talking about
actual threads within one process, or different processes, which the word
"cluster" above hints?  And are you using a framework like Rails or Merb, or
are you building your own registered handlers in Mongrel?

Best regards,

/David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080320/a4391a00/attachment.html 

From matte at ruckuswireless.com  Wed Mar 19 19:22:33 2008
From: matte at ruckuswireless.com (Matte Edens)
Date: Wed, 19 Mar 2008 16:22:33 -0700
Subject: [Mongrel] Process environment requires procfs(5)
Message-ID: <47E1A039.2010306@ruckuswireless.com>

I know this has been brought up before but I'm receiving the error ...

Process environment requires procfs(5)

... when trying to restart my cluster.  I've narrowed it down to 
'find_pid' method in init.rb.  I'm running a FreeBSD 6.2 server and 
mongrel_cluster version 1.0.5.  The command that gives me the error is ...

ps -ewwo pid,command

... but I've been able to narrow it even further to just the "e" flag.  
When I run a `ps e` I get the error as well.  Now, this is possibly a 
server setup issue.  I believe it's related to the fact that my ps 
output for each mongrel instance only says "[ruby18]" and not the much 
longer but expected output of ...

/usr/local/bin/ruby18 /usr/local/bin/mongrel_rails start -d -e 
production ...

I'm fully ready to call my hosting provider to find out if they can help 
but I'm wondering if anyone on the list has thoughts about this.  My 
restarts won't run successfully because the pidfile isn't being found 
due to the fact that the process isn't being fully listed.  Just the 
"[ruby18]" is showing up for my mongrel processes and cluster can't 
parse that for the pid filename

matte

From rgkaufman at gmail.com  Thu Mar 20 22:09:35 2008
From: rgkaufman at gmail.com (Rob Kaufman)
Date: Thu, 20 Mar 2008 19:09:35 -0700
Subject: [Mongrel] Running multiple mongrel to "serve" the same rail
	application with active scaffold
In-Reply-To: <47E02FAA.30304@intertrade.com>
References: <47E02FAA.30304@intertrade.com>
Message-ID: <17112e2f0803201909r677c2be3wcfe394edf2903889@mail.gmail.com>

Hi Yves,
  Those assets should only be writen the very first time your app loads, and
not on each load.  No matter what the various mongrels will not interfere
with each other if run from the same rails directory unless active scaffold
is doing something truly horrible with they're file writing.  This seems
pretty unlikely.

Rob

On Tue, Mar 18, 2008 at 2:10 PM, Yves D'Astous <yves.dastous at intertrade.com>
wrote:

> Hi,
>
> I want to run more then one mongrel for the same rail application on the
> same machine, using the same rails files.
> This application use "active scaffold"  when I start one mongrel some
> files are writes in the  public/plugin_assets/  folder.
>
> If I start more then one mongrel it is possible these files become
> scrambles ?, there is also some files in tmp/{cache,pids,sessions
> ,sockets, webfile}
> who can i make sure the 2 mongrels will not interfere each other ?, Do i
> really need to use a separated copy of the same rail application for
> each mongrel ?
>
> Yves
>
>
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080320/2b642b0c/attachment.html 

From www at saepia.net  Fri Mar 21 06:52:44 2008
From: www at saepia.net (Marcin Lewandowski)
Date: Fri, 21 Mar 2008 11:52:44 +0100
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <47D836F8.6070809@saepia.net>
References: <47D7953B.4020600@saepia.net>	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>	<47D82BE7.8050803@saepia.net>	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
	<47D836F8.6070809@saepia.net>
Message-ID: <47E3937C.7050902@saepia.net>

It still crashes... :(

/usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4: 
[BUG] gc_sweep(): unknown data type 0x0(0xac53841c)
ruby 1.8.6 (2008-03-03) [i686-linux]


Once, there was a crash message related to gettext, but I've made an 
upgrade and it crashes like that.

How to trace or debug that?

m.

From wyhaines at gmail.com  Fri Mar 21 10:34:49 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 21 Mar 2008 08:34:49 -0600
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <47E3937C.7050902@saepia.net>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
	<47D82BE7.8050803@saepia.net>
	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
	<47D836F8.6070809@saepia.net> <47E3937C.7050902@saepia.net>
Message-ID: <f4cd26df0803210734m5c137b8p7ac8c8cf51794372@mail.gmail.com>

On Fri, Mar 21, 2008 at 4:52 AM, Marcin Lewandowski <www at saepia.net> wrote:
> It still crashes... :(
>
> /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
> [BUG] gc_sweep(): unknown data type 0x0(0xac53841c)
> ruby 1.8.6 (2008-03-03) [i686-linux]
>
>
> Once, there was a crash message related to gettext, but I've made an
> upgrade and it crashes like that.
>
> How to trace or debug that?

Look at the exception.  Since you've gotten the same identical
exception twice, start there.  It's in ActiveSupport, not Mongrel,
BTW.

Here's the line in question:

ObjectSpace.each_object(Class) { |k| classes << k if
k.included_modules.include?(self) }

This is an interesting place for that error to occur.  I don't have an
immediate hypothesis, but somehow something is getting into the Ruby
heap that isn't a recognized node type.


Kirk Haines

From evan at cloudbur.st  Fri Mar 21 12:50:18 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 21 Mar 2008 12:50:18 -0400
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <f4cd26df0803210734m5c137b8p7ac8c8cf51794372@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
	<47D82BE7.8050803@saepia.net>
	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
	<47D836F8.6070809@saepia.net> <47E3937C.7050902@saepia.net>
	<f4cd26df0803210734m5c137b8p7ac8c8cf51794372@mail.gmail.com>
Message-ID: <b6f68fc60803210950q19db47dbmbb7b5ea8b003796f@mail.gmail.com>

Do you have any GC patches in Ruby, use of DL, shaky C libs, or a
weird Ruby patchlevel?

Evan

On Fri, Mar 21, 2008 at 10:34 AM, Kirk Haines <wyhaines at gmail.com> wrote:
> On Fri, Mar 21, 2008 at 4:52 AM, Marcin Lewandowski <www at saepia.net> wrote:
>  > It still crashes... :(
>  >
>  > /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
>  > [BUG] gc_sweep(): unknown data type 0x0(0xac53841c)
>  > ruby 1.8.6 (2008-03-03) [i686-linux]
>  >
>  >
>  > Once, there was a crash message related to gettext, but I've made an
>  > upgrade and it crashes like that.
>  >
>  > How to trace or debug that?
>
>  Look at the exception.  Since you've gotten the same identical
>  exception twice, start there.  It's in ActiveSupport, not Mongrel,
>  BTW.
>
>  Here's the line in question:
>
>  ObjectSpace.each_object(Class) { |k| classes << k if
>  k.included_modules.include?(self) }
>
>  This is an interesting place for that error to occur.  I don't have an
>  immediate hypothesis, but somehow something is getting into the Ruby
>  heap that isn't a recognized node type.
>
>
>  Kirk Haines
>
>
> _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From evan at cloudbur.st  Fri Mar 21 13:01:20 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 21 Mar 2008 13:01:20 -0400
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <b6f68fc60803210950q19db47dbmbb7b5ea8b003796f@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
	<47D82BE7.8050803@saepia.net>
	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
	<47D836F8.6070809@saepia.net> <47E3937C.7050902@saepia.net>
	<f4cd26df0803210734m5c137b8p7ac8c8cf51794372@mail.gmail.com>
	<b6f68fc60803210950q19db47dbmbb7b5ea8b003796f@mail.gmail.com>
Message-ID: <b6f68fc60803211001k167c8cc5s9fd7a41511307a5@mail.gmail.com>

Some things to try:

Downgrade Ruby to p111 (unlikely).

Avoid using gettext (more likely).

Are you using any RubyInline plugins or vendorized gems?

Evan

On Fri, Mar 21, 2008 at 12:50 PM, Evan Weaver <evan at cloudbur.st> wrote:
> Do you have any GC patches in Ruby, use of DL, shaky C libs, or a
>  weird Ruby patchlevel?
>
>  Evan
>
>
>
>  On Fri, Mar 21, 2008 at 10:34 AM, Kirk Haines <wyhaines at gmail.com> wrote:
>  > On Fri, Mar 21, 2008 at 4:52 AM, Marcin Lewandowski <www at saepia.net> wrote:
>  >  > It still crashes... :(
>  >  >
>  >  > /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.4/lib/active_support/core_ext/module/inclusion.rb:4:
>  >  > [BUG] gc_sweep(): unknown data type 0x0(0xac53841c)
>  >  > ruby 1.8.6 (2008-03-03) [i686-linux]
>  >  >
>  >  >
>  >  > Once, there was a crash message related to gettext, but I've made an
>  >  > upgrade and it crashes like that.
>  >  >
>  >  > How to trace or debug that?
>  >
>  >  Look at the exception.  Since you've gotten the same identical
>  >  exception twice, start there.  It's in ActiveSupport, not Mongrel,
>  >  BTW.
>  >
>  >  Here's the line in question:
>  >
>  >  ObjectSpace.each_object(Class) { |k| classes << k if
>  >  k.included_modules.include?(self) }
>  >
>  >  This is an interesting place for that error to occur.  I don't have an
>  >  immediate hypothesis, but somehow something is getting into the Ruby
>  >  heap that isn't a recognized node type.
>  >
>  >
>  >  Kirk Haines
>  >
>  >
>  > _______________________________________________
>  >  Mongrel-users mailing list
>  >  Mongrel-users at rubyforge.org
>  >  http://rubyforge.org/mailman/listinfo/mongrel-users
>  >
>
>
>
>  --
>  Evan Weaver
>  Cloudburst, LLC
>



-- 
Evan Weaver
Cloudburst, LLC

From wyhaines at gmail.com  Fri Mar 21 13:11:22 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 21 Mar 2008 11:11:22 -0600
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <b6f68fc60803210950q19db47dbmbb7b5ea8b003796f@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
	<47D82BE7.8050803@saepia.net>
	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
	<47D836F8.6070809@saepia.net> <47E3937C.7050902@saepia.net>
	<f4cd26df0803210734m5c137b8p7ac8c8cf51794372@mail.gmail.com>
	<b6f68fc60803210950q19db47dbmbb7b5ea8b003796f@mail.gmail.com>
Message-ID: <f4cd26df0803211011nd042ab0yf2030cba349161dc@mail.gmail.com>

On Fri, Mar 21, 2008 at 10:50 AM, Evan Weaver <evan at cloudbur.st> wrote:
> Do you have any GC patches in Ruby, use of DL, shaky C libs, or a
> weird Ruby patchlevel?

Evan, if you look back in the thread, he listed his gems on Mar 12th.


Kirk Haines

From evan at cloudbur.st  Fri Mar 21 13:21:11 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 21 Mar 2008 13:21:11 -0400
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <f4cd26df0803211011nd042ab0yf2030cba349161dc@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>
	<f4cd26df0803120933w663787ddyd1228dace2191122@mail.gmail.com>
	<8be813400803120952w7f96d6c4u951c732d173970d4@mail.gmail.com>
	<47D82BE7.8050803@saepia.net>
	<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6@mail.gmail.com>
	<47D836F8.6070809@saepia.net> <47E3937C.7050902@saepia.net>
	<f4cd26df0803210734m5c137b8p7ac8c8cf51794372@mail.gmail.com>
	<b6f68fc60803210950q19db47dbmbb7b5ea8b003796f@mail.gmail.com>
	<f4cd26df0803211011nd042ab0yf2030cba349161dc@mail.gmail.com>
Message-ID: <b6f68fc60803211021t53e490a7rd3e6760375f600ca@mail.gmail.com>

I saw that, so I mentioned gettext as the only one that stands out.
There can be more gems than just system gems though.

Evan

On Fri, Mar 21, 2008 at 1:11 PM, Kirk Haines <wyhaines at gmail.com> wrote:
> On Fri, Mar 21, 2008 at 10:50 AM, Evan Weaver <evan at cloudbur.st> wrote:
>  > Do you have any GC patches in Ruby, use of DL, shaky C libs, or a
>  > weird Ruby patchlevel?
>
>  Evan, if you look back in the thread, he listed his gems on Mar 12th.
>
>
>
>
>  Kirk Haines
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From swindsor at gmail.com  Fri Mar 21 14:12:19 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Fri, 21 Mar 2008 11:12:19 -0700
Subject: [Mongrel] mongrel garbage collection
Message-ID: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>

Sorry, for the re-post, but I'm new to the mailing list and wanted to bring
back up and old topic I saw in the archives.

http://rubyforge.org/pipermail/mongrel-users/2008-February/004991.html

I think a patch to delay garbage collection and run it later is pretty
important for high performance web applications.  I do understand the
trade-offs of having explicit vs. implicit garbage collection running, and
would much prefer to off-load my garbage collection until later point (when
users are not waiting for a request).

I agree from the previous points that this could very well be
rails-specific, but isn't this a feature that would benefit all of the
frameworks that use mongrel?

This could be easily added as a configuration option to run after N number
of requests or let the GC behave as normal and run when needed, the default
of course, allowing the GC when it deems necessary.  Adding the collection
would be explicit after processing a request, but before listening to any
new requests.

- scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080321/f1ad87d5/attachment.html 

From wyhaines at gmail.com  Fri Mar 21 14:49:11 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 21 Mar 2008 12:49:11 -0600
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
Message-ID: <f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>

On Fri, Mar 21, 2008 at 12:12 PM, Scott Windsor <swindsor at gmail.com> wrote:
> Sorry, for the re-post, but I'm new to the mailing list and wanted to bring
> back up and old topic I saw in the archives.
>
> http://rubyforge.org/pipermail/mongrel-users/2008-February/004991.html
>
> I think a patch to delay garbage collection and run it later is pretty
> important for high performance web applications.  I do understand the

In the vast majority of cases you are going to do a worse job of
determining when and how often to run the GC than even MRI Ruby's
simple algorithms.  MRI garbage collection stops the world -- nothing
else happens while the GC runs --  so when talking about overall
throughput on an application, you don't want it to run any more than
necessary.

I don't use Rails, but in the past I have experimented with this quite
a lot under IOWA, and in my normal applications (i.e. not using
RMagick) I could never come up with an algorithm of self-managed
GC.disable/GC.enable/GC.start that gave the same overall level of
throughput that I got by letting Ruby start the GC according to its
own algorithms.  That experience makes me skeptical of that approach
in the general case, though there are occasional specific cases where
it can be useful.


Kirk Haines

From swindsor at gmail.com  Fri Mar 21 15:23:16 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Fri, 21 Mar 2008 12:23:16 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
Message-ID: <aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>

On Fri, Mar 21, 2008 at 11:49 AM, Kirk Haines <wyhaines at gmail.com> wrote:

> On Fri, Mar 21, 2008 at 12:12 PM, Scott Windsor <swindsor at gmail.com>
> wrote:
> > Sorry, for the re-post, but I'm new to the mailing list and wanted to
> bring
> > back up and old topic I saw in the archives.
> >
> > http://rubyforge.org/pipermail/mongrel-users/2008-February/004991.html
> >
> > I think a patch to delay garbage collection and run it later is pretty
> > important for high performance web applications.  I do understand the
>
> In the vast majority of cases you are going to do a worse job of
> determining when and how often to run the GC than even MRI Ruby's
> simple algorithms.  MRI garbage collection stops the world -- nothing
> else happens while the GC runs --  so when talking about overall
> throughput on an application, you don't want it to run any more than
> necessary.
>
> I don't use Rails, but in the past I have experimented with this quite
> a lot under IOWA, and in my normal applications (i.e. not using
> RMagick) I could never come up with an algorithm of self-managed
> GC.disable/GC.enable/GC.start that gave the same overall level of
> throughput that I got by letting Ruby start the GC according to its
> own algorithms.  That experience makes me skeptical of that approach
> in the general case, though there are occasional specific cases where
> it can be useful.
>
>
> Kirk Haines


I understand that the GC is quite knowledgeable about when to run garbage
collection when examining the heap.  But, the GC doesn't know anything about
my application or it's state.  The fact that when the GC runs everything
stops is why I'd prefer to limit when the GC will run.  I'd rather it run
outside of serving a web request rather then when it's right in the middle
of serving requests.

I know that the ideal situation is to not need to run the GC, but the
reality is that I'm using various gems and plugins and not all are well
behaved and free of memory leaks.  Rails itself may also have regular leaks
from time to time and I'd prefer to have my application consistently be slow
than randomly (and unexpectedly) be slow.  The alternative is to terminate
your application after N number of requests and never run the GC, which I'm
not a fan of.

- scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080321/50aab13b/attachment.html 

From wyhaines at gmail.com  Fri Mar 21 16:19:01 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Fri, 21 Mar 2008 14:19:01 -0600
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
Message-ID: <f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>

On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <swindsor at gmail.com> wrote:

> I understand that the GC is quite knowledgeable about when to run garbage
> collection when examining the heap.  But, the GC doesn't know anything about
> my application or it's state.  The fact that when the GC runs everything
> stops is why I'd prefer to limit when the GC will run.  I'd rather it run
> outside of serving a web request rather then when it's right in the middle
> of serving requests.

It doesn't matter, if one is looking at overall throughput.  And how
long do your GC runs take?  If you have a GC invocation that is
noticable on a single request, your processes must be gigantic, which
would suggest to me that there's a more fundamental problem with the
app.

> I know that the ideal situation is to not need to run the GC, but the
> reality is that I'm using various gems and plugins and not all are well
> behaved and free of memory leaks.  Rails itself may also have regular leaks

No, it's impractical to never run the GC.  The ideal situation, at
least where execution performance and throughput on a high performance
app is concerned, is to just intelligently reduce how often it needs
to run by paying attention to your object creation.  In particular,
pay attention to the throwaway object creation.

> from time to time and I'd prefer to have my application consistently be slow
> than randomly (and unexpectedly) be slow.  The alternative is to terminate
> your application after N number of requests and never run the GC, which I'm
> not a fan of.

If your goal is to deal with memory leaks, then you really need to
define what that means in a GC'd language like Ruby.
To me, a leak is something that consumes memory in a way that eludes
the GC's ability to track it and reuse it.  The fundamental nature of
that sort of thing is that the GC can't help you with it.

If by leaks, you mean code that just creates a lot of objects that the
GC needs to clean up, then those aren't leaks.  It may be inefficient
code, but it's not a memory leak.

And in the end, while disabling GC over the course of a request may
result in processing that one request more quickly than it would have
been processed otherwise, the disable/enable dance is going to cost
you something.

You'll likely either end up using more RAM than you otherwise would
have in between GC calls, resulting in bigger processes, or you end up
calling GC more often than you otherwise would have, reducing your
high performance app's throughput.

And for the general cases, that's not an advantageous situation.

To be more specific, if excessive RAM usage and GC costs that are
noticable to the user during requests is a common thing for Rails
apps, and the reason for that is bad code in Rails and not just bad
user code, then the Rails folks should be the targets of a
conversation on the matter.  Mongrel itself, though, does not need to
be, and should not be playing manual memory management games on the
behalf of a web framework.


Kirk Haines

From evan at cloudbur.st  Fri Mar 21 16:22:41 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 21 Mar 2008 16:22:41 -0400
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
Message-ID: <b6f68fc60803211322x27ac1851r7280c3238696cbc1@mail.gmail.com>

> The alternative is to terminate your application after N number of requests and never run the > GC, which I'm not a fan of.

WSGI (Python) can do that, and it's a pretty nice alternative to
having Monit kill a leaky app that may have a bunch of requests queued
up (Mongrel soft shutdown not withstanding).

Evan

On Fri, Mar 21, 2008 at 3:23 PM, Scott Windsor <swindsor at gmail.com> wrote:
> On Fri, Mar 21, 2008 at 11:49 AM, Kirk Haines <wyhaines at gmail.com> wrote:
>
> >
> > On Fri, Mar 21, 2008 at 12:12 PM, Scott Windsor <swindsor at gmail.com>
> wrote:
> > > Sorry, for the re-post, but I'm new to the mailing list and wanted to
> bring
> > > back up and old topic I saw in the archives.
> > >
> > > http://rubyforge.org/pipermail/mongrel-users/2008-February/004991.html
> > >
> > > I think a patch to delay garbage collection and run it later is pretty
> > > important for high performance web applications.  I do understand the
> >
> > In the vast majority of cases you are going to do a worse job of
> > determining when and how often to run the GC than even MRI Ruby's
> > simple algorithms.  MRI garbage collection stops the world -- nothing
> > else happens while the GC runs --  so when talking about overall
> > throughput on an application, you don't want it to run any more than
> > necessary.
> >
> > I don't use Rails, but in the past I have experimented with this quite
> > a lot under IOWA, and in my normal applications (i.e. not using
> > RMagick) I could never come up with an algorithm of self-managed
> > GC.disable/GC.enable/GC.start that gave the same overall level of
> > throughput that I got by letting Ruby start the GC according to its
> > own algorithms.  That experience makes me skeptical of that approach
> > in the general case, though there are occasional specific cases where
> > it can be useful.
> >
> >
> > Kirk Haines
>
> I understand that the GC is quite knowledgeable about when to run garbage
> collection when examining the heap.  But, the GC doesn't know anything about
> my application or it's state.  The fact that when the GC runs everything
> stops is why I'd prefer to limit when the GC will run.  I'd rather it run
> outside of serving a web request rather then when it's right in the middle
> of serving requests.
>
> I know that the ideal situation is to not need to run the GC, but the
> reality is that I'm using various gems and plugins and not all are well
> behaved and free of memory leaks.  Rails itself may also have regular leaks
> from time to time and I'd prefer to have my application consistently be slow
> than randomly (and unexpectedly) be slow.  The alternative is to terminate
> your application after N number of requests and never run the GC, which I'm
> not a fan of.
>
> - scott
>
> _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From evan at cloudbur.st  Fri Mar 21 16:40:40 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Fri, 21 Mar 2008 16:40:40 -0400
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
Message-ID: <b6f68fc60803211340j43c50694ia0e6c03ffa647a46@mail.gmail.com>

> You'll likely either end up using more RAM than you otherwise would
> have in between GC calls, resulting in bigger processes

This is definitely true. Keep in mind that the in-struct mark phase
means that the entire process has to lurch out of swap whenever the GC
runs. Since the process is now much bigger, and the pages idled longer
and are more likely to be swapped out, that can be pretty a brutal
hit.

Evan

On Fri, Mar 21, 2008 at 4:19 PM, Kirk Haines <wyhaines at gmail.com> wrote:
> On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <swindsor at gmail.com> wrote:
>
>  > I understand that the GC is quite knowledgeable about when to run garbage
>  > collection when examining the heap.  But, the GC doesn't know anything about
>  > my application or it's state.  The fact that when the GC runs everything
>  > stops is why I'd prefer to limit when the GC will run.  I'd rather it run
>  > outside of serving a web request rather then when it's right in the middle
>  > of serving requests.
>
>  It doesn't matter, if one is looking at overall throughput.  And how
>  long do your GC runs take?  If you have a GC invocation that is
>  noticable on a single request, your processes must be gigantic, which
>  would suggest to me that there's a more fundamental problem with the
>  app.
>
>
>  > I know that the ideal situation is to not need to run the GC, but the
>  > reality is that I'm using various gems and plugins and not all are well
>  > behaved and free of memory leaks.  Rails itself may also have regular leaks
>
>  No, it's impractical to never run the GC.  The ideal situation, at
>  least where execution performance and throughput on a high performance
>  app is concerned, is to just intelligently reduce how often it needs
>  to run by paying attention to your object creation.  In particular,
>  pay attention to the throwaway object creation.
>
>
>  > from time to time and I'd prefer to have my application consistently be slow
>  > than randomly (and unexpectedly) be slow.  The alternative is to terminate
>  > your application after N number of requests and never run the GC, which I'm
>  > not a fan of.
>
>  If your goal is to deal with memory leaks, then you really need to
>  define what that means in a GC'd language like Ruby.
>  To me, a leak is something that consumes memory in a way that eludes
>  the GC's ability to track it and reuse it.  The fundamental nature of
>  that sort of thing is that the GC can't help you with it.
>
>  If by leaks, you mean code that just creates a lot of objects that the
>  GC needs to clean up, then those aren't leaks.  It may be inefficient
>  code, but it's not a memory leak.
>
>  And in the end, while disabling GC over the course of a request may
>  result in processing that one request more quickly than it would have
>  been processed otherwise, the disable/enable dance is going to cost
>  you something.
>
>  You'll likely either end up using more RAM than you otherwise would
>  have in between GC calls, resulting in bigger processes, or you end up
>  calling GC more often than you otherwise would have, reducing your
>  high performance app's throughput.
>
>  And for the general cases, that's not an advantageous situation.
>
>  To be more specific, if excessive RAM usage and GC costs that are
>  noticable to the user during requests is a common thing for Rails
>  apps, and the reason for that is bad code in Rails and not just bad
>  user code, then the Rails folks should be the targets of a
>  conversation on the matter.  Mongrel itself, though, does not need to
>  be, and should not be playing manual memory management games on the
>  behalf of a web framework.
>
>
>
>
>  Kirk Haines
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From public at misuse.org  Fri Mar 21 17:19:13 2008
From: public at misuse.org (Steve Midgley)
Date: Fri, 21 Mar 2008 14:19:13 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <mailman.268.1206130744.19550.mongrel-users@rubyforge.org>
References: <mailman.268.1206130744.19550.mongrel-users@rubyforge.org>
Message-ID: <20080321211920.E3CDE18586C0@rubyforge.org>

At 01:19 PM 3/21/2008, mongrel-users-request at rubyforge.org wrote:
>Date: Fri, 21 Mar 2008 14:19:01 -0600
>From: "Kirk Haines" <wyhaines at gmail.com>
>Subject: Re: [Mongrel] mongrel garbage collection
>To: mongrel-users at rubyforge.org
>Message-ID:
>         <f4cd26df0803211319o491ad3c1u92ef5509c433c76b at mail.gmail.com>
>Content-Type: text/plain; charset=ISO-8859-1
>On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <swindsor at gmail.com> 
>wrote:
>
> > I understand that the GC is quite knowledgeable about when to run 
> garbage
> > collection when examining the heap.  But, the GC doesn't know 
> anything about
> > my application or it's state.  The fact that when the GC runs 
> everything
> > stops is why I'd prefer to limit when the GC will run.  I'd rather 
> it run
> > outside of serving a web request rather then when it's right in the 
> middle
> > of serving requests.
>
>It doesn't matter, if one is looking at overall throughput.

Hi Kirk,

One thought on this - would it be possible to schedule GC to run just 
after all the html has been rendered to the client from Rails, but 
while leaving open the connection (so that mongrel is blocked on 
Rails)?

If so, it seems like if one were using something like nginx fair proxy, 
then the mongrel would be running it's garbage collection AFTER the 
client got all its html but BEFORE any new requests were sent to it.

In a fully loaded server it wouldn't matter at all, but most 
environments have a little headroom at least, so that nginx fair proxy 
would just route around the mongrel that is still running a GC at the 
end of it's Rails loop.

So total throughput for a given (non-max) volume of requests might be 
unaffected since nothing would ever pile up behind a rails process that 
has slowed down to run GC (and the client will be happy since they got 
all their html before the GC started).

I have no idea if this is meaningful, but I've been playing with some 
performance tests against mongrel + nginx fair proxy and it occurs to 
me that this might be relevant..

Best,

Steve


From dave at cheney.net  Sat Mar 22 06:39:41 2008
From: dave at cheney.net (Dave Cheney)
Date: Sat, 22 Mar 2008 21:39:41 +1100
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <20080321211920.E3CDE18586C0@rubyforge.org>
References: <mailman.268.1206130744.19550.mongrel-users@rubyforge.org>
	<20080321211920.E3CDE18586C0@rubyforge.org>
Message-ID: <D161E0D0-3927-4D35-A64C-B33D26F3181F@cheney.net>


On 22/03/2008, at 8:19 AM, Steve Midgley wrote:

> If so, it seems like if one were using something like nginx fair  
> proxy,
> then the mongrel would be running it's garbage collection AFTER the
> client got all its html but BEFORE any new requests were sent to it.
>
> In a fully loaded server it wouldn't matter at all, but most
> environments have a little headroom at least, so that nginx fair proxy
> would just route around the mongrel that is still running a GC at the
> end of it's Rails loop.

That would only be true if you set the connect timeout on the backend  
to 1 second AND your GC pass took longer than 1 second.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080322/67f7c34a/attachment.html 

From www at saepia.net  Sat Mar 22 08:05:06 2008
From: www at saepia.net (Marcin Lewandowski)
Date: Sat, 22 Mar 2008 13:05:06 +0100
Subject: [Mongrel] Mongrel has crashed
In-Reply-To: <b6f68fc60803211021t53e490a7rd3e6760375f600ca@mail.gmail.com>
References: <47D7953B.4020600@saepia.net>
Message-ID: <20080322120506.B654CC649B@hosting.spine.pl>

<f4cd26df0803120933w663787ddyd1228dace2191122 at mail.gmail.com>
<8be813400803120952w7f96d6c4u951c732d173970d4 at mail.gmail.com>
<47D82BE7.8050803 at saepia.net>
<f4cd26df0803121248h30e3a30dm6a9837a6843a57b6 at mail.gmail.com>
<47D836F8.6070809 at saepia.net> <47E3937C.7050902 at saepia.net>
<f4cd26df0803210734m5c137b8p7ac8c8cf51794372 at mail.gmail.com>
<b6f68fc60803210950q19db47dbmbb7b5ea8b003796f at mail.gmail.com>
<f4cd26df0803211011nd042ab0yf2030cba349161dc at mail.gmail.com> <b6f68fc60803211021t53e490a7rd3e6760375f600ca at mail.gmail.com>
Message-ID: <1185b4255b2b2872750d836f6a20846b at localhost>
X-Sender: www at saepia.net
Received: from cie46.neoplus.adsl.tpnet.pl [83.31.28.46] with HTTP/1.1 (POST);
	Sat, 22 Mar 2008 13:05:06 +0100
User-Agent: RoundCube Webmail/0.1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Here comes my current gem list after upgrade:

hosting cur # gem list

*** LOCAL GEMS ***

actionmailer (1.3.6)
    Service layer for easy email delivery and testing.

actionpack (1.13.6)
    Web-flow and rendering framework putting the VC in MVC.

actionwebservice (1.2.6)
    Web service support for Action Pack.

activerecord (1.15.6)
    Implements the ActiveRecord pattern for ORM.

activesupport (1.4.4)
    Support and utility classes used by the Rails framework.

cgi_multipart_eof_fix (2.5.0)
    Fix an exploitable bug in CGI multipart parsing.

daemons (1.0.5)
    A toolkit to create and control daemons in different ways

fastthread (1.0.1)
    Optimized replacement for thread.rb primitives

gem_plugin (0.2.3)
    A plugin system based on rubygems that uses dependencies only

gettext (1.90.0, 1.9.0)
    Ruby-GetText-Package is a libary and tools to localize messages.

icalendar (1.0.2)
    A ruby implementation of the iCalendar specification (RFC-2445).

mongrel (1.1.4)
    A small fast HTTP library and server that runs Rails, Camping, Nitro
    and Iowa apps.

mongrel_upload_progress (0.2.2)
    The mongrel_upload_progress gemplugin

rails (1.2.6)
    Web-application framework with template engine, control-flow layer,
    and ORM.

rake (0.7.3)
    Ruby based make-like utility.

sources (0.0.1)
    This package provides download sources for remote gem installation



hosting cur # ruby -v
ruby 1.8.6 (2008-03-03 patchlevel 114) [i686-linux]


I have 2 versions of gettext, I'll try to uninstall older one, and if it
won't help, the newer one.

I have latest patchlevel of ruby, only custom patches could be from
gentoo's portage.

It happened a few times before I've started to use gettext, but my mongrel
was also older. Arghh seems to be difficult to find, what's going on. I'll
try to do this gettext stuff...

m.


From lists at ruby-forum.com  Mon Mar 24 03:55:16 2008
From: lists at ruby-forum.com (Vapor ..)
Date: Mon, 24 Mar 2008 08:55:16 +0100
Subject: [Mongrel] Doesnt show log on command prompt
Message-ID: <9a21d8570bf0b02692d29d7301bf9b19@ruby-forum.com>

I have latest version of mongrel installed.
But when I run 'ruby script/server' it starts the server and shows
nothing else, like development log it used to show on my previous dev
machine.

Any idea?
-- 
Posted via http://www.ruby-forum.com/.

From swindsor at gmail.com  Mon Mar 24 11:21:52 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Mon, 24 Mar 2008 08:21:52 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
Message-ID: <aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>

On Fri, Mar 21, 2008 at 1:19 PM, Kirk Haines <wyhaines at gmail.com> wrote:

> On Fri, Mar 21, 2008 at 1:23 PM, Scott Windsor <swindsor at gmail.com> wrote:
>
> > I understand that the GC is quite knowledgeable about when to run
> garbage
> > collection when examining the heap.  But, the GC doesn't know anything
> about
> > my application or it's state.  The fact that when the GC runs everything
> > stops is why I'd prefer to limit when the GC will run.  I'd rather it
> run
> > outside of serving a web request rather then when it's right in the
> middle
> > of serving requests.
>
> It doesn't matter, if one is looking at overall throughput.  And how
> long do your GC runs take?  If you have a GC invocation that is
> noticable on a single request, your processes must be gigantic, which
> would suggest to me that there's a more fundamental problem with the
> app.


Right now, my processes aren't gigantic... I'm preparing for a 'worst case'
scenario when I have a extremely large processes or memory usage.  This can
easily happen on specific applications such as an image server (using image
magick) or parsing/creating large xml payloads (a large REST server).  For
those applications, I may have a large amount of memory used for each
request, which will increase until the GC is run.


> > I know that the ideal situation is to not need to run the GC, but the
> > reality is that I'm using various gems and plugins and not all are well
> > behaved and free of memory leaks.  Rails itself may also have regular
> leaks
>
> No, it's impractical to never run the GC.  The ideal situation, at
> least where execution performance and throughput on a high performance
> app is concerned, is to just intelligently reduce how often it needs
> to run by paying attention to your object creation.  In particular,
> pay attention to the throwaway object creation.
>

There may be perfectly good reasons to have intermediate object creation
(good encapsulation, usage of a another library/gem you can't modify, large
operations that you need to keep atomic).  While ideally you'd fix the
memory usage problem, this doesn't solve all cases.


>
> > from time to time and I'd prefer to have my application consistently be
> slow
> > than randomly (and unexpectedly) be slow.  The alternative is to
> terminate
> > your application after N number of requests and never run the GC, which
> I'm
> > not a fan of.
>
> If your goal is to deal with memory leaks, then you really need to
> define what that means in a GC'd language like Ruby.
> To me, a leak is something that consumes memory in a way that eludes
> the GC's ability to track it and reuse it.  The fundamental nature of
> that sort of thing is that the GC can't help you with it.
>

Yes, for Ruby (and other GC'd languages), it's much harder to leak memory
such that the GC can never clean it up - but it does (and has) happened.
This case I'm less concerned about as a leak of this magnitude should be
considered a bug and fixed.

>
> If by leaks, you mean code that just creates a lot of objects that the
> GC needs to clean up, then those aren't leaks.  It may be inefficient
> code, but it's not a memory leak.
>

Inefficient it may be - but it might be just optimizing for a different
problem.  For example, take ActiveRecord's association cache and it's query
cache.  If you're doing a large number of queries each page load,
ActiveRecord is still going to cache them for each request - this is far
better than further round trips to the database, but may lead to a large
amount of memory consumed per each request.


>
> And in the end, while disabling GC over the course of a request may
> result in processing that one request more quickly than it would have
> been processed otherwise, the disable/enable dance is going to cost
> you something.
>

Agreed.  But again, I'd rather it be a constant cost outside of processing a
request than a variable cost inside of processing a request.


>
> You'll likely either end up using more RAM than you otherwise would
> have in between GC calls, resulting in bigger processes, or you end up
> calling GC more often than you otherwise would have, reducing your
> high performance app's throughput.
>
> And for the general cases, that's not an advantageous situation.
>

This can vary from application to application - all the more reason to make
this a configurable option (and not the default).

>
> To be more specific, if excessive RAM usage and GC costs that are
> noticable to the user during requests is a common thing for Rails
> apps, and the reason for that is bad code in Rails and not just bad
> user code, then the Rails folks should be the targets of a
> conversation on the matter.  Mongrel itself, though, does not need to
> be, and should not be playing manual memory management games on the
> behalf of a web framework.
>
>
> Kirk Haines
>

I still disagree on this point - I doubt that Rails is the only web
framework that would benefit from being able to control when the GC is run.
This is going to be a common problem across frameworks whenever web
applications are consuming then releasing large amounts of memory - I'd say
it can be a pretty common use case for certain types of web applications.

- scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080324/4fa5c50e/attachment-0001.html 

From swindsor at gmail.com  Mon Mar 24 11:27:50 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Mon, 24 Mar 2008 08:27:50 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <b6f68fc60803211322x27ac1851r7280c3238696cbc1@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<b6f68fc60803211322x27ac1851r7280c3238696cbc1@mail.gmail.com>
Message-ID: <aa2e18ec0803240827q5d54a2fdq35bb0ff60d7eb815@mail.gmail.com>

If you plan on regularly killing your application (for whatever reason),
then this is a pretty good option.  This is a pretty common practice for
apache modules and fastcgi applications as a hold-over from dealing with
older leaky C apps.

I'd personally prefer for my Ruby web apps to re-run the GC rather than have
to startup/shutdown/parse configs/connect to external resources costs, but
it's because they are far less likely to leak memory that the GC can't catch
or get into an unstable state.

- scott

On Fri, Mar 21, 2008 at 1:22 PM, Evan Weaver <evan at cloudbur.st> wrote:

> > The alternative is to terminate your application after N number of
> requests and never run the > GC, which I'm not a fan of.
>
> WSGI (Python) can do that, and it's a pretty nice alternative to
> having Monit kill a leaky app that may have a bunch of requests queued
> up (Mongrel soft shutdown not withstanding).
>
> Evan
>
> On Fri, Mar 21, 2008 at 3:23 PM, Scott Windsor <swindsor at gmail.com> wrote:
> > On Fri, Mar 21, 2008 at 11:49 AM, Kirk Haines <wyhaines at gmail.com>
> wrote:
> >
> > >
> > > On Fri, Mar 21, 2008 at 12:12 PM, Scott Windsor <swindsor at gmail.com>
> > wrote:
> > > > Sorry, for the re-post, but I'm new to the mailing list and wanted
> to
> > bring
> > > > back up and old topic I saw in the archives.
> > > >
> > > >
> http://rubyforge.org/pipermail/mongrel-users/2008-February/004991.html
> > > >
> > > > I think a patch to delay garbage collection and run it later is
> pretty
> > > > important for high performance web applications.  I do understand
> the
> > >
> > > In the vast majority of cases you are going to do a worse job of
> > > determining when and how often to run the GC than even MRI Ruby's
> > > simple algorithms.  MRI garbage collection stops the world -- nothing
> > > else happens while the GC runs --  so when talking about overall
> > > throughput on an application, you don't want it to run any more than
> > > necessary.
> > >
> > > I don't use Rails, but in the past I have experimented with this quite
> > > a lot under IOWA, and in my normal applications (i.e. not using
> > > RMagick) I could never come up with an algorithm of self-managed
> > > GC.disable/GC.enable/GC.start that gave the same overall level of
> > > throughput that I got by letting Ruby start the GC according to its
> > > own algorithms.  That experience makes me skeptical of that approach
> > > in the general case, though there are occasional specific cases where
> > > it can be useful.
> > >
> > >
> > > Kirk Haines
> >
> > I understand that the GC is quite knowledgeable about when to run
> garbage
> > collection when examining the heap.  But, the GC doesn't know anything
> about
> > my application or it's state.  The fact that when the GC runs everything
> > stops is why I'd prefer to limit when the GC will run.  I'd rather it
> run
> > outside of serving a web request rather then when it's right in the
> middle
> > of serving requests.
> >
> > I know that the ideal situation is to not need to run the GC, but the
> > reality is that I'm using various gems and plugins and not all are well
> > behaved and free of memory leaks.  Rails itself may also have regular
> leaks
> > from time to time and I'd prefer to have my application consistently be
> slow
> > than randomly (and unexpectedly) be slow.  The alternative is to
> terminate
> > your application after N number of requests and never run the GC, which
> I'm
> > not a fan of.
> >
> > - scott
> >
> > _______________________________________________
> >  Mongrel-users mailing list
> >  Mongrel-users at rubyforge.org
> >  http://rubyforge.org/mailman/listinfo/mongrel-users
> >
>
>
>
> --
> Evan Weaver
> Cloudburst, LLC
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080324/fae9ef17/attachment.html 

From swindsor at gmail.com  Mon Mar 24 11:32:03 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Mon, 24 Mar 2008 08:32:03 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <D161E0D0-3927-4D35-A64C-B33D26F3181F@cheney.net>
References: <mailman.268.1206130744.19550.mongrel-users@rubyforge.org>
	<20080321211920.E3CDE18586C0@rubyforge.org>
	<D161E0D0-3927-4D35-A64C-B33D26F3181F@cheney.net>
Message-ID: <aa2e18ec0803240832i31186e7ak4ebc7c3ae062dbff@mail.gmail.com>

On Sat, Mar 22, 2008 at 3:39 AM, Dave Cheney <dave at cheney.net> wrote:

>
> On 22/03/2008, at 8:19 AM, Steve Midgley wrote:
>
> If so, it seems like if one were using something like nginx fair proxy,
> then the mongrel would be running it's garbage collection AFTER the
> client got all its html but BEFORE any new requests were sent to it.
>
> In a fully loaded server it wouldn't matter at all, but most
> environments have a little headroom at least, so that nginx fair proxy
> would just route around the mongrel that is still running a GC at the
> end of it's Rails loop.
>
>
> That would only be true if you set the connect timeout on the backend to 1
> second AND your GC pass took longer than 1 second.
>

Yes, but worse case here is that another request gets delayed before
processing.  Still potentially better (IMHO) than dealing with this delaying
when processing a request.

- scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080324/77dafa2d/attachment.html 

From public at misuse.org  Mon Mar 24 11:50:17 2008
From: public at misuse.org (Steve Midgley)
Date: Mon, 24 Mar 2008 08:50:17 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <mailman.161.1206372117.18952.mongrel-users@rubyforge.org>
References: <mailman.161.1206372117.18952.mongrel-users@rubyforge.org>
Message-ID: <20080324155226.B007318586C2@rubyforge.org>

At 08:21 AM 3/24/2008, mongrel-users-request at rubyforge.org wrote:
>Message: 7
>Date: Mon, 24 Mar 2008 08:21:52 -0700
>From: "Scott Windsor" <swindsor at gmail.com>
>Subject: Re: [Mongrel] mongrel garbage collection
>To: mongrel-users at rubyforge.org
>Message-ID:
>         <aa2e18ec0803240821o1f4f5921tf4156780ed0928d at mail.gmail.com>
>[snip]
>Right now, my processes aren't gigantic... I'm preparing for a 'worst 
>case'
>scenario when I have a extremely large processes or memory 
>usage.  This can
>easily happen on specific applications such as an image server (using 
>image
>magick) or parsing/creating large xml payloads (a large REST 
>server).  For
>those applications, I may have a large amount of memory used for each
>request, which will increase until the GC is run.
>
>[snip]
>There may be perfectly good reasons to have intermediate object 
>creation
>(good encapsulation, usage of a another library/gem you can't modify, 
>large
>operations that you need to keep atomic).  While ideally you'd fix the
>memory usage problem, this doesn't solve all cases.

Hi Scott,

I hope this somewhat OT post is ok (feedback welcome). I've had memory 
problems with image magick too - even when it runs out of process. On 
certain (rare but reasonably sized) image files it seems to go memory 
haywire, eating too much memory and throwing my app stack into swap.

So I wrote this simple rails plug-in which is very limited in function, 
but does mostly what I needed from an image processor. Notable for your 
issue above, it lets you easily specify limits on how much memory image 
magick is allowed to consume while doing its work (thanks to Ara Howard 
on initial direction on that one). It might be interest to you:

http://www.misuse.org/science/2008/01/30/mojomagick-ruby-image-library-for-imagemagick/

Best,

Steve


From wyhaines at gmail.com  Mon Mar 24 12:51:29 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 24 Mar 2008 10:51:29 -0600
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
Message-ID: <f4cd26df0803240951p24da5136r11ebd911d8359e43@mail.gmail.com>

On Mon, Mar 24, 2008 at 9:21 AM, Scott Windsor <swindsor at gmail.com> wrote:

> Right now, my processes aren't gigantic... I'm preparing for a 'worst case'
> scenario when I have a extremely large processes or memory usage.  This can
> easily happen on specific applications such as an image server (using image
> magick) or parsing/creating large xml payloads (a large REST server).  For
> those applications, I may have a large amount of memory used for each
> request, which will increase until the GC is run.

(*nod*)  image magick is a well known bad citizen.  Either don't use
it at all, or use it in an external process from your web app
processes.

And if, for whatever reason, you must use it inside of your web app
process, and your use case really does create processes so enormous
that you can perceive a response lag from a manual GC.start inside of
your request processing, then create a custom Rails handler that does
it.  You can trivially alter it to do whatever GC.foo actions you
desire.  The code is simple and easy to follow, so just make your own
Mongrel::Rails::RailsHandlerWithParanoidGCManagement.

> There may be perfectly good reasons to have intermediate object creation
> (good encapsulation, usage of a another library/gem you can't modify, large
> operations that you need to keep atomic).  While ideally you'd fix the
> memory usage problem, this doesn't solve all cases.

Obviously.  It's easy and convenient to ignore the issue, and often
the issue doesn't matter for a given piece of code.  But if memory
usage or execution speed becomes an issue for one's code, going back
and taking a look at the throwaway object creation, and addressing it,
can net considerable improvements.

> Yes, for Ruby (and other GC'd languages), it's much harder to leak memory
> such that the GC can never clean it up - but it does (and has) happened.
> This case I'm less concerned about as a leak of this magnitude should be
> considered a bug and fixed.

Oh, I know.  That's why I brought it up, though.  You were talking
about memory leaks, so I wanted to make a distinction.  Real leaks,
like the Array#shift bug, or leaky continuations,  or badly behaved
Ruby extensions, aren't affected by GC manipulations.

> Inefficient it may be - but it might be just optimizing for a different
> problem.  For example, take ActiveRecord's association cache and it's query
> cache.  If you're doing a large number of queries each page load,
> ActiveRecord is still going to cache them for each request - this is far
> better than further round trips to the database, but may lead to a large
> amount of memory consumed per each request.

Sure.  And if it's optimizing for a different problem, then that's
fine, so long as the optimization isn't creating a worse probablem
than the issue it's trying to address.

But that's also largely irrelevant, I think.  I just did a quick test.
 I created a program that creates 10 million objects.  It has a
footprint of about a gigabyte of RAM usage.  It takes Ruby 0.5 seconds
to walk that 10 million objects on my server.

If you have a web app that has processes anywhere near that large, you
have bigger problems to deal with.  And if you have a more reasonably
large, million object app, then on my server, the GC cost would be
0.05 seconds.  Given the typical speed of Rails apps, an occasional
0.05 second delay is going to be unnoticable.

> Agreed.  But again, I'd rather it be a constant cost outside of processing a
> request than a variable cost inside of processing a request.

You're worrying about something that just isn't a problem in the vast,
vast majority of cases.  Again, testing on my server, even with a very
simple, very fast piece of code creating objects, it takes almost 20x
as long to create the objects as to GC them.

> This can vary from application to application - all the more reason to make
> this a configurable option (and not the default).

It's still my position that it's not Mongrel's job to be implementing
a manual memory management scheme that is almost always going to be a
performance loser over just leaving it alone.

It's still my position that if one has an application that, through
testing, has been shown to have a use case where it can actually
benefit from manual GC.foo management, then one can trivially create a
mongrel handler that will do this for you.

> I still disagree on this point - I doubt that Rails is the only web
> framework that would benefit from being able to control when the GC is run.
> This is going to be a common problem across frameworks whenever web
> applications are consuming then releasing large amounts of memory - I'd say
> it can be a pretty common use case for certain types of web applications.

My point is that if it is _Rails_ code that is causing the problem,
that's a _Rails_ problem.

My point is also that manual GC.foo management is going to cause more
problems than it helps for the vast majority of applications.  GC
cycles aren't that slow, especially compared to the speed of a typical
Rails app, and certainly not when compared to the speed of a Rails
request that makes a lot of objects and does any sort of intensive,
time consuming operations.


Kirk Haines

From zedshaw at zedshaw.com  Tue Mar 25 14:53:23 2008
From: zedshaw at zedshaw.com (Zed A. Shaw)
Date: Tue, 25 Mar 2008 14:53:23 -0400
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
Message-ID: <20080325145323.e05fe8b5.zedshaw@zedshaw.com>

On Mon, 24 Mar 2008 08:21:52 -0700
"Scott Windsor" <swindsor at gmail.com> wrote:

> Right now, my processes aren't gigantic... I'm preparing for a 'worst case'
> scenario when I have a extremely large processes or memory usage.  This can
> easily happen on specific applications such as an image server (using image
> magick) or parsing/creating large xml payloads (a large REST server).  For
> those applications, I may have a large amount of memory used for each
> request, which will increase until the GC is run.

Well, does that mean you DO have this problem or DO NOT have this
problem?  If you aren't actually facing a real problem that could be
solved by this change then you most likely won't get very far.  Any
imagined scenario you come up with could easily just be avoided.

If you want to do this then you'll have to write code and you'll have
to learn how to make a Mongrel handler that is registered before and
after the regular rails handler.  All you do is have this before handler
analyze the request and disable the GC on the way in.  In the after
handler you just have to renable the GC and make it do the work.

It's pretty simple, but *you* will have to go read Mongrel code and
understand it first.  Otherwise you're just wasting your time really.

-- 
Zed A. Shaw
- Hate: http://savingtheinternetwithhate.com/
- Good: http://www.zedshaw.com/
- Evil: http://yearofevil.com/

From luislavena at gmail.com  Mon Mar 24 14:02:27 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Mon, 24 Mar 2008 15:02:27 -0300
Subject: [Mongrel] Doesnt show log on command prompt
In-Reply-To: <9a21d8570bf0b02692d29d7301bf9b19@ruby-forum.com>
References: <9a21d8570bf0b02692d29d7301bf9b19@ruby-forum.com>
Message-ID: <71166b3b0803241102m3ca3eb03wb14bad662ae4ded3@mail.gmail.com>

On Mon, Mar 24, 2008 at 4:55 AM, Vapor .. <lists at ruby-forum.com> wrote:
> I have latest version of mongrel installed.
>  But when I run 'ruby script/server' it starts the server and shows
>  nothing else, like development log it used to show on my previous dev
>  machine.
>

Are you expecting script/server dump information to the console? or in
the log file?

>  Any idea?

Can you rephrase your question? Is not quite clear what are you
getting and what are you expecting...

Btw, if you plan to post from anonymous posting sites (like
ruby-forum) at least provide a better alias than "Vapor".

-- 
Luis Lavena
Multimedia systems
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

From lists at ruby-forum.com  Mon Mar 24 14:19:34 2008
From: lists at ruby-forum.com (Vapor ..)
Date: Mon, 24 Mar 2008 19:19:34 +0100
Subject: [Mongrel] Doesnt show log on command prompt
In-Reply-To: <71166b3b0803241102m3ca3eb03wb14bad662ae4ded3@mail.gmail.com>
References: <9a21d8570bf0b02692d29d7301bf9b19@ruby-forum.com>
	<71166b3b0803241102m3ca3eb03wb14bad662ae4ded3@mail.gmail.com>
Message-ID: <a94207a98c205f1349b2ba5ba257a9d7@ruby-forum.com>

well actually I used to have a dev machine where it would show 
development log on command prompt (console) as I would use the app ... 
displaying behind the scene info like queries and stuff

but it doesnt do on new machine which is rather fresh..

somebody suggested tail --follow log/development.log

but that takes a separate window, it used to display info right after 
ruby script/serve

i hope it makes sense this time
-- 
Posted via http://www.ruby-forum.com/.

From evan at cloudbur.st  Mon Mar 24 14:32:13 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Mon, 24 Mar 2008 14:32:13 -0400
Subject: [Mongrel] Doesnt show log on command prompt
In-Reply-To: <a94207a98c205f1349b2ba5ba257a9d7@ruby-forum.com>
References: <9a21d8570bf0b02692d29d7301bf9b19@ruby-forum.com>
	<71166b3b0803241102m3ca3eb03wb14bad662ae4ded3@mail.gmail.com>
	<a94207a98c205f1349b2ba5ba257a9d7@ruby-forum.com>
Message-ID: <b6f68fc60803241132j6422258fr9db2cc0a7f4d9a2a@mail.gmail.com>

Put this in ~/.irbrc

script_console_running = ENV.include?('RAILS_ENV') &&
IRB.conf[:LOAD_MODULES] &&
IRB.conf[:LOAD_MODULES].include?('console_with_helpers')
rails_running = ENV.include?('RAILS_ENV') && !(IRB.conf[:LOAD_MODULES]
&& IRB.conf[:LOAD_MODULES].include?('console_with_helpers'))
irb_standalone_running = !script_console_running && !rails_running

if script_console_running
  require 'logger'
  Object.const_set(:RAILS_DEFAULT_LOGGER, Logger.new(STDOUT))
end

It doesn't have anything to do with Mongrel.

Evan

On Mon, Mar 24, 2008 at 2:19 PM, Vapor .. <lists at ruby-forum.com> wrote:
> well actually I used to have a dev machine where it would show
>  development log on command prompt (console) as I would use the app ...
>  displaying behind the scene info like queries and stuff
>
>  but it doesnt do on new machine which is rather fresh..
>
>  somebody suggested tail --follow log/development.log
>
>  but that takes a separate window, it used to display info right after
>  ruby script/serve
>
>  i hope it makes sense this time
>
> --
>  Posted via http://www.ruby-forum.com/.
>  _______________________________________________
>
>
> Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From evan at cloudbur.st  Mon Mar 24 14:34:01 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Mon, 24 Mar 2008 14:34:01 -0400
Subject: [Mongrel] Doesnt show log on command prompt
In-Reply-To: <b6f68fc60803241132j6422258fr9db2cc0a7f4d9a2a@mail.gmail.com>
References: <9a21d8570bf0b02692d29d7301bf9b19@ruby-forum.com>
	<71166b3b0803241102m3ca3eb03wb14bad662ae4ded3@mail.gmail.com>
	<a94207a98c205f1349b2ba5ba257a9d7@ruby-forum.com>
	<b6f68fc60803241132j6422258fr9db2cc0a7f4d9a2a@mail.gmail.com>
Message-ID: <b6f68fc60803241134u264a6a33k2d876cbb4e234cce@mail.gmail.com>

...if you meant the Rails console. You say 'console' and 'server'
both; I should have read more carefully.

If you want it in the script/server STDOUT you could add something
similar to config/environment.rb.

Evan

On Mon, Mar 24, 2008 at 2:32 PM, Evan Weaver <evan at cloudbur.st> wrote:
> Put this in ~/.irbrc
>
>  script_console_running = ENV.include?('RAILS_ENV') &&
>  IRB.conf[:LOAD_MODULES] &&
>  IRB.conf[:LOAD_MODULES].include?('console_with_helpers')
>  rails_running = ENV.include?('RAILS_ENV') && !(IRB.conf[:LOAD_MODULES]
>  && IRB.conf[:LOAD_MODULES].include?('console_with_helpers'))
>  irb_standalone_running = !script_console_running && !rails_running
>
>  if script_console_running
>   require 'logger'
>   Object.const_set(:RAILS_DEFAULT_LOGGER, Logger.new(STDOUT))
>  end
>
>  It doesn't have anything to do with Mongrel.
>
>  Evan
>
>
>
>  On Mon, Mar 24, 2008 at 2:19 PM, Vapor .. <lists at ruby-forum.com> wrote:
>  > well actually I used to have a dev machine where it would show
>  >  development log on command prompt (console) as I would use the app ...
>  >  displaying behind the scene info like queries and stuff
>  >
>  >  but it doesnt do on new machine which is rather fresh..
>  >
>  >  somebody suggested tail --follow log/development.log
>  >
>  >  but that takes a separate window, it used to display info right after
>  >  ruby script/serve
>  >
>  >  i hope it makes sense this time
>  >
>  > --
>  >  Posted via http://www.ruby-forum.com/.
>  >  _______________________________________________
>  >
>  >
>  > Mongrel-users mailing list
>  >  Mongrel-users at rubyforge.org
>  >  http://rubyforge.org/mailman/listinfo/mongrel-users
>  >
>
>
>
>  --
>  Evan Weaver
>  Cloudburst, LLC
>



-- 
Evan Weaver
Cloudburst, LLC

From swindsor at gmail.com  Mon Mar 24 14:58:48 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Mon, 24 Mar 2008 11:58:48 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <20080325145323.e05fe8b5.zedshaw@zedshaw.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
Message-ID: <aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>

On Tue, Mar 25, 2008 at 11:53 AM, Zed A. Shaw <zedshaw at zedshaw.com> wrote:

> On Mon, 24 Mar 2008 08:21:52 -0700
> "Scott Windsor" <swindsor at gmail.com> wrote:
>
> > Right now, my processes aren't gigantic... I'm preparing for a 'worst
> case'
> > scenario when I have a extremely large processes or memory usage.  This
> can
> > easily happen on specific applications such as an image server (using
> image
> > magick) or parsing/creating large xml payloads (a large REST server).
>  For
> > those applications, I may have a large amount of memory used for each
> > request, which will increase until the GC is run.
>
> Well, does that mean you DO have this problem or DO NOT have this
> problem?  If you aren't actually facing a real problem that could be
> solved by this change then you most likely won't get very far.  Any
> imagined scenario you come up with could easily just be avoided.



Right now my current deployment configuration for all my rails applications
is using apache + fastcgi.

With this deployment strategy,  if I don't set the garbage collection in my
dispatch.fcgi, any rails application I use that uses image magick (for
resizing/effects/etc) eats memory like a hog.
In my dispatch...
http://dev.rubyonrails.org/browser/trunk/railties/dispatches/dispatch.fcgi
I usually set this to around 50 executions per gc run and my rails apps seem
pretty happy.

This has been working great for me thus far, but using mod_fastcgi leaves
zombies processes occasionally during restart.  Checking in with the docs,
mod_fastcgi is more or less deprecated, and mod_fcgid is prefered.
mod_fcgid has all sorts of issues (random 500s and the like), and to boot
the documentation is quite poor.

So, I've decieded to move my apps over to using nginx with proxy with
mongrel.  The decsion to move the nginx is pretty minor (it's lighter weight
and easier to configure), but my decision to move to mongrel warrented a bit
of research.  I do want to ensure that all of my applications behave
properly in terms of memory consumption and the first thing I've noticed is
that mongrel doesn't have the same options available for customizing when
the GC runs.

This leads me to believe that either there's something specific to rails
running under FastCGI that requires the GC to disabled/enabled during
processes execution or mongrel hasn't implemented the feature yet.


>
> If you want to do this then you'll have to write code and you'll have
> to learn how to make a Mongrel handler that is registered before and
> after the regular rails handler.  All you do is have this before handler
> analyze the request and disable the GC on the way in.  In the after
> handler you just have to renable the GC and make it do the work.
>
> It's pretty simple, but *you* will have to go read Mongrel code and
> understand it first.  Otherwise you're just wasting your time really.

--
> Zed A. Shaw
> - Hate: http://savingtheinternetwithhate.com/
> - Good: http://www.zedshaw.com/
> - Evil: http://yearofevil.com/
>

Sounds good to me - I don't mind writing code, I just want to see if I do
spend the time if it's something the mongrel community would accept...

Quick question about the code change....

Counting the number of processes served and determining the GC behavior
should be done inside a mutex (or we start to run the risk of running the GC
twice or mis-counting the number of requests processed).

I don't see any common mutex used for all mongrel dispatchers, but the logic
is specific within each type of http handler (rails, camping, etc).  Would
it make sense then to put the optional GC run check (and GC run, if
applicable) within the syncronize block for each http handler or is the
something that should live in the base HTTPHandler class?

- scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080324/2604c5f2/attachment.html 

From luislavena at gmail.com  Mon Mar 24 15:18:33 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Mon, 24 Mar 2008 16:18:33 -0300
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
Message-ID: <71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>

On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor <swindsor at gmail.com> wrote:
>
> Right now my current deployment configuration for all my rails applications
> is using apache + fastcgi.
>
> With this deployment strategy,  if I don't set the garbage collection in my
> dispatch.fcgi, any rails application I use that uses image magick (for
> resizing/effects/etc) eats memory like a hog.
>  In my dispatch...
> http://dev.rubyonrails.org/browser/trunk/railties/dispatches/dispatch.fcgi
> I usually set this to around 50 executions per gc run and my rails apps seem
> pretty happy.
>

You're using *RMagick*, not ImageMagick directly. If you used the
later (via system calls) there will no be memory leakage you can worry
about.

> This has been working great for me thus far, but using mod_fastcgi leaves
> zombies processes occasionally during restart.  Checking in with the docs,
> mod_fastcgi is more or less deprecated, and mod_fcgid is prefered.
> mod_fcgid has all sorts of issues (random 500s and the like), and to boot
> the documentation is quite poor.
>

Moving from FastCGi to Mongrel will also require you monitor your
cluster processes with external tools, since you're suing things that
leak too much memory like RMagick and requires restart of the process.

To make it clear: the memory leaked by RMagick cannot be recovered
with garbage collection mechanism. I tried that several times but in
the long run, required to restart and hunt down all the zombies
processes left by Apache.

> So, I've decieded to move my apps over to using nginx with proxy with
> mongrel.  The decsion to move the nginx is pretty minor (it's lighter weight
> and easier to configure), but my decision to move to mongrel warrented a bit
> of research.  I do want to ensure that all of my applications behave
> properly in terms of memory consumption and the first thing I've noticed is
> that mongrel doesn't have the same options available for customizing when
> the GC runs.
>

Can you tell me how you addressed the "schedule" of the garbage
collection execution on your previous scenario? AFAIK most of the
frameworks or servers don't impose to the user how often GC should be
performed.

> This leads me to believe that either there's something specific to rails
> running under FastCGI that requires the GC to disabled/enabled during
> processes execution or mongrel hasn't implemented the feature yet.
>

I'll bet is rails specific, or you should take a look at the fcgi ruby
extension, since it is responsible, ruby-side, of bridging both
worlds.

On a personal note, I believe is not responsibility of Mongrel, as a
webserver, take care of the garbage collection and leakage issues of
the Vm on which your application runs. In any case, the GC of the VM
(MRI Ruby) should be enhanced to work better with heavy load and long
running environments.

-- 
Luis Lavena
Multimedia systems
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

From swindsor at gmail.com  Mon Mar 24 15:59:24 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Mon, 24 Mar 2008 12:59:24 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
Message-ID: <aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>

On Mon, Mar 24, 2008 at 12:18 PM, Luis Lavena <luislavena at gmail.com> wrote:

> On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor <swindsor at gmail.com> wrote:
>
>
> You're using *RMagick*, not ImageMagick directly. If you used the
> later (via system calls) there will no be memory leakage you can worry
> about.


You're correct - I'm using 'RMagick' - and it uses a large amount of
memory.  But that's not really the overall point.  My overall point is how
to properly handle a rails app that uses a great deal of memory during each
request.  I'm pretty sure this happens in other rails applications that
don't happen to use 'RMagick'.


>
> Moving from FastCGi to Mongrel will also require you monitor your
> cluster processes with external tools, since you're suing things that
> leak too much memory like RMagick and requires restart of the process.
>

Yes, although all monitoring will be able to do is kill off a mis-behaved
application.  I'd much rather run garbage collection rather than kill of my
application.

>
> To make it clear: the memory leaked by RMagick cannot be recovered
> with garbage collection mechanism. I tried that several times but in
> the long run, required to restart and hunt down all the zombies
> processes left by Apache.
>

So far, running the GC under fastcgi has given me pretty good results.  The
zombing issue with fast cgi is a known issue with mod_fastcgi and I'm pretty
sure unrelated to RMagick or garbage collection.

Can you tell me how you addressed the "schedule" of the garbage
> collection execution on your previous scenario? AFAIK most of the
> frameworks or servers don't impose to the user how often GC should be
> performed.
>

In the previous scenario I was using fast_cgi with rails.  In my previous
reply I provided a link to the rails fastcgi dispatcher.
http://dev.rubyonrails.org/browser/trunk/railties/dispatches/dispatch.fcgi

In addtion, in other languages and other language web frameworks there are
provisions to control garbage collection (for languages that have garbage
collections, of course).


> I'll bet is rails specific, or you should take a look at the fcgi ruby
> extension, since it is responsible, ruby-side, of bridging both
> worlds.
>

This is done in the Rails FastCGI dispatcher.  I believe that the equivalent
of this in Mongrel is the Mongrel Rails dispatcher.  Since the Mongrel Rails
dispatcher is distributed as a part of Mongrel, I'd say this code is owned
by Mongrel, which bridges these two worlds when using mongrel as a
webserver.


>
> On a personal note, I believe is not responsibility of Mongrel, as a
> webserver, take care of the garbage collection and leakage issues of
> the Vm on which your application runs. In any case, the GC of the VM
> (MRI Ruby) should be enhanced to work better with heavy load and long
> running environments.
>

Ruby provides an API to access and call the Garbage Collector.  This
provides ruby application developers the ability to control when the garbage
collection is run because in some cases, there may be an
application-specific reason to prevent or explicity run the GC.  Web servers
are a good example of these applications where state may help determine a
better time to run the GC.  As you're serving each request, you're generally
allocating a number of objects, then rendering output, then moving on to the
next request.

By limiting the GC to run in between requests rather than during requests
you are trading request time for latency between requests.  This is a
trade-off that I think web application developers should deciede, but by no
means should this be a default or silver bullet for all.  My position is
that this just be an option within Mongrel as a web server.

- scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080324/8035dbd3/attachment-0001.html 

From luislavena at gmail.com  Mon Mar 24 16:37:45 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Mon, 24 Mar 2008 17:37:45 -0300
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
Message-ID: <71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>

On Mon, Mar 24, 2008 at 4:59 PM, Scott Windsor <swindsor at gmail.com> wrote:
> On Mon, Mar 24, 2008 at 12:18 PM, Luis Lavena <luislavena at gmail.com> wrote:
>
> >
> >
> > On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor <swindsor at gmail.com> wrote:
> >
> >
> >
> > You're using *RMagick*, not ImageMagick directly. If you used the
> > later (via system calls) there will no be memory leakage you can worry
> > about.
>
> You're correct - I'm using 'RMagick' - and it uses a large amount of memory.
> But that's not really the overall point.  My overall point is how to
> properly handle a rails app that uses a great deal of memory during each
> request.  I'm pretty sure this happens in other rails applications that
> don't happen to use 'RMagick'.
>

Yes, I faced huge memory usage issues with other things non related to
image processing and found that a good thing was move them out of the
request-response cycle and into a out-of-bound background job.

>
> So far, running the GC under fastcgi has given me pretty good results.  The
> zombing issue with fast cgi is a known issue with mod_fastcgi and I'm pretty
> sure unrelated to RMagick or garbage collection.
>

Yes, but even you "reclaim" the memory with GC, there will be pieces
that wouldn't be GC'ed ever, since the leaked in the C side, outside
GC control (some of the RMagick and ImageMagick mysteries).

>
> > Can you tell me how you addressed the "schedule" of the garbage
> > collection execution on your previous scenario? AFAIK most of the
> > frameworks or servers don't impose to the user how often GC should be
> > performed.
> >
> >
>
> In the previous scenario I was using fast_cgi with rails.  In my previous
> reply I provided a link to the rails fastcgi dispatcher.
>
> http://dev.rubyonrails.org/browser/trunk/railties/dispatches/dispatch.fcgi
>
> In addtion, in other languages and other language web frameworks there are
> provisions to control garbage collection (for languages that have garbage
> collections, of course).
>
>
> > I'll bet is rails specific, or you should take a look at the fcgi ruby
> > extension, since it is responsible, ruby-side, of bridging both
> > worlds.
> >
>
> This is done in the Rails FastCGI dispatcher.  I believe that the equivalent
> of this in Mongrel is the Mongrel Rails dispatcher.  Since the Mongrel Rails
> dispatcher is distributed as a part of Mongrel, I'd say this code is owned
> by Mongrel, which bridges these two worlds when using mongrel as a
> webserver.
>

Then you could provide a different Mongrel Handler that could perform
that, or even a series of GemPlugins that provide a gc:start instead
of plain 'start' command mongrel_rails scripts provides.

> >
> > On a personal note, I believe is not responsibility of Mongrel, as a
> > webserver, take care of the garbage collection and leakage issues of
> > the Vm on which your application runs. In any case, the GC of the VM
> > (MRI Ruby) should be enhanced to work better with heavy load and long
> > running environments.
> >
>
> Ruby provides an API to access and call the Garbage Collector.  This
> provides ruby application developers the ability to control when the garbage
> collection is run because in some cases, there may be an
> application-specific reason to prevent or explicity run the GC.  Web servers
> are a good example of these applications where state may help determine a
> better time to run the GC.  As you're serving each request, you're generally
> allocating a number of objects, then rendering output, then moving on to the
> next request.
>
> By limiting the GC to run in between requests rather than during requests
> you are trading request time for latency between requests.  This is a
> trade-off that I think web application developers should deciede, but by no
> means should this be a default or silver bullet for all.  My position is
> that this just be an option within Mongrel as a web server.
>

--gc-interval maybe?

Now that you convinced me and proved your point, having the option to
perform it (optionally, not forced) will be something good to have.

Patches are Welcome ;-)

-- 
Luis Lavena
Multimedia systems
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

From lists at ruby-forum.com  Mon Mar 24 18:58:26 2008
From: lists at ruby-forum.com (Fernando Perez)
Date: Mon, 24 Mar 2008 23:58:26 +0100
Subject: [Mongrel] mongrel_rails not starting
In-Reply-To: <9d33458ed1ad8a6edd8d510cfaf094d3@ruby-forum.com>
References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com>
	<9d33458ed1ad8a6edd8d510cfaf094d3@ruby-forum.com>
Message-ID: <8a8e8e8a3125b8fc8dccf30199cc0b00@ruby-forum.com>

Hi same problem for me. Has anyone found a solution to it?
-- 
Posted via http://www.ruby-forum.com/.

From jftucker at gmail.com  Tue Mar 25 06:40:50 2008
From: jftucker at gmail.com (James Tucker)
Date: Tue, 25 Mar 2008 10:40:50 +0000
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211149m6890a236va7bbc53116db8ea6@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
	<71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
Message-ID: <E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>

Forgive me for not having read the whole thread, however, there is one  
thing that seems to be really important, and that is, ruby hardly ever  
runs the damned GC. It certainly doesn't do full runs nearly often  
enough (IMO).

Also, implicit OOMEs or GC runs quite often DO NOT affect the  
extensions correctly. I don't know what rmagick is doing under the  
hood in this area, but having been generating large portions of  
country maps with it (and moving away from it very rapidly), I know  
the GC doesn't do "The Right Thing".

First call of address is GC_MALLOC_LIMIT and friends. For any small  
script that doesn't breach that value, the GC simply doesn't run. More  
than this, RMagick, in it's apparent 'wisdom' never frees memory if  
the GC never runs. Seriously, check it out. Make a tiny script, and  
make a huge image with it. Hell, make 20, get an OOME, and watch for a  
run of the GC. The OOME will reach your code before the GC calls on  
RMagick to free.

Now, add a call to GC.start, and no OOME. Despite the limitations of  
it (ruby performance only IMO), most of the above experience was built  
up on windows, and last usage was about 6 months ago, FYI.

On 24 Mar 2008, at 20:37, Luis Lavena wrote:
> On Mon, Mar 24, 2008 at 4:59 PM, Scott Windsor <swindsor at gmail.com>  
> wrote:
>> On Mon, Mar 24, 2008 at 12:18 PM, Luis Lavena  
>> <luislavena at gmail.com> wrote:
>>
>>>
>>>
>>> On Mon, Mar 24, 2008 at 3:58 PM, Scott Windsor  
>>> <swindsor at gmail.com> wrote:
>>>
>>>
>>>
>>> You're using *RMagick*, not ImageMagick directly. If you used the
>>> later (via system calls) there will no be memory leakage you can  
>>> worry
>>> about.
>>
>> You're correct - I'm using 'RMagick' - and it uses a large amount  
>> of memory.
>> But that's not really the overall point.  My overall point is how to
>> properly handle a rails app that uses a great deal of memory during  
>> each
>> request.  I'm pretty sure this happens in other rails applications  
>> that
>> don't happen to use 'RMagick'.

Personally, I'll simply say call the GC more often. Seriously. I mean  
it. It's not *that* slow, not at all. In fact, I call GC.start  
explicitly inside of by ubygems.rb due to stuff I have observed before:

http://blog.ra66i.org/archives/informatics/2007/10/05/calling-on-the-gc-after-rubygems/ 
    - N.B. This isn't "FIXED" it's still a good idea (gem 1.0.1).
http://zdavatz.wordpress.com/2007/07/18/heap-fragmentation-in-a-long-running-ruby-process/

Now, by my reckoning (and a few production apps seem to be showing  
emperically (purely emperical, sorry)) we should be calling on the GC  
whilst loading up the apps. I mean come on, when are a really serious  
number of temporary objects being created. Actually, it's when  
rubygems loads, and that's the first thing that happens in, hmm,  
probably over 90% of ruby processes out there.

>
> Yes, I faced huge memory usage issues with other things non related to
> image processing and found that a good thing was move them out of the
> request-response cycle and into a out-of-bound background job.
>
>>
>> So far, running the GC under fastcgi has given me pretty good  
>> results.  The
>> zombing issue with fast cgi is a known issue with mod_fastcgi and  
>> I'm pretty
>> sure unrelated to RMagick or garbage collection.
>>
>
> Yes, but even you "reclaim" the memory with GC, there will be pieces
> that wouldn't be GC'ed ever, since the leaked in the C side, outside
> GC control (some of the RMagick and ImageMagick mysteries).

Sure, but leaks are odd things. Some processes that appear to be  
leaking are really just fragmenting (allocating more ram due to lack  
of 'usable' space on 'the heap'. Call the GC more often, take a 0.01%  
performance hit, and monitor. I bet it'll get better. In fact, you can  
drop fragmentation the first allocated segment significantly just by  
calling GC.start after a rubygems load, if you have more than a few  
gems.

>>>
>>> Can you tell me how you addressed the "schedule" of the garbage
>>> collection execution on your previous scenario? AFAIK most of the
>>> frameworks or servers don't impose to the user how often GC should  
>>> be
>>> performed.

In fact there are many rubyists who hate the idea of splatting  
GC.start into processes. Given what I've seen, I'm willing to reject  
that notion completely. Test yourself, YMMV.

FYI, even on windows under the OCI, where performance for the  
interpreter sucks, really really hard, I couldn't reliably measure the  
runtime of a call to GC.start after loading rubygems. I don't know  
what kind of 'performance' people are after, but I can't see the point  
in not running the GC more often, especially for 'more common' daemon  
load. Furthermore, hitting the kernel for more allocations more often,  
is actually pretty slow too, so this may actually even result in  
faster processes under *certain* conditions.

Running a lib like RMagick, I would say you *should* be doing this,  
straight up, no arguments.

>>
>> In the previous scenario I was using fast_cgi with rails.  In my  
>> previous
>> reply I provided a link to the rails fastcgi dispatcher.
>>
>> http://dev.rubyonrails.org/browser/trunk/railties/dispatches/dispatch.fcgi
>>
>> In addtion, in other languages and other language web frameworks  
>> there are
>> provisions to control garbage collection (for languages that have  
>> garbage
>> collections, of course).
>>
>>
>>> I'll bet is rails specific, or you should take a look at the fcgi  
>>> ruby
>>> extension, since it is responsible, ruby-side, of bridging both
>>> worlds.
>>>
>>
>> This is done in the Rails FastCGI dispatcher.  I believe that the  
>> equivalent
>> of this in Mongrel is the Mongrel Rails dispatcher.  Since the  
>> Mongrel Rails
>> dispatcher is distributed as a part of Mongrel, I'd say this code  
>> is owned
>> by Mongrel, which bridges these two worlds when using mongrel as a
>> webserver.

It doesn't *really* matter where you run the GC. It matters that it  
runs, how often, and what it's doing. If you're actually calling on  
the GC and freeing nothing, that's stupid, but if you've run RMagick  
up, just call GC.start anyway, and I'm pretty sure it'll help. There's  
certainly no harm in investigating this, unless you're doing something  
silly with weakrefs.


> Then you could provide a different Mongrel Handler that could perform
> that, or even a series of GemPlugins that provide a gc:start instead
> of plain 'start' command mongrel_rails scripts provides.


$occasional_gc_run_counter = 0
before_filter :occasional_gc_run

def occasional_gc_run
   $occasional_gc_run_counter += 1
   if $occasional_gc_run_counter > 1_000
     $occasional_gc_run_counter = 0
     GC.start
   end
end

Or whatever. It doesn't really matter that much where you do this, or  
when, it just needs to happen every now and then. More importantly,  
add a GC.start to the end of environment.rb, and you will have  
literally half the number of objects in ObjectSpace.

>>> On a personal note, I believe is not responsibility of Mongrel, as a
>>> webserver, take care of the garbage collection and leakage issues of
>>> the Vm on which your application runs. In any case, the GC of the VM
>>> (MRI Ruby) should be enhanced to work better with heavy load and  
>>> long
>>> running environments.

Right, and it's not just the interpreter, although indirection around  
this stuff can help. (such as compacting).

>>
>> Ruby provides an API to access and call the Garbage Collector.  This
>> provides ruby application developers the ability to control when  
>> the garbage
>> collection is run because in some cases, there may be an
>> application-specific reason to prevent or explicity run the GC.   
>> Web servers
>> are a good example of these applications where state may help  
>> determine a
>> better time to run the GC.  As you're serving each request, you're  
>> generally
>> allocating a number of objects, then rendering output, then moving  
>> on to the
>> next request.
>>
>> By limiting the GC to run in between requests rather than during  
>> requests
>> you are trading request time for latency between requests.  This is a
>> trade-off that I think web application developers should deciede,  
>> but by no
>> means should this be a default or silver bullet for all.  My  
>> position is
>> that this just be an option within Mongrel as a web server.
>>

Right, I think this is important too. You're absolutely right that  
there's no specific place to provide a generic solution. In rails the  
answer may be simple, but that's because rails outer architecture is  
simplistic. No threads, no out-of-request processing, and so on.
>
> --gc-interval maybe?
>
> Now that you convinced me and proved your point, having the option to
> perform it (optionally, not forced) will be something good to have.

Surely you can just:

require 'thread'
Thread.new { loop { sleep GC_FORCE_INTERVAL; GC.start } }

In environment.rb in that case.

Of course, this is going to kill performance under evented_mongrel,  
thin and so on. I'd stay away from threaded solutions. _why blogged  
years ago about the GC, trying to remind people that we actually have  
control. I know ruby is supposed to abstract memory problems etc away  
from us, and for the most part it does, but hey, no one's perfect,  
right? :-)

http://whytheluckystiff.net/articles/theFullyUpturnedBin.html

> Patches are Welcome ;-)

Have fun! :o)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080325/75800781/attachment-0001.html 

From wyhaines at gmail.com  Tue Mar 25 11:26:01 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Tue, 25 Mar 2008 09:26:01 -0600
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
	<71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
	<E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>
Message-ID: <f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>

On Tue, Mar 25, 2008 at 4:40 AM, James Tucker <jftucker at gmail.com> wrote:
> Forgive me for not having read the whole thread, however, there is one thing
> that seems to be really important, and that is, ruby hardly ever runs the
> damned GC. It certainly doesn't do full runs nearly often enough (IMO).

There's only one kind of garbage collection sweep.  And yeah,
depending on what's happening, GC may not run very often.  That's not
generally a problem.

> Also, implicit OOMEs or GC runs quite often DO NOT affect the extensions
> correctly. I don't know what rmagick is doing under the hood in this area,
> but having been generating large portions of country maps with it (and
> moving away from it very rapidly), I know the GC doesn't do "The Right
> Thing".

There should be no difference between a GC run that is initiated by
the interpreter and one that is initiated by one's code.  It ends up
calling the same thing in gc.c.  Extensions can easily mismanage
memory, though, and I have a hunch about what's happening with
rmagick.

> First call of address is GC_MALLOC_LIMIT and friends. For any small script
> that doesn't breach that value, the GC simply doesn't run. More than this,
> RMagick, in it's apparent 'wisdom' never frees memory if the GC never runs.
> Seriously, check it out. Make a tiny script, and make a huge image with it.
> Hell, make 20, get an OOME, and watch for a run of the GC. The OOME will
> reach your code before the GC calls on RMagick to free.
>
> Now, add a call to GC.start, and no OOME. Despite the limitations of it
> (ruby performance only IMO), most of the above experience was built up on
> windows, and last usage was about 6 months ago, FYI.

My hunch is that rmagick is allocating large amounts of RAM ouside of
Ruby.  It registers its objects with the interpreter, but the RAM
usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because
Ruby didn't allocate it, so doesn't know about it.

So, it uses huge amounts of RAM, but doesn't use huge numbers of
objects.  Thus you never trigger a GC cycle by exceeding the
GC_MALLOC_LIMIT nor by running our of object slots in the heap.  I'd
have to go look at the code to be sure, but the theory fits the
behavior that is described very well.

I don't think this is a case for building GC.foo memory management
into Mongrel, though.  As I think you are suggesting, just call
GC.start yourself in your code when necessary.  In a typical Rails app
doing big things with rmagick, the extra time to do GC.start at the
end of the image manipulation, in the request handling, isn't going to
be noticable.

> But that's not really the overall point.  My overall point is how to
> properly handle a rails app that uses a great deal of memory during each
> request.  I'm pretty sure this happens in other rails applications that
> don't happen to use 'RMagick'.
>
> Personally, I'll simply say call the GC more often. Seriously. I mean it.
> It's not *that* slow, not at all. In fact, I call GC.start explicitly inside
> of by ubygems.rb due to stuff I have observed before:

I completely concur with this.  If there are issues with huge memory
use (most likely caused by extensions making RAM allocations outside
of Ruby's accounting, so implicit GC isn't triggered), just call
GC.start in one's own code.

> Now, by my reckoning (and a few production apps seem to be showing
> emperically (purely emperical, sorry)) we should be calling on the GC whilst
> loading up the apps. I mean come on, when are a really serious number of
> temporary objects being created. Actually, it's when rubygems loads, and
> that's the first thing that happens in, hmm, probably over 90% of ruby
> processes out there.

Just as a tangent, I do this in Swiftiply.  I make an explicit call to
GC.start after everything is loaded and all configs are parsed, just
to make sure execution is going into the main event loop with as much
junk cleaned out as possible.

> Or whatever. It doesn't really matter that much where you do this, or when,
> it just needs to happen every now and then. More importantly, add a GC.start
> to the end of environment.rb, and you will have literally half the number of
> objects in ObjectSpace.

This makes sense to me.

I could also see providing a 2nd Rails handler that had some GC
management stuff in it, along with some documentation on what it
actually does or does not do, so people can make an explicit choice to
use it, if they need it.  I'm still completely against throwing code
into Mongrel itself for this sort of thing.  I just prefer not to
throw more things into Mongrel than we really _need_ to, when there is
no strong argument for them being inside of Mongrel itself.  GC.start
stuff is simple enough to put into one's own code at appropriate
locations, or to put into a customized Mongrel handler if one needs
it.

Maybe this simply needs to be documented in the body of Mongrel documentation?


Kirk Haines

From evan at cloudbur.st  Tue Mar 25 13:02:54 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Tue, 25 Mar 2008 13:02:54 -0400
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
	<71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
	<E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>
	<f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>
Message-ID: <b6f68fc60803251002i6373310q900134ea0c0b9adc@mail.gmail.com>

>  My hunch is that rmagick is allocating large amounts of RAM ouside of
>  Ruby.  It registers its objects with the interpreter, but the RAM
>  usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because
>  Ruby didn't allocate it, so doesn't know about it.

It's allocating opaque objects on the Ruby heap but not using Ruby's
built-in malloc? That seems pretty evil.

Evan

>
>  So, it uses huge amounts of RAM, but doesn't use huge numbers of
>  objects.  Thus you never trigger a GC cycle by exceeding the
>  GC_MALLOC_LIMIT nor by running our of object slots in the heap.  I'd
>  have to go look at the code to be sure, but the theory fits the
>  behavior that is described very well.
>
>  I don't think this is a case for building GC.foo memory management
>  into Mongrel, though.  As I think you are suggesting, just call
>  GC.start yourself in your code when necessary.  In a typical Rails app
>  doing big things with rmagick, the extra time to do GC.start at the
>  end of the image manipulation, in the request handling, isn't going to
>  be noticable.
>
>
>  > But that's not really the overall point.  My overall point is how to
>  > properly handle a rails app that uses a great deal of memory during each
>  > request.  I'm pretty sure this happens in other rails applications that
>  > don't happen to use 'RMagick'.
>  >
>  > Personally, I'll simply say call the GC more often. Seriously. I mean it.
>  > It's not *that* slow, not at all. In fact, I call GC.start explicitly inside
>  > of by ubygems.rb due to stuff I have observed before:
>
>  I completely concur with this.  If there are issues with huge memory
>  use (most likely caused by extensions making RAM allocations outside
>  of Ruby's accounting, so implicit GC isn't triggered), just call
>  GC.start in one's own code.
>
>
>  > Now, by my reckoning (and a few production apps seem to be showing
>  > emperically (purely emperical, sorry)) we should be calling on the GC whilst
>  > loading up the apps. I mean come on, when are a really serious number of
>  > temporary objects being created. Actually, it's when rubygems loads, and
>  > that's the first thing that happens in, hmm, probably over 90% of ruby
>  > processes out there.
>
>  Just as a tangent, I do this in Swiftiply.  I make an explicit call to
>  GC.start after everything is loaded and all configs are parsed, just
>  to make sure execution is going into the main event loop with as much
>  junk cleaned out as possible.
>
>
>  > Or whatever. It doesn't really matter that much where you do this, or when,
>  > it just needs to happen every now and then. More importantly, add a GC.start
>  > to the end of environment.rb, and you will have literally half the number of
>  > objects in ObjectSpace.
>
>  This makes sense to me.
>
>  I could also see providing a 2nd Rails handler that had some GC
>  management stuff in it, along with some documentation on what it
>  actually does or does not do, so people can make an explicit choice to
>  use it, if they need it.  I'm still completely against throwing code
>  into Mongrel itself for this sort of thing.  I just prefer not to
>  throw more things into Mongrel than we really _need_ to, when there is
>  no strong argument for them being inside of Mongrel itself.  GC.start
>  stuff is simple enough to put into one's own code at appropriate
>  locations, or to put into a customized Mongrel handler if one needs
>  it.
>
>  Maybe this simply needs to be documented in the body of Mongrel documentation?
>
>
>  Kirk Haines
>
>
> _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From public at misuse.org  Tue Mar 25 13:05:21 2008
From: public at misuse.org (Steve Midgley)
Date: Tue, 25 Mar 2008 10:05:21 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <mailman.94.1206441664.913.mongrel-users@rubyforge.org>
References: <mailman.94.1206441664.913.mongrel-users@rubyforge.org>
Message-ID: <20080325170601.5AF6B18585FA@rubyforge.org>

At 03:41 AM 3/25/2008, mongrel-users-request at rubyforge.org wrote:
>Date: Tue, 25 Mar 2008 10:40:50 +0000
>From: James Tucker <jftucker at gmail.com>
>Subject: Re: [Mongrel] mongrel garbage collection
>To: mongrel-users at rubyforge.org
>Message-ID: <E5223BA4-90BD-4976-8651-33226BC23299 at gmail.com>
>Content-Type: text/plain; charset="us-ascii"
>[snip]
>Also, implicit OOMEs or GC runs quite often DO NOT affect the
>extensions correctly. I don't know what rmagick is doing under the
>hood in this area, but having been generating large portions of
>country maps with it (and moving away from it very rapidly), I know
>the GC doesn't do "The Right Thing".
>[snip]

Hi James,

My understanding with RMagick is that it is hooking the Imagemagick 
libs directly in process. As a result, memory is not always freed when 
you'd expect it to be. I haven't read up on the details, having chosen 
to just use out of process image management, but you might find this 
link interesting - in it, there is a claim that the latest releases of 
RMagick do *not* in fact leak any memory and that running a full GC 
manually will reclaim all memory it uses after the references are out 
of scope.

http://rubyforge.org/forum/forum.php?thread_id=1374&forum_id=1618

Steve




From public at misuse.org  Tue Mar 25 13:05:21 2008
From: public at misuse.org (Steve Midgley)
Date: Tue, 25 Mar 2008 10:05:21 -0700
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <mailman.94.1206441664.913.mongrel-users@rubyforge.org>
References: <mailman.94.1206441664.913.mongrel-users@rubyforge.org>
Message-ID: <20080325170607.7741F185868E@rubyforge.org>

At 03:41 AM 3/25/2008, mongrel-users-request at rubyforge.org wrote:
>Date: Tue, 25 Mar 2008 10:40:50 +0000
>From: James Tucker <jftucker at gmail.com>
>Subject: Re: [Mongrel] mongrel garbage collection
>To: mongrel-users at rubyforge.org
>Message-ID: <E5223BA4-90BD-4976-8651-33226BC23299 at gmail.com>
>Content-Type: text/plain; charset="us-ascii"
>[snip]
>Also, implicit OOMEs or GC runs quite often DO NOT affect the
>extensions correctly. I don't know what rmagick is doing under the
>hood in this area, but having been generating large portions of
>country maps with it (and moving away from it very rapidly), I know
>the GC doesn't do "The Right Thing".
>[snip]

Hi James,

My understanding with RMagick is that it is hooking the Imagemagick 
libs directly in process. As a result, memory is not always freed when 
you'd expect it to be. I haven't read up on the details, having chosen 
to just use out of process image management, but you might find this 
link interesting - in it, there is a claim that the latest releases of 
RMagick do *not* in fact leak any memory and that running a full GC 
manually will reclaim all memory it uses after the references are out 
of scope.

http://rubyforge.org/forum/forum.php?thread_id=1374&forum_id=1618

Steve




From wyhaines at gmail.com  Tue Mar 25 13:20:29 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Tue, 25 Mar 2008 11:20:29 -0600
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <b6f68fc60803251002i6373310q900134ea0c0b9adc@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
	<71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
	<E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>
	<f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>
	<b6f68fc60803251002i6373310q900134ea0c0b9adc@mail.gmail.com>
Message-ID: <f4cd26df0803251020v1737e8fbxd6ffa89e947ec306@mail.gmail.com>

On Tue, Mar 25, 2008 at 11:02 AM, Evan Weaver <evan at cloudbur.st> wrote:
> >  My hunch is that rmagick is allocating large amounts of RAM ouside of
> >  Ruby.  It registers its objects with the interpreter, but the RAM
> >  usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because
> >  Ruby didn't allocate it, so doesn't know about it.
>
> It's allocating opaque objects on the Ruby heap but not using Ruby's
> built-in malloc? That seems pretty evil.

Not really.  It's pretty common in extensions.  You alloc your
structures in whatever way is appropriate for the library you are
using, then use Data_Wrap_Struct with a mark and a free function to
hook your stuff into the Ruby garbage collector.

Your objects are thus known to Ruby as Ruby objects, but you have
potentially large chunks of memory that Ruby itself knows nothing
about.


Kirk Haines

From jftucker at gmail.com  Tue Mar 25 13:26:23 2008
From: jftucker at gmail.com (James Tucker)
Date: Tue, 25 Mar 2008 17:26:23 +0000
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<aa2e18ec0803211223p51c46af5h81986fe3d2a00cfa@mail.gmail.com>
	<f4cd26df0803211319o491ad3c1u92ef5509c433c76b@mail.gmail.com>
	<aa2e18ec0803240821o1f4f5921tf4156780ed0928d@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
	<71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
	<E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>
	<f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>
Message-ID: <8F863F0A-27C1-454D-97A3-2C9221F9E980@gmail.com>


On 25 Mar 2008, at 15:26, Kirk Haines wrote:
> On Tue, Mar 25, 2008 at 4:40 AM, James Tucker <jftucker at gmail.com>  
> wrote:
>> Forgive me for not having read the whole thread, however, there is  
>> one thing
>> that seems to be really important, and that is, ruby hardly ever  
>> runs the
>> damned GC. It certainly doesn't do full runs nearly often enough  
>> (IMO).
>
> There's only one kind of garbage collection sweep.  And yeah,
> depending on what's happening, GC may not run very often.  That's not
> generally a problem.

Sure, inside ruby there's only one kind of run, but....

>> Also, implicit OOMEs or GC runs quite often DO NOT affect the  
>> extensions
>> correctly. I don't know what rmagick is doing under the hood in  
>> this area,
>> but having been generating large portions of country maps with it  
>> (and
>> moving away from it very rapidly), I know the GC doesn't do "The  
>> Right
>> Thing".
>
> There should be no difference between a GC run that is initiated by
> the interpreter and one that is initiated by one's code.  It ends up
> calling the same thing in gc.c.  Extensions can easily mismanage
> memory, though, and I have a hunch about what's happening with
> rmagick.

I just realised the obvious truth, that ruby isn't actually running  
the GC under those OOME conditions.

>> First call of address is GC_MALLOC_LIMIT and friends. For any small  
>> script
>> that doesn't breach that value, the GC simply doesn't run. More  
>> than this,
>> RMagick, in it's apparent 'wisdom' never frees memory if the GC  
>> never runs.
>> Seriously, check it out. Make a tiny script, and make a huge image  
>> with it.
>> Hell, make 20, get an OOME, and watch for a run of the GC. The OOME  
>> will
>> reach your code before the GC calls on RMagick to free.
>>
>> Now, add a call to GC.start, and no OOME. Despite the limitations  
>> of it
>> (ruby performance only IMO), most of the above experience was built  
>> up on
>> windows, and last usage was about 6 months ago, FYI.
>
> My hunch is that rmagick is allocating large amounts of RAM ouside of
> Ruby.  It registers its objects with the interpreter, but the RAM
> usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because
> Ruby didn't allocate it, so doesn't know about it.

Yup, it's ImageMagick, un-patched and they don't provide afaik a  
callback to replace malloc, or maybe that's an rmagick issue.

> So, it uses huge amounts of RAM, but doesn't use huge numbers of
> objects.  Thus you never trigger a GC cycle by exceeding the
> GC_MALLOC_LIMIT nor by running our of object slots in the heap.  I'd
> have to go look at the code to be sure, but the theory fits the
> behavior that is described very well.

Right, in fact, I think the OOME actually comes from outside of ruby  
(unverified), and ruby can't or won't run the GC before going down. As  
the free() calls inside RMagick / ImageMagick aren't happening without  
calling GC.start. The GC.start call, somewhere/how is being used to  
trigger frees in the framework. Personally, this is bad design, and  
the really common complaints may also suggest so, however, I don't  
know what their domain specific issues and limitations are. Maybe it's  
an ImageMagick thing.

Creating an OOME inside ruby, the interpreter calls on GC.start prior  
to going down. I started talking to zenspider about this stuff, and  
eventually he just pointed me at gc.c, fair enough. I still hold the  
opinion that an OOME hitting the interpreter (from whatever source)  
should attempt to invoke the GC. Of course, a hell of a lot of  
software doesn't check the result of a call to malloc(), tut tut.

Tool: http://ideas.water-powered.com/projects/libgreat

> I don't think this is a case for building GC.foo memory management
> into Mongrel, though.  As I think you are suggesting, just call
> GC.start yourself in your code when necessary.  In a typical Rails app
> doing big things with rmagick, the extra time to do GC.start at the
> end of the image manipulation, in the request handling, isn't going to
> be noticable.

Absolutely right, and yes, this is my opinion.

>> But that's not really the overall point.  My overall point is how to
>> properly handle a rails app that uses a great deal of memory during  
>> each
>> request.  I'm pretty sure this happens in other rails applications  
>> that
>> don't happen to use 'RMagick'.
>>
>> Personally, I'll simply say call the GC more often. Seriously. I  
>> mean it.
>> It's not *that* slow, not at all. In fact, I call GC.start  
>> explicitly inside
>> of by ubygems.rb due to stuff I have observed before:
>
> I completely concur with this.  If there are issues with huge memory
> use (most likely caused by extensions making RAM allocations outside
> of Ruby's accounting, so implicit GC isn't triggered), just call
> GC.start in one's own code.
>
>> Now, by my reckoning (and a few production apps seem to be showing
>> emperically (purely emperical, sorry)) we should be calling on the  
>> GC whilst
>> loading up the apps. I mean come on, when are a really serious  
>> number of
>> temporary objects being created. Actually, it's when rubygems  
>> loads, and
>> that's the first thing that happens in, hmm, probably over 90% of  
>> ruby
>> processes out there.
>
> Just as a tangent, I do this in Swiftiply.  I make an explicit call to
> GC.start after everything is loaded and all configs are parsed, just
> to make sure execution is going into the main event loop with as much
> junk cleaned out as possible.

I've done similar in anything that is running as a fire and forget  
style daemon. You know, the kinds of things that get setup once, and  
run for 1 to 20 years. There are several that I have never restarted.  
No rails, though. These kinds of things I also simply don't want to  
waste the ram to silly fragmentation, the next allocation takes you up  
to a registerable percentage on medium aged machines. IIRC there's one  
in my copy of analogger too, or maybe you had that in there already :-)

>> Or whatever. It doesn't really matter that much where you do this,  
>> or when,
>> it just needs to happen every now and then. More importantly, add a  
>> GC.start
>> to the end of environment.rb, and you will have literally half the  
>> number of
>> objects in ObjectSpace.
>
> This makes sense to me.
>
> I could also see providing a 2nd Rails handler that had some GC
> management stuff in it, along with some documentation on what it
> actually does or does not do, so people can make an explicit choice to
> use it, if they need it.  I'm still completely against throwing code
> into Mongrel itself for this sort of thing.  I just prefer not to
> throw more things into Mongrel than we really _need_ to, when there is
> no strong argument for them being inside of Mongrel itself.  GC.start
> stuff is simple enough to put into one's own code at appropriate
> locations, or to put into a customized Mongrel handler if one needs
> it.

If it wasn't app specific I'd say put it in mongrel. It is though, and  
peoples tendency to pre-optimize probably makes this pointless.

I mean the cost of doing it in a thread under eventmachine is way  
higher than the ram usage costs for pure ruby apps, at least for my  
pure ruby apps. 20-40mb vs. lots of req. / sec.

But then, one could check for better alternatives, like add_timer(),  
etc, but that route tends towards bloat, so your original assertion of  
put it in the app configuration, is what I would choose.

> Maybe this simply needs to be documented in the body of Mongrel  
> documentation?

Maybe not even there. I think research needs to be done into the  
longer running effects of the GC under real environments. I know some  
people have done some (including myself), but the results are never  
released in public. The GC also seems to be one of those topics, as  
it's so close to performance, where people are happy to see how high  
up the wall they can go, prior to doing research.

With regard to mongrel and this stuff, it's really not a mongrel  
issue. Mongrel is a great citizen wrt the GC (at least by comparison  
to a lot of other code).

Particularly bad citizens in this area include:
  - Every single pure ruby pdf lib I've seen
  - rubygems (by way of the spec loading semantics, not rubygems  
itself, kinda (lets just say, I'd do it different, but by design, not  
implementation))
  - rails
  - rmagick

>
>
>
> Kirk Haines
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From jftucker at gmail.com  Tue Mar 25 13:45:26 2008
From: jftucker at gmail.com (James Tucker)
Date: Tue, 25 Mar 2008 17:45:26 +0000
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <20080325170607.7741F185868E@rubyforge.org>
References: <mailman.94.1206441664.913.mongrel-users@rubyforge.org>
	<20080325170607.7741F185868E@rubyforge.org>
Message-ID: <57B2A448-EFFE-454D-B452-CFF5C7D6AB7C@gmail.com>


On 25 Mar 2008, at 17:05, Steve Midgley wrote:
> At 03:41 AM 3/25/2008, mongrel-users-request at rubyforge.org wrote:
>> Date: Tue, 25 Mar 2008 10:40:50 +0000
>> From: James Tucker <jftucker at gmail.com>
>> Subject: Re: [Mongrel] mongrel garbage collection
>> To: mongrel-users at rubyforge.org
>> Message-ID: <E5223BA4-90BD-4976-8651-33226BC23299 at gmail.com>
>> Content-Type: text/plain; charset="us-ascii"
>> [snip]
>> Also, implicit OOMEs or GC runs quite often DO NOT affect the
>> extensions correctly. I don't know what rmagick is doing under the
>> hood in this area, but having been generating large portions of
>> country maps with it (and moving away from it very rapidly), I know
>> the GC doesn't do "The Right Thing".
>> [snip]
>
> Hi James,
>
> My understanding with RMagick is that it is hooking the Imagemagick
> libs directly in process. As a result, memory is not always freed when
> you'd expect it to be. I haven't read up on the details, having chosen
> to just use out of process image management, but you might find this
> link interesting - in it, there is a claim that the latest releases of
> RMagick do *not* in fact leak any memory and that running a full GC
> manually will reclaim all memory it uses after the references are out
> of scope.

Thank you for kindly ensuring that I got this. We actually moved  
completely away from anything ImageMagick based. There really wasn't  
any sensible way to 'fix' it.

Whilst destroy! looks ok, even when doing what we were (high res  
tiling, covering around 250 square miles), we found performance was  
fine and we could avoid all allocation issues by using the crazy  
thread solution (Thread.new { loop { sleep some_time; GC.start } }).

This is all good in most scenarios but then there are times when  
running a framework like eventmachine, where threads (yes, even green  
ones) can be total performance killers too. Mind you, under rails,  
there's always a linear reaction run, so I'm not going to speculate  
more on that detail. It's also OT for here, mostly...

> http://rubyforge.org/forum/forum.php?thread_id=1374&forum_id=1618
>
> Steve

Thanks again,

James.

P.S. Personally, if I was coming up against this problem today, I'd  
drop out to a separate process, driven by something like background  
job if under rails. If under pure ruby, I'd use drb or eventmachine +  
a marshalling protocol, depending on specific requirements. The  
biggest issue for our old project was hitting swap / page file. Image  
rendering, when you're already working on the per-pixel layer, is  
really easy to divide up, though, so optimizing for speed is pretty  
easy really.

When it comes to background concurrent scheduling, staying away from  
ACID can really help, too, but that really is another topic for  
another time. Lets just say, allow slack, and life will be easier if  
you ever hit a silly scale. I've seen people trying to recover broken  
ACID implementations by trawling logs, and my god, tearful.

From evan at cloudbur.st  Wed Mar 26 08:48:17 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Wed, 26 Mar 2008 08:48:17 -0400
Subject: [Mongrel] mongrel garbage collection
In-Reply-To: <f4cd26df0803251020v1737e8fbxd6ffa89e947ec306@mail.gmail.com>
References: <aa2e18ec0803211112g50959802oe74ba431c3280647@mail.gmail.com>
	<20080325145323.e05fe8b5.zedshaw@zedshaw.com>
	<aa2e18ec0803241158y2f597e9dgf1f7dbc61437f2f7@mail.gmail.com>
	<71166b3b0803241218l7a3451b5kff7f50bc5020b588@mail.gmail.com>
	<aa2e18ec0803241259j550af314nfcdf5ee99d87da61@mail.gmail.com>
	<71166b3b0803241337w72a3b97dp8146fedf2781d3de@mail.gmail.com>
	<E5223BA4-90BD-4976-8651-33226BC23299@gmail.com>
	<f4cd26df0803250826t59a8c8f9h1ecb98bae10bbba4@mail.gmail.com>
	<b6f68fc60803251002i6373310q900134ea0c0b9adc@mail.gmail.com>
	<f4cd26df0803251020v1737e8fbxd6ffa89e947ec306@mail.gmail.com>
Message-ID: <b6f68fc60803260548l38f6f9edp94d4220634f28530@mail.gmail.com>

You're right, ok. So the memory causing the OOM error isn't actually
on the Ruby heap, but it can't get freed until the opaque object gets
GC'd.

Evan

On Tue, Mar 25, 2008 at 1:20 PM, Kirk Haines <wyhaines at gmail.com> wrote:
> On Tue, Mar 25, 2008 at 11:02 AM, Evan Weaver <evan at cloudbur.st> wrote:
>  > >  My hunch is that rmagick is allocating large amounts of RAM ouside of
>  > >  Ruby.  It registers its objects with the interpreter, but the RAM
>  > >  usage in rmagick itself doesn't count against GC_MALLOC_LIMIT because
>  > >  Ruby didn't allocate it, so doesn't know about it.
>  >
>  > It's allocating opaque objects on the Ruby heap but not using Ruby's
>  > built-in malloc? That seems pretty evil.
>
>  Not really.  It's pretty common in extensions.  You alloc your
>  structures in whatever way is appropriate for the library you are
>  using, then use Data_Wrap_Struct with a mark and a free function to
>  hook your stuff into the Ruby garbage collector.
>
>  Your objects are thus known to Ruby as Ruby objects, but you have
>  potentially large chunks of memory that Ruby itself knows nothing
>  about.
>
>
>
>
>  Kirk Haines
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From mike.engelhart at gmail.com  Wed Mar 26 17:53:31 2008
From: mike.engelhart at gmail.com (Michael Engelhart)
Date: Wed, 26 Mar 2008 17:53:31 -0400
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
	<46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>
Message-ID: <42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>

Does anyone have any tips for checking to see if the mongrel instance
is responding to HTTP?  I just tried doing this:

if failed host 127.0.0.1 port 8001 protocol http
    with timeout 10 seconds
  then restart

but when I have that code in my monit script it gives me this error:
mongrel_rails_8001' failed protocol test [HTTP] at INET[127.0.0.1:8018] via TCP


Thanks for any tips.

Mike
On Sun, Mar 9, 2008 at 1:31 AM, Nathan Esquenazi <lists at ruby-forum.com> wrote:
> Here is a good recipe for a mongrel process, as provided on some site I
>  found:
>
>  (I believe the forum post was by Ezra)
>
>  check process mongrel_cluster_8000
>    with pidfile /var/apps/twocb/shared/pids/mongrel.8000.pid
>    start program = "/usr/bin/mongrel_rails cluster::start -C
>  /var/apps/twocb/current/config/mongrel_cluster.yml --clean --only 8000"
>    stop program = "/usr/bin/mongrel_rails cluster::stop -C
>  /var/apps/twocb/current/config/mongrel_cluster.yml --clean --only 8000"
>    if totalmem is greater than 110.0 MB for 4 cycles then  restart
>  # eating up memory?
>    if cpu is greater than 50% for 2 cycles then  alert
>   # send an email to admin
>    if cpu is greater than 80% for 3 cycles then  restart
>  # hung process?
>    if loadavg(5min) greater than 10 for 8 cycles then  restart
>  # bad, bad, bad
>    if 20 restarts within 20 cycles then timeout
>  # something is wrong, call the sys-admin
>    group mongrel_cluster
>  --
>  Posted via http://www.ruby-forum.com/.
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From john at smokinggun.com  Wed Mar 26 18:27:43 2008
From: john at smokinggun.com (John Weir)
Date: Wed, 26 Mar 2008 18:27:43 -0400
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
	<46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>
	<42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>
Message-ID: <65D8ADBE-1EE3-4042-8583-5F93123B844C@smokinggun.com>

well, something doesn't seem right:

-> mongrel_rails_8001' failed protocol test [HTTP] at  
INET[127.0.0.1:8018]
                                                                                                          ^ 
^^^^^^^^^^^^

127.0.0.1:8018 <- port 8018? or do you want port 8001?

also i leave the host parameter out of the monit config.

   if failed port 8015 protocol http
      with timeout 20 seconds
      then restart


On Mar 26, 2008, at 5:53 PM, Michael Engelhart wrote:

> Does anyone have any tips for checking to see if the mongrel instance
> is responding to HTTP?  I just tried doing this:
>
> if failed host 127.0.0.1 port 8001 protocol http
>    with timeout 10 seconds
>  then restart
>
> but when I have that code in my monit script it gives me this error:
> mongrel_rails_8001' failed protocol test [HTTP] at  
> INET[127.0.0.1:8018] via TCP
>
>
> Thanks for any tips.
>
> Mike

From mike.engelhart at gmail.com  Wed Mar 26 19:09:27 2008
From: mike.engelhart at gmail.com (Michael Engelhart)
Date: Wed, 26 Mar 2008 19:09:27 -0400
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <65D8ADBE-1EE3-4042-8583-5F93123B844C@smokinggun.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
	<46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>
	<42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>
	<65D8ADBE-1EE3-4042-8583-5F93123B844C@smokinggun.com>
Message-ID: <42642c790803261609sa53e12ewe14a28d225c0afb3@mail.gmail.com>

Sorry - I was copying  and pasting from an alert email.    The ports
"do" match up in my monit script

Here's a real example:

#8002
check process mongrel_rails_8002 with pidfile /var/run/mongrel/mongrel.8002.pid
  group mongrel
  start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
start -d -e production -a 127.0.0.1 -c /opt/rails/radius --user
mongrel --group mongrel -p 8002 -P /var/run/mongrel/mongrel.8002.pid
-l /var/log/mongrel/mongrel.8002.log"
  stop program  = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
stop -P /var/run/mongrel/mongrel.8002.pid"
  if totalmem > 100.0 MB for 5 cycles then restart
  if failed host 127.0.0.1 port 8002 protocol http
    with timeout 10 seconds
  then restart

On Wed, Mar 26, 2008 at 6:27 PM, John Weir <john at smokinggun.com> wrote:
> well, something doesn't seem right:
>
>  -> mongrel_rails_8001' failed protocol test [HTTP] at
>
> INET[127.0.0.1:8018]
>                                                                                                           ^
>  ^^^^^^^^^^^^
>
>  127.0.0.1:8018 <- port 8018? or do you want port 8001?
>
>  also i leave the host parameter out of the monit config.
>
>    if failed port 8015 protocol http
>       with timeout 20 seconds
>       then restart
>
>
>
>  On Mar 26, 2008, at 5:53 PM, Michael Engelhart wrote:
>
>  > Does anyone have any tips for checking to see if the mongrel instance
>  > is responding to HTTP?  I just tried doing this:
>  >
>  > if failed host 127.0.0.1 port 8001 protocol http
>  >    with timeout 10 seconds
>  >  then restart
>  >
>  > but when I have that code in my monit script it gives me this error:
>  > mongrel_rails_8001' failed protocol test [HTTP] at
>  > INET[127.0.0.1:8018] via TCP
>  >
>  >
>  > Thanks for any tips.
>  >
>  > Mike
>
>
> _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From jgeiger at gmail.com  Wed Mar 26 22:46:21 2008
From: jgeiger at gmail.com (Joey Geiger)
Date: Wed, 26 Mar 2008 21:46:21 -0500
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <42642c790803261609sa53e12ewe14a28d225c0afb3@mail.gmail.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
	<46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>
	<42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>
	<65D8ADBE-1EE3-4042-8583-5F93123B844C@smokinggun.com>
	<42642c790803261609sa53e12ewe14a28d225c0afb3@mail.gmail.com>
Message-ID: <466af3440803261946k4e6aed45p8bafe6028edece47@mail.gmail.com>

Well, there are a few things to think about.

1. I've got a script that checks nginx with a "success" message.
2. Are you checking if mongrel is up, or rails?
If it's mongrel, you might want to write a custom route so it's not
loading the framework stack every time.
If it's rails, you should create something like a MonitController that
will respond with "success" when asked for the index action.

Doing those things has worked quite well for me so far. Just be sure
to disable the logging in the monit controller so it doesn't fill your
logs.
def logger
end


On Wed, Mar 26, 2008 at 6:09 PM, Michael Engelhart
<mike.engelhart at gmail.com> wrote:
> Sorry - I was copying  and pasting from an alert email.    The ports
>  "do" match up in my monit script
>
>  Here's a real example:
>
>  #8002
>  check process mongrel_rails_8002 with pidfile /var/run/mongrel/mongrel.8002.pid
>   group mongrel
>   start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
>  start -d -e production -a 127.0.0.1 -c /opt/rails/radius --user
>  mongrel --group mongrel -p 8002 -P /var/run/mongrel/mongrel.8002.pid
>  -l /var/log/mongrel/mongrel.8002.log"
>   stop program  = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
>  stop -P /var/run/mongrel/mongrel.8002.pid"
>   if totalmem > 100.0 MB for 5 cycles then restart
>   if failed host 127.0.0.1 port 8002 protocol http
>
>     with timeout 10 seconds
>   then restart
>
>
>
> On Wed, Mar 26, 2008 at 6:27 PM, John Weir <john at smokinggun.com> wrote:
>  > well, something doesn't seem right:
>  >
>  >  -> mongrel_rails_8001' failed protocol test [HTTP] at
>  >
>  > INET[127.0.0.1:8018]
>  >                                                                                                           ^
>  >  ^^^^^^^^^^^^
>  >
>  >  127.0.0.1:8018 <- port 8018? or do you want port 8001?
>  >
>  >  also i leave the host parameter out of the monit config.
>  >
>  >    if failed port 8015 protocol http
>  >       with timeout 20 seconds
>  >       then restart
>  >
>  >
>  >
>  >  On Mar 26, 2008, at 5:53 PM, Michael Engelhart wrote:
>  >
>  >  > Does anyone have any tips for checking to see if the mongrel instance
>  >  > is responding to HTTP?  I just tried doing this:
>  >  >
>  >  > if failed host 127.0.0.1 port 8001 protocol http
>  >  >    with timeout 10 seconds
>  >  >  then restart
>  >  >
>  >  > but when I have that code in my monit script it gives me this error:
>  >  > mongrel_rails_8001' failed protocol test [HTTP] at
>  >  > INET[127.0.0.1:8018] via TCP
>  >  >
>  >  >
>  >  > Thanks for any tips.
>  >  >
>  >  > Mike
>  >
>  >
>  > _______________________________________________
>  >  Mongrel-users mailing list
>  >  Mongrel-users at rubyforge.org
>  >  http://rubyforge.org/mailman/listinfo/mongrel-users
>  >
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From mike.engelhart at gmail.com  Thu Mar 27 12:53:38 2008
From: mike.engelhart at gmail.com (Michael Engelhart)
Date: Thu, 27 Mar 2008 12:53:38 -0400
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <466af3440803261946k4e6aed45p8bafe6028edece47@mail.gmail.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>
	<46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>
	<42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>
	<65D8ADBE-1EE3-4042-8583-5F93123B844C@smokinggun.com>
	<42642c790803261609sa53e12ewe14a28d225c0afb3@mail.gmail.com>
	<466af3440803261946k4e6aed45p8bafe6028edece47@mail.gmail.com>
Message-ID: <42642c790803270953k66b0ff01sd4bcbad28e089fa0@mail.gmail.com>

Thanks Joey -

I have my apache setup to respond "OK" to monit to test my Apache
instances but how do I write my monit script to hit the
MonitController.index action?  Is it the same config as I would use
for Apache?

Thanks
Mike

On Wed, Mar 26, 2008 at 10:46 PM, Joey Geiger <jgeiger at gmail.com> wrote:
> Well, there are a few things to think about.
>
>  1. I've got a script that checks nginx with a "success" message.
>  2. Are you checking if mongrel is up, or rails?
>  If it's mongrel, you might want to write a custom route so it's not
>  loading the framework stack every time.
>  If it's rails, you should create something like a MonitController that
>  will respond with "success" when asked for the index action.
>
>  Doing those things has worked quite well for me so far. Just be sure
>  to disable the logging in the monit controller so it doesn't fill your
>  logs.
>  def logger
>  end
>
>
>
>
>  On Wed, Mar 26, 2008 at 6:09 PM, Michael Engelhart
>  <mike.engelhart at gmail.com> wrote:
>  > Sorry - I was copying  and pasting from an alert email.    The ports
>  >  "do" match up in my monit script
>  >
>  >  Here's a real example:
>  >
>  >  #8002
>  >  check process mongrel_rails_8002 with pidfile /var/run/mongrel/mongrel.8002.pid
>  >   group mongrel
>  >   start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
>  >  start -d -e production -a 127.0.0.1 -c /opt/rails/radius --user
>  >  mongrel --group mongrel -p 8002 -P /var/run/mongrel/mongrel.8002.pid
>  >  -l /var/log/mongrel/mongrel.8002.log"
>  >   stop program  = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
>  >  stop -P /var/run/mongrel/mongrel.8002.pid"
>  >   if totalmem > 100.0 MB for 5 cycles then restart
>  >   if failed host 127.0.0.1 port 8002 protocol http
>  >
>  >     with timeout 10 seconds
>  >   then restart
>  >
>  >
>  >
>  > On Wed, Mar 26, 2008 at 6:27 PM, John Weir <john at smokinggun.com> wrote:
>  >  > well, something doesn't seem right:
>  >  >
>  >  >  -> mongrel_rails_8001' failed protocol test [HTTP] at
>  >  >
>  >  > INET[127.0.0.1:8018]
>  >  >                                                                                                           ^
>  >  >  ^^^^^^^^^^^^
>  >  >
>  >  >  127.0.0.1:8018 <- port 8018? or do you want port 8001?
>  >  >
>  >  >  also i leave the host parameter out of the monit config.
>  >  >
>  >  >    if failed port 8015 protocol http
>  >  >       with timeout 20 seconds
>  >  >       then restart
>  >  >
>  >  >
>  >  >
>  >  >  On Mar 26, 2008, at 5:53 PM, Michael Engelhart wrote:
>  >  >
>  >  >  > Does anyone have any tips for checking to see if the mongrel instance
>  >  >  > is responding to HTTP?  I just tried doing this:
>  >  >  >
>  >  >  > if failed host 127.0.0.1 port 8001 protocol http
>  >  >  >    with timeout 10 seconds
>  >  >  >  then restart
>  >  >  >
>  >  >  > but when I have that code in my monit script it gives me this error:
>  >  >  > mongrel_rails_8001' failed protocol test [HTTP] at
>  >  >  > INET[127.0.0.1:8018] via TCP
>  >  >  >
>  >  >  >
>  >  >  > Thanks for any tips.
>  >  >  >
>  >  >  > Mike
>  >  >
>  >  >
>  >  > _______________________________________________
>  >  >  Mongrel-users mailing list
>  >  >  Mongrel-users at rubyforge.org
>  >  >  http://rubyforge.org/mailman/listinfo/mongrel-users
>  >  >
>  >  _______________________________________________
>  >  Mongrel-users mailing list
>  >  Mongrel-users at rubyforge.org
>  >  http://rubyforge.org/mailman/listinfo/mongrel-users
>  >
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From andy.koch at pc-doctor.com  Thu Mar 27 14:49:00 2008
From: andy.koch at pc-doctor.com (Andy Koch)
Date: Thu, 27 Mar 2008 11:49:00 -0700
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <466af3440803261946k4e6aed45p8bafe6028edece47@mail.gmail.com>
References: <57babe2b0803041837y4058975apb04de9d94ae5c42d@mail.gmail.com>	<46c0e1e6bfacaacbe3ed5c6bf3699f19@ruby-forum.com>	<42642c790803261453v7ea6b69fy9eea0e4fb18d39e7@mail.gmail.com>	<65D8ADBE-1EE3-4042-8583-5F93123B844C@smokinggun.com>	<42642c790803261609sa53e12ewe14a28d225c0afb3@mail.gmail.com>
	<466af3440803261946k4e6aed45p8bafe6028edece47@mail.gmail.com>
Message-ID: <fsgq6s$c6v$3@ger.gmane.org>

here's a blog post on suppressing the logging...

"http://www.ruby-forum.com/topic/95146"

here's my bit from my monitrc

******************
if failed port 8000 protocol http and request "/pulse"
     with timeout 10 seconds
     then restart
*******************

  where "pulse" is defined in my routes.rb


Joey Geiger wrote:
> Well, there are a few things to think about.
> 
> 1. I've got a script that checks nginx with a "success" message.
> 2. Are you checking if mongrel is up, or rails?
> If it's mongrel, you might want to write a custom route so it's not
> loading the framework stack every time.
> If it's rails, you should create something like a MonitController that
> will respond with "success" when asked for the index action.
> 
> Doing those things has worked quite well for me so far. Just be sure
> to disable the logging in the monit controller so it doesn't fill your
> logs.
> def logger
> end
> 
> 
> On Wed, Mar 26, 2008 at 6:09 PM, Michael Engelhart
> <mike.engelhart at gmail.com> wrote:
>> Sorry - I was copying  and pasting from an alert email.    The ports
>>  "do" match up in my monit script
>>
>>  Here's a real example:
>>
>>  #8002
>>  check process mongrel_rails_8002 with pidfile /var/run/mongrel/mongrel.8002.pid
>>   group mongrel
>>   start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
>>  start -d -e production -a 127.0.0.1 -c /opt/rails/radius --user
>>  mongrel --group mongrel -p 8002 -P /var/run/mongrel/mongrel.8002.pid
>>  -l /var/log/mongrel/mongrel.8002.log"
>>   stop program  = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails
>>  stop -P /var/run/mongrel/mongrel.8002.pid"
>>   if totalmem > 100.0 MB for 5 cycles then restart
>>   if failed host 127.0.0.1 port 8002 protocol http
>>
>>     with timeout 10 seconds
>>   then restart
>>
>>
>>
>> On Wed, Mar 26, 2008 at 6:27 PM, John Weir <john at smokinggun.com> wrote:
>>  > well, something doesn't seem right:
>>  >
>>  >  -> mongrel_rails_8001' failed protocol test [HTTP] at
>>  >
>>  > INET[127.0.0.1:8018]
>>  >                                                                                                           ^
>>  >  ^^^^^^^^^^^^
>>  >
>>  >  127.0.0.1:8018 <- port 8018? or do you want port 8001?
>>  >
>>  >  also i leave the host parameter out of the monit config.
>>  >
>>  >    if failed port 8015 protocol http
>>  >       with timeout 20 seconds
>>  >       then restart
>>  >
>>  >
>>  >
>>  >  On Mar 26, 2008, at 5:53 PM, Michael Engelhart wrote:
>>  >
>>  >  > Does anyone have any tips for checking to see if the mongrel instance
>>  >  > is responding to HTTP?  I just tried doing this:
>>  >  >
>>  >  > if failed host 127.0.0.1 port 8001 protocol http
>>  >  >    with timeout 10 seconds
>>  >  >  then restart
>>  >  >
>>  >  > but when I have that code in my monit script it gives me this error:
>>  >  > mongrel_rails_8001' failed protocol test [HTTP] at
>>  >  > INET[127.0.0.1:8018] via TCP
>>  >  >
>>  >  >
>>  >  > Thanks for any tips.
>>  >  >
>>  >  > Mike
>>  >
>>  >
>>  > _______________________________________________
>>  >  Mongrel-users mailing list
>>  >  Mongrel-users at rubyforge.org
>>  >  http://rubyforge.org/mailman/listinfo/mongrel-users
>>  >
>>  _______________________________________________
>>  Mongrel-users mailing list
>>  Mongrel-users at rubyforge.org
>>  http://rubyforge.org/mailman/listinfo/mongrel-users
>>





From public at misuse.org  Thu Mar 27 15:06:30 2008
From: public at misuse.org (Steve Midgley)
Date: Thu, 27 Mar 2008 12:06:30 -0700
Subject: [Mongrel] best practices for monitoring mongrels with  monit?
In-Reply-To: <mailman.218.1206643812.17414.mongrel-users@rubyforge.org>
References: <mailman.218.1206643812.17414.mongrel-users@rubyforge.org>
Message-ID: <20080327192749.7D9C31858670@rubyforge.org>

At 11:50 AM 3/27/2008, mongrel-users-request at rubyforge.org wrote:
>Date: Thu, 27 Mar 2008 11:49:00 -0700
>From: Andy Koch <andy.koch at pc-doctor.com>
>Subject: Re: [Mongrel] best practices for monitoring mongrels with
>         monit?
>To: mongrel-users at rubyforge.org
>Message-ID: <fsgq6s$c6v$3 at ger.gmane.org>
>Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>here's a blog post on suppressing the logging...
>
>"http://www.ruby-forum.com/topic/95146"
>
>here's my bit from my monitrc
>
>******************
>if failed port 8000 protocol http and request "/pulse"
>      with timeout 10 seconds
>      then restart
>*******************
>
>   where "pulse" is defined in my routes.rb

Hi,

I wrote a very simple Rails controller (+ test!) that lets my load 
balancers check that my stack is up. Note that I intentionally make a 
database call here b/c I want this monitor to fail if the db is not up. 
But you could easily remove that part of the system. I chose 
"schema_info" to query b/c it's always available in Rails and as small 
as can be.

http://pastie.caboo.se/132646

hth,

Steve 


From normalperson at yhbt.net  Thu Mar 27 17:50:27 2008
From: normalperson at yhbt.net (Eric Wong)
Date: Thu, 27 Mar 2008 14:50:27 -0700
Subject: [Mongrel] Mongrel can't handle some URLs that IE6 sends
In-Reply-To: <20061016020505.50ed39d3.zedshaw@zedshaw.com>
References: <565dbdd40610150856t41656c74k9f091b2d49b8fb09@mail.gmail.com>
	<48fe25b0610150931w473f34d3q5423c6f76eca46b0@mail.gmail.com>
	<eef0b5c0610151036u35b4d641t2a254fad6121df00@mail.gmail.com>
	<48fe25b0610151202m44773de8k75baab56babb9d2c@mail.gmail.com>
	<565dbdd40610151607g40223662ia0c948214b1599e0@mail.gmail.com>
	<565dbdd40610151648t28c43eaub3ccdcb8dd0aae57@mail.gmail.com>
	<20061016020505.50ed39d3.zedshaw@zedshaw.com>
Message-ID: <20080327215027.GA14531@untitled>

"Zed A. Shaw" <zedshaw-dd7LMGGEL7NBDgjK7y7TUQ at public.gmane.org> wrote:
> On Mon, 16 Oct 2006 07:48:16 +0800
> "Eden Li" <eden.li-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
> 
> > Here's a patch of http_parser.rl against r358 that allows these three
> > characters and associated unit tests.
> 
> 
> Rock on!  That's what I'm talking about.  I'll apply this soon and
> post up a pre-release.
>  
> Thanks Eden.

I was late to the party, but just committed this to trunk:

------------------------------------------------------------------------
r996 | normalperson | 2008-03-27 14:46:28 -0700 (Thu, 27 Mar 2008) | 12 lines
Changed paths:
   M ext/http11/http11_parser.c
   M ext/http11/http11_parser_common.rl
   M test/test_http11.rb

http11_parser: accept '"' (double-quote), '<', and '>' characters in URLs

Some broken web browsers don't properly escape ", <, and > characters
in URLs, however these URLs to occasionally legitimate and sometimes
show up.

This patch was submitted by Eden Li here:
  http://rubyforge.org/pipermail/mongrel-users/2006-October/001845.html

This patch was accepted by Zed Shaw here:
  http://rubyforge.org/pipermail/mongrel-users/2006-October/001847.html

------------------------------------------------------------------------

-- 
Eric Wong

From evan at cloudbur.st  Thu Mar 27 18:18:28 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Thu, 27 Mar 2008 18:18:28 -0400
Subject: [Mongrel] Mongrel can't handle some URLs that IE6 sends
In-Reply-To: <20080327215027.GA14531@untitled>
References: <565dbdd40610150856t41656c74k9f091b2d49b8fb09@mail.gmail.com>
	<48fe25b0610150931w473f34d3q5423c6f76eca46b0@mail.gmail.com>
	<eef0b5c0610151036u35b4d641t2a254fad6121df00@mail.gmail.com>
	<48fe25b0610151202m44773de8k75baab56babb9d2c@mail.gmail.com>
	<565dbdd40610151607g40223662ia0c948214b1599e0@mail.gmail.com>
	<565dbdd40610151648t28c43eaub3ccdcb8dd0aae57@mail.gmail.com>
	<20061016020505.50ed39d3.zedshaw@zedshaw.com>
	<20080327215027.GA14531@untitled>
Message-ID: <b6f68fc60803271518i3cbd7bdcq7ea59bdc582f64a0@mail.gmail.com>

Oh wow; good catch.

Evan

On Thu, Mar 27, 2008 at 5:50 PM, Eric Wong <normalperson at yhbt.net> wrote:
> "Zed A. Shaw" <zedshaw-dd7LMGGEL7NBDgjK7y7TUQ at public.gmane.org> wrote:
>  > On Mon, 16 Oct 2006 07:48:16 +0800
>  > "Eden Li" <eden.li-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
>  >
>  > > Here's a patch of http_parser.rl against r358 that allows these three
>  > > characters and associated unit tests.
>  >
>  >
>  > Rock on!  That's what I'm talking about.  I'll apply this soon and
>  > post up a pre-release.
>  >
>  > Thanks Eden.
>
>  I was late to the party, but just committed this to trunk:
>
>  ------------------------------------------------------------------------
>  r996 | normalperson | 2008-03-27 14:46:28 -0700 (Thu, 27 Mar 2008) | 12 lines
>  Changed paths:
>    M ext/http11/http11_parser.c
>    M ext/http11/http11_parser_common.rl
>    M test/test_http11.rb
>
>  http11_parser: accept '"' (double-quote), '<', and '>' characters in URLs
>
>  Some broken web browsers don't properly escape ", <, and > characters
>  in URLs, however these URLs to occasionally legitimate and sometimes
>  show up.
>
>  This patch was submitted by Eden Li here:
>   http://rubyforge.org/pipermail/mongrel-users/2006-October/001845.html
>
>  This patch was accepted by Zed Shaw here:
>   http://rubyforge.org/pipermail/mongrel-users/2006-October/001847.html
>
>  ------------------------------------------------------------------------
>
>  --
>  Eric Wong
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC

From dave at cheney.net  Thu Mar 27 18:24:54 2008
From: dave at cheney.net (Dave Cheney)
Date: Fri, 28 Mar 2008 09:24:54 +1100
Subject: [Mongrel] Mongrel can't handle some URLs that IE6 sends
In-Reply-To: <b6f68fc60803271518i3cbd7bdcq7ea59bdc582f64a0@mail.gmail.com>
References: <565dbdd40610150856t41656c74k9f091b2d49b8fb09@mail.gmail.com>
	<48fe25b0610150931w473f34d3q5423c6f76eca46b0@mail.gmail.com>
	<eef0b5c0610151036u35b4d641t2a254fad6121df00@mail.gmail.com>
	<48fe25b0610151202m44773de8k75baab56babb9d2c@mail.gmail.com>
	<565dbdd40610151607g40223662ia0c948214b1599e0@mail.gmail.com>
	<565dbdd40610151648t28c43eaub3ccdcb8dd0aae57@mail.gmail.com>
	<20061016020505.50ed39d3.zedshaw@zedshaw.com>
	<20080327215027.GA14531@untitled>
	<b6f68fc60803271518i3cbd7bdcq7ea59bdc582f64a0@mail.gmail.com>
Message-ID: <56AACD8A-4A89-4C53-8878-2953D685447E@cheney.net>

Sweet, when can I remove these lines from my nginx.conf ?

--

# nginx does not appear to properly sanitize urls in the form http://www.redbubble.com/people/photography/art/ 
</p
# which causes mongrel to slam the door all the way down the cluster,  
causing a minor hickup. Both behaviours are
# wrong, but until one party fixes the bug, this hack protects us

location ~ '<' {
    return 400;
}

location ~ '>' {
    return 400;
}

location ~ '"' {
    return 400;
}

--

Cheers

Dave

On 28/03/2008, at 9:18 AM, Evan Weaver wrote:

> Oh wow; good catch.
>
> Evan
>
> On Thu, Mar 27, 2008 at 5:50 PM, Eric Wong <normalperson at yhbt.net>  
> wrote:
>> "Zed A. Shaw" <zedshaw-dd7LMGGEL7NBDgjK7y7TUQ at public.gmane.org>  
>> wrote:
>>> On Mon, 16 Oct 2006 07:48:16 +0800
>>> "Eden Li" <eden.li-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
>>>
>>>> Here's a patch of http_parser.rl against r358 that allows these  
>>>> three
>>>> characters and associated unit tests.
>>>
>>>
>>> Rock on!  That's what I'm talking about.  I'll apply this soon and
>>> post up a pre-release.
>>>
>>> Thanks Eden.
>>
>> I was late to the party, but just committed this to trunk:
>>
>> ------------------------------------------------------------------------
>> r996 | normalperson | 2008-03-27 14:46:28 -0700 (Thu, 27 Mar 2008)  
>> | 12 lines
>> Changed paths:
>>   M ext/http11/http11_parser.c
>>   M ext/http11/http11_parser_common.rl
>>   M test/test_http11.rb
>>
>> http11_parser: accept '"' (double-quote), '<', and '>' characters  
>> in URLs
>>
>> Some broken web browsers don't properly escape ", <, and > characters
>> in URLs, however these URLs to occasionally legitimate and sometimes
>> show up.
>>
>> This patch was submitted by Eden Li here:
>>  http://rubyforge.org/pipermail/mongrel-users/2006-October/ 
>> 001845.html
>>
>> This patch was accepted by Zed Shaw here:
>>  http://rubyforge.org/pipermail/mongrel-users/2006-October/ 
>> 001847.html
>>
>> ------------------------------------------------------------------------
>>
>> --
>> Eric Wong
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>
>
>
>
> -- 
> Evan Weaver
> Cloudburst, LLC
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From luislavena at gmail.com  Thu Mar 27 18:28:17 2008
From: luislavena at gmail.com (Luis Lavena)
Date: Thu, 27 Mar 2008 19:28:17 -0300
Subject: [Mongrel] Mongrel can't handle some URLs that IE6 sends
In-Reply-To: <56AACD8A-4A89-4C53-8878-2953D685447E@cheney.net>
References: <565dbdd40610150856t41656c74k9f091b2d49b8fb09@mail.gmail.com>
	<48fe25b0610150931w473f34d3q5423c6f76eca46b0@mail.gmail.com>
	<eef0b5c0610151036u35b4d641t2a254fad6121df00@mail.gmail.com>
	<48fe25b0610151202m44773de8k75baab56babb9d2c@mail.gmail.com>
	<565dbdd40610151607g40223662ia0c948214b1599e0@mail.gmail.com>
	<565dbdd40610151648t28c43eaub3ccdcb8dd0aae57@mail.gmail.com>
	<20061016020505.50ed39d3.zedshaw@zedshaw.com>
	<20080327215027.GA14531@untitled>
	<b6f68fc60803271518i3cbd7bdcq7ea59bdc582f64a0@mail.gmail.com>
	<56AACD8A-4A89-4C53-8878-2953D685447E@cheney.net>
Message-ID: <71166b3b0803271528t2f9d34es1891804213a7afef@mail.gmail.com>

On Thu, Mar 27, 2008 at 7:24 PM, Dave Cheney <dave at cheney.net> wrote:
> Sweet, when can I remove these lines from my nginx.conf ?
>

Hold down a bit man, it just got into the code base ;-)

Regards,
-- 
Luis Lavena
Multimedia systems
-
Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

From mike.engelhart at gmail.com  Thu Mar 27 23:26:30 2008
From: mike.engelhart at gmail.com (Michael Engelhart)
Date: Thu, 27 Mar 2008 23:26:30 -0400
Subject: [Mongrel] best practices for monitoring mongrels with monit?
In-Reply-To: <20080327192749.7D9C31858670@rubyforge.org>
References: <mailman.218.1206643812.17414.mongrel-users@rubyforge.org>
	<20080327192749.7D9C31858670@rubyforge.org>
Message-ID: <42642c790803272026y79b471beh78442d13e2835739@mail.gmail.com>

Thanks guys - this works great for me.

Much appreciated.

Mike

On Thu, Mar 27, 2008 at 3:06 PM, Steve Midgley <public at misuse.org> wrote:
> At 11:50 AM 3/27/2008, mongrel-users-request at rubyforge.org wrote:
>  >Date: Thu, 27 Mar 2008 11:49:00 -0700
>  >From: Andy Koch <andy.koch at pc-doctor.com>
>  >Subject: Re: [Mongrel] best practices for monitoring mongrels with
>  >         monit?
>  >To: mongrel-users at rubyforge.org
>  >Message-ID: <fsgq6s$c6v$3 at ger.gmane.org>
>  >Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> >
>  >here's a blog post on suppressing the logging...
>  >
>  >"http://www.ruby-forum.com/topic/95146"
>  >
>  >here's my bit from my monitrc
>  >
>  >******************
>  >if failed port 8000 protocol http and request "/pulse"
>  >      with timeout 10 seconds
>  >      then restart
>  >*******************
>  >
>  >   where "pulse" is defined in my routes.rb
>
>  Hi,
>
>  I wrote a very simple Rails controller (+ test!) that lets my load
>  balancers check that my stack is up. Note that I intentionally make a
>  database call here b/c I want this monitor to fail if the db is not up.
>  But you could easily remove that part of the system. I chose
>  "schema_info" to query b/c it's always available in Rails and as small
>  as can be.
>
>  http://pastie.caboo.se/132646
>
>  hth,
>
>  Steve
>
>
>
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From lists at ruby-forum.com  Mon Mar 31 12:15:32 2008
From: lists at ruby-forum.com (James Testa)
Date: Mon, 31 Mar 2008 18:15:32 +0200
Subject: [Mongrel] To surf to URL mydomain.com instead of mydomain.com:3000?
Message-ID: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>

My production server is using mongrel_rails on Ubuntu linux.
With Firefox I can reach my production server with the URL:

www.mydomain.com:3000/

but I can't reach my production server with the URL

www.mydomain.com/

Is there a way to configure mongrel so that I can reach my
production server with the URL www.mydomain.com?
-- 
Posted via http://www.ruby-forum.com/.

From rochkind at jhu.edu  Mon Mar 31 12:25:02 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Mon, 31 Mar 2008 12:25:02 -0400
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000?
In-Reply-To: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
Message-ID: <47F1105E.5040304@jhu.edu>

That's usually done by setting up mod_proxy in Apache to reverse proxy 
from Apache to mongrel. I believe that is the typical mongrel production 
scenario?

Here's one page I found on the mongrel wiki (it warns it may be 
outdated, but I think it's still good) leading you through how to do that.

http://mongrel.rubyforge.org/wiki/Apache

Note that once you've turned on mod_proxy in apache, it's very important 
to include the line "ProxyRequests off" in your apache config. We're 
using mod_proxy as a reverse proxy, but by default it'll be an unsecured 
forward proxy too, which is something you definitely don't want.  
"ProxyRequests off" turns this off.

Hope this helps. Some other mongrel experts correct me if I'm wrong, 
I've been puzzling through this stuff myself.

Jonathan


James Testa wrote:
> My production server is using mongrel_rails on Ubuntu linux.
> With Firefox I can reach my production server with the URL:
>
> www.mydomain.com:3000/
>
> but I can't reach my production server with the URL
>
> www.mydomain.com/
>
> Is there a way to configure mongrel so that I can reach my
> production server with the URL www.mydomain.com?
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From wyhaines at gmail.com  Mon Mar 31 12:25:10 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 31 Mar 2008 10:25:10 -0600
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000?
In-Reply-To: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
Message-ID: <f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>

On Mon, Mar 31, 2008 at 10:15 AM, James Testa <lists at ruby-forum.com> wrote:
> My production server is using mongrel_rails on Ubuntu linux.
> With Firefox I can reach my production server with the URL:
>
> www.mydomain.com:3000/
>
> but I can't reach my production server with the URL
>
> www.mydomain.com/
>
> Is there a way to configure mongrel so that I can reach my
> production server with the URL www.mydomain.com?

Run it on port 80 instead of port 3000.


Kirk Haines

From lists at ruby-forum.com  Mon Mar 31 12:28:42 2008
From: lists at ruby-forum.com (James Testa)
Date: Mon, 31 Mar 2008 18:28:42 +0200
Subject: [Mongrel] how to get mongrel to use maintenance.html file?
Message-ID: <5bb732c6ac868a8ac3e1c1f51fdc7c1d@ruby-forum.com>

Does someone know how to tell mongrel_rails to read the
public/system/maintenance.html file?
-- 
Posted via http://www.ruby-forum.com/.

From njvack at wisc.edu  Mon Mar 31 12:34:24 2008
From: njvack at wisc.edu (Nate Vack)
Date: Mon, 31 Mar 2008 11:34:24 -0500
Subject: [Mongrel] how to get mongrel to use maintenance.html file?
In-Reply-To: <5bb732c6ac868a8ac3e1c1f51fdc7c1d@ruby-forum.com>
References: <5bb732c6ac868a8ac3e1c1f51fdc7c1d@ruby-forum.com>
Message-ID: <8be813400803310934r36c90a1cr153bca6a7d976413@mail.gmail.com>

You'd probably ask Apache / ngnix to do that, actually, via some
mod_rewrite-fu I can't muster this morning :/

Cheers,
-Nate

On Mon, Mar 31, 2008 at 11:28 AM, James Testa <lists at ruby-forum.com> wrote:
> Does someone know how to tell mongrel_rails to read the
>  public/system/maintenance.html file?
>  --
>  Posted via http://www.ruby-forum.com/.
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>

From wyhaines at gmail.com  Mon Mar 31 13:34:02 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 31 Mar 2008 11:34:02 -0600
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000?
In-Reply-To: <47F1105E.5040304@jhu.edu>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<47F1105E.5040304@jhu.edu>
Message-ID: <f4cd26df0803311034s32138bdcrbea0c0ec9b5e2ff@mail.gmail.com>

On Mon, Mar 31, 2008 at 10:25 AM, Jonathan Rochkind <rochkind at jhu.edu> wrote:
> That's usually done by setting up mod_proxy in Apache to reverse proxy
> from Apache to mongrel. I believe that is the typical mongrel production
> scenario?

Yes, the typical production architecture is to have some sort of
webserver front end that proxies to a pool of backend processes.

Apache is a common choice, as is nginx (unless you must use Apache,
nginx will often be a better choice).  Some people use lighttpd, and a
small but growing group of people are using my project, Swiftiply.


Kirk Haines

From wyhaines at gmail.com  Mon Mar 31 14:33:20 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 31 Mar 2008 12:33:20 -0600
Subject: [Mongrel] how to get mongrel to use maintenance.html file?
In-Reply-To: <8be813400803310934r36c90a1cr153bca6a7d976413@mail.gmail.com>
References: <5bb732c6ac868a8ac3e1c1f51fdc7c1d@ruby-forum.com>
	<8be813400803310934r36c90a1cr153bca6a7d976413@mail.gmail.com>
Message-ID: <f4cd26df0803311133y1858b6dfi80d962a09f29c5e0@mail.gmail.com>

On Mon, Mar 31, 2008 at 10:34 AM, Nate Vack <njvack at wisc.edu> wrote:
> You'd probably ask Apache / ngnix to do that, actually, via some
> mod_rewrite-fu I can't muster this morning :/
>
> Cheers,
> -Nate
>
>
> On Mon, Mar 31, 2008 at 11:28 AM, James Testa <lists at ruby-forum.com> wrote:
> > Does someone know how to tell mongrel_rails to read the
> >  public/system/maintenance.html file?

Generally, yes.

If you are doing some sort of maintenance, you probably want that
served at the highest level possible.  i.e. at the web server level.

Swiftiply w/ rewrite support (totally experimental, but will probably
end up in a release soon)

:rewrites:
- :match: "*"
  :sub: "/public/system/maintenance.html"


Apache:

RewriteEngine On
RewriteRule * /public/system/maintenance.html
(or something similar; unless you do it a lot, always check the
rewrite rule docs when writing them)



Kirk Haines

From lists at ruby-forum.com  Mon Mar 31 16:31:42 2008
From: lists at ruby-forum.com (James Testa)
Date: Mon, 31 Mar 2008 22:31:42 +0200
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
Message-ID: <72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>

Kirk,
    Do you mean run

% mongrel_rails start -d -p 80

I tried that and mongrel won't start.   But mongrel will start with

% mongrel_rails start -d -p 3000

Is there a mongrel configuration file somewhere that is overriding
the port setting?

Jim




Kirk Haines wrote:
> On Mon, Mar 31, 2008 at 10:15 AM, James Testa <lists at ruby-forum.com> 
> wrote:
>> production server with the URL www.mydomain.com?
> Run it on port 80 instead of port 3000.
> 
> 
> Kirk Haines

-- 
Posted via http://www.ruby-forum.com/.

From g.vishnu at gmail.com  Mon Mar 31 16:36:26 2008
From: g.vishnu at gmail.com (Vishnu Gopal)
Date: Tue, 1 Apr 2008 02:06:26 +0530
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
Message-ID: <d9605de70803311336j67345e0aw5855501abebb7dd1@mail.gmail.com>

You probably need root privileges. Try sudo.

Vish

On Tue, Apr 1, 2008 at 2:01 AM, James Testa <lists at ruby-forum.com> wrote:

> Kirk,
>    Do you mean run
>
> % mongrel_rails start -d -p 80
>
> I tried that and mongrel won't start.   But mongrel will start with
>
> % mongrel_rails start -d -p 3000
>
> Is there a mongrel configuration file somewhere that is overriding
> the port setting?
>
> Jim
>
>
>
>
> Kirk Haines wrote:
> > On Mon, Mar 31, 2008 at 10:15 AM, James Testa <lists at ruby-forum.com>
> > wrote:
> >> production server with the URL www.mydomain.com?
> > Run it on port 80 instead of port 3000.
> >
> >
> > Kirk Haines
>
> --
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080401/82a5e15b/attachment.html 

From wyhaines at gmail.com  Mon Mar 31 16:43:34 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 31 Mar 2008 14:43:34 -0600
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
Message-ID: <f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>

On Mon, Mar 31, 2008 at 2:31 PM, James Testa <lists at ruby-forum.com> wrote:
> Kirk,
>    Do you mean run
>
> % mongrel_rails start -d -p 80
>
> I tried that and mongrel won't start.   But mongrel will start with
>
> % mongrel_rails start -d -p 3000
>
> Is there a mongrel configuration file somewhere that is overriding
> the port setting?

Okay, if it won't start, then do you get an error?

Odds are that either there is already something running there, or you
are starting mongrel without sufficient permissions. To bind to a port
below 1024, a process has to be running as root.


Kirk Haines

From rochkind at jhu.edu  Mon Mar 31 16:51:45 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Mon, 31 Mar 2008 16:51:45 -0400
Subject: [Mongrel] To surf to URL mydomain.com instead
	of	mydomain.com:3000
In-Reply-To: <f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
Message-ID: <47F14EE1.7000509@jhu.edu>

I really do still think you're going down the wrong path running mongrel 
directly on 80, instead of sending it through a proxy on your web 
server. I think it's going to cause you headaches in the long run. Do 
you have any web server running on that machine?  What web server?  Is 
there any reason you are choosing not to go the route of having apache 
(or other specialized) web server be a front end to your mongrel?

Jonathan

Kirk Haines wrote:
> On Mon, Mar 31, 2008 at 2:31 PM, James Testa <lists at ruby-forum.com> wrote:
>   
>> Kirk,
>>    Do you mean run
>>
>> % mongrel_rails start -d -p 80
>>
>> I tried that and mongrel won't start.   But mongrel will start with
>>
>> % mongrel_rails start -d -p 3000
>>
>> Is there a mongrel configuration file somewhere that is overriding
>> the port setting?
>>     
>
> Okay, if it won't start, then do you get an error?
>
> Odds are that either there is already something running there, or you
> are starting mongrel without sufficient permissions. To bind to a port
> below 1024, a process has to be running as root.
>
>
> Kirk Haines
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From lists at ruby-forum.com  Mon Mar 31 17:03:04 2008
From: lists at ruby-forum.com (James Testa)
Date: Mon, 31 Mar 2008 23:03:04 +0200
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
Message-ID: <778d84f47613eba8de67415669a9878f@ruby-forum.com>

Kirk,
    There wasn't an error message but I wasn't running it
as root.  Once I ran it as root it worked fine.
    Thanks,

Jim

Kirk Haines wrote:
> On Mon, Mar 31, 2008 at 2:31 PM, James Testa <lists at ruby-forum.com> 
> wrote:
>> the port setting?
> Okay, if it won't start, then do you get an error?
> 
> Odds are that either there is already something running there, or you
> are starting mongrel without sufficient permissions. To bind to a port
> below 1024, a process has to be running as root.
> 
> 
> Kirk Haines

-- 
Posted via http://www.ruby-forum.com/.

From lists at ruby-forum.com  Mon Mar 31 17:08:01 2008
From: lists at ruby-forum.com (James Testa)
Date: Mon, 31 Mar 2008 23:08:01 +0200
Subject: [Mongrel]
	=?utf-8?q?To_surf_to_URL_mydomain=2Ecom_instead_of=09my?=
	=?utf-8?q?domain=2Ecom=3A3000?=
In-Reply-To: <47F14EE1.7000509@jhu.edu>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
	<47F14EE1.7000509@jhu.edu>
Message-ID: <ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>

Jonathan,
    At the moment I am only running mongrel, but we are
just beta testing.

1.  What sort of headaches do you foresee just running mongrel?  Do
    you mean from a load balancing, high traffic point of view?
2.  I was planning on running nginx as the front end.  Do you see any 
issues
    with nginx vs Apache?


Jim

Jonathan Rochkind wrote:
> I really do still think you're going down the wrong path running mongrel
> directly on 80, instead of sending it through a proxy on your web
> server. I think it's going to cause you headaches in the long run. Do
> you have any web server running on that machine?  What web server?  Is
> there any reason you are choosing not to go the route of having apache
> (or other specialized) web server be a front end to your mongrel?
> 
> Jonathan
> 
> Kirk Haines wrote:
>>>
>>
>> Kirk Haines
>> _______________________________________________
>> Mongrel-users mailing list
>> Mongrel-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>   
> 
> --
> Jonathan Rochkind
> Digital Services Software Engineer
> The Sheridan Libraries
> Johns Hopkins University
> 410.516.8886
> rochkind (at) jhu.edu

-- 
Posted via http://www.ruby-forum.com/.

From rochkind at jhu.edu  Mon Mar 31 17:15:21 2008
From: rochkind at jhu.edu (Jonathan Rochkind)
Date: Mon, 31 Mar 2008 17:15:21 -0400
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
	<47F14EE1.7000509@jhu.edu>
	<ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>
Message-ID: <47F15469.4030500@jhu.edu>

I have no experience with nginx, but someone else on this list said it 
was common, so that'd probably work.

If you plan to go that route eventually anyway, I'd just do that now, 
because what you need to do to get your mongrel to answer on port 80 
will change completely in the presence of nginx or apache vs. now with a 
bare mongrel. So the work you're doing now to figure out how to get 
mongrel to run bare on 80 seems kind of unproductive.

I don't honestly have any specifics in mind as far as problems from 
running a bare mongrel on port 80 in production. Maybe others on the 
list do.  Just that that doesn't seem to be the popular or recommended 
way to do it, so you might run into weird problems that others don't. As 
you've noticed, one is that you will need to run the mongrel as root, 
which is really pretty undesirable. Or come up with some custom solution 
to start mongrel as root and attach it to 80 then exec it as another 
user. But now you're creating weird solutions just to get mongrel to run 
bare on 80, when if you're planning on running it under nginx or apache 
anyway you won't ultimately need, so why deal with it unneccesarily?

Jonathan

James Testa wrote:
> Jonathan,
>     At the moment I am only running mongrel, but we are
> just beta testing.
>
> 1.  What sort of headaches do you foresee just running mongrel?  Do
>     you mean from a load balancing, high traffic point of view?
> 2.  I was planning on running nginx as the front end.  Do you see any 
> issues
>     with nginx vs Apache?
>
>
> Jim
>
> Jonathan Rochkind wrote:
>   
>> I really do still think you're going down the wrong path running mongrel
>> directly on 80, instead of sending it through a proxy on your web
>> server. I think it's going to cause you headaches in the long run. Do
>> you have any web server running on that machine?  What web server?  Is
>> there any reason you are choosing not to go the route of having apache
>> (or other specialized) web server be a front end to your mongrel?
>>
>> Jonathan
>>
>> Kirk Haines wrote:
>>     
>>> Kirk Haines
>>> _______________________________________________
>>> Mongrel-users mailing list
>>> Mongrel-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/mongrel-users
>>>   
>>>       
>> --
>> Jonathan Rochkind
>> Digital Services Software Engineer
>> The Sheridan Libraries
>> Johns Hopkins University
>> 410.516.8886
>> rochkind (at) jhu.edu
>>     
>
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu


From wyhaines at gmail.com  Mon Mar 31 17:22:27 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 31 Mar 2008 15:22:27 -0600
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <47F15469.4030500@jhu.edu>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
	<47F14EE1.7000509@jhu.edu>
	<ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>
	<47F15469.4030500@jhu.edu>
Message-ID: <f4cd26df0803311422y7c6ad667o6fcbdc4e1d8284a4@mail.gmail.com>

On Mon, Mar 31, 2008 at 3:15 PM, Jonathan Rochkind <rochkind at jhu.edu> wrote:

> I don't honestly have any specifics in mind as far as problems from
> running a bare mongrel on port 80 in production. Maybe others on the
> list do.  Just that that doesn't seem to be the popular or recommended
> way to do it, so you might run into weird problems that others don't. As

There should be any particular problems.  Just be aware of the
limitations.  It will work fine for simple testing, though.

> you've noticed, one is that you will need to run the mongrel as root,
> which is really pretty undesirable. Or come up with some custom solution
> to start mongrel as root and attach it to 80 then exec it as another
> user. But now you're creating weird solutions just to get mongrel to run
> bare on 80, when if you're planning on running it under nginx or apache
> anyway you won't ultimately need, so why deal with it unneccesarily?

Mongrel has support for changing it's userid/group after starting.
So, no custom solution is necessary.


Kirk Haines

From swindsor at gmail.com  Mon Mar 31 17:22:58 2008
From: swindsor at gmail.com (Scott Windsor)
Date: Mon, 31 Mar 2008 14:22:58 -0700
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <47F15469.4030500@jhu.edu>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
	<47F14EE1.7000509@jhu.edu>
	<ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>
	<47F15469.4030500@jhu.edu>
Message-ID: <aa2e18ec0803311422i223b077am35ec7fca03e6c60e@mail.gmail.com>

+1 from me for nginx...

The configuration is pretty straighforward and easy to understand.

Here's the example config from the nginx cookbook.
http://wiki.codemongers.com/NginxRubyonRailsMongrel

It's got proxying, gzip compression, & sendfile out of the box (very nice if
you'd had to configure for apache).

- scott

On Mon, Mar 31, 2008 at 2:15 PM, Jonathan Rochkind <rochkind at jhu.edu> wrote:

> I have no experience with nginx, but someone else on this list said it
> was common, so that'd probably work.
>
> If you plan to go that route eventually anyway, I'd just do that now,
> because what you need to do to get your mongrel to answer on port 80
> will change completely in the presence of nginx or apache vs. now with a
> bare mongrel. So the work you're doing now to figure out how to get
> mongrel to run bare on 80 seems kind of unproductive.
>
> I don't honestly have any specifics in mind as far as problems from
> running a bare mongrel on port 80 in production. Maybe others on the
> list do.  Just that that doesn't seem to be the popular or recommended
> way to do it, so you might run into weird problems that others don't. As
> you've noticed, one is that you will need to run the mongrel as root,
> which is really pretty undesirable. Or come up with some custom solution
> to start mongrel as root and attach it to 80 then exec it as another
> user. But now you're creating weird solutions just to get mongrel to run
> bare on 80, when if you're planning on running it under nginx or apache
> anyway you won't ultimately need, so why deal with it unneccesarily?
>
> Jonathan
>
> James Testa wrote:
> > Jonathan,
> >     At the moment I am only running mongrel, but we are
> > just beta testing.
> >
> > 1.  What sort of headaches do you foresee just running mongrel?  Do
> >     you mean from a load balancing, high traffic point of view?
> > 2.  I was planning on running nginx as the front end.  Do you see any
> > issues
> >     with nginx vs Apache?
> >
> >
> > Jim
> >
> > Jonathan Rochkind wrote:
> >
> >> I really do still think you're going down the wrong path running
> mongrel
> >> directly on 80, instead of sending it through a proxy on your web
> >> server. I think it's going to cause you headaches in the long run. Do
> >> you have any web server running on that machine?  What web server?  Is
> >> there any reason you are choosing not to go the route of having apache
> >> (or other specialized) web server be a front end to your mongrel?
> >>
> >> Jonathan
> >>
> >> Kirk Haines wrote:
> >>
> >>> Kirk Haines
> >>> _______________________________________________
> >>> Mongrel-users mailing list
> >>> Mongrel-users at rubyforge.org
> >>> http://rubyforge.org/mailman/listinfo/mongrel-users
> >>>
> >>>
> >> --
> >> Jonathan Rochkind
> >> Digital Services Software Engineer
> >> The Sheridan Libraries
> >> Johns Hopkins University
> >> 410.516.8886
> >> rochkind (at) jhu.edu
> >>
> >
> >
>
> --
> Jonathan Rochkind
> Digital Services Software Engineer
> The Sheridan Libraries
> Johns Hopkins University
> 410.516.8886
> rochkind (at) jhu.edu
>
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20080331/3f5989d6/attachment.html 

From wyhaines at gmail.com  Mon Mar 31 17:25:06 2008
From: wyhaines at gmail.com (Kirk Haines)
Date: Mon, 31 Mar 2008 15:25:06 -0600
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <47F15469.4030500@jhu.edu>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
	<47F14EE1.7000509@jhu.edu>
	<ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>
	<47F15469.4030500@jhu.edu>
Message-ID: <f4cd26df0803311425m74857e07h5dfbaeb934f15305@mail.gmail.com>

On Mon, Mar 31, 2008 at 3:15 PM, Jonathan Rochkind <rochkind at jhu.edu> wrote:
> I have no experience with nginx, but someone else on this list said it
> was common, so that'd probably work.

Despite (or perhaps because of) being an Apache user for nigh on a
decade, I'd advocate for looking at something else first, for a
production app.  The reason is, quite simply, because there are other
things that are faster and/or easier to use.


Kirk Haines

From dave at cheney.net  Mon Mar 31 17:47:02 2008
From: dave at cheney.net (Dave Cheney)
Date: Tue, 1 Apr 2008 08:47:02 +1100
Subject: [Mongrel] how to get mongrel to use maintenance.html file?
In-Reply-To: <f4cd26df0803311133y1858b6dfi80d962a09f29c5e0@mail.gmail.com>
References: <5bb732c6ac868a8ac3e1c1f51fdc7c1d@ruby-forum.com>
	<8be813400803310934r36c90a1cr153bca6a7d976413@mail.gmail.com>
	<f4cd26df0803311133y1858b6dfi80d962a09f29c5e0@mail.gmail.com>
Message-ID: <79DB07B5-9D14-4B54-B8C9-2E42997AFF09@cheney.net>

For nginx, try something like this,

error_page   503          /maintenance.html;

location /maintenance.html {

}

location / {
    if ( -f $document_root/system/maintenance.html ) {
       return 503;
    }

    if (!-f $request_filename) {
       rewrite ^(.+)$ $uri break;
       proxy_pass http://mongrel_app;
   }
}

This is slightly different to the configuration that is out there for  
doing /maintenance.html because those return a 200 code in maintenance  
mode, which upsets paypal

Cheers

Dave

On 01/04/2008, at 5:33 AM, Kirk Haines wrote:

> On Mon, Mar 31, 2008 at 10:34 AM, Nate Vack <njvack at wisc.edu> wrote:
>> You'd probably ask Apache / ngnix to do that, actually, via some
>> mod_rewrite-fu I can't muster this morning :/
>>
>> Cheers,
>> -Nate
>>
>>
>> On Mon, Mar 31, 2008 at 11:28 AM, James Testa <lists at ruby- 
>> forum.com> wrote:
>>> Does someone know how to tell mongrel_rails to read the
>>> public/system/maintenance.html file?
>
> Generally, yes.
>
> If you are doing some sort of maintenance, you probably want that
> served at the highest level possible.  i.e. at the web server level.
>
> Swiftiply w/ rewrite support (totally experimental, but will probably
> end up in a release soon)
>
> :rewrites:
> - :match: "*"
>  :sub: "/public/system/maintenance.html"
>
>
> Apache:
>
> RewriteEngine On
> RewriteRule * /public/system/maintenance.html
> (or something similar; unless you do it a lot, always check the
> rewrite rule docs when writing them)
>
>
>
> Kirk Haines
> _______________________________________________
> Mongrel-users mailing list
> Mongrel-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-users


From lists at ruby-forum.com  Mon Mar 31 18:21:56 2008
From: lists at ruby-forum.com (James Testa)
Date: Tue, 1 Apr 2008 00:21:56 +0200
Subject: [Mongrel] To surf to URL mydomain.com instead of
	mydomain.com:3000
In-Reply-To: <47F15469.4030500@jhu.edu>
References: <139ae55c66fb6a1436e82c9b991771b5@ruby-forum.com>
	<f4cd26df0803310925n401bd075h4be9cc5d381f4469@mail.gmail.com>
	<72e89dcd9f94eb30c5f31994eaf99acd@ruby-forum.com>
	<f4cd26df0803311343s449aafaepca9f2efd95239854@mail.gmail.com>
	<47F14EE1.7000509@jhu.edu>
	<ec0d5009e3fcaf7c01daff0fa73b6704@ruby-forum.com>
	<47F15469.4030500@jhu.edu>
Message-ID: <2928a5281a65f15892920e1217b4037f@ruby-forum.com>

Jonathan,
    You make a good point about running mongrel as root.
Previously I wasn't doing that and was using Capistrano
to restart mongrel.  So you are right, I had to start
thinking about these mongrel/root issues.
    I'll begin adding a front end, since I need to get
there anyway, sooner is better.
    Thanks,


Jim

Jonathan Rochkind wrote:
> I have no experience with nginx, but someone else on this list said it
> was common, so that'd probably work.
> 
> If you plan to go that route eventually anyway, I'd just do that now,
> because what you need to do to get your mongrel to answer on port 80
> will change completely in the presence of nginx or apache vs. now with a
> bare mongrel. So the work you're doing now to figure out how to get
> mongrel to run bare on 80 seems kind of unproductive.
> 
> I don't honestly have any specifics in mind as far as problems from
> running a bare mongrel on port 80 in production. Maybe others on the
> list do.  Just that that doesn't seem to be the popular or recommended
> way to do it, so you might run into weird problems that others don't. As
> you've noticed, one is that you will need to run the mongrel as root,
> which is really pretty undesirable. Or come up with some custom solution
> to start mongrel as root and attach it to 80 then exec it as another
> user. But now you're creating weird solutions just to get mongrel to run
> bare on 80, when if you're planning on running it under nginx or apache
> anyway you won't ultimately need, so why deal with it unneccesarily?
> 
> Jonathan
> 
> James Testa wrote:
>>
>>>
>>>>       
>>> --
>>> Jonathan Rochkind
>>> Digital Services Software Engineer
>>> The Sheridan Libraries
>>> Johns Hopkins University
>>> 410.516.8886
>>> rochkind (at) jhu.edu
>>>     
>>
>>   
> 
> --
> Jonathan Rochkind
> Digital Services Software Engineer
> The Sheridan Libraries
> Johns Hopkins University
> 410.516.8886
> rochkind (at) jhu.edu

-- 
Posted via http://www.ruby-forum.com/.

From lists at ruby-forum.com  Mon Mar 31 18:30:44 2008
From: lists at ruby-forum.com (Michael Kinney)
Date: Tue, 1 Apr 2008 00:30:44 +0200
Subject: [Mongrel] Mongrels stop responding
Message-ID: <a537a17d57dd941695f37149ff859bb8@ruby-forum.com>

I have a Rails 1.2.3 app (due for an update, I know) running on a
Mongrel 1.1.4 behind Apache 2.2.3 mod_proxy on Debian 4.0. It keeps
hanging up for no reason I can tell. I have tried everything that anyone
has ever hinted at being a solution for this problem:

- The application doesn't use MySQL, so setting the connection timeout
won't help.
- Using AR not PStore for sessions.
- Logger isn't rotating the logs itself.
- I added the no-keepalive and force 1.0 setenv commands to apache.

In the apache log, I get the typical "proxy: error reading status line
from remote server localhost". lsof shows lots of sockets in CLOSE_WAIT.
strace gives this sequence over and over:

time(NULL)                              = 1207001155
select(4, [3], [], [], {0, 0})          = 0 (Timeout)
sigprocmask(SIG_BLOCK, NULL, [])        = 0
time(NULL)                              = 1207001155
gettimeofday({1207001155, 980832}, NULL) = 0
gettimeofday({1207001155, 980855}, NULL) = 0
select(4, [3], [], [], {0, 999977})     = 0 (Timeout)
gettimeofday({1207001156, 980679}, NULL) = 0
select(4, [3], [], [], {0, 153})        = 0 (Timeout)

I tried letting them run after sending a USR1, but that doesn't seem to
generate any extra information.

At first, I thought it would only happen if the Mongrels were allowed to
sit idle - similar to the MySQL problem others reported - but then I saw
one go silent within a half hour of a restart.

Any suggestions?
-- 
Posted via http://www.ruby-forum.com/.

From evan at cloudbur.st  Mon Mar 31 18:40:23 2008
From: evan at cloudbur.st (Evan Weaver)
Date: Mon, 31 Mar 2008 18:40:23 -0400
Subject: [Mongrel] Mongrels stop responding
In-Reply-To: <a537a17d57dd941695f37149ff859bb8@ruby-forum.com>
References: <a537a17d57dd941695f37149ff859bb8@ruby-forum.com>
Message-ID: <b6f68fc60803311540o1f12332cwfb1f481881bcbeb8@mail.gmail.com>

Gdb the stuck mongrel and force it to raise a backtrace. It looks like
maybe you're stuck in a timeout loop somewhere that never excepts
properly?

Evan

On Mon, Mar 31, 2008 at 6:30 PM, Michael Kinney <lists at ruby-forum.com> wrote:
> I have a Rails 1.2.3 app (due for an update, I know) running on a
>  Mongrel 1.1.4 behind Apache 2.2.3 mod_proxy on Debian 4.0. It keeps
>  hanging up for no reason I can tell. I have tried everything that anyone
>  has ever hinted at being a solution for this problem:
>
>  - The application doesn't use MySQL, so setting the connection timeout
>  won't help.
>  - Using AR not PStore for sessions.
>  - Logger isn't rotating the logs itself.
>  - I added the no-keepalive and force 1.0 setenv commands to apache.
>
>  In the apache log, I get the typical "proxy: error reading status line
>  from remote server localhost". lsof shows lots of sockets in CLOSE_WAIT.
>  strace gives this sequence over and over:
>
>  time(NULL)                              = 1207001155
>  select(4, [3], [], [], {0, 0})          = 0 (Timeout)
>  sigprocmask(SIG_BLOCK, NULL, [])        = 0
>  time(NULL)                              = 1207001155
>  gettimeofday({1207001155, 980832}, NULL) = 0
>  gettimeofday({1207001155, 980855}, NULL) = 0
>  select(4, [3], [], [], {0, 999977})     = 0 (Timeout)
>  gettimeofday({1207001156, 980679}, NULL) = 0
>  select(4, [3], [], [], {0, 153})        = 0 (Timeout)
>
>  I tried letting them run after sending a USR1, but that doesn't seem to
>  generate any extra information.
>
>  At first, I thought it would only happen if the Mongrels were allowed to
>  sit idle - similar to the MySQL problem others reported - but then I saw
>  one go silent within a half hour of a restart.
>
>  Any suggestions?
>  --
>  Posted via http://www.ruby-forum.com/.
>  _______________________________________________
>  Mongrel-users mailing list
>  Mongrel-users at rubyforge.org
>  http://rubyforge.org/mailman/listinfo/mongrel-users
>



-- 
Evan Weaver
Cloudburst, LLC