From larseidnes at gmail.com Mon Oct 1 10:07:55 2007 From: larseidnes at gmail.com (Lars Eidnes) Date: Mon, 01 Oct 2007 16:07:55 +0200 Subject: [Mongrel] Getting started on win32 - without Rails. Message-ID: Hi, I'm trying to get started developing my first ruby-powered web-app. I wont be using Rails for this, at least I don't see any reason why I should. However, the "getting started" sections in the documentation only shows you how to start a Rails application. My mindset comes from PHP/JSP-type languages where you embed code into HTML. It seems the way to do this with ruby is to use Apache + mod_ruby + eruby. It also seems this is not the preferred way to do things. (Right?) So: How can I get a simple hello world webpage running from Mongrel, without any framework? Should I not be using Mongrel to do this? Am I asking the wrong questions? -Lars From _ at whats-your.name Mon Oct 1 11:23:09 2007 From: _ at whats-your.name (cdr) Date: Mon, 1 Oct 2007 11:23:09 -0400 Subject: [Mongrel] Getting started on win32 - without Rails. In-Reply-To: References: Message-ID: <20071001152308.GA2803@replic.net> > My mindset comes from PHP/JSP-type languages where you embed code into > HTML. It seems the way to do this with ruby is to use Apache + mod_ruby > + eruby. It also seems this is not the preferred way to do things. (Right?) its not the 'Rails' way of doing things[1], but its a way that will be alive as long as the web. even new projects, such as IBM's Project Zero "focused on the agile development of the next generation of dynamic Web applications" take this approach with a directory containing more or less one PHP file for each resource class, organized by HTTP path.. [1] the Rails way is to use synthetic URLs, and Ruby code to map those to ActiveRecord objects, which reside in SQL tables. Merb, Iowa also share similar paradigm.. you can adjust the GET method below to do something like: path = PathName.new('./resources/'+r.params["REQUEST_PATH"]) (path.exist && path.extname.match(/\.mab$/) && MarkabyHandler || Mongrel::DirHandler).new(Path).process(r,re) MarkabyHandler is an exercise to you. i'd do (Templates[path] ||= eval("lambda{"+path.read+"}")).call or similar to reuse (memoize) the parsed file.. of course you could make a controllers/ dir to match resources (renaming that to views), consolidating both to only have a file per controller/view per resource class [2], etc.. Mongrel lets you build up exactly what you need, instead of what framework authors think you need. [2] Mongrel facilitates this due to a feature of its URIClassifier. '/trinkets/3' will actually match a '/trinkets' registry. afaik you can even get the 'excess' portion of the match so you dont need to do regex of your own (it returns 3 results on a lookup). forget the details as its been a couple months since i dug into the URIClassifier code. > So: How can I get a simple hello world webpage running from Mongrel, > without any framework? setup a configurator line and a class to match, eg: #!/usr/bin/env ruby Path=File.dirname __FILE__ %w{rubygems markaby mongrel}.each{|r|require r} class Resource < Mongrel::HttpHandler def GET r,e e.start{|h,o| o.write "hello world"} end def process r,e send r.params["REQUEST_METHOD"], r, e end end if __FILE__== $0 Mongrel::Configurator.new :host => (ARGV[0] || "0.0.0.0") do listener :port => (ARGV[1] || 80) do uri '/', :handler => Resource.new run end end.join end cheers, and welcome to Mongrel land - where all the animals are ugly > > -Lars > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users From sid.webnews at gmail.com Mon Oct 1 15:44:26 2007 From: sid.webnews at gmail.com (Sid Jay) Date: Mon, 01 Oct 2007 14:44:26 -0500 Subject: [Mongrel] Error with Mongrel_cluster Message-ID: Hi Guys, I?m getting this error with mongrel_cluster. I am able to start mongrel through mongrel_rails start but the cluster is not working. Initially mongrel_rails wouldn?t work but after I added $ORACLE_HOME and $PD_LIBRARY_PATH variables it worked but mongrel_cluster isn?t working still. The mongrel log file looks like below. Any help is appreciated. Thanks. -----------begin mongrel.8000.log--------------- ** Daemonized, any open files are closed. Look at tmp/pids/mongrel.8000.pid and log/mongrel.8000.log for info. ** Starting Mongrel listening at 0.0.0.0:8000 ** Changing group to xxxxx. ** Changing user to xxxxx. ** Starting Rails with production environment... /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection _adapters/oracle_adapter.rb:536: undefined method `define_a_column' for class `OCI8::Cursor' (NameError) from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependenc ies.rb:495:in `require' from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependenc ies.rb:342:in `new_constants_in' from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependenc ies.rb:495:in `require' from /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record.rb:81 from /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record.rb:80 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require' from /usr/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/initializer.rb:162:in `require_frameworks' ... 10 levels... from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:83:in `run' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/command.rb:211:in `run' from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:243 from /usr/bin/mongrel_rails:16 --------------------------end log ------------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/f7eb9ab3/attachment.html From wayneeseguin at gmail.com Mon Oct 1 15:58:57 2007 From: wayneeseguin at gmail.com (Wayne E. Seguin) Date: Mon, 1 Oct 2007 15:58:57 -0400 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: References: Message-ID: On 10/1/07, Sid Jay wrote: > > Hi Guys, > I'm getting this error with mongrel_cluster. I am able to start mongrel > through mongrel_rails start but the cluster is not working. Initially > mongrel_rails wouldn't work but after I added $ORACLE_HOME and > $PD_LIBRARY_PATH variables it worked but mongrel_cluster isn't working > still. The mongrel log file looks like below. > Any help is appreciated. > Thanks. > Sid, See if you can startup the console without errors. Then see if you can start a single instance without errors. This does not appear to be a Mongrel related issue but instead something missing or incorrectly configured in the application. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/c1410f76/attachment.html From piyush.pr at gmail.com Mon Oct 1 16:21:19 2007 From: piyush.pr at gmail.com (Piyush Ranjan) Date: Tue, 2 Oct 2007 01:51:19 +0530 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: References: Message-ID: <325148f70710011321xe3c1291qee0c34309d770f6f@mail.gmail.com> $PD_LIBRARY_PATH or $LD_LIBRARY_PATH? or is it just a typo ? I have faced this problem is past but setting these two correctly does the trick. Are u sure u did add these variables in your .bashrc ? On 10/2/07, Sid Jay wrote: > > Hi Guys, > I'm getting this error with mongrel_cluster. I am able to start mongrel > through mongrel_rails start but the cluster is not working. Initially > mongrel_rails wouldn't work but after I added $ORACLE_HOME and > $PD_LIBRARY_PATH variables it worked but mongrel_cluster isn't working > still. The mongrel log file looks like below. > Any help is appreciated. > Thanks. > > -----------begin mongrel.8000.log--------------- > ** Daemonized, any open files are closed. Look at > tmp/pids/mongrel.8000.pid and log/mongrel.8000.log for info. > ** Starting Mongrel listening at 0.0.0.0:8000 > ** Changing group to xxxxx. > ** Changing user to xxxxx. > ** Starting Rails with production environment... > /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/oracle_adapter.rb:536: > undefined method `define_a_column' for class `OCI8::Cursor' (NameError) > from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in > `require' > from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:495:in > `require' > from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:342:in > `new_constants_in' > from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:495:in > `require' > from /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3 > /lib/active_record.rb:81 > from /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3 > /lib/active_record.rb:80 > from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in > `require' > from /usr/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/initializer.rb:162:in > `require_frameworks' > ... 10 levels... > from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:83:in > `run' > from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/command.rb:211:in > `run' > from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1 > /bin/mongrel_rails:243 > from /usr/bin/mongrel_rails:16 > > --------------------------end log ------------------------------------ > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071002/2c92e05b/attachment.html From piyush.pr at gmail.com Mon Oct 1 16:22:36 2007 From: piyush.pr at gmail.com (Piyush Ranjan) Date: Tue, 2 Oct 2007 01:52:36 +0530 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: References: Message-ID: <325148f70710011322o7b9df78dqe43aff66962af760@mail.gmail.com> ** Changing group to xxxxx. ** Changing user to xxxxx. Guess this is what the problem is Did you set those variables for those userspace also ? On 10/2/07, Sid Jay wrote: > > Hi Guys, > I'm getting this error with mongrel_cluster. I am able to start mongrel > through mongrel_rails start but the cluster is not working. Initially > mongrel_rails wouldn't work but after I added $ORACLE_HOME and > $PD_LIBRARY_PATH variables it worked but mongrel_cluster isn't working > still. The mongrel log file looks like below. > Any help is appreciated. > Thanks. > > -----------begin mongrel.8000.log--------------- > ** Daemonized, any open files are closed. Look at > tmp/pids/mongrel.8000.pid and log/mongrel.8000.log for info. > ** Starting Mongrel listening at 0.0.0.0:8000 > ** Changing group to xxxxx. > ** Changing user to xxxxx. > ** Starting Rails with production environment... > /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3/lib/active_record/connection_adapters/oracle_adapter.rb:536: > undefined method `define_a_column' for class `OCI8::Cursor' (NameError) > from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in > `require' > from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:495:in > `require' > from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:342:in > `new_constants_in' > from /usr/lib/ruby/gems/1.8/gems/activesupport-1.4.2/lib/active_support/dependencies.rb:495:in > `require' > from /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3 > /lib/active_record.rb:81 > from /usr/lib/ruby/gems/1.8/gems/activerecord-1.15.3 > /lib/active_record.rb:80 > from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in > `require' > from /usr/lib/ruby/gems/1.8/gems/rails-1.2.3/lib/initializer.rb:162:in > `require_frameworks' > ... 10 levels... > from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:83:in > `run' > from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel/command.rb:211:in > `run' > from /usr/lib/ruby/gems/1.8/gems/mongrel-1.0.1 > /bin/mongrel_rails:243 > from /usr/bin/mongrel_rails:16 > > --------------------------end log ------------------------------------ > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071002/58673021/attachment-0001.html From wayneeseguin at gmail.com Mon Oct 1 16:25:39 2007 From: wayneeseguin at gmail.com (Wayne E. Seguin) Date: Mon, 1 Oct 2007 16:25:39 -0400 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: References: Message-ID: On 10/1/07, Wayne E. Seguin wrote: > > On 10/1/07, Sid Jay wrote: > > > > Hi Guys, > > I'm getting this error with mongrel_cluster. I am able to start mongrel > > through mongrel_rails start but the cluster is not working. Initially > > mongrel_rails wouldn't work but after I added $ORACLE_HOME and > > $PD_LIBRARY_PATH variables it worked but mongrel_cluster isn't working > > still. The mongrel log file looks like below. > > Any help is appreciated. > > Thanks. > > > Also, are you also making sure that you use "export " when assigning the $ORACLE_HOME and $PD_LIBRARY_PATH variables? (assuming you're using bash) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/af40c202/attachment.html From sid.webnews at gmail.com Mon Oct 1 16:48:55 2007 From: sid.webnews at gmail.com (Sid News) Date: Mon, 1 Oct 2007 15:48:55 -0500 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: References: Message-ID: <8ce27c380710011348m418c14d5hd067ef51c5b9835e@mail.gmail.com> Guys, Thanks for the replies. This is how my .bashrc looks like -----begin .bashrc .............. >more .bashrc #/etc/profile: system-wide .profile file for the Bourne shell (sh(1)) # and Bourne compatible shells (bash(1), ksh(1), ash(1), ...). ORACLE_HOME=/usr/lib/oracle/10.2.0.3/client PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11:/usr/games:/usr/lib/oracle/10.2.0.3/client/bin LD_LIBRARY_PATH=/usr/lib/oracle/10.2.0.3/client/lib export ORACLE_HOME export LD_LIBRARY_PATH if [ "$PS1" ]; then if [ "$BASH" ]; then PS1='>' fi fi umask 022 -------------------------------------------------- I follow these instructions to setup my mongrel_cluster http://mongrel.rubyforge.org/docs/mongrel_cluster.html The mongrel_cluster and mongrel_rails both worked fine, but my supervisor installed oci8 or oracle and then when i try to start mongrel_rails it would give me the same error. Googling it led me to add the environment variables $ORACLE_HOME and $LD_LIBRARY_PATH , after that mongrel_rails works. But mongrel_cluster is still broken and I'm not sure what more do I need to do to get it back running. I must be missing something about mongrel_cluster. My setup is Ubuntu 6.06.1 LTS , nginx (webserver) ruby 1.8.4 Thanks. On 10/1/07, Wayne E. Seguin wrote: > > > > On 10/1/07, Wayne E. Seguin wrote: > > > > On 10/1/07, Sid Jay wrote: > > > > > > Hi Guys, > > > I'm getting this error with mongrel_cluster. I am able to start > > > mongrel through mongrel_rails start but the cluster is not working. > > > Initially mongrel_rails wouldn't work but after I added $ORACLE_HOME and > > > $PD_LIBRARY_PATH variables it worked but mongrel_cluster isn't working > > > still. The mongrel log file looks like below. > > > Any help is appreciated. > > > Thanks. > > > > > > > Also, are you also making sure that you use "export " when assigning the > $ORACLE_HOME and $PD_LIBRARY_PATH variables? (assuming you're using bash) > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/804131c3/attachment.html From wayneeseguin at gmail.com Mon Oct 1 16:52:50 2007 From: wayneeseguin at gmail.com (Wayne E. Seguin) Date: Mon, 1 Oct 2007 16:52:50 -0400 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: <8ce27c380710011348m418c14d5hd067ef51c5b9835e@mail.gmail.com> References: <8ce27c380710011348m418c14d5hd067ef51c5b9835e@mail.gmail.com> Message-ID: On 10/1/07, Sid News wrote: > > I follow these instructions to setup my mongrel_cluster > http://mongrel.rubyforge.org/docs/mongrel_cluster.html > > The mongrel_cluster and mongrel_rails both worked fine, but my supervisor > installed oci8 or oracle and then when i try to start mongrel_rails it would > give me the same error. Googling it led me to add the environment variables > $ORACLE_HOME and $LD_LIBRARY_PATH , after that mongrel_rails works. > But mongrel_cluster is still broken and I'm not sure what more do I need > to do to get it back running. I must be missing something about > mongrel_cluster. > > My setup is Ubuntu 6.06.1 LTS , nginx (webserver) ruby 1.8.4 > Thanks. Sid, I would try to re-install (or install if you don't have it) the oci8 adaptor to force it to rebuild against the new oracle installation. sudo gem uninstall ruby-oci8 sudo gem install ruby-oci8 http://rubyforge.org/projects/ruby-oci8/ Let us know if this works. Best, ~Wayne -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/c512f8ab/attachment.html From rogerpack2005 at gmail.com Mon Oct 1 16:59:33 2007 From: rogerpack2005 at gmail.com (Roger Pack) Date: Mon, 1 Oct 2007 14:59:33 -0600 Subject: [Mongrel] Error with Mongrel_cluster Message-ID: <966599840710011359j44d92390s2667ff5c5d7d5970@mail.gmail.com> Here's a link for some code that I once used to fight a somewhat similar problem. http://dev.rubyonrails.org/ticket/7452#comment:41 From sid.webnews at gmail.com Mon Oct 1 17:08:02 2007 From: sid.webnews at gmail.com (Sid News) Date: Mon, 1 Oct 2007 16:08:02 -0500 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: References: <8ce27c380710011348m418c14d5hd067ef51c5b9835e@mail.gmail.com> Message-ID: <8ce27c380710011408j60c21242u3418786ee264badb@mail.gmail.com> Wayne, surprisingly i'm getting this error now: --------------begin error----------------- >sudo gem install ruby-oci8 ERROR: While executing gem ... (Gem::GemNotFoundException) Could not find ruby-oci8 (> 0) in any repository ---------end error ----------------------------- On 10/1/07, Wayne E. Seguin wrote: > > On 10/1/07, Sid News wrote: > > > > I follow these instructions to setup my mongrel_cluster http://mongrel.rubyforge.org > > /docs/mongrel_cluster.html > > > > The mongrel_cluster and mongrel_rails both worked fine, but my > > supervisor installed oci8 or oracle and then when i try to start > > mongrel_rails it would give me the same error. Googling it led me to add the > > environment variables $ORACLE_HOME and $LD_LIBRARY_PATH , after that > > mongrel_rails works. > > But mongrel_cluster is still broken and I'm not sure what more do I need > > to do to get it back running. I must be missing something about > > mongrel_cluster. > > > > My setup is Ubuntu 6.06.1 LTS , nginx (webserver) ruby 1.8.4 > > Thanks. > > > Sid, > > I would try to re-install (or install if you don't have it) the oci8 > adaptor to force it to rebuild against the new oracle installation. > > sudo gem uninstall ruby-oci8 > sudo gem install ruby-oci8 > > http://rubyforge.org/projects/ruby-oci8/ > > Let us know if this works. > > Best, > > ~Wayne > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/d883d7dd/attachment.html From wayneeseguin at gmail.com Mon Oct 1 21:12:17 2007 From: wayneeseguin at gmail.com (Wayne E. Seguin) Date: Mon, 1 Oct 2007 21:12:17 -0400 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: <8ce27c380710011408j60c21242u3418786ee264badb@mail.gmail.com> References: <8ce27c380710011348m418c14d5hd067ef51c5b9835e@mail.gmail.com> <8ce27c380710011408j60c21242u3418786ee264badb@mail.gmail.com> Message-ID: On 10/1/07, Sid News wrote: > > Wayne, > surprisingly i'm getting this error now: You can download the gem manually from here and install from local: http://rubyforge.org/frs/?group_id=256&release_id=12694 The ruby-oci8 project page is here: http://ruby-oci8.rubyforge.org/en/ For example, on Linux you might do: wget http://rubyforge.org/frs/download.php/22320/ruby-oci8-1.0.0-rc3.tar.gz tar zxvf ruby-oci8-1.0.0-rc3.tar.gz cd ruby-oci8-1.0.0-rc3 make sudo make install I hope this helps. ~Wayne -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071001/0c5d0249/attachment-0001.html From ctmailinglists at googlemail.com Tue Oct 2 05:08:42 2007 From: ctmailinglists at googlemail.com (Chris T) Date: Tue, 02 Oct 2007 10:08:42 +0100 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? Message-ID: <47020A9A.5080609@googlemail.com> I've been trying to track down the culprit of erratic behaviour and crashes on my production server (which is split into a number of Xen instances), so set up a staging server so that I could really try to get to the bottom of it. The staging server (also split with Xen) is set up pretty much identically as far as the mongrel_cluster server is concerned (the production box has two servers/instances for mongrel_cluster each with 4 mongrels per cluster vs one server on the staging box with 2 mongrels per cluster -- otherwise they are the same: both Debian Etch, same gems, same Ruby - 1.8.5, same mongrel versions - 1.0.1, same mongrel_cluster - 1.0.2). Oh, and the data on the staging one is a clone of the production one. On the production box, pretty much immediately (and I'm talking about within one or two requests), the mongrels climb up to about 150-160MB. One the staging server, even when I'm hammering it with a benchmarking suite (have tried with httperf, and bench and crawl) the mongrels sit comfortably at about 60MB each. Here's some stats from top: Production server: top - 08:38:43 up 22 days, 13:21, 1 user, load average: 0.24, 0.23, 0.15 Tasks: 33 total, 3 running, 30 sleeping, 0 stopped, 0 zombie Cpu(s): 16.6%us, 24.5%sy, 0.0%ni, 51.0%id, 0.0%wa, 0.0%hi, 3.6%si, 4.3%st Mem: 524288k total, 384528k used, 139760k free, 1576k buffers Swap: 262136k total, 1352k used, 260784k free, 19644k cached and from ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND mongrel 20387 2.6 18.5 148780 97228 ? Sl 08:36 0:11 /usr/bin/ruby1.8 /usr/bin/mongrel_rail mongrel 20390 2.8 19.4 153396 101940 ? Sl 08:36 0:13 /usr/bin/ruby1.8 /usr/bin/mongrel_rail mongrel 20393 4.1 19.6 154356 102936 ? Sl 08:36 0:18 /usr/bin/ruby1.8 /usr/bin/mongrel_rail mongrel 20396 4.7 18.8 150124 98576 ? Rl 08:36 0:21 /usr/bin/ruby1.8 /usr/bin/mongrel_rail And here's the corresponding staging server ones: top - 10:00:55 up 15:22, 1 user, load average: 0.00, 0.01, 0.15 Tasks: 29 total, 2 running, 27 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni, 99.0%id, 0.0%wa, 0.0%hi, 0.0%si, 1.0%st Mem: 262336k total, 203064k used, 59272k free, 1156k buffers Swap: 262136k total, 4k used, 262132k free, 58636k cached USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND mongrel 3617 7.2 16.8 58808 44296 ? Sl 09:28 2:28 /usr/bin/ruby1.8 /usr/bin/mongrel_rail mongrel 3620 7.1 16.7 58424 43912 ? Sl 09:28 2:26 /usr/bin/ruby1.8 /usr/bin/mongrel_rail Anybody got any suggestions, or at least thoughts on tracking down the issues? Thanks, Chris -- ---------------------------- Autopendium :: Stuff about old cars http://autopendium.com From johan at johansorensen.com Tue Oct 2 05:27:20 2007 From: johan at johansorensen.com (=?ISO-8859-1?Q?Johan_S=F8rensen?=) Date: Tue, 2 Oct 2007 11:27:20 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: <47020A9A.5080609@googlemail.com> References: <47020A9A.5080609@googlemail.com> Message-ID: <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> On Oct 2, 2007, at 11:08 AM, Chris T wrote: > The staging server (also split with Xen) is set up pretty much > identically > > On the production box, pretty much immediately (and I'm talking about > within one or two requests), the mongrels climb up to about 150-160MB. > One the staging server, even when I'm hammering it with a benchmarking > suite (have tried with httperf, and bench and crawl) the mongrels sit > comfortably at about 60MB each. Same box, hardware+kernel wise (32 vs 64 bit)? Libraries built the same way? JS From dave at cheney.net Tue Oct 2 06:06:36 2007 From: dave at cheney.net (Dave Cheney) Date: Tue, 2 Oct 2007 06:06:36 -0400 Subject: [Mongrel] Mongrel using way more memory on production thanstaging. Any ideas why? In-Reply-To: <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> References: <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> Message-ID: <2034dbd20cd61125d50a1cedf42c1474@localhost> >> The staging server (also split with Xen) is set up pretty much >> identically >> >> On the production box, pretty much immediately (and I'm talking about >> within one or two requests), the mongrels climb up to about 150-160MB. >> One the staging server, even when I'm hammering it with a benchmarking >> suite (have tried with httperf, and bench and crawl) the mongrels sit >> comfortably at about 60MB each. Your issue may be related to the amount of data in your production database. In our codebase we're constantly be hit by AR calls that bring the whole table into memory, then filter out the bits they don't want rails side. In development we don't notice the bloat or delay but under a production dataset the effect was pronounced. Cheers Dave From ctmailinglists at googlemail.com Tue Oct 2 06:31:30 2007 From: ctmailinglists at googlemail.com (Chris T) Date: Tue, 02 Oct 2007 11:31:30 +0100 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> Message-ID: <47021E02.2000905@googlemail.com> Johan S?rensen wrote: > > On Oct 2, 2007, at 11:08 AM, Chris T wrote: > >> The staging server (also split with Xen) is set up pretty much >> identically >> >> On the production box, pretty much immediately (and I'm talking about >> within one or two requests), the mongrels climb up to about 150-160MB. >> One the staging server, even when I'm hammering it with a benchmarking >> suite (have tried with httperf, and bench and crawl) the mongrels sit >> comfortably at about 60MB each. > > Same box, hardware+kernel wise (32 vs 64 bit)? Libraries built the > same way? > > JS > > No, the production box is Athlon64 X2 3800, the staging one i386 (actually a Pentium 4 I had lying around). However the ruby libraries were built the same way: sudo apt-get install ruby1.8 libzlib-ruby rdoc irb ruby1.8-dev libopenssl-ruby1.8 On production: #ruby -v ruby 1.8.5 (2006-08-25) [x86_64-linux] On staging: #ruby -v ruby 1.8.5 (2006-08-25) [i486-linux] -- ---------------------------- Autopendium :: Stuff about old cars http://autopendium.com From lists at ruby-forum.com Tue Oct 2 06:35:53 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Tue, 2 Oct 2007 12:35:53 +0200 Subject: [Mongrel] Mongrel using way more memory on production thanstaging. In-Reply-To: <2034dbd20cd61125d50a1cedf42c1474@localhost> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <2034dbd20cd61125d50a1cedf42c1474@localhost> Message-ID: <9547f6bad80fba92abba9a5f2fd70926@ruby-forum.com> Dave Cheney wrote: >>> The staging server (also split with Xen) is set up pretty much >>> identically >>> >>> On the production box, pretty much immediately (and I'm talking about >>> within one or two requests), the mongrels climb up to about 150-160MB. >>> One the staging server, even when I'm hammering it with a benchmarking >>> suite (have tried with httperf, and bench and crawl) the mongrels sit >>> comfortably at about 60MB each. > > Your issue may be related to the amount of data in your production > database. In our codebase we're constantly be hit by AR calls that bring > the whole table into memory, then filter out the bits they don't want > rails > side. > > In development we don't notice the bloat or delay but under a production > dataset the effect was pronounced. > > Cheers > > Dave The staging server is using a copy of the production database, so that can't be it (though I thought it might be before setting up the staging server). -- Posted via http://www.ruby-forum.com/. From dee.zsombor at gmail.com Tue Oct 2 10:56:11 2007 From: dee.zsombor at gmail.com (Dee Zsombor) Date: Tue, 02 Oct 2007 17:56:11 +0300 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: <47021E02.2000905@googlemail.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> Message-ID: <47025C0B.1000701@gmail.com> Chris T wrote: > No, the production box is Athlon64 X2 3800, the staging one i386 > (actually a Pentium 4 I had lying around). However the ruby libraries > were built the same way: > > sudo apt-get install ruby1.8 libzlib-ruby rdoc irb ruby1.8-dev > libopenssl-ruby1.8 > > On production: > #ruby -v > ruby 1.8.5 (2006-08-25) [x86_64-linux] > > On staging: > #ruby -v > ruby 1.8.5 (2006-08-25) [i486-linux] This explains it well, pointers are 8 byte long on 64 bit while only 4 byte long on 32 bit. Each ruby object is essentially a pointer (plus some other magic encoded in the byte alignment and misc other data). A ruby process is bound to be more heavy on 64 bit. You can use a 64 bit db server and keep the application servers 32, this should give some benefits while trading in some uniformity. -- Company - http://primalgrasp.com Thoughts - http://deezsombor.blogspot.com From cremes.devlist at mac.com Tue Oct 2 10:56:34 2007 From: cremes.devlist at mac.com (Chuck Remes) Date: Tue, 2 Oct 2007 09:56:34 -0500 Subject: [Mongrel] performance observation on redhat In-Reply-To: <20070923104135.e7c28jtlno4w8wcc@lcgalaska.com> References: <20070923063054.43DE710A3D2@ws6-4.us4.outblaze.com> <0451432B-F007-443E-9831-F9B26B028F1A@gmail.com> <20070923104135.e7c28jtlno4w8wcc@lcgalaska.com> Message-ID: On Sep 23, 2007, at 1:41 PM, barsalou wrote: > Armin, > > Since your goofing around with that, how about some of the other > settings like maxtime and ttl values? > > Seems like you have deleayd when some garbage collection operations > are > taking place, so maybe tweeking that a little more will give you the > performance your looking for. > > I've never used this feature, but thought that might interesting as > well. > > Mike B. > > Quoting "Wayne E. Seguin" : > >> On Sep 23, 2007, at 02:30 , armin roehrl wrote: >>> Hi, >>> >>> I made an interesting observation using webservers (not just >>> mongrel) under red hat enterprise >>> linux ES release 4 (Nahant Update 5). Maybe this is helpful or >>> somebody with deeper networking >>> expertise can comment on this. >>> >>> [snip] >>> echo 500000 > /proc/sys/net/ipv4/inet_peer_threshold >>> >>> There is a trade-off here. A too small value causes too many delays >>> from inet-peer-storage cleaning >>> and a too big value makes life well for some limited time, but when >>> it hits you, it becomes really expensive. >>> >>> Did you ever see this? >>> >>> Thanks, >>> -Armin >> >> Armin, >> >> We might put this in the documentation, will discuss with the dev >> team. I've searched all over the place to confirm this issue with RHEL 4 update 5 and have come up empty. What's the original source of the "fix" ? Also, any suggestions on how to build a test harness to confirm new values actually *improve* the situation rather than make it worse? cr From wayneeseguin at gmail.com Tue Oct 2 12:44:12 2007 From: wayneeseguin at gmail.com (Wayne E. Seguin) Date: Tue, 2 Oct 2007 12:44:12 -0400 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: <47021E02.2000905@googlemail.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> Message-ID: On 10/2/07, Chris T wrote: > > Johan S?rensen wrote: > > > > On Oct 2, 2007, at 11:08 AM, Chris T wrote: > > > >> The staging server (also split with Xen) is set up pretty much > >> identically > >> > >> On the production box, pretty much immediately (and I'm talking about > >> within one or two requests), the mongrels climb up to about 150-160MB. > >> One the staging server, even when I'm hammering it with a benchmarking > >> suite (have tried with httperf, and bench and crawl) the mongrels sit > >> comfortably at about 60MB each. > > > > Same box, hardware+kernel wise (32 vs 64 bit)? Libraries built the > > same way? > > > > JS > > > > > No, the production box is Athlon64 X2 3800, the staging one i386 > (actually a Pentium 4 I had lying around). However the ruby libraries > were built the same way: > > sudo apt-get install ruby1.8 libzlib-ruby rdoc irb ruby1.8-dev > libopenssl-ruby1.8 > > On production: > #ruby -v > ruby 1.8.5 (2006-08-25) [x86_64-linux] > > On staging: > #ruby -v > ruby 1.8.5 (2006-08-25) [i486-linux] > I have not personally encountered this issue, however I also do not use apt-get to install anything in the Ruby application stack. I compile everything by hand. I'd recommend trying that next (compile-install Ruby and RubyGems) on the production server. Most production servers I work with are also 64 bit Linux boxes and as I said, I compile everything and haven't seen this issue. I hope that this helps, ~Wayne -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071002/1dbe9356/attachment.html From alexey.verkhovsky at gmail.com Tue Oct 2 13:07:00 2007 From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky) Date: Tue, 2 Oct 2007 11:07:00 -0600 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: <47020A9A.5080609@googlemail.com> References: <47020A9A.5080609@googlemail.com> Message-ID: <3945c4270710021007p67e327a6l8e8932bd7c913774@mail.gmail.com> On 10/2/07, Chris T wrote: > On the production box, pretty much immediately (and I'm talking about > within one or two requests), the mongrels climb up to about 150-160MB. > One the staging server, even when I'm hammering it with a benchmarking > suite (have tried with httperf, and bench and crawl) the mongrels sit > comfortably at about 60MB each. It is the difference between x86_64 and i386, mostly. Some of it may also be that you are not hitting Mongrel hard enough in staging. An overloaded Mongrel with default --num-procs setting of 1024 should eventually allocate enough memory for 1024 request handler threads, which takes even a skeleton Rails app above 60 Mb level. -- Alexey Verkhovsky CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com] RubyWorks [http://rubyworks.thoughtworks.com] From lists at ruby-forum.com Tue Oct 2 15:21:53 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Tue, 2 Oct 2007 21:21:53 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <47025C0B.1000701@gmail.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <47025C0B.1000701@gmail.com> Message-ID: <856fa1fb938b79b13c8fbb81e2170dff@ruby-forum.com> Dee Zsombor wrote: > Chris T wrote: >> >> On staging: >> #ruby -v >> ruby 1.8.5 (2006-08-25) [i486-linux] > > > This explains it well, pointers are 8 byte long on 64 bit while only 4 > byte long on 32 bit. Each ruby object is essentially a pointer (plus > some > other magic encoded in the byte alignment and misc other data). A ruby > process is bound to be more heavy on 64 bit. > > You can use a 64 bit db server and keep the application servers 32, this > should give some benefits while trading in some uniformity. OK. I think I get this. How do i ensure the app servers are 32 bit. Do I need to use a 32-bit kernel when creating up the instances? -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Tue Oct 2 15:23:18 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Tue, 2 Oct 2007 21:23:18 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> Message-ID: <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> Wayne E. Seguin wrote: > On 10/2/07, Chris T wrote: >> >> One the staging server, even when I'm hammering it with a benchmarking >> (actually a Pentium 4 I had lying around). However the ruby libraries >> #ruby -v >> ruby 1.8.5 (2006-08-25) [i486-linux] >> > > I have not personally encountered this issue, however I also do not use > apt-get to install anything in the Ruby application stack. I compile > everything by hand. I'd recommend trying that next (compile-install Ruby > and > RubyGems) on the production server. > > Most production servers I work with are also 64 bit Linux boxes and as I > said, I compile everything and haven't seen this issue. > > I hope that this helps, > > ~Wayne I may try that -- set up a test instance same as other ones but with Ruby compiled by hand. Did it once before and seem to remember it wasn't too bad. -- Posted via http://www.ruby-forum.com/. From evan at cloudbur.st Tue Oct 2 16:09:45 2007 From: evan at cloudbur.st (Evan Weaver) Date: Tue, 2 Oct 2007 16:09:45 -0400 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: <47025C0B.1000701@gmail.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <47025C0B.1000701@gmail.com> Message-ID: I really doubt the pointers are giving you 120% more memory usage. Overhead from switching from 32bit to 64bit addressing is usually about 15%. Evan On 10/2/07, Dee Zsombor wrote: > Chris T wrote: > > No, the production box is Athlon64 X2 3800, the staging one i386 > > (actually a Pentium 4 I had lying around). However the ruby libraries > > were built the same way: > > > > sudo apt-get install ruby1.8 libzlib-ruby rdoc irb ruby1.8-dev > > libopenssl-ruby1.8 > > > > On production: > > #ruby -v > > ruby 1.8.5 (2006-08-25) [x86_64-linux] > > > > On staging: > > #ruby -v > > ruby 1.8.5 (2006-08-25) [i486-linux] > > > This explains it well, pointers are 8 byte long on 64 bit while only 4 > byte long on 32 bit. Each ruby object is essentially a pointer (plus some > other magic encoded in the byte alignment and misc other data). A ruby > process is bound to be more heavy on 64 bit. > > You can use a 64 bit db server and keep the application servers 32, this > should give some benefits while trading in some uniformity. > > > -- > Company - http://primalgrasp.com > Thoughts - http://deezsombor.blogspot.com > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Evan Weaver Cloudburst, LLC From alexey.verkhovsky at gmail.com Tue Oct 2 16:39:07 2007 From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky) Date: Tue, 2 Oct 2007 14:39:07 -0600 Subject: [Mongrel] Mongrel using way more memory on production than staging. Any ideas why? In-Reply-To: References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <47025C0B.1000701@gmail.com> Message-ID: <3945c4270710021339h7473be51na1414850b2f6dc32@mail.gmail.com> On 10/2/07, Evan Weaver wrote: > I really doubt the pointers are giving you 120% more memory usage. > Overhead from switching from 32bit to 64bit addressing is usually > about 15%. Usually, yes. And yet RSS of a Mongrel process running a skeleton Rails app is ~20 Mb on i386 and ~32 Mb on x86_64. -- Alexey Verkhovsky CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com] RubyWorks [http://rubyworks.thoughtworks.com] From wayneeseguin at gmail.com Tue Oct 2 17:23:47 2007 From: wayneeseguin at gmail.com (Wayne E. Seguin) Date: Tue, 2 Oct 2007 17:23:47 -0400 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> Message-ID: This is something like what I use, modify to your own designs: mkdir ~/src package=ruby cd ~/src && curl -O ftp://ftp.ruby-lang.org/pub/ruby/stable-snapshot.tar.gz tar zxf stable-snapshot.tar.gz && cd ruby && ./configure --prefix=/usr/local --disable-pthread --with-readline-dir=/usr/local make && sudo make install && cd ~/src -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071002/a709f7b4/attachment.html From lists at ruby-forum.com Tue Oct 2 18:15:13 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Wed, 3 Oct 2007 00:15:13 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> Message-ID: <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> Wayne E. Seguin wrote: > This is something like what I use, modify to your own designs: > > mkdir ~/src > > package=ruby > > cd ~/src && curl -O > ftp://ftp.ruby-lang.org/pub/ruby/stable-snapshot.tar.gz > > tar zxf stable-snapshot.tar.gz && cd ruby && ./configure > --prefix=/usr/local > --disable-pthread --with-readline-dir=/usr/local > > make && sudo make install && cd ~/src OK. Will try a few things out tomorrow and report back. -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Wed Oct 3 09:12:01 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Wed, 3 Oct 2007 15:12:01 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> Message-ID: <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> Chris Taggart wrote: > Wayne E. Seguin wrote: >> This is something like what I use, modify to your own designs: >> >> mkdir ~/src >> >> package=ruby >> >> cd ~/src && curl -O >> ftp://ftp.ruby-lang.org/pub/ruby/stable-snapshot.tar.gz >> >> tar zxf stable-snapshot.tar.gz && cd ruby && ./configure >> --prefix=/usr/local >> --disable-pthread --with-readline-dir=/usr/local >> >> make && sudo make install && cd ~/src > > OK. Will try a few things out tomorrow and report back. Update on what I've tried/found out so far. Set up a testbed xen server instance on the same box (AMD64x2) as my mongrel clusters. It's not quite a direct comparison as I compiled Ruby from source, whereas the staging server was installed from binaries via apt-get. I then ran stand alone mongrel in the same production environment with the same data on both server, and threw various requests at it via curl. The results are here: http://spreadsheets.google.com/pub?key=pHDcT1cywdrovBLlL7QV8Ow but briefly: -- on startup (before any requests have been made), the testbed (64-bit) server uses 87% more memory than the staging one (78MB vs 42MB) -- on the first request, both increase memory usage by 17%, and then both gradually climb by varying percentages, though the testbed's mem usage is always 80-95% more than the 32-bit staging server. -- memory usage increases inconsistently, sometimes staying the same when the requests are repeated, sometimes increasing. I'm wondering if this could be a memory leak to do with caching (which might explain why it wasn't showing up when I was httperf'ing the staging server). There's obviously more work to be done here, but all comments and suggestions are welcome. This is new territory for me, so I may be missing something completely obvious. Cheers Chris -- Posted via http://www.ruby-forum.com/. From zedshaw at zedshaw.com Wed Oct 3 11:59:52 2007 From: zedshaw at zedshaw.com (Zed A. Shaw) Date: Wed, 3 Oct 2007 11:59:52 -0400 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> Message-ID: <20071003115952.08217eef.zedshaw@zedshaw.com> On Wed, 3 Oct 2007 15:12:01 +0200 Chris Taggart wrote: > Update on what I've tried/found out so far. > > http://spreadsheets.google.com/pub?key=pHDcT1cywdrovBLlL7QV8Ow What's funny is I've been stuggling with JRuby Rails applications running under *any* application server requring 2G (yes gigabytes) of RAM just to function. Many times we'd have to do complete restarts about 1 or 2 times a week. So it's strange to hear someone say that 120M is too much. :-) -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/ From wyhaines at gmail.com Wed Oct 3 12:35:08 2007 From: wyhaines at gmail.com (Kirk Haines) Date: Wed, 3 Oct 2007 09:35:08 -0700 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> Message-ID: On 10/3/07, Chris Taggart wrote: > -- on startup (before any requests have been made), the testbed (64-bit) > server uses 87% more memory than the staging one (78MB vs 42MB) This appears to be in the reasonable ballpark of what to expect, based on my testing with a bunch of different frameworks. > -- on the first request, both increase memory usage by 17%, and then > both gradually climb by varying percentages, though the testbed's mem > usage is always 80-95% more than the 32-bit staging server. The initial bump is probably a factor of Rails and of your application's object initializations. An ongoing memory increase, though, is caused either by your application intentionally caching things in RAM, or by something leaking. RAM usage should stay pretty stable. > -- memory usage increases inconsistently, sometimes staying the same > when the requests are repeated, sometimes increasing. I'm wondering if > this could be a memory leak to do with caching (which might explain why > it wasn't showing up when I was httperf'ing the staging server). It could be. It could be that something your app is using is leaking, or that you are hitting a memory leak in Ruby. As of 1.8.6, the Array#shift bug is fixed, but there are probably others. I demonstrated a leak a few months ago in one of the get*by* methods. gethostbyname, IIRC, and I have no doubt that if one took the time to look into this intently, other leaky libs could be found. Do you use rmagick? Historically, people have had a lot of memory leak issues with it. > There's obviously more work to be done here, but all comments and > suggestions are welcome. This is new territory for me, so I may be > missing something completely obvious. Tracking down memory leaks in Ruby is a labor intensive process. Good luck. Kirk Haines From lists at ruby-forum.com Wed Oct 3 14:52:20 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Wed, 3 Oct 2007 20:52:20 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <20071003115952.08217eef.zedshaw@zedshaw.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> <20071003115952.08217eef.zedshaw@zedshaw.com> Message-ID: <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> Zed A. Shaw wrote: > On Wed, 3 Oct 2007 15:12:01 +0200 > Chris Taggart wrote: > >> Update on what I've tried/found out so far. >> >> http://spreadsheets.google.com/pub?key=pHDcT1cywdrovBLlL7QV8Ow > > What's funny is I've been stuggling with JRuby Rails applications > running under *any* application server requring 2G (yes gigabytes) of > RAM just to function. Many times we'd have to do complete restarts > about 1 or 2 times a week. > > So it's strange to hear someone say that 120M is too much. :-) > > -- > Zed A. Shaw > - Hate: http://savingtheinternetwithhate.com/ > - Good: http://www.zedshaw.com/ > - Evil: http://yearofevil.com/ The problem isn't 120M -- but that it seems to keep climbing until much instability ensues... That's possibly a leak, but I'm still a bit confused as to why the 64-bit server is using so much more... and whether I should maybe run the mongrel_cluster instances on 32-bit kernel (assuming that's possible). -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Wed Oct 3 15:05:47 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Wed, 3 Oct 2007 21:05:47 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> Message-ID: <7a1c5334d7ac6ca7a91ba11f4ab934d8@ruby-forum.com> Kirk Haines wrote: > On 10/3/07, Chris Taggart wrote: >> -- on startup (before any requests have been made), the testbed (64-bit) >> server uses 87% more memory than the staging one (78MB vs 42MB) > > This appears to be in the reasonable ballpark of what to expect, based > on my testing with a bunch of different frameworks. So just to clarify, the 80%+ is not uncommon for 64-bit vs 32-bit. Any benefits (as far as mongrel is concerned) to being on 64-bt, or (if it's possible) is it worth running the app servers on 32-bit kernels? > >> -- on the first request, both increase memory usage by 17%, and then >> both gradually climb by varying percentages, though the testbed's mem >> usage is always 80-95% more than the 32-bit staging server. > > The initial bump is probably a factor of Rails and of your > application's object initializations. > That's what I figured > An ongoing memory increase, though, is caused either by your > application intentionally caching things in RAM, or by something > leaking. RAM usage should stay pretty stable. > The caching is all done using fragment caching (on disk -- haven't yet investigated memcached). >> -- memory usage increases inconsistently, sometimes staying the same >> when the requests are repeated, sometimes increasing. I'm wondering if >> this could be a memory leak to do with caching (which might explain why >> it wasn't showing up when I was httperf'ing the staging server). > > It could be. It could be that something your app is using is leaking, > or that you are hitting a memory leak in Ruby. As of 1.8.6, the > Array#shift bug is fixed, but there are probably others. I > demonstrated a leak a few months ago in one of the get*by* methods. > gethostbyname, IIRC, and I have no doubt that if one took the time to > look into this intently, other leaky libs could be found. Do you use > rmagick? Historically, people have had a lot of memory leak issues > with it. > I'd read about the Array#shift bug and had removed it everywhere but one place (must do that), but doing a search of the app see that it's used elsewhere, including a number of plugins and rails itself. I ditched rmagick some time ago due to concerns about it and no use ImageScience. >> There's obviously more work to be done here, but all comments and >> suggestions are welcome. This is new territory for me, so I may be >> missing something completely obvious. > > Tracking down memory leaks in Ruby is a labor intensive process. Good > luck. > Any suggestions for best practice/howtos/etc? > > Kirk Haines -- Posted via http://www.ruby-forum.com/. From zedshaw at zedshaw.com Wed Oct 3 16:15:26 2007 From: zedshaw at zedshaw.com (Zed A. Shaw) Date: Wed, 3 Oct 2007 16:15:26 -0400 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> <20071003115952.08217eef.zedshaw@zedshaw.com> <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> Message-ID: <20071003161526.374fb911.zedshaw@zedshaw.com> On Wed, 3 Oct 2007 20:52:20 +0200 Chris Taggart wrote: > Zed A. Shaw wrote: > The problem isn't 120M -- but that it seems to keep climbing until much > instability ensues... > > That's possibly a leak, but I'm still a bit confused as to why the > 64-bit server is using so much more... and whether I should maybe run > the mongrel_cluster instances on 32-bit kernel (assuming that's > possible). I haven't tracked all the different things you've done, but have you tried: 1) Running with mongrel_rails -B and looking in the log/mongrel_debug/* files? Specifically objects.log 2) Trying bleakhouse? 3) Running it on jruby? See if you still have the leak there. If you've got a leak under jruby then it's your code dude. Apart from that, I've got no idea. Last time I dealt with this crap with the horrible Ruby GC implementation the entire Ruby world took out torches and chased me down the street screaming that I was ruining their party be exposing how crap the code is. But hey, that's just me. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/ From wyhaines at gmail.com Wed Oct 3 16:26:17 2007 From: wyhaines at gmail.com (Kirk Haines) Date: Wed, 3 Oct 2007 13:26:17 -0700 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <20071003161526.374fb911.zedshaw@zedshaw.com> References: <47020A9A.5080609@googlemail.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> <20071003115952.08217eef.zedshaw@zedshaw.com> <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> <20071003161526.374fb911.zedshaw@zedshaw.com> Message-ID: On 10/3/07, Zed A. Shaw wrote: > Apart from that, I've got no idea. Last time I dealt with this crap with the > horrible Ruby GC implementation the entire Ruby world took out > torches and chased me down the street screaming that I was ruining > their party be exposing how crap the code is. Hey now. I was one of those torch carriers, but all I was worried about was making sure we were pointing in the right direction in identifying the real memory leak instead of just villifying poor, innocent Mutex. In the end, it was the influence of that dastardly Array#shift that had turned Mutex into the problem causing bad boy. What's really appalling is how long it took after that time before Array#shift was actually fixed in a ruby release. It should have been fixed in 1.8.5. I suspect that the leak in the gethostbyname code is a similar sort of sloppiness that had been overlooked (and probably still exists in 1.9). Chris, you asked about suggestions on how track down memory leaks. What Zed said. In addition, you can manually write code to check ObjectSpace object counts. If you suspect that the problem is actually at Ruby or C/C++ extension level, you can also use a tool like valgrind to analyze running code and see if you can pinpoint anything that is actualy a problem. Kirk Haines From lists at ruby-forum.com Wed Oct 3 16:32:43 2007 From: lists at ruby-forum.com (Roger Pack) Date: Wed, 3 Oct 2007 22:32:43 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> <20071003115952.08217eef.zedshaw@zedshaw.com> <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> Message-ID: > The problem isn't 120M -- but that it seems to keep climbing until much > instability ensues... > > That's possibly a leak, but I'm still a bit confused as to why the > 64-bit server is using so much more... and whether I should maybe run > the mongrel_cluster instances on 32-bit kernel (assuming that's > possible). Could try googl'ing for ruby memory profiler and possibly (if desperate) running ruby with valgrind. GL! -Roger -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Wed Oct 3 16:59:23 2007 From: lists at ruby-forum.com (Chris Taggart) Date: Wed, 3 Oct 2007 22:59:23 +0200 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> <20071003115952.08217eef.zedshaw@zedshaw.com> <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> <20071003161526.374fb911.zedshaw@zedshaw.com> Message-ID: <68276713504a5f8c0d3e13d9150d23e8@ruby-forum.com> Kirk Haines wrote: > On 10/3/07, Zed A. Shaw wrote: > >> Apart from that, I've got no idea. Last time I dealt with this crap with the >> horrible Ruby GC implementation the entire Ruby world took out >> torches and chased me down the street screaming that I was ruining >> their party be exposing how crap the code is. > > Hey now. I was one of those torch carriers, but all I was worried > about was making sure we were pointing in the right direction in > identifying the real memory leak instead of just villifying poor, > innocent Mutex. In the end, it was the influence of that dastardly > Array#shift that had turned Mutex into the problem causing bad boy. > > What's really appalling is how long it took after that time before > Array#shift was actually fixed in a ruby release. It should have been > fixed in 1.8.5. > > I suspect that the leak in the gethostbyname code is a similar sort of > sloppiness that had been overlooked (and probably still exists in > 1.9). > > Chris, you asked about suggestions on how track down memory leaks. > What Zed said. In addition, you can manually write code to check > ObjectSpace object counts. If you suspect that the problem is > actually at Ruby or C/C++ extension level, you can also use a tool > like valgrind to analyze running code and see if you can pinpoint > anything that is actualy a problem. > > > Kirk Haines Thanks for all the suggestions. Will try them out, and perhaps try some direct comparisons of 1.8.5 and 1.8.6 to see how the Array#shift pro is affecting things. Anybody got any thoughts re the idea of running the app servers under 32-bit kernel? May seem naive, but would seem to improve the memory profile. -- Posted via http://www.ruby-forum.com/. From ml at syntheticplayground.com Thu Oct 4 02:37:12 2007 From: ml at syntheticplayground.com (Corey Jewett) Date: Wed, 3 Oct 2007 23:37:12 -0700 Subject: [Mongrel] Mongrel using way more memory on production than staging In-Reply-To: <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> References: <47020A9A.5080609@googlemail.com> <154F54F6-63DB-4C0A-AF3B-BED65136F4AF@johansorensen.com> <47021E02.2000905@googlemail.com> <74f8c4f45c848a72d39f6f258628d8f4@ruby-forum.com> <067354a69ac7d64ccd5c202218ede245@ruby-forum.com> <5fa4f4f91a463cf08f87851d6975e77c@ruby-forum.com> <20071003115952.08217eef.zedshaw@zedshaw.com> <28135f2b83f4ec97d8b1dd2f3d5c25c1@ruby-forum.com> Message-ID: <20071003233712926973.6d48faef@syntheticplayground.com> On Wed, 3 Oct 2007 20:52:20 +0200, Chris Taggart wrote: > That's possibly a leak, but I'm still a bit confused as to why the > 64-bit server is using so much more... and whether I should maybe run > the mongrel_cluster instances on 32-bit kernel (assuming that's > possible). On the Java side of the world I have an app that consumes 40% more memory running on 64-bit over 32-bit. This is under 64-bit linux. Our JVMs average around 5G so 32-bit kernels aren't really an option. (32-bit Java has a 2G per jvm cap). Corey From sid.webnews at gmail.com Thu Oct 4 14:05:01 2007 From: sid.webnews at gmail.com (Sid Jay) Date: Thu, 04 Oct 2007 13:05:01 -0500 Subject: [Mongrel] Error with Mongrel_cluster In-Reply-To: Message-ID: Wayne, I tried the below and still it won?t work. On 10/1/07 8:12 PM, "Wayne E. Seguin" wrote: > On 10/1/07, Sid News wrote: >> Wayne, >> surprisingly i'm getting this error now: > > You can download the gem manually from here and install from local: > http://rubyforge.org/frs/?group_id=256&release_id=12694 > > > The ruby-oci8 project page is here: > http://ruby-oci8.rubyforge.org/en/ > > > For example, on Linux you might do: > wget http://rubyforge.org/frs/download.php/22320/ruby-oci8-1.0.0-rc3.tar.gz > > tar zxvf ruby-oci8-1.0.0-rc3.tar.gz > cd ruby-oci8-1.0.0-rc3 > make > sudo make install > > > > I hope this helps. > > ~Wayne > > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071004/3498f536/attachment.html From rogerpack2005 at gmail.com Fri Oct 5 12:43:46 2007 From: rogerpack2005 at gmail.com (Roger Pack) Date: Fri, 5 Oct 2007 10:43:46 -0600 Subject: [Mongrel] multi threaded theoretically useful? Message-ID: <966599840710050943o53510e2am582b8b2910ee348e@mail.gmail.com> Thanks to all for your helpful answers on this question. It's nice to belong to a mailing list that has smart users (not that others don't, of course). So overall the answer to my question is that a multi-threaded rails app might well not be significantly faster because say it has 10 threads, if 1 thread is waiting on a long-taking DB query, the other 9 are blocked anyway, so the benefits of a multi-threaded model are moot (so a single-threaded rails is perhaps about as fast as a multi-threaded rails). Bring on Ruby 2.0 :) Further question: would providing the option of having mongrel bind to UNIX sockets perhaps speed it? (i.e. bind to port 3000 and unix socket x). Just a thought. Probably not enough speed difference to matter, but perhaps useful for local processes and simple to implement. Thoughts? > Here's one question that I think would have an impact on how useful > green threads are for many server apps: > > If you are blocking on something like a database SELECT in one green > thread, will the Ruby interpreter switch to another green thread > while it's waiting for the response? > > Or does a blocking request block all green threads? >When the flow of execution leaves Ruby and goes into some 3rd party >extension (like that of a DB library), then ruby can't context switch. >So yes, when one does a DB query that is going through a C extension, >all of the other green threads are blocked, so that latency can't be >captured. >If it could be, then yes, there would be some benefit from some number >of green threads per process for typical web apps. From wyhaines at gmail.com Fri Oct 5 12:55:37 2007 From: wyhaines at gmail.com (Kirk Haines) Date: Fri, 5 Oct 2007 09:55:37 -0700 Subject: [Mongrel] multi threaded theoretically useful? In-Reply-To: <966599840710050943o53510e2am582b8b2910ee348e@mail.gmail.com> References: <966599840710050943o53510e2am582b8b2910ee348e@mail.gmail.com> Message-ID: On 10/5/07, Roger Pack wrote: > Thanks to all for your helpful answers on this question. It's nice to > belong to a mailing list that has smart users (not that others don't, > of course). > So overall the answer to my question is that a multi-threaded rails > app might well not be significantly faster because say it has 10 > threads, if 1 thread is waiting on a long-taking DB query, the other 9 > are blocked anyway, so the benefits of a multi-threaded model are moot > (so a single-threaded rails is perhaps about as fast as a > multi-threaded rails). Bring on Ruby 2.0 :) Multi-threaded may even be slower, in fact. :) In a lot of the cases, it will definitely be slower. Even on a platform with native threads, multithreading may be no faster than a multiprocess model. It's not a magic bullet. > Further question: > would providing the option of having mongrel bind to UNIX sockets > perhaps speed it? (i.e. bind to port 3000 and unix socket x). Just a > thought. Probably not enough speed difference to matter, but perhaps > useful for local processes and simple to implement. Thoughts? I used to run all of my IOWA apps via unix sockets. I don't do that anymore, though. There are situations where it can arguably be useful, but most of the time it just doesn't matter. Any differences between 127.0.0.1:11111 and /path/to/a/unix/socket are lost in the noise of the rest of the performance affecting load -- they are just too small to matter much. Kirk Haines From garbagecat10 at gmail.com Fri Oct 5 13:11:01 2007 From: garbagecat10 at gmail.com (Francis Cianfrocca) Date: Fri, 5 Oct 2007 13:11:01 -0400 Subject: [Mongrel] multi threaded theoretically useful? In-Reply-To: <966599840710050943o53510e2am582b8b2910ee348e@mail.gmail.com> References: <966599840710050943o53510e2am582b8b2910ee348e@mail.gmail.com> Message-ID: <3a94cf510710051011r34da220ch7f3acedb7da621d3@mail.gmail.com> On 10/5/07, Roger Pack wrote: > > > Further question: > would providing the option of having mongrel bind to UNIX sockets > perhaps speed it? (i.e. bind to port 3000 and unix socket x). Just a > thought. Probably not enough speed difference to matter, but perhaps > useful for local processes and simple to implement. Thoughts? Depending on the kernel, you may find Unix sockets to be less performant than localhost TCP. The dominant effect on performance comes from getting the network out of the picture. Beyond that, I've often found that unix-domain I/O stacks don't seem to be as well-tuned as TCP stacks are. And use localhost/tcp instead of an IP address on a network interface. On most kernels, that saves you a trip through the network interface driver. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071005/22dd81de/attachment-0001.html From francois.beausoleil at gmail.com Fri Oct 5 14:41:02 2007 From: francois.beausoleil at gmail.com (=?UTF-8?Q?Fran=C3=A7ois_Beausoleil?=) Date: Fri, 5 Oct 2007 14:41:02 -0400 Subject: [Mongrel] Mongrel PID file permissions Message-ID: <41d5fadf0710051141t10ef5219t8ccbd1bbfd985bd@mail.gmail.com> Hi ! I'm using Mongrel 1.0.1 with --pid and --user/--group. The PID files are correctly owned by the user/group, but their permissions is 0666. Is that normal ? Shouldn't it be something like 0664 ? Just curious to know if I'm wrong. Configurator has this (line 77): # Writes the PID file but only if we're on windows. def write_pid_file if RUBY_PLATFORM !~ /mswin/ log "Writing PID file to #{@pid_file}" open(@pid_file,"w") {|f| f.write(Process.pid) } end end The comment's wrong, and we probably need a File.chmod call there somewhere. Should I submit a patch ? Thanks ! -- Fran?ois Beausoleil http://blog.teksol.info/ http://piston.rubyforge.org/ From luislavena at gmail.com Fri Oct 5 21:39:27 2007 From: luislavena at gmail.com (Luis Lavena) Date: Fri, 5 Oct 2007 22:39:27 -0300 Subject: [Mongrel] Mongrel PID file permissions In-Reply-To: <41d5fadf0710051141t10ef5219t8ccbd1bbfd985bd@mail.gmail.com> References: <41d5fadf0710051141t10ef5219t8ccbd1bbfd985bd@mail.gmail.com> Message-ID: <71166b3b0710051839x70a82e14p891c3e2d9dc9adee@mail.gmail.com> On 10/5/07, Fran?ois Beausoleil wrote: > Hi ! > > I'm using Mongrel 1.0.1 with --pid and --user/--group. The PID files > are correctly owned by the user/group, but their permissions is 0666. > Is that normal ? Shouldn't it be something like 0664 ? > > Just curious to know if I'm wrong. > > Configurator has this (line 77): > > # Writes the PID file but only if we're on windows. > def write_pid_file > if RUBY_PLATFORM !~ /mswin/ > log "Writing PID file to #{@pid_file}" > open(@pid_file,"w") {|f| f.write(Process.pid) } > end > end > > The comment's wrong, and we probably need a File.chmod call there > somewhere. Should I submit a patch ? > Yes please! I don't see any reference to the file permission for the file, so maybe you require File.chmod after writting to it. Post it to RubyForge tracker (under patches) and someone will check and test it before commiting. Again, thank you for your time. -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From chrisangileri at yahoo.com Tue Oct 9 15:23:09 2007 From: chrisangileri at yahoo.com (Eire Angel) Date: Tue, 9 Oct 2007 12:23:09 -0700 (PDT) Subject: [Mongrel] Monit reporting that pid changed Message-ID: <538044.86856.qm@web53409.mail.re2.yahoo.com> Hi, I was wondering if this is normal or if it something i need to wonder about. I get Monit alerts that my mongrels PID files have changed. Changed Service MONGREL_2 Date: Tue, 09 Oct 2007 14:19:38 +0000 Action: alert Host: sa.greenling.com Description: 'MONGREL_2' process PID changed to 18368 Your faithful employee, monit --------------------------------- Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071009/fd9939ab/attachment.html From kamil.kukura at gmail.com Tue Oct 9 16:40:09 2007 From: kamil.kukura at gmail.com (Kamil Kukura) Date: Tue, 9 Oct 2007 22:40:09 +0200 Subject: [Mongrel] Fix for using memcache store in mongrel cluster Message-ID: Hi, I was having problem with disappearing sessions when storing them in memcache and when runnning as mongrel cluster in production environment. I've created a patch and it seems it's now okay. A message to rails-core including the link to patch can be found in the group here: http://groups.google.com/group/rubyonrails-core/browse_thread/thread/6427c8fda42683c2 -- Kamil -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071009/3bb2828e/attachment.html From alexey.verkhovsky at gmail.com Tue Oct 9 21:52:58 2007 From: alexey.verkhovsky at gmail.com (Alexey Verkhovsky) Date: Tue, 9 Oct 2007 19:52:58 -0600 Subject: [Mongrel] Monit reporting that pid changed In-Reply-To: <538044.86856.qm@web53409.mail.re2.yahoo.com> References: <538044.86856.qm@web53409.mail.re2.yahoo.com> Message-ID: <3945c4270710091852h84a02a6y9479c15e33581ad9@mail.gmail.com> On 10/9/07, Eire Angel wrote: > Hi, I was wondering if this is normal > or if it something i need to wonder about. > I get Monit alerts that my mongrels PID files > have changed. It means that Mongrel was restarted. If nobody actually wanted it to restart, it possibly crashed. I'd say, you do need to wonder about it. :) -- Alexey Verkhovsky CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com] RubyWorks [http://rubyworks.thoughtworks.com] From rob at robmela.com Wed Oct 10 10:48:21 2007 From: rob at robmela.com (Robert Mela) Date: Wed, 10 Oct 2007 10:48:21 -0400 Subject: [Mongrel] Mongrel SVN? In-Reply-To: <3945c4270710091852h84a02a6y9479c15e33581ad9@mail.gmail.com> References: <538044.86856.qm@web53409.mail.re2.yahoo.com> <3945c4270710091852h84a02a6y9479c15e33581ad9@mail.gmail.com> Message-ID: <470CE635.70302@robmela.com> What's the URL for checking things out of the Mongrel SVN? thx. -------------- next part -------------- A non-text attachment was scrubbed... Name: rob.vcf Type: text/x-vcard Size: 116 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20071010/58c51105/attachment.vcf From todd.fisher at gmail.com Wed Oct 10 10:59:34 2007 From: todd.fisher at gmail.com (Todd Fisher) Date: Wed, 10 Oct 2007 10:59:34 -0400 Subject: [Mongrel] Mongrel SVN? In-Reply-To: <470CE635.70302@robmela.com> References: <538044.86856.qm@web53409.mail.re2.yahoo.com> <3945c4270710091852h84a02a6y9479c15e33581ad9@mail.gmail.com> <470CE635.70302@robmela.com> Message-ID: svn://rubyforge.org/var/svn/mongrel/trunk On 10/10/07, Robert Mela wrote: > > What's the URL for checking things out of the Mongrel SVN? > > thx. > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071010/fafc590a/attachment.html From evan at cloudbur.st Thu Oct 11 02:55:36 2007 From: evan at cloudbur.st (Evan Weaver) Date: Thu, 11 Oct 2007 02:55:36 -0400 Subject: [Mongrel] 1.0.2 release candidates Message-ID: Hello Mongrels, Release candidates for Mongrel 1.0.2 and its dependencies are now available. Note that gems are now signed, so please add the Mongrel public certificate via: $ wget http://rubyforge.org/frs/download.php/25325/mongrel-public_cert.pem $ gem cert --add mongrel-public_cert.pem Now you can verify and install the candidates via gem install: $ sudo gem install mongrel --include-dependencies --source http://mongrel.rubyforge.org/releases/ -P HighSecurity Manual downloads are available from http://mongrel.rubyforge.org/releases/ . Please report problems on the Rubyforge tracker at http://rubyforge.org/tracker/?group_id=1306 . We will finalize the release in a week or two, barring unforeseen difficulties. Thanks, Evan, and the Mongrel Team PS. If you really don't care about authenticity, and don't want to install the public certificate, you can omit the '-P HighSecurity' option in 'gem install'. -- Evan Weaver Cloudburst, LLC From 2828628 at gmail.com Thu Oct 11 03:03:50 2007 From: 2828628 at gmail.com (Ken) Date: Thu, 11 Oct 2007 15:03:50 +0800 Subject: [Mongrel] 1.0.2 release candidates In-Reply-To: References: Message-ID: <2a0834610710110003o214fcae8h1c5a47c1a3d0891f@mail.gmail.com> great! On 10/11/07, Evan Weaver wrote: > > Hello Mongrels, > > Release candidates for Mongrel 1.0.2 and its dependencies are now > available. Note that gems are now signed, so please add the Mongrel > public certificate via: > > $ wget > http://rubyforge.org/frs/download.php/25325/mongrel-public_cert.pem > $ gem cert --add mongrel-public_cert.pem > > Now you can verify and install the candidates via gem install: > > $ sudo gem install mongrel --include-dependencies --source > http://mongrel.rubyforge.org/releases/ -P HighSecurity > > Manual downloads are available from http://mongrel.rubyforge.org/releases/. > > Please report problems on the Rubyforge tracker at > http://rubyforge.org/tracker/?group_id=1306 . We will finalize the > release in a week or two, barring unforeseen difficulties. > > Thanks, > > Evan, and the Mongrel Team > > > PS. If you really don't care about authenticity, and don't want to > install the public certificate, you can omit the '-P HighSecurity' > option in 'gem install'. > > -- > Evan Weaver > Cloudburst, LLC > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- 君問歸期未有期，巴山夜雨漲秋池。何當共剪西窗燭，卻話巴山夜雨時。 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071011/8af2cbac/attachment.html From lists at lastonepicked.com Thu Oct 11 03:05:30 2007 From: lists at lastonepicked.com (Hunter Hillegas) Date: Thu, 11 Oct 2007 00:05:30 -0700 Subject: [Mongrel] 1.0.2 release candidates In-Reply-To: References: Message-ID: What's the word on the changes? On Oct 10, 2007, at 11:55 PM, Evan Weaver wrote: > Hello Mongrels, > > Release candidates for Mongrel 1.0.2 and its dependencies are now > available. Note that gems are now signed, so please add the Mongrel > public certificate via: > > $ wget http://rubyforge.org/frs/download.php/25325/mongrel- > public_cert.pem > $ gem cert --add mongrel-public_cert.pem > > Now you can verify and install the candidates via gem install: > > $ sudo gem install mongrel --include-dependencies --source > http://mongrel.rubyforge.org/releases/ -P HighSecurity > > Manual downloads are available from http://mongrel.rubyforge.org/ > releases/ . > > Please report problems on the Rubyforge tracker at > http://rubyforge.org/tracker/?group_id=1306 . We will finalize the > release in a week or two, barring unforeseen difficulties. > > Thanks, > > Evan, and the Mongrel Team > > > PS. If you really don't care about authenticity, and don't want to > install the public certificate, you can omit the '-P HighSecurity' > option in 'gem install'. > > -- > Evan Weaver > Cloudburst, LLC > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users From evan at cloudbur.st Thu Oct 11 03:59:12 2007 From: evan at cloudbur.st (Evan Weaver) Date: Thu, 11 Oct 2007 03:59:12 -0400 Subject: [Mongrel] 1.0.2 release candidates In-Reply-To: References: Message-ID: The CHANGELOG message for Mongrel is just: v1.0.2. Signed gem; various bugfixes and patches. There should be no breaking changes. If you're interested in a specific bug, check the tracker and see what the resolution was. The detailed SVN log up to and including 1.0.2 is at http://mongrel.rubyforge.org/releases/Changelog . Mongrel_cluster v1.0.3 includes Capistrano 2 recipes. That's probably the most significant new feature. Evan On 10/11/07, Hunter Hillegas wrote: > What's the word on the changes? > > On Oct 10, 2007, at 11:55 PM, Evan Weaver wrote: > > > Hello Mongrels, > > > > Release candidates for Mongrel 1.0.2 and its dependencies are now > > available. Note that gems are now signed, so please add the Mongrel > > public certificate via: > > > > $ wget http://rubyforge.org/frs/download.php/25325/mongrel- > > public_cert.pem > > $ gem cert --add mongrel-public_cert.pem > > > > Now you can verify and install the candidates via gem install: > > > > $ sudo gem install mongrel --include-dependencies --source > > http://mongrel.rubyforge.org/releases/ -P HighSecurity > > > > Manual downloads are available from http://mongrel.rubyforge.org/ > > releases/ . > > > > Please report problems on the Rubyforge tracker at > > http://rubyforge.org/tracker/?group_id=1306 . We will finalize the > > release in a week or two, barring unforeseen difficulties. > > > > Thanks, > > > > Evan, and the Mongrel Team > > > > > > PS. If you really don't care about authenticity, and don't want to > > install the public certificate, you can omit the '-P HighSecurity' > > option in 'gem install'. > > > > -- > > Evan Weaver > > Cloudburst, LLC > > _______________________________________________ > > Mongrel-users mailing list > > Mongrel-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/mongrel-users > > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Evan Weaver Cloudburst, LLC From francois.beausoleil at gmail.com Thu Oct 11 13:54:19 2007 From: francois.beausoleil at gmail.com (=?UTF-8?Q?Fran=C3=A7ois_Beausoleil?=) Date: Thu, 11 Oct 2007 13:54:19 -0400 Subject: [Mongrel] Mongrel PID file permissions In-Reply-To: <71166b3b0710051839x70a82e14p891c3e2d9dc9adee@mail.gmail.com> References: <41d5fadf0710051141t10ef5219t8ccbd1bbfd985bd@mail.gmail.com> <71166b3b0710051839x70a82e14p891c3e2d9dc9adee@mail.gmail.com> Message-ID: <41d5fadf0710111054g3cd690dbu9c1aa139fd2a4831@mail.gmail.com> Hi ! 2007/10/5, Luis Lavena : > I don't see any reference to the file permission for the file, so > maybe you require File.chmod after writting to it. > > Post it to RubyForge tracker (under patches) and someone will check > and test it before commiting. Take the patch from here: http://rubyforge.org/tracker/index.php?func=detail&aid=14641&group_id=1306&atid=5147 Bye ! Fran?ois From lists at ruby-forum.com Thu Oct 11 19:23:57 2007 From: lists at ruby-forum.com (Greg Willits) Date: Fri, 12 Oct 2007 01:23:57 +0200 Subject: [Mongrel] mongrel_rails not starting Message-ID: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> os x 10.4.10 used gem to install mongrel 1.0.1. seemed to go w/o errors. "mongrel_rails start -d" spits out this error /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.bundle: Failed to load /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.bundle (LoadError) from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:32:in `require' from /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:11 from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:32:in `gem_original_require' from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:32:in `require' from /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/bin/mongrel_rails:9 from /usr/local/bin/mongrel_rails:16:in `load' from /usr/local/bin/mongrel_rails:16 installed again with --include-dependencies still getting same error. lost. need a flashlight. -- gw -- Posted via http://www.ruby-forum.com/. From luislavena at gmail.com Thu Oct 11 19:33:30 2007 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 11 Oct 2007 20:33:30 -0300 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> Message-ID: <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> On 10/11/07, Greg Willits wrote: > os x 10.4.10 > > used gem to install mongrel 1.0.1. seemed to go w/o errors. > > "mongrel_rails start -d" spits out this error > > /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.bundle: > Failed to load > /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.bundle > (LoadError) > from > /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:32:in > `require' > from > /usr/local/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/mongrel.rb:11 It seems there is no bundle (compiled extension) for http11, which is need by Mongrel. > > installed again with --include-dependencies > > still getting same error. > > lost. need a flashlight. > Do you have everything to compile ruby extensions in place? Which version of ruby are you using? the broken one that came with OSX or compiled by hand? What's the output of gem install mongrel? You should see the details about "building native extensions"... -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From lists at ruby-forum.com Thu Oct 11 20:08:49 2007 From: lists at ruby-forum.com (Greg Willits) Date: Fri, 12 Oct 2007 02:08:49 +0200 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> Message-ID: <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> Luis Lavena wrote: > On 10/11/07, Greg Willits wrote: > It seems there is no bundle (compiled extension) for http11, which is > need by Mongrel. > Do you have everything to compile ruby extensions in place? Unknown. (ruby newbie). > Which version of ruby are you using? The one from here: http://rubyosx.rubyforge.org/ -- which I see should already have Mongrel in it, but then I installed... oh, hang on. Maybe I broke it by monkeying with gem installs afterwards. OK, I just re-installed using that installer, and now mongrel launches, but complains about a log path being invalid. So, I'm further along. I'll see if I can figure out the log thing. Thanks Luis, that got me to re-evaluate what I had done. -- gw -- Posted via http://www.ruby-forum.com/. From luislavena at gmail.com Thu Oct 11 20:15:57 2007 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 11 Oct 2007 21:15:57 -0300 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> Message-ID: <71166b3b0710111715j5e341ba5g8d6a7d320c5f172a@mail.gmail.com> On 10/11/07, Greg Willits wrote: > Luis Lavena wrote: > > > Which version of ruby are you using? > > The one from here: http://rubyosx.rubyforge.org/ -- which I see should > already have Mongrel in it, but then I installed... oh, hang on. Maybe I > broke it by monkeying with gem installs afterwards. > Yeah, Ruby for OSX have Mongrel pre-compiled (along with Ruby) so you don't need a compilation/build environment in place. Is like One-Click-Installer on Windows ;-) > OK, I just re-installed using that installer, and now mongrel launches, > but complains about a log path being invalid. > you're runnin mongrel_rails start inside the root of your Rails application? maybe there isn't a log folder inside it, or the permissions aren't valid. > So, I'm further along. I'll see if I can figure out the log thing. Could you post the messages mongrel_rails start is showing you? that could help us provide you better support :-) > Thanks Luis, that got me to re-evaluate what I had done. No problem, I'm glad being helpful. -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From lists at ruby-forum.com Thu Oct 11 20:24:31 2007 From: lists at ruby-forum.com (Greg Willits) Date: Fri, 12 Oct 2007 02:24:31 +0200 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: <71166b3b0710111715j5e341ba5g8d6a7d320c5f172a@mail.gmail.com> References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> <71166b3b0710111715j5e341ba5g8d6a7d320c5f172a@mail.gmail.com> Message-ID: <072d207c3e12240f2af7b9058ef59791@ruby-forum.com> >> OK, I just re-installed using that installer, and now mongrel launches, >> but complains about a log path being invalid. >> > > you're runnin mongrel_rails start inside the root of your Rails > application? maybe there isn't a log folder inside it, or the > permissions aren't valid. Yeah I was one dir above my app, so that was the problem. All looks good now. (lots of voodoo to get this stuff running compared to what I am used to -- Lasso) -- gw -- Posted via http://www.ruby-forum.com/. From luislavena at gmail.com Thu Oct 11 20:29:14 2007 From: luislavena at gmail.com (Luis Lavena) Date: Thu, 11 Oct 2007 21:29:14 -0300 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: <072d207c3e12240f2af7b9058ef59791@ruby-forum.com> References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> <71166b3b0710111715j5e341ba5g8d6a7d320c5f172a@mail.gmail.com> <072d207c3e12240f2af7b9058ef59791@ruby-forum.com> Message-ID: <71166b3b0710111729r6440ae52p7e632a0162392f@mail.gmail.com> On 10/11/07, Greg Willits wrote: > Yeah I was one dir above my app, so that was the problem. All looks good > now. > > (lots of voodoo to get this stuff running compared to what I am used to > -- Lasso) > Hehehe, Please, file a bug, feature-request at mongrel tracker: http://rubyforge.org/tracker/?group_id=1306 I guess we could make "mongrel_rails" script more intelligent about where you are and if that folder is a Rails one ;-) There is also Locomotive [1] which provides a simpler way to configure mongrels for your applications. It differs from Ruby OSX is that is tailored for Rails (and rubyosx is more ruby generic). [1] http://locomotive.raaum.org/ -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From lists at ruby-forum.com Thu Oct 11 20:40:40 2007 From: lists at ruby-forum.com (Greg Willits) Date: Fri, 12 Oct 2007 02:40:40 +0200 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: <71166b3b0710111729r6440ae52p7e632a0162392f@mail.gmail.com> References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> <71166b3b0710111715j5e341ba5g8d6a7d320c5f172a@mail.gmail.com> <072d207c3e12240f2af7b9058ef59791@ruby-forum.com> <71166b3b0710111729r6440ae52p7e632a0162392f@mail.gmail.com> Message-ID: Luis Lavena wrote: > There is also Locomotive [1] which provides a simpler way to configure > mongrels for your applications. It differs from Ruby OSX is that is > tailored for Rails (and rubyosx is more ruby generic). My thought was maybe that would isolate me too much from what's going on. After I get my feet wet a little bit with just getting some stuff running and working in Rails, I'll probably go back and uninstall everything and try it by hand. Wandering off into another (related) topic... Right now I'm trying to figure out how to get multiple apps running on mongrels with dev URLs I'm used to using www.XYZ.dev and www.ABC.dev in my browser, and then using NetInfo machines or even hosts to map those to 127.0.0.1 I thought 127.0.0.1:3000 www.abc.dev would work but it didn't. I understand that I'll have to run multiple mongrels on unique ports, but haven't figured out how to map a URL to the localhost:port yet -- gw -- Posted via http://www.ruby-forum.com/. From lists at ruby-forum.com Thu Oct 11 20:48:53 2007 From: lists at ruby-forum.com (Greg Willits) Date: Fri, 12 Oct 2007 02:48:53 +0200 Subject: [Mongrel] mongrel_rails not starting In-Reply-To: References: <3d1be0c7af64aabda8dcab0cecbd9084@ruby-forum.com> <71166b3b0710111633l445346cdu8693a182e98f8cc1@mail.gmail.com> <4a746b688910bfc7c3ee6b7000265536@ruby-forum.com> <71166b3b0710111715j5e341ba5g8d6a7d320c5f172a@mail.gmail.com> <072d207c3e12240f2af7b9058ef59791@ruby-forum.com> <71166b3b0710111729r6440ae52p7e632a0162392f@mail.gmail.com> Message-ID: <678706ff4eae953532c5f414325227c0@ruby-forum.com> > Right now I'm trying to figure out how to get multiple apps running on > mongrels with dev URLs > > I'm used to using www.XYZ.dev and www.ABC.dev in my browser, and then > using NetInfo machines or even hosts to map those to 127.0.0.1 > > I thought 127.0.0.1:3000 www.abc.dev would work but it didn't. btw, I know I can use Apache 2.2 mod_proxy_balancer -- I actually helped set one of those up (though I didn't do the mongerl stuff then), so I can just do that I guess. Just more junk to install since using Apache 1.3 is fine for everything else I do on my dev system. -- gw -- Posted via http://www.ruby-forum.com/. From carlos at bueno.org Fri Oct 12 14:29:22 2007 From: carlos at bueno.org (Carlos) Date: Fri, 12 Oct 2007 11:29:22 -0700 Subject: [Mongrel] deciphering objects.log Message-ID: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> Howdy -- I've been tracking down mem leaks (oh, the fun...), and I think there is a clue in objects.log. There are a few mentions of this file, but usually Zed saying "look at this file and it will help you". Can anyone clue me in to what the actual columns mean? 18,Float,143952,256821,112869,,, 18,String,39543,41693,2150,24.727076,55.526376,2308.000000 18,Array,3014,4232,1218,4.445888,23.633335,612.000000 18,Proc,1699,1750,51,,, 18,Regexp,1300,1310,10,,, 18,Hash,691,730,39,8.008219,36.218131,519.000000 18,Time,134,168,34,,, 18,YAML::Syck::Scalar,60,108,48,,, 18,Mongrel::Stats,44,57,13,,, 18,MatchData,14,24,10,1.416667,1.138904,5.000000 18,StringIO,9,12,3,166.833333,178.322401,408.000000 19,Float,256821,107916,-148905,,, 19,String,41693,39700,-1993,23.864257,52.626340,2308.000000 19,Array,4232,3086,-1146,5.254051,27.529250,612.000000 19,Proc,1750,1697,-53,,, 19,Regexp,1310,1298,-12,,, 19,Hash,730,702,-28,8.056980,36.770207,519.000000 19,Time,168,122,-46,,, 19,YAML::Syck::Scalar,108,46,-62,,, 19,Mongrel::Stats,57,31,-26,,, 19,MatchData,24,12,-12,1.333333,1.154701,5.000000 TIA, Carlos -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071012/9191987d/attachment-0001.html From carlos at bueno.org Fri Oct 12 14:39:20 2007 From: carlos at bueno.org (Carlos) Date: Fri, 12 Oct 2007 11:39:20 -0700 Subject: [Mongrel] deciphering objects.log In-Reply-To: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> References: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> Message-ID: <887494480710121139y4954da86pf3cd7aa9c84de27a@mail.gmail.com> I RTFCd and got this: $objects_out.puts "run,classname,last,count,delta,lenmean,lensd,lenmax" So, run: a serial number of the request classname: obvious last: number of object from previous run count: number of objects in this run delta: last - count lenmean: average length of object lensd: std deviation lenmax: largest-seen (or maybe largest-existing?) object of this class Right? On 10/12/07, Carlos wrote: > > Howdy -- I've been tracking down mem leaks (oh, the fun...), and I think > there is a clue in objects.log. There are a few mentions of this file, but > usually Zed saying "look at this file and it will help you". Can anyone clue > me in to what the actual columns mean? > > 18,Float,143952,256821,112869,,, > 18,String,39543,41693,2150, 24.727076,55.526376,2308.000000 > 18,Array,3014,4232,1218,4.445888,23.633335,612.000000 > 18,Proc,1699,1750,51,,, > 18,Regexp,1300,1310,10,,, > 18,Hash,691,730,39,8.008219,36.218131,519.000000 > 18,Time,134,168,34,,, > 18,YAML::Syck::Scalar,60,108,48,,, > 18,Mongrel::Stats,44,57,13,,, > 18,MatchData,14,24,10,1.416667,1.138904,5.000000 > 18,StringIO,9,12,3,166.833333,178.322401,408.000000 > 19,Float,256821,107916,-148905,,, > 19,String,41693,39700,-1993,23.864257,52.626340,2308.000000 > 19,Array,4232,3086,-1146,5.254051,27.529250,612.000000 > 19,Proc,1750,1697,-53,,, > 19,Regexp,1310,1298,-12,,, > 19,Hash,730,702,-28,8.056980,36.770207,519.000000 > 19,Time,168,122,-46,,, > 19,YAML::Syck::Scalar,108,46,-62,,, > 19,Mongrel::Stats,57,31,-26,,, > 19,MatchData,24,12,-12,1.333333,1.154701,5.000000 > > > TIA, > Carlos > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071012/9cbf18ac/attachment.html From zedshaw at zedshaw.com Fri Oct 12 22:15:07 2007 From: zedshaw at zedshaw.com (Zed A. Shaw) Date: Fri, 12 Oct 2007 22:15:07 -0400 Subject: [Mongrel] deciphering objects.log In-Reply-To: <887494480710121139y4954da86pf3cd7aa9c84de27a@mail.gmail.com> References: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> <887494480710121139y4954da86pf3cd7aa9c84de27a@mail.gmail.com> Message-ID: <20071012221507.8b4dcd70.zedshaw@zedshaw.com> On Fri, 12 Oct 2007 11:39:20 -0700 Carlos wrote: > I RTFCd and got this: > > $objects_out.puts "run,classname,last,count,delta,lenmean,lensd,lenmax" > > So, > run: a serial number of the request > classname: obvious > last: number of object from previous run > count: number of objects in this run > delta: last - count > lenmean: average length of object > lensd: std deviation > lenmax: largest-seen (or maybe largest-existing?) object of this class > > Right? Yep, except len(mean|sd|max) are used for objects that report a length, and that header should be created if the file is first created. You must have truncated it at some point. How you use this file is you run a *sequential* number of hits and do it in *production* mode. When it's done you load it into a spreadsheet or R or some Ruby and you look for the following: 1) Any class which seems to have a count who's delta mean isn't close to 0 for all runs. A leak can actually be defined in a GC language as: any class who's object count delta over time do not have a mean of 0. 2) Any class who's average length or sd.dev length is gigantic (send_file anyone?). 3) Any classes who have a mean of 0, but have a sd.dev that's massive. This shows poor GC management since it's a huge number of object being created and destroyed. 4) Any other column that for any class seems to be not like the others. Also look at bleakhouse. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/ From evan at cloudbur.st Fri Oct 12 23:52:24 2007 From: evan at cloudbur.st (Evan Weaver) Date: Fri, 12 Oct 2007 23:52:24 -0400 Subject: [Mongrel] deciphering objects.log In-Reply-To: <20071012221507.8b4dcd70.zedshaw@zedshaw.com> References: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> <887494480710121139y4954da86pf3cd7aa9c84de27a@mail.gmail.com> <20071012221507.8b4dcd70.zedshaw@zedshaw.com> Message-ID: Bleakhouse isn't as detailed as it could be, but it's better than using Objectspace. Zed, do you think we should remove the object.log? My guess is that it leads people astray as much as it helps. Ara Howard has some good ideas in his leak checker "Dike", but again, it uses Objectspace. Evan On 10/12/07, Zed A. Shaw wrote: > On Fri, 12 Oct 2007 11:39:20 -0700 > Carlos wrote: > > > I RTFCd and got this: > > > > $objects_out.puts "run,classname,last,count,delta,lenmean,lensd,lenmax" > > > > So, > > run: a serial number of the request > > classname: obvious > > last: number of object from previous run > > count: number of objects in this run > > delta: last - count > > lenmean: average length of object > > lensd: std deviation > > lenmax: largest-seen (or maybe largest-existing?) object of this class > > > > Right? > > Yep, except len(mean|sd|max) are used for objects that report a length, and that header should be created if the file is first created. You must have truncated it at some point. > > How you use this file is you run a *sequential* number of hits and do it in *production* mode. When it's done you load it into a spreadsheet or R or some Ruby and you look for the following: > > 1) Any class which seems to have a count who's delta mean isn't close to 0 for all runs. A leak can actually be defined in a GC language as: any class who's object count delta over time do not have a mean of 0. > 2) Any class who's average length or sd.dev length is gigantic (send_file anyone?). > 3) Any classes who have a mean of 0, but have a sd.dev that's massive. This shows poor GC management since it's a huge number of object being created and destroyed. > 4) Any other column that for any class seems to be not like the others. > > Also look at bleakhouse. > > -- > Zed A. Shaw > - Hate: http://savingtheinternetwithhate.com/ > - Good: http://www.zedshaw.com/ > - Evil: http://yearofevil.com/ > _______________________________________________ > Mongrel-users mailing list > Mongrel-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mongrel-users > -- Evan Weaver Cloudburst, LLC From luislavena at gmail.com Fri Oct 12 23:57:51 2007 From: luislavena at gmail.com (Luis Lavena) Date: Sat, 13 Oct 2007 00:57:51 -0300 Subject: [Mongrel] deciphering objects.log In-Reply-To: References: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> <887494480710121139y4954da86pf3cd7aa9c84de27a@mail.gmail.com> <20071012221507.8b4dcd70.zedshaw@zedshaw.com> Message-ID: <71166b3b0710122057i1a6ab49bi261f371d9a292525@mail.gmail.com> On 10/13/07, Evan Weaver wrote: > Bleakhouse isn't as detailed as it could be, but it's better than > using Objectspace. > > Zed, do you think we should remove the object.log? My guess is that it > leads people astray as much as it helps. > > Ara Howard has some good ideas in his leak checker "Dike", but again, > it uses Objectspace. > Objectspace is like Heisenberg principle [1], you cannot touch it without altering the results. Maybe Dike and Bleak could merge and get something more powerful? ;-) [1] http://en.wikipedia.org/wiki/Uncertainty_principle -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From luislavena at gmail.com Sat Oct 13 13:29:00 2007 From: luislavena at gmail.com (Luis Lavena) Date: Sat, 13 Oct 2007 14:29:00 -0300 Subject: [Mongrel] vista on mongrel In-Reply-To: <540203.56039.qm@web33012.mail.mud.yahoo.com> References: <540203.56039.qm@web33012.mail.mud.yahoo.com> Message-ID: <71166b3b0710131029k6e334e60o6514b407d680bbfe@mail.gmail.com> On 10/13/07, Raj Prakash wrote: > Hello, > > I am having the exact same problem described by Jonah Fox here: > > http://rubyforge.org/pipermail/mongrel-users/2007-May/003647.html > > I was wondering if you have found a solution yet. It is not the session > issue. > Are you sure about the session storage? have you tried changing it? On XP with cookie store happens, and get fixed when I change to pstore. Since I don't have Vista, I couldn't test it. (Also willb e useful know the ruby and rails version that you're using). -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From luislavena at gmail.com Sat Oct 13 14:12:39 2007 From: luislavena at gmail.com (Luis Lavena) Date: Sat, 13 Oct 2007 15:12:39 -0300 Subject: [Mongrel] vista on mongrel In-Reply-To: <278233.10573.qm@web33004.mail.mud.yahoo.com> References: <71166b3b0710131029k6e334e60o6514b407d680bbfe@mail.gmail.com> <278233.10573.qm@web33004.mail.mud.yahoo.com> Message-ID: <71166b3b0710131112h6c0a8ear604fd7115c29a554@mail.gmail.com> On 10/13/07, Raj Prakash wrote: > Thank you for a a quick response, Luis. > > I am using Ruby 1.8.5 and Rails 1.2.3 > > According to the comment in ruby source In 1.8.5 the default is pstore. > Could you please tell me the specific changes that you did to the > environment > to force pstore. > > Apparently, Jonah Fox is still having the problem. I see a post from him > and a few replies just today at > > http://thread.gmane.org/gmane.comp.lang.ruby.rails/156245 > > There is one difference though I have problem in both IE and firefox > Let me explain this better: Latest versions of Rails (1.2.3 and Edge) sport a cookie-based session handling as default. The combination of this fact with Firefox on windows shows problems. On the other hand, webrick don't show this issue because cookies are handled in a different way. To workaround this problem, until deeper investigation of it, I suggest you switch for local/development to PStore based session (that was the default on previous versions). You need to uncomment and modify the config/environment.rb around lines 29-31: config.action_controller.session_store = :PStore Please note: that will work OK on development, but this will not under production since multiple mongrel instances will have problems accessing the session files (stored in tmp/sessions). For production I suggest change to :active_record_storage as suggested in environment.rb comments and perform the rake task needed to create these tables on the DB. Also, I'll like some volunteers to try the new release candidate of Mongrel (1.0.2) [1] and let us know [2] if worked for you under the same situation. [1] http://rubyforge.org/pipermail/mongrel-users/2007-October/004108.html [2] mongrel-users at rubyforge.org (mailing list) Regards, -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From luislavena at gmail.com Sat Oct 13 15:11:00 2007 From: luislavena at gmail.com (Luis Lavena) Date: Sat, 13 Oct 2007 16:11:00 -0300 Subject: [Mongrel] vista on mongrel In-Reply-To: <60937.75029.qm@web33001.mail.mud.yahoo.com> References: <71166b3b0710131112h6c0a8ear604fd7115c29a554@mail.gmail.com> <60937.75029.qm@web33001.mail.mud.yahoo.com> Message-ID: <71166b3b0710131211h48e56087h9c6d0c0e83ba69c7@mail.gmail.com> On 10/13/07, Raj Prakash wrote: > Luis, > > I just tried the mongrel 1.0.2 rc. Unfortunately, it does not improve the > speed > with or without ...= :PStore. > > With this and the previous version of mongrel at start, I see the message > ** Ruby version is not up-to-date: loading cgi_multipart_eof_fix > > Could that cause speed problem? > It's weird, so I'll check later today at the office. With 1.8.6, cgi eof fix don't load but I'm using PStore... I'll check and let you know. -- Luis Lavena Multimedia systems - Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile. Vince Lombardi From zedshaw at zedshaw.com Sat Oct 13 20:53:15 2007 From: zedshaw at zedshaw.com (Zed A. Shaw) Date: Sat, 13 Oct 2007 20:53:15 -0400 Subject: [Mongrel] deciphering objects.log In-Reply-To: References: <887494480710121129i2af727f2o19c320922fd1968@mail.gmail.com> <887494480710121139y4954da86pf3cd7aa9c84de27a@mail.gmail.com> <20071012221507.8b4dcd70.zedshaw@zedshaw.com> Message-ID: <20071013205315.bdee762a.zedshaw@zedshaw.com> On Fri, 12 Oct 2007 23:52:24 -0400 "Evan Weaver" wrote: > > Zed, do you think we should remove the object.log? My guess is that it > leads people astray as much as it helps. I don't hear anyone complaining it leads them astray. I've used it to find plenty of leaks. If you could improve it that'd be better than just removing it. -- Zed A. Shaw - Hate: http://savingtheinternetwithhate.com/ - Good: http://www.zedshaw.com/ - Evil: http://yearofevil.com/ From basic70 at gmail.com Mon Oct 15 08:00:20 2007 From: basic70 at gmail.com (Daniel Brahneborg) Date: Mon, 15 Oct 2007 14:00:20 +0200 Subject: [Mongrel] POST with huge HTTP body Message-ID: Hi, I found the following page describing how to stream data from the server to the client: http://api.rubyonrails.com/classes/ActionController/Streaming.html I want to do the opposite, streaming data from the client to the server, letting the controller saving the data while it's being received. In particular, I want to upload large files to the RailsDav (http://www.liverail.net/railsdav) plugin, without having to store the entire file in memory during the operation. This seems to be a Mongrel problem, so I started looking at this plugin: http://mongrel.rubyforge.org/docs/upload_progress.html How do I get Mongrel to store the incoming POST data in a tempfile, passing a handler to that file as the raw_post field in the Rails request? Is it even possible? How is the "request" parameter to the process() method related to the "request" object in the Rails controllers? /Daniel From david at vrensk.com Mon Oct 15 08:49:27 2007 From: david at vrensk.com (David Vrensk) Date: Mon, 15 Oct 2007 14:49:27 +0200 Subject: [Mongrel] POST with huge HTTP body In-Reply-To: References: Message-ID: <81b453920710150549y32ebb0b3ta14b9d634ab437fc@mail.gmail.com> On 10/15/07, Daniel Brahneborg wrote: > > I want to do the opposite, streaming data from the client to the server, > letting the controller saving the data while it's being received. > In particular, I want to upload large files to the RailsDav > (http://www.liverail.net/railsdav) plugin, without having to store > the entire file in memory during the operation. > > This seems to be a Mongrel problem, [...] > I don't have the solution to your problem, but I think you are looking in the wrong place. Unless you plan to have mongrel listening on a public port, this is not a mongrel problem. The standard way to use mongrel, with Rails or not, is to set up a few of them behind a reverse proxy like nginx or apache+mod_proxy_balancer. In those cases, the proxy will not forward the data to mongrel before it has received it all, and once it has all the data, the latency between nginx and mongrel is negligible. On the other hand, the reverse proxy (I'm only speaking from experience with nginx here) saves the data to file while it is receiving it. I think you should look for a solution that lets nginx pass the path to that file to mongrel rather than sending the data again. I believe I have read about such a solution on this very list a few months ago. So, not the help you wanted, but I hope it leads you in the right direction. Well, if you actually intend to have mongrel face the public, then just forget what I said. Good luck! /David -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mongrel-users/attachments/20071015/af934fb6/attachment.html From rob at robmela.com Mon Oct 15 09:20:18 2007 From: rob at robmela.com (Robert Mela) Date: Mon, 15 Oct 2007 09:20:18 -0400 Subject: [Mongrel] Design flaw? - num_processors, accept/close Message-ID: <47136912.6080107@robmela.com> Rails instances themselves are almost always single-threaded, whereas Mongrel, and it's acceptor, are multithreaded. In a situation with long-running Rails pages this presents a problem for mod_proxy_balancer. If num_processors is greater than 1 ( default: 950 ), then Mongrel will gladly accept incoming requests and queue them if its rails instance is currently busy. So even though there are non-busy mongrel instances, a busy one can accept a new request and queue it behind a long-running request. I tried setting num_processors to 1. But it looks like this is less than ideal -- I need to dig into mod_proxy_balancer to be sure. But at first glance, it appears this replaces queuing problem with a proxy error. That's because Mongrel still accepts the incoming request -- only to close the new socket immediately if Rails is busy. Once again, I do need to set up a test and see exactly how mod_proxy_balancer handles this... but... If I understand the problem correctly, then one solution might be moving lines 721 thru 734 into a loop, possibly in its own method, which does sth like this: def myaccept while true return @socket.accept if worker_list.length < num_processors ## check first to see if we can handle the request. Let client worry about connect timeouts. while @num_processors < reap_dead_workers sleep @loop_throttle end end end 720 @acceptor = Thread.new do 721 while true 722 begin * 723 client = @socket.accept * 724 725 if $tcp_cork_opts 726 client.setsockopt(*$tcp_cork_opts) rescue nil 727 end 728 729 worker_list = @workers.list 730 731 if worker_list.length >= @num_processors 732 STDERR.puts "Server overloaded with #{worker_list.length} processors (#@num_processors max). Dropping connection." * 733 client.close rescue Object* 734 reap_dead_workers("max processors") 735 else 736 thread = Thread.new(client) {|c| process_client(c) } 737 thread[:started_on] = Time.now 738 @workers.add(thread) 739 740 sleep @timeout/100 if @timeout > 0 741 end -------------- next part -------------- A non-text attachment was scrubbed... Name: rob.vcf Type: text/x-vcard Size: 116 bytes Desc: not available Url : http://rubyforge.org/pipermail/mongrel-users/attachments/20071015/40912d0f/attachment.vcf From kuijsten at yahoo.com Mon Oct 15 09:33:18 2007 From: kuijsten at yahoo.com (Tim Kuijsten) Date: Mon, 15 Oct 2007 06:33:18 -0700 (PDT) Subject: [Mongrel] POST with huge HTTP body Message-ID: <445886.78918.qm@web30907.mail.mud.yahoo.com> A while ago I wrote a plugin to limit uploads in Mongrel. Mongrel does save uploads to a tempfile on disk - if the upload is bigger than about 12 KB - using a Ruby TempFile-object. (which stores in the $TMPDIR, /tmp on most systems). The request is handed over to Rails after it's fully received by Mongrel. I'm not sure if this "saving to disk" works the same with chunked uploads (uploads without a Content-Length header). Maybe my plugin can help you: http://slasaus.netsend.nl/articles/show/7 (warning: it works, but is not very elegant). Gl, Tim ----- Original Message ---- From: Daniel Brahneborg To: mongrel-users at rubyforge.org Sent: Monday, October 15, 2007 2:00:20 PM Subject: [Mongrel] POST with huge HTTP body Hi, I found the following page describing how to stream data from the server to the client: http://api.rubyonrails.com/classes/ActionController/Streaming.html I want to do the opposite, streaming data from the client to the server, letting the controller saving the data while it's being received. In particular, I want to upload large files to the RailsDav (http://www.liverail.net/railsdav) plugin, without having to store the entire file in memory during the operation. This seems to be a Mongrel problem, so I started looking at this plugin: http://mongrel.rubyforge.org/docs/upload_progress.html How do I get Mongrel to store the incoming POST data in a tempfile, passing a handler to that file as the raw_post field in the Rails request? Is it even possible? How is the "request" parameter to the process() method related to the "request" object in the Rails controllers? /Daniel _______________________________________________ Mongrel-users mailing list Mongrel-users at rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-users ____________________________________________________________________________________ Shape Yahoo! in your own image. Join our Network Research Panel today! http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 From kuijsten at yahoo.com Mon Oct 15 10:02:03 2007 From: kuijsten at yahoo.com (Tim Kuijsten) Date: Mon, 15 Oct 2007 07:02:03 -0700 (PDT) Subject: [Mongrel] POST with huge HTTP body Message-ID: <702678.58438.qm@web30908.mail.mud.yahoo.com> "... The standard way to use mongrel, with Rails or not, is to set up a few of them behind a reverse proxy like nginx or apache+mod_proxy_balancer. In those cases, the proxy will not forward the data to mongrel before it has received it all, and once it has all the data, the latency between nginx and mongrel is negligible. ..." My Apache 2.2 load balancing reverse proxy forwards all data immediately, without storing it in an "intermediate-file". I believe this is standard behavior. So this should not be the problem. I think the problem is in the fact that Mongrel stores the whole file-object before passing it through to Rails, right? Unfortunately I have no clue how to fix that. Tim ----- Original Message ---- From: David Vrensk To: mongrel-users at rubyforge.org Sent: Monday, October 15, 2007 2:49:27 PM Subject: Re: [Mongrel] POST with huge HTTP body On 10/15/07, Daniel Brahneborg wrote: I want to do the opposite, streaming data from the client to the server, letting the controller saving the data while it's being received. In particular, I want to upload large files to the RailsDav ( http://www.liverail.net/railsdav) plugin, without having to store the entire file in memory during the operation. This seems to be a Mongrel problem, [...] I don't have the solution to your problem, but I think you are looking in the wrong place. Unless you plan to have mongrel listening on a public port, this is not a mongrel problem. The standard way to use mongrel, with Rails or not, is to set up a few of them behind a reverse proxy like nginx or apache+mod_proxy_balancer. In those cases, the proxy will not forward the data to mongrel before it has received it all, and once it has all the data, the latency between nginx and mongrel is negligible. On the other hand, the reverse proxy (I'm only speaking from experience with nginx here) saves the data to file while it is receiving it. I think you should look for a solution that lets nginx pass the path to that file to mongrel rather than sending the data again. I believe I have read about such a solution on this very list a few months ago. So, not the help you wanted, but I hope it leads you in the right direction. Well, if you actually intend to have mongrel face the public, then just forget what I said. Good luck! /David ____________________________________________________________________________________ Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 From basic70 at gmail.com Mon Oct 15 10:10:06 2007 From: basic70 at gmail.com (Daniel Brahneborg) Date: Mon, 15 Oct 2007 16:10:06 +0200 Subject: [Mongrel] POST with huge HTTP body In-Reply-To: <445886.78918.qm@web30907.mail.mud.yahoo.com> References: <445886.78918.qm@web30907.mail.mud.yahoo.com> Message-ID: Thanks for the replies. The fact that Mongrel shouldn't be the first one to get the POST/PUT data was a good point that I hadn't thought of. However, even when talking directly to Mongrel, the memory consumption of the application increases until it crashes if too much data is sent. In my case, I was sending a 500MB file from a Webdav client. At some point all of it was loaded into RAM, which doesn't work. Are there any more productive alternatives than writing the entire thing from scratch in C? Please? :) /Daniel On 10/15/07, Tim Kuijsten wrote: > A while ago I wrote a plugin to limit uploads in Mongrel. Mongrel does save uploads to a tempfile on disk - if the upload is bigger than about 12 KB - using a Ruby TempFile-object. (which stores in the $TMPDIR, /tmp on most systems). > > > The request is handed over to Rails after it's fully received by Mongrel. > > > I'm not sure if this "saving to disk" works the same with chunked uploads (uploads without a Content-Length header). > > > > Maybe my plugin can help you: http://slasaus.netsend.nl/articles/show/7 (warning: it works, but is not very elegant). > > > Gl, > > > Tim > > ----- Original Message ---- > From: Daniel Brahneborg > To: mongrel-users at rubyforge.org > Sent: Monday, October 15, 2007 2:00:20 PM > Subject: [Mongrel] POST with huge HTTP body > > > Hi, > > I found the following page describing how to stream data from the > server to the client: > > http://api.rubyonrails.com/classes/ActionController/Streaming.html > > I want to do the opposite, streaming data from the client to the > server, > letting the controller saving the data while it's being received. > In particular, I want to upload large files to the RailsDav > (http://www.liverail.net/railsdav) plugin, without having to store > the entire file in memory during the operation. > > This seems to be a Mongrel problem, so I started looking at this > plugin: > > http://mongrel.rubyforge.org/docs/upload_progress.html > > How do I get Mongrel to store the incoming POST data i