From cremes.devlist at mac.com Sun Apr 1 10:43:59 2007 From: cremes.devlist at mac.com (cremes.devlist at mac.com) Date: Sun, 1 Apr 2007 09:43:59 -0500 Subject: [Backgroundrb-devel] example code for file upload In-Reply-To: References: Message-ID: On Mar 30, 2007, at 4:36 PM, Walter McGinnis wrote: > My real problem is uploads over crap network connections turning into > CPU eating zombie mongrels. Man, I love it when geek speak sounds > like a horror movie premise, but in this case it's a real pain. I > suspect that the solution for that is going to be mongrel based > though. I'll RTFM over in mongrel doc land, but any hints are > appreciated. Walter, for very large files (200+ MB) I recommend using Apache and the mod_put extension [1]. I added an extension to mod_put so it would support resumable PUT operations which is very handy for "crap network connections." :-) I also created a patch for Mongrel to allow it to better handle large file uploads, but it hasn't been accepted by Zed. I can provide a copy to you if you are interested. cr [1] http://www.gknw.at/development/apache/httpd-2.0/unix/modules/ mod_put-2.0.8.tar.gz From cschlaefcke at wms-network.de Tue Apr 3 06:26:17 2007 From: cschlaefcke at wms-network.de (Christian Schlaefcke) Date: Tue, 3 Apr 2007 12:26:17 +0200 (CEST) Subject: [Backgroundrb-devel] Problem with blocking workers Message-ID: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> Hi Folks, I found this thread http://rubyforge.org/pipermail/backgroundrb-devel/2007-March/000790.html on this mailing list that describes pretty much the same behaviour that I experience as well. My problem is, that I have no chance for putting a "sleep 0" in an iteration because I have no Iteration. What I?m doing is executing a stored procedure in a sybase db that could take from few seconds up to hours to complete. The stored procedure gets executed and spools the result to another table where the user could access it later. So in my worker I don?t actually iterate over the results instead it waits until the stored procedure has finished. I need to get different jobs with different execution times executed at the same time. This is how I understand concurrency. How could I solve my problem? Are concurrent tasks possible anyway? Thanks & Regards, Christian Schlaefcke From Marc at SoftwareHackery.Com Tue Apr 3 11:21:18 2007 From: Marc at SoftwareHackery.Com (Marc Evans) Date: Tue, 3 Apr 2007 11:21:18 -0400 (EDT) Subject: [Backgroundrb-devel] Problem with blocking workers In-Reply-To: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> References: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> Message-ID: <20070403111910.M1048@me.softwarehackery.com> On Tue, 3 Apr 2007, Christian Schlaefcke wrote: > Hi Folks, > > I found this thread > > http://rubyforge.org/pipermail/backgroundrb-devel/2007-March/000790.html > > on this mailing list that describes pretty much the same behaviour that I > experience as well. > > My problem is, that I have no chance for putting a "sleep 0" in an > iteration because I have no Iteration. > > What I?m doing is executing a stored procedure in a sybase db that could > take from few seconds up to hours to complete. The stored procedure gets > executed and spools the result to another table where the user could > access it later. So in my worker I don?t actually iterate over the results > instead it waits until the stored procedure has finished. > > I need to get different jobs with different execution times executed at > the same time. This is how I understand concurrency. > > How could I solve my problem? Are concurrent tasks possible anyway? > > Thanks & Regards, > > Christian Schlaefcke Based on my experiences, you will have a difficult time doing what you want to do, at least if trying to use backgroundrb. If I were attacking this problem, I probably would use a far less elegant approach, that being to fork a new child in which the query is run. Hopefully, others on this list have a better suggestion than I do... - Marc From ezmobius at gmail.com Tue Apr 3 14:46:05 2007 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Tue, 3 Apr 2007 11:46:05 -0700 Subject: [Backgroundrb-devel] Problem with blocking workers In-Reply-To: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> References: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> Message-ID: <7E3E7BA0-3681-45D8-AAC1-753BB3F5D031@brainspl.at> It sounds to me like the sybase-ruby driver blocks when being called. If something blocks in a ruby thread then it can block all other ruby threads. That appears to be what it happening here. -Ezra On Apr 3, 2007, at 3:26 AM, Christian Schlaefcke wrote: > Hi Folks, > > I found this thread > > http://rubyforge.org/pipermail/backgroundrb-devel/2007-March/ > 000790.html > > on this mailing list that describes pretty much the same behaviour > that I > experience as well. > > My problem is, that I have no chance for putting a "sleep 0" in an > iteration because I have no Iteration. > > What I?m doing is executing a stored procedure in a sybase db that > could > take from few seconds up to hours to complete. The stored procedure > gets > executed and spools the result to another table where the user could > access it later. So in my worker I don?t actually iterate over the > results > instead it waits until the stored procedure has finished. > > I need to get different jobs with different execution times > executed at > the same time. This is how I understand concurrency. > > How could I solve my problem? Are concurrent tasks possible anyway? > > Thanks & Regards, > > Christian Schlaefcke > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273) From cschlaefcke at wms-network.de Tue Apr 3 15:22:27 2007 From: cschlaefcke at wms-network.de (Christian Schlaefcke) Date: Tue, 03 Apr 2007 21:22:27 +0200 Subject: [Backgroundrb-devel] Problem with blocking workers In-Reply-To: <20070403111910.M1048@me.softwarehackery.com> References: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> <20070403111910.M1048@me.softwarehackery.com> Message-ID: <4612A973.1060607@wms-network.de> Marc Evans schrieb: > Based on my experiences, you will have a difficult time doing what you > want to do, at least if trying to use backgroundrb. If I were > attacking this problem, I probably would use a far less elegant > approach, that being to fork a new child in which the query is run. > > Hopefully, others on this list have a better suggestion than I do... Could you provide an example for your suggestion? I played around with ruby threads before I switched to backgroundrb. I set up a test where have two threads that should download a large file from my server. This should pretty much simulate what I want to do at work. It seems to me that one thread works after the other and not concurrently. I also encountered the situation where none of the thread started (race condition?). I would be glad for any hint on this. Hopefully it is possible to solve with Backgroundrb... Regards, Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: cschlaefcke.vcf Type: text/x-vcard Size: 368 bytes Desc: not available Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070403/75a60675/attachment-0001.vcf From Marc at SoftwareHackery.Com Tue Apr 3 16:02:29 2007 From: Marc at SoftwareHackery.Com (Marc Evans) Date: Tue, 3 Apr 2007 16:02:29 -0400 (EDT) Subject: [Backgroundrb-devel] Problem with blocking workers In-Reply-To: <4612A973.1060607@wms-network.de> References: <40349.159.103.208.26.1175595977.squirrel@www.wms-network.de> <20070403111910.M1048@me.softwarehackery.com> <4612A973.1060607@wms-network.de> Message-ID: <20070403155728.L1048@me.softwarehackery.com> On Tue, 3 Apr 2007, Christian Schlaefcke wrote: > Marc Evans schrieb: >> Based on my experiences, you will have a difficult time doing what you want >> to do, at least if trying to use backgroundrb. If I were attacking this >> problem, I probably would use a far less elegant approach, that being to >> fork a new child in which the query is run. >> >> Hopefully, others on this list have a better suggestion than I do... > Could you provide an example for your suggestion? I played around with ruby > threads before I switched to backgroundrb. I set up a test where have two > threads that should download a large file from my server. This should pretty > much simulate what I want to do at work. It seems to me that one thread works > after the other and not concurrently. I also encountered the situation where > none of the thread started (race condition?). > > I would be glad for any hint on this. Hopefully it is possible to solve with > Backgroundrb... My suggestion was to use Process.fork, e.g. avoid the ruby thread issue. It's much heavier weight, and requires that you re-invent status tracking and related mechanisms. That said, I don't believe that you will encounter the blocking problems observed in thread-style code. - Marc From mengkuan at gmail.com Wed Apr 4 00:01:17 2007 From: mengkuan at gmail.com (Meng Kuan) Date: Wed, 4 Apr 2007 12:01:17 +0800 Subject: [Backgroundrb-devel] autostart Message-ID: <7D1B8557-9560-471B-9729-4347C1BC8470@gmail.com> Hi, Does backgroundrb 0.2.1 support autostart? I've tried to set up autostart by adding something like the following in backgroundrb.yml: autostart: 1: class: bar_worker args: '' However, that does not seem to work. I also tried the scheduling mechanism like this in backgroundrb_schedules.yml: bar: :class: :bar_worker :job_key: :bar1 :trigger_args: 0 * * * * * * This works, but after around 30 to 40 mins, the scheduler will fail and I get errors like this in the logs: 20070404-11:37:00 (23325) failed to find slave socket - (RuntimeError) 20070404-11:37:00 (23325) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:435:in `initialize' If I can get autostart to work, I will not need to use the scheduling mechanism. Any hints? If autostart is not available, how can one start a worker when the backgroundrb daemon is started? cheers, mengkuan From saimonmoore at gmail.com Wed Apr 4 04:04:32 2007 From: saimonmoore at gmail.com (Saimon Moore) Date: Wed, 4 Apr 2007 10:04:32 +0200 Subject: [Backgroundrb-devel] Password required for dejavu repo Message-ID: Since this morning, the server at http://svn.devjavu.com/backgroundrb is requiring authorized access. Could you guys look into it? Thanks, Saimon -- Saimon Moore Freelance Web Developer (Available for hire - For details visit http://saimonmoore.net) Skype: saimonmoore Yahoo IM: saimonmoore Google IM: saimonmoore From sergio.espeja at gmail.com Wed Apr 4 07:47:23 2007 From: sergio.espeja at gmail.com (Sergio Espeja) Date: Wed, 4 Apr 2007 13:47:23 +0200 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker Message-ID: Hello, I am trying to make DRb calls to a external server from a backgrounDRb worker, but I'm getting always the next exception when I make the remote call. > ERROR: > drbunix:///tmp/backgroundrb.11175/experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422051 > - # ///tmp/backgroundrb.11175/experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422051> > > 20070404-13:12:15 (11179) (druby://externalserver:43696) > /usr/lib/ruby/1.8/drb/drb.rb:733:in `open' > Doing the same DRb call from a script outside a backgrounDRb worker works. I don't know what to do in order to fix this, any idea? Thanks a lot! Sergio. -- Sergio Espeja http://spejman-on-rails.blogspot.com http://spejman.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070404/73190c53/attachment.html From ruby at geoffgarside.co.uk Wed Apr 4 08:09:00 2007 From: ruby at geoffgarside.co.uk (Geoff Garside) Date: Wed, 4 Apr 2007 13:09:00 +0100 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker In-Reply-To: References: Message-ID: <50C72281-FC8C-45E2-9798-8353261090AC@geoffgarside.co.uk> On 4 Apr 2007, at 12:47, Sergio Espeja wrote: > Hello, > > I am trying to make DRb calls to a external server from a > backgrounDRb worker, but I'm getting always the next exception when > I make the remote call. > ERROR: drbunix:///tmp/backgroundrb.11175/ > experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422 > 051 - # backgroundrb.11175/ > experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422 > 051> > 20070404-13:12:15 (11179) (druby://externalserver:43696) /usr/lib/ > ruby/1.8/drb/drb.rb:733:in `open' > > Doing the same DRb call from a script outside a backgrounDRb worker > works. > > I don't know what to do in order to fix this, any idea? > > Thanks a lot! > Sergio. > -- > Sergio Espeja > http://spejman-on-rails.blogspot.com > http://spejman.blogspot.com > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel Its likely that something in your worker is raising an exception and the worker is crashing out. Check the backgroundrb and backgroundrb_server logs. Regards, Geoff -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070404/5b63a38a/attachment.html From sergio.espeja at gmail.com Wed Apr 4 09:15:53 2007 From: sergio.espeja at gmail.com (Sergio Espeja) Date: Wed, 4 Apr 2007 15:15:53 +0200 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker In-Reply-To: <50C72281-FC8C-45E2-9798-8353261090AC@geoffgarside.co.uk> References: <50C72281-FC8C-45E2-9798-8353261090AC@geoffgarside.co.uk> Message-ID: Hi Geoff, thanks for your reply! The exception in the worker occurs when I call a method of a remote server object (DRbObject). Any idea? Thanks for your help! Sergio. On 4/4/07, Geoff Garside wrote: > > On 4 Apr 2007, at 12:47, Sergio Espeja wrote: > > Hello, > > I am trying to make DRb calls to a external server from a backgrounDRb > worker, but I'm getting always the next exception when I make the remote > call. > > > ERROR: > > drbunix:///tmp/backgroundrb.11175/experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422051 > > - # > ///tmp/backgroundrb.11175/experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928422051> > > > > 20070404-13:12:15 (11179) (druby://externalserver:43696) > > /usr/lib/ruby/1.8/drb/drb.rb:733:in `open' > > > > Doing the same DRb call from a script outside a backgrounDRb worker works. > > I don't know what to do in order to fix this, any idea? > > Thanks a lot! > Sergio. > -- > Sergio Espeja > http://spejman-on-rails.blogspot.com > http://spejman.blogspot.com > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > > Its likely that something in your worker is raising an exception and the > worker is crashing out. Check the backgroundrb and backgroundrb_server logs. > > Regards, > Geoff > -- Sergio Espeja http://spejman-on-rails.blogspot.com http://spejman.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070404/805bcc9a/attachment.html From ruby at geoffgarside.co.uk Wed Apr 4 09:29:20 2007 From: ruby at geoffgarside.co.uk (Geoff Garside) Date: Wed, 4 Apr 2007 14:29:20 +0100 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker In-Reply-To: References: <50C72281-FC8C-45E2-9798-8353261090AC@geoffgarside.co.uk> Message-ID: <2C8FBBC0-5FCD-4E52-BAF5-752A2751A4DE@geoffgarside.co.uk> Hi Sergio, Are you getting this error when you do something like w = MiddleMan.worker(:worker_key) w.do_work or when you actually call `MiddleMan.worker`? Usually its happened like this to me key = MiddleMan.new_worker(:class => :my_worker) -- Exception has been thrown here inside BackgrounDRb w = MiddleMan.worker(key) -- Get ERRNO exception here The exception thrown by the worker in BackgrounDRb is dumped out to log/backgroundrb.log. Because the worker crashes out my call to `worker` fails and raises an exception. Does that help? Geoff On 4 Apr 2007, at 14:15, Sergio Espeja wrote: > Hi Geoff, thanks for your reply! > > The exception in the worker occurs when I call a method of a remote > server object (DRbObject). Any idea? > > Thanks for your help! > Sergio. > > On 4/4/07, Geoff Garside wrote: >> On 4 Apr 2007, at 12:47, Sergio Espeja wrote: >> >>> Hello, >>> >>> I am trying to make DRb calls to a external server from a >>> backgrounDRb worker, but I'm getting always the next exception >>> when I make the remote call. >>>> ERROR: drbunix:///tmp/backgroundrb.11175/ >>>> experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928 >>>> 422051 - #>>> backgroundrb.11175/ >>>> experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928 >>>> 422051> >>>> 20070404-13:12:15 (11179) (druby://externalserver:43696) /usr/ >>>> lib/ruby/1.8/drb/drb.rb:733:in `open' >>> >>> Doing the same DRb call from a script outside a backgrounDRb >>> worker works. >>> >>> I don't know what to do in order to fix this, any idea? >>> >>> Thanks a lot! >>> Sergio. >>> -- >>> Sergio Espeja >>> http://spejman-on-rails.blogspot.com >>> http://spejman.blogspot.com >>> _______________________________________________ >>> Backgroundrb-devel mailing list >>> Backgroundrb-devel at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel >> >> Its likely that something in your worker is raising an exception >> and the worker is crashing out. Check the backgroundrb and >> backgroundrb_server logs. >> >> Regards, >> Geoff > -- > Sergio Espeja > http://spejman-on-rails.blogspot.com > http://spejman.blogspot.com From sergio.espeja at gmail.com Wed Apr 4 09:49:14 2007 From: sergio.espeja at gmail.com (Sergio Espeja) Date: Wed, 4 Apr 2007 15:49:14 +0200 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker In-Reply-To: <2C8FBBC0-5FCD-4E52-BAF5-752A2751A4DE@geoffgarside.co.uk> References: <50C72281-FC8C-45E2-9798-8353261090AC@geoffgarside.co.uk> <2C8FBBC0-5FCD-4E52-BAF5-752A2751A4DE@geoffgarside.co.uk> Message-ID: Hi Geoff, The error is inside the worker and is dumped into log/backgroundrb.log, the rails application doesn't notice it. I think I didn't explain me well, the backgroundrb worker starts and do its work well until it haves to make a remote call to a remote object using ruby DRb. The whole dump is: ERROR: drbunix:///tmp/backgroundrb.11251/experimenter_worker_fd50c992d69cc4395c8798878fb4231b_0_0.338539203351759 - # 20070404-13:41:30 (11255) (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:733:in `open' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:726:in `open' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1186:in `initialize' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1166:in `open' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' (druby://externalserver:36865) ./drb_method_executer.rb:27:in `method_missing' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' (druby://externalserver:36865) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' (druby://externalserver:36865) ./drb_method_executer.rb:101:in `start_up_executer_service' (druby://externalserver:36865) ./startup:5 /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/gga4r/drb_method_executer.rb:66:in `execute' /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/gga4r/gga4r_main.rb:152:in `evaluation_with_drb' /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/gga4r/gga4r_main.rb:147:in `evaluation_with_drb' /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/gga4r/gga4r_main.rb:85:in `evolve' /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/gga4r/gga4r_main.rb:82:in `evolve' /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/ga_for_aaile.rb:257:in `ga_for_signature' /home/espeja/Workspaces/eclipseRoR/wsaaile/config/../lib/ga_for_aaile.rb:256:in `ga_for_signature' /home/espeja/Workspaces/eclipseRoR/wsaaile/lib/workers/experimenter_worker.rb:248:in `do_calculate_experiment_results_for_ga' /home/espeja/Workspaces/eclipseRoR/wsaaile/lib/workers/experimenter_worker.rb:246:in `do_calculate_experiment_results_for_ga' /home/espeja/Workspaces/eclipseRoR/wsaaile/lib/workers/experimenter_worker.rb:59:in `do_work' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in `work_thread' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in `work_thread' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in `work_thread' /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/slave.rb:396:in `initialize' /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/slave.rb:391:in `initialize' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in `new_worker' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in `new_worker' /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/application.rb:186:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/daemonize.rb:208:in `call_as_daemon' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/application.rb:190:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons- 1.0.3/lib/daemons/application.rb:226:in `start' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/controller.rb:69:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons.rb:179:in `run_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons/cmdline.rb:94:in `catch_exceptions' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.3/lib/daemons.rb:178:in `run_proc' /home/espeja/Workspaces/eclipseRoR/wsaaile/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' script/backgroundrb:29 20070404-13:41:30 (11255) ERROR: drbunix:///tmp/backgroundrb.11251/experimenter_worker_fd50c992d69cc4395c8798878fb4231b_0_0.338539203351759 - # I've marked what I think are the important lines. The orange one is executed in the external server and is: DRb.start_service executer_service_url, drb_me And the yellow one executes a remote object call: @drb_querier.execute(chromosome, :fitness) where @drb_querier.execute makes a call to a DRbObject I hope I clarify more the question ;) I think that the problem is that isn't still posible to execute methods on remote objects using DRb with backgrounDRb. Do you agree? Do you have any idea to solve this? Thanks, Sergio. On 4/4/07, Geoff Garside wrote: > > Hi Sergio, > Are you getting this error when you do something like > > w = MiddleMan.worker(:worker_key) > w.do_work > > or when you actually call `MiddleMan.worker`? Usually its happened > like this to me > > key = MiddleMan.new_worker(:class => :my_worker) > -- Exception has been thrown here inside BackgrounDRb > w = MiddleMan.worker(key) > -- Get ERRNO exception here > > The exception thrown by the worker in BackgrounDRb is dumped out to > log/backgroundrb.log. Because the worker crashes out my call to > `worker` fails and raises an exception. > > Does that help? > > Geoff > > On 4 Apr 2007, at 14:15, Sergio Espeja wrote: > > > Hi Geoff, thanks for your reply! > > > > The exception in the worker occurs when I call a method of a remote > > server object (DRbObject). Any idea? > > > > Thanks for your help! > > Sergio. > > > > On 4/4/07, Geoff Garside wrote: > >> On 4 Apr 2007, at 12:47, Sergio Espeja wrote: > >> > >>> Hello, > >>> > >>> I am trying to make DRb calls to a external server from a > >>> backgrounDRb worker, but I'm getting always the next exception > >>> when I make the remote call. > >>>> ERROR: drbunix:///tmp/backgroundrb.11175/ > >>>> experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928 > >>>> 422051 - # >>>> backgroundrb.11175/ > >>>> experimenter_worker_3e899de945f0803a82cd257cfac316d0_0_0.0738904928 > >>>> 422051> > >>>> 20070404-13:12:15 (11179) (druby://externalserver:43696) /usr/ > >>>> lib/ruby/1.8/drb/drb.rb:733:in `open' > >>> > >>> Doing the same DRb call from a script outside a backgrounDRb > >>> worker works. > >>> > >>> I don't know what to do in order to fix this, any idea? > >>> > >>> Thanks a lot! > >>> Sergio. > >>> -- > >>> Sergio Espeja > >>> http://spejman-on-rails.blogspot.com > >>> http://spejman.blogspot.com > >>> _______________________________________________ > >>> Backgroundrb-devel mailing list > >>> Backgroundrb-devel at rubyforge.org > >>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > >> > >> Its likely that something in your worker is raising an exception > >> and the worker is crashing out. Check the backgroundrb and > >> backgroundrb_server logs. > >> > >> Regards, > >> Geoff > > -- > > Sergio Espeja > > http://spejman-on-rails.blogspot.com > > http://spejman.blogspot.com > > > > > -- Sergio Espeja http://spejman-on-rails.blogspot.com http://spejman.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070404/aa4af0fb/attachment-0001.html From phlip2005 at gmail.com Wed Apr 4 13:44:55 2007 From: phlip2005 at gmail.com (Phlip) Date: Wed, 4 Apr 2007 10:44:55 -0700 Subject: [Backgroundrb-devel] what version of BackgrounDRb do I have? Message-ID: <860c114f0704041044g388ae419y6fe947d6a34b7a80@mail.gmail.com> How to detect that from the files? -- Phlip http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!! From phlip2005 at gmail.com Wed Apr 4 14:32:38 2007 From: phlip2005 at gmail.com (Phlip) Date: Wed, 4 Apr 2007 11:32:38 -0700 Subject: [Backgroundrb-devel] unit tests? Message-ID: <860c114f0704041132x5eeff611vc2d4d7f0b0ad6897@mail.gmail.com> Ezra et al: I have this problem: http://rubyforge.org/pipermail/backgroundrb-devel/2006-December/000578.html Any progress on it? Or are unit tests simply impossible for now? -- Phlip http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!! From jason at heydsg.com Wed Apr 4 15:01:27 2007 From: jason at heydsg.com (Jason Sydes) Date: Wed, 4 Apr 2007 12:01:27 -0700 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker In-Reply-To: References: <50C72281-FC8C-45E2-9798-8353261090AC@geoffgarside.co.uk> <2C8FBBC0-5FCD-4E52-BAF5-752A2751A4DE@geoffgarside.co.uk> Message-ID: On 4/4/07, Sergio Espeja wrote: > I've marked what I think are the important lines. The orange one is > executed in the external server and is: > DRb.start_service executer_service_url, drb_me > And the yellow one executes a remote object call: > @drb_querier.execute(chromosome, :fitness) > where @drb_querier.execute makes a call to a DRbObject > > I hope I clarify more the question ;) I think that the problem is that > isn't still posible to execute methods on remote objects using DRb with > backgrounDRb. Do you agree? Do you have any idea to solve this? > My experience has been that the syntax is finicky. (... a bdrb worker is just a Slave object, which is a drbunix DRb_server, but you're trying to establish contact w/ a druby DRb_server...). I've seen the same kind of errors in the past when I was trying to do similar things. But we've lately been using BackgrounDRb:: MiddleManDRbObject.init(:uri => some_uri) inside of a worker to reliably establish contact w/ a second middleman. (And a second middleman is just a druby DRb_server, which is what you're trying to do.) You might want to take a look at the exact syntax of that init method. (That approach won't work with the 'parent' middleman of the worker; you'll get recycled object errors, as previously discussed on this list.) Hope that helps, Jason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070404/a00bc67d/attachment.html From gethemant at gmail.com Thu Apr 5 18:53:29 2007 From: gethemant at gmail.com (hemant) Date: Fri, 6 Apr 2007 04:23:29 +0530 Subject: [Backgroundrb-devel] unit tests? In-Reply-To: <860c114f0704041132x5eeff611vc2d4d7f0b0ad6897@mail.gmail.com> References: <860c114f0704041132x5eeff611vc2d4d7f0b0ad6897@mail.gmail.com> Message-ID: On 4/5/07, Phlip wrote: > Ezra et al: > > I have this problem: > > http://rubyforge.org/pipermail/backgroundrb-devel/2006-December/000578.html > > Any progress on it? Or are unit tests simply impossible for now? > I did found a way to test my workers eventually, i thought of writting a nice blog entry about that, but well... ;) Here is how i do it: # mocker code: # RAILS_ROOT/test/mocks/test/backgroundrn_mocker.rb module BackgrounDRb module Worker class RailsBase # class methods on the class attr_accessor :logger class << self def register return true end end def ivar var instance_variable_get :"@#{var}" end end end end # RAILS_ROOT/test/unit/message_worker_test.rb require File.dirname(__FILE__) + '/../mocks/test/backgroundrb_mocker' require File.dirname(__FILE__) + '/../test_helper' rails_lib = "#{RAILS_ROOT}/lib" bdrb_worker = "#{RAILS_ROOT}/lib/workers" $:.unshift(File.dirname(__FILE__),rails_lib,bdrb_worker) require "mocha" require "edit_message_worker" class TestEditMessageWorker < Test::Unit::TestCase def setup @edit_worker = EditMessageWorker.new @logger = mock @logger.stubs(:info).returns(true) @edit_worker.logger = @logger @edit_worker.stubs(:exit).returns(true) end def test_do_work end end -- gnufied ----------- There was only one Road; that it was like a great river: its springs were at every doorstep, and every path was its tributary. http://people.inxsasia.com/~hemant From gethemant at gmail.com Thu Apr 5 19:00:56 2007 From: gethemant at gmail.com (hemant) Date: Fri, 6 Apr 2007 04:30:56 +0530 Subject: [Backgroundrb-devel] unit tests? In-Reply-To: References: <860c114f0704041132x5eeff611vc2d4d7f0b0ad6897@mail.gmail.com> Message-ID: On 4/6/07, hemant wrote: > On 4/5/07, Phlip wrote: > > Ezra et al: > > > > I have this problem: > > > > http://rubyforge.org/pipermail/backgroundrb-devel/2006-December/000578.html > > > > Any progress on it? Or are unit tests simply impossible for now? > > > > I did found a way to test my workers eventually, i thought of writting > a nice blog entry about that, but well... ;) > > Here is how i do it: > > > # mocker code: > # RAILS_ROOT/test/mocks/test/backgroundrn_mocker.rb > > module BackgrounDRb > module Worker > > class RailsBase > # class methods on the class > attr_accessor :logger > class << self > def register > return true > end > end > > def ivar var > instance_variable_get :"@#{var}" > end > end > end > > end > > > > # RAILS_ROOT/test/unit/message_worker_test.rb > > require File.dirname(__FILE__) + '/../mocks/test/backgroundrb_mocker' > require File.dirname(__FILE__) + '/../test_helper' > > rails_lib = "#{RAILS_ROOT}/lib" > bdrb_worker = "#{RAILS_ROOT}/lib/workers" > $:.unshift(File.dirname(__FILE__),rails_lib,bdrb_worker) > > > require "mocha" > require "edit_message_worker" > > > class TestEditMessageWorker < Test::Unit::TestCase > def setup > @edit_worker = EditMessageWorker.new > @logger = mock > @logger.stubs(:info).returns(true) > @edit_worker.logger = @logger > @edit_worker.stubs(:exit).returns(true) > end > def test_do_work > > end > end > I have also patched bdrb to generate test cases for workers on above lines. I think, it was simply too much to load entire bdrb to run our puny tests, and hence i took the shortcut. Works for me. From gethemant at gmail.com Thu Apr 5 19:05:39 2007 From: gethemant at gmail.com (hemant) Date: Fri, 6 Apr 2007 04:35:39 +0530 Subject: [Backgroundrb-devel] Scheduled workers only run once unless you call self.delete inside the worker In-Reply-To: <998B296F-9C9D-43A2-9551-78A139CF6F58@brainspl.at> References: <572c318d0703051424q44d250c9mee4883cbb247182b@mail.gmail.com> <998B296F-9C9D-43A2-9551-78A139CF6F58@brainspl.at> Message-ID: On 3/6/07, Ezra Zygmuntowicz wrote: > > You are right you need to self.delete at the end of a worker to > destropy it. Bdrb doesn't auto desatroy any workers currently. > > -Ezra > > > On Mar 5, 2007, at 2:24 PM, David Balatero wrote: > > > I had a worker scheduled to run every minute with > > backgroundrb_schedules.yml: > > > > ebay_runner: > > :class: :ebay_auction_worker > > :job_key: :ebay_auction_runner > > :trigger_type: :cron_trigger > > :trigger_args: 0 * * * * * * > > > > This worker posts an auction to eBay from a queue of auctions every > > minute. I was having a problem where the worker would run ok the > > first time, but never any subsequent minutes after that. The way I > > fixed it was adding a call to self.delete at the end of every > > do_work() method: > > > > ---- > > class EbayAuctionWorker < BackgrounDRb::Rails > > # Set this worker to run every minute. > > attr_accessor :progress, :description > > > > def do_work(args) > > # This method is called in it's own new thread when you > > # call new worker. args is set to :args > > > > @progress = 0 > > @description = "Checking for eBay auctions and posting" > > > > logger.info("Checking to post an auction at #{Time.now.to_s}.") > > auction = EbayAuction.find(:first, :conditions => > > ["auction_status = ? AND post_on < ?", EbayAuction::STATUS_STRINGS > > [:queued], Time.now]) > > if auction > > logger.info("--- Posting auction: #{auction.title}") > > > > auction.post > > else > > logger.info("--- No auctions currently need posting.") > > end > > > > @progress = 100 > > > > logger.info("--- Finished auction check.") > > > > # Exit the thread, see what happens > > self.delete > > end > > end > > > > EbayAuctionWorker.register > > -------- > > > > > > Does this make sense? I never saw any mention of having to do this > > with scheduled workers in the documentation. Shouldn't the > > scheduler automatically clean up the worker with a self.delete, > > after @progress is >= 100? > > He is right to auto destroy the worker. But shouldn't bdrb use same worker for scheduling run, if method to be called is anything apart from do_work? it looks like an overhead to recreate the worker for repeated tasks. Assume, I am reading from a TCP server, and that would mean disconnecting and reconnecting on each iteration, not a very good thing. Also, I have seen for workers that run at time gap of more than 30 minutes or so, mysql connection gets dropped and worker throws couple of exceptions and fails to iterate the method execution. Is there any simple solution to get rid of this behaviour? -- gnufied ----------- There was only one Road; that it was like a great river: its springs were at every doorstep, and every path was its tributary. http://people.inxsasia.com/~hemant From jim.scott at yahoo.com Thu Apr 5 20:57:02 2007 From: jim.scott at yahoo.com (Jim Scott) Date: Thu, 5 Apr 2007 18:57:02 -0600 Subject: [Backgroundrb-devel] Best way to start a worker Message-ID: <006d01c777e6$7d623500$6a01a8c0@LIGHTFOOT> I have installed backgroundrb. I can start it. I have read everything I can get my hands on, and have read the list for a few months. I have written a worker, and am ready to start. This worker reads a queue, and processes it (sends queued emails). Sends 10, sleeps for a minute, sends 10 until done - very straightforward. My question is, what would be the best way to run this? I have assumed a cron like scheduled worker with a named key, but I don?t know. My worker looks like this: class EmailProcessorWorker < BackgrounDRb::Worker::RailsBase def do_work(args) queue = EmailQueue.new queue.process_queue end end EmailProcessorWorker.register I do all the work in a Rails Model, since I can unit test it outside of backgroundrb. Does anyone have a backgroundrb_schedules.yml file I could use as an example, and should this run something like every 15 minutes, etc., or should it start once, and loop endlessly inside do_work? Any thoughts or experience? Thanks for your input ? Jim Scott -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.446 / Virus Database: 268.18.26/746 - Release Date: 4/4/2007 1:09 PM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070405/6297ea40/attachment-0001.html From ezmobius at gmail.com Thu Apr 5 21:43:16 2007 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Thu, 5 Apr 2007 18:43:16 -0700 Subject: [Backgroundrb-devel] Best way to start a worker In-Reply-To: <006d01c777e6$7d623500$6a01a8c0@LIGHTFOOT> References: <006d01c777e6$7d623500$6a01a8c0@LIGHTFOOT> Message-ID: Jim- For a worker like that it shoudl just run all the time in a loop. Have it sleep for x second s and then wake up and process the queue, then sleep again. etc.. Cheers- -Ezra On Apr 5, 2007, at 5:57 PM, Jim Scott wrote: > I have installed backgroundrb. I can start it. I have read > everything I can get my hands on, and have read the list for a few > months. I have written a worker, and am ready to start. > > > > This worker reads a queue, and processes it (sends queued emails). > Sends 10, sleeps for a minute, sends 10 until done - very > straightforward. > > > > My question is, what would be the best way to run this? I have > assumed a cron like scheduled worker with a named key, but I don?t > know. My worker looks like this: > > > > class EmailProcessorWorker < BackgrounDRb::Worker::RailsBase > > def do_work(args) > > queue = EmailQueue.new > > queue.process_queue > > end > > end > > EmailProcessorWorker.register > > > > I do all the work in a Rails Model, since I can unit test it > outside of backgroundrb. > > > > Does anyone have a backgroundrb_schedules.yml file I could use as > an example, and should this run something like every 15 minutes, > etc., or should it start once, and loop endlessly inside do_work? > Any thoughts or experience? > > > > Thanks for your input ? > > > > Jim Scott > > > > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.5.446 / Virus Database: 268.18.26/746 - Release Date: > 4/4/2007 1:09 PM > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273) From me at seebq.com Fri Apr 6 00:22:37 2007 From: me at seebq.com (Charles Brian Quinn) Date: Fri, 6 Apr 2007 00:22:37 -0400 Subject: [Backgroundrb-devel] Background Processing Chapter in upcoming book Message-ID: <3a2de0cd0704052122g48ee80e0xae054765a37e3133@mail.gmail.com> Hi BackgrounDRber's, I'm helping Obie Fernandez write a chapter on Background Processing in his upcoming Pro Rails book, and wanted to see if I could run it past the group. Any input is appreciated. See attached (word doc, owch)! p.s. thanks Ezra skaar, and File.read(CONTRIBUTORS).collect Cheers, -- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios: www.highgroove.com slingshot hosting: www.slingshothosting.com main: 678.389.9462 fax: 678.826.0969 Ruby on Rails Bootcamp at the Big Nerd Ranch Intensive Ruby on Rails Training: http://www.bignerdranch.com/classes/ruby.shtml -------------- next part -------------- A non-text attachment was scrubbed... Name: Background Processes v3.doc Type: application/msword Size: 92672 bytes Desc: not available Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070406/bd90c1ed/attachment-0001.doc From me at seebq.com Fri Apr 6 00:27:10 2007 From: me at seebq.com (Charles Brian Quinn) Date: Fri, 6 Apr 2007 00:27:10 -0400 Subject: [Backgroundrb-devel] Best way to start a worker In-Reply-To: <006d01c777e6$7d623500$6a01a8c0@LIGHTFOOT> References: <006d01c777e6$7d623500$6a01a8c0@LIGHTFOOT> Message-ID: <3a2de0cd0704052127u40e01aect4afe6506c787675d@mail.gmail.com> Hi Jim, No offense to BackgrounDRb, but you might be better off with creating a class method in Email to find the last 10 (and send 'em), and then use script/runner in a cron job ( you can do every minute easy): the crontab entry might look like: */1 * * * * user /var/www/apps/your_app/script/runner 'EmailQueue.find_and_send_ten()' cheers! On 4/5/07, Jim Scott wrote: > > > > > I have installed backgroundrb. I can start it. I have read everything I > can get my hands on, and have read the list for a few months. I have > written a worker, and am ready to start. > > > > This worker reads a queue, and processes it (sends queued emails). Sends 10, > sleeps for a minute, sends 10 until done - very straightforward. > > > > My question is, what would be the best way to run this? I have assumed a > cron like scheduled worker with a named key, but I don't know. My worker > looks like this: > > > > class EmailProcessorWorker < BackgrounDRb::Worker::RailsBase > > def do_work(args) > > queue = EmailQueue.new > > queue.process_queue > > end > > end > > EmailProcessorWorker.register > > > > I do all the work in a Rails Model, since I can unit test it outside of > backgroundrb. > > > > Does anyone have a backgroundrb_schedules.yml file I could use as an > example, and should this run something like every 15 minutes, etc., or > should it start once, and loop endlessly inside do_work? Any thoughts or > experience? > > > > Thanks for your input ? > > > > Jim Scott > > > > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.5.446 / Virus Database: 268.18.26/746 - Release Date: 4/4/2007 > 1:09 PM > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > -- Charles Brian Quinn self-promotion: www.seebq.com highgroove studios: www.highgroove.com slingshot hosting: www.slingshothosting.com main: 678.389.9462 fax: 678.826.0969 Ruby on Rails Bootcamp at the Big Nerd Ranch Intensive Ruby on Rails Training: http://www.bignerdranch.com/classes/ruby.shtml From phlip2005 at gmail.com Fri Apr 6 14:19:23 2007 From: phlip2005 at gmail.com (Phlip) Date: Fri, 6 Apr 2007 11:19:23 -0700 Subject: [Backgroundrb-devel] Data changes in backgroundrb task don't show up in Rails Message-ID: <860c114f0704061119m6c2d5dd0r435fb8dac608b4ed@mail.gmail.com> BackgrounDRbers: We pass a data batch to a background task for processing. When it finishes, our GUI updates correctly (using the results hash). But the data records don't appear in the Rails connection to the database. Our stack is the HEAD BackgrounDRb, Rails 1.1.6, MySQL, and Apache/mongrel. As soon as I whack the background process with script/backgroundrb stop, the data records show up. This implies that a database connection somehow buffered them. The task worker uses a Record.transaction block once per record, and the entire system was working back when we use BackgrounDRb Classic. -- Phlip http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!! From cschlaefcke at wms-network.de Sat Apr 7 15:20:29 2007 From: cschlaefcke at wms-network.de (Christian Schlaefcke) Date: Sat, 07 Apr 2007 21:20:29 +0200 Subject: [Backgroundrb-devel] Error when starting Backgroundrb Message-ID: <4617EEFD.8050304@wms-network.de> Hi Folks, I want to set up backgroundrb on my laptop to do some tests. But when I run "scripts/background start" I get this error: ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in `setsid': the setsid() function is unimplemented on this machine (NotImplementedError) from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in `call_as_daemon' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:191:in `start_proc' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in `start' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:69:in `run' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in `run_proc' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in `call' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in `catch_exceptions' from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in `run_proc' from Test/vendor/plugins/backgroundrb-0.2.1/server/lib/backgroundrb_server.rb:301:in `run' from Test/vendor/plugins/backgroundrb-0.2.1/script/backgroundrb:31 I had the fork-not-implemented-error before so I use "require 'win32/process'" on my WinXP Notebook. This "setsid is unimplemented" is new and even google could not tell me much about it. Anybody here who experienced this error and could help with a workaround? Thanks & Regards, Christian -------------- next part -------------- A non-text attachment was scrubbed... Name: cschlaefcke.vcf Type: text/x-vcard Size: 368 bytes Desc: not available Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070407/e7170100/attachment.vcf From gethemant at gmail.com Sun Apr 8 09:41:58 2007 From: gethemant at gmail.com (hemant) Date: Sun, 8 Apr 2007 19:11:58 +0530 Subject: [Backgroundrb-devel] Error when starting Backgroundrb In-Reply-To: <4617EEFD.8050304@wms-network.de> References: <4617EEFD.8050304@wms-network.de> Message-ID: On 4/8/07, Christian Schlaefcke wrote: > Hi Folks, > > I want to set up backgroundrb on my laptop to do some tests. But when I > run "scripts/background start" I get this error: > > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in > `setsid': the setsid() function is unimplemented on this machine > (NotImplementedError) > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in > `call_as_daemon' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:191:in > `start_proc' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in > `start' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:69:in > `run' > from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in > `run_proc' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in > `call' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in > `catch_exceptions' > from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in > `run_proc' > from > Test/vendor/plugins/backgroundrb-0.2.1/server/lib/backgroundrb_server.rb:301:in > > `run' > from Test/vendor/plugins/backgroundrb-0.2.1/script/backgroundrb:31 > > I had the fork-not-implemented-error before so I use "require > 'win32/process'" on my WinXP Notebook. This "setsid is unimplemented" is > new and even google could not tell me much about it. Anybody here who > experienced this error and could help with a workaround? > > Thanks & Regards, > > Christian > You do realize that, bdrb is not supported on Windows platforms. From ruby at geoffgarside.co.uk Sun Apr 8 10:06:08 2007 From: ruby at geoffgarside.co.uk (Geoff Garside) Date: Sun, 8 Apr 2007 15:06:08 +0100 Subject: [Backgroundrb-devel] Error when starting Backgroundrb In-Reply-To: <4617EEFD.8050304@wms-network.de> References: <4617EEFD.8050304@wms-network.de> Message-ID: On 7 Apr 2007, at 20:20, Christian Schlaefcke wrote: > Hi Folks, > > I want to set up backgroundrb on my laptop to do some tests. But > when I > run "scripts/background start" I get this error: > > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb: > 160:in > `setsid': the setsid() function is unimplemented on this machine > (NotImplementedError) > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb: > 160:in > `call_as_daemon' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:191:in > `start_proc' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:227:in > `start' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb: > 69:in > `run' > from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb: > 182:in > `run_proc' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb: > 105:in > `call' > from > ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb: > 105:in > `catch_exceptions' > from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb: > 181:in > `run_proc' > from > Test/vendor/plugins/backgroundrb-0.2.1/server/lib/ > backgroundrb_server.rb:301:in > `run' > from Test/vendor/plugins/backgroundrb-0.2.1/script/backgroundrb:31 > > I had the fork-not-implemented-error before so I use "require > 'win32/process'" on my WinXP Notebook. This "setsid is > unimplemented" is > new and even google could not tell me much about it. Anybody here who > experienced this error and could help with a workaround? > > Thanks & Regards, > > Christian > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel As far as I know `sesid` is a unix call which sets the session id of a "leader" process. All processes then spawned by that process belong to the same session. Think a unix shell and any programs it starts up. I'm not sure what you need to do to get this work properly on windows but that might help you turn up something. - Geoff From sergio.espeja at gmail.com Sat Apr 7 13:55:33 2007 From: sergio.espeja at gmail.com (Sergio Espeja) Date: Sat, 7 Apr 2007 19:55:33 +0200 Subject: [Backgroundrb-devel] Exception when doing DRb remote calls from a BackgrounDRb worker Message-ID: Thanks for your reply I'll try to look inside the MiddleManDRbObject.initmethod for a solution. Sergio. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070407/6ab321c3/attachment.html From cschlaefcke at wms-network.de Sun Apr 8 14:15:49 2007 From: cschlaefcke at wms-network.de (Christian Schlaefcke) Date: Sun, 08 Apr 2007 20:15:49 +0200 Subject: [Backgroundrb-devel] Error when starting Backgroundrb In-Reply-To: References: <4617EEFD.8050304@wms-network.de> Message-ID: <46193155.1050009@wms-network.de> Geoff Garside schrieb: > On 7 Apr 2007, at 20:20, Christian Schlaefcke wrote: > >> Hi Folks, >> >> I want to set up backgroundrb on my laptop to do some tests. But when I >> run "scripts/background start" I get this error: >> >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in >> >> `setsid': the setsid() function is unimplemented on this machine >> (NotImplementedError) >> from >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in >> >> `call_as_daemon' >> from >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:191:in >> >> `start_proc' >> from >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in >> >> `start' >> from >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:69:in >> >> `run' >> from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in >> `run_proc' >> from >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in >> `call' >> from >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in >> `catch_exceptions' >> from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in >> `run_proc' >> from >> Test/vendor/plugins/backgroundrb-0.2.1/server/lib/backgroundrb_server.rb:301:in >> >> `run' >> from Test/vendor/plugins/backgroundrb-0.2.1/script/backgroundrb:31 >> >> I had the fork-not-implemented-error before so I use "require >> 'win32/process'" on my WinXP Notebook. This "setsid is unimplemented" is >> new and even google could not tell me much about it. Anybody here who >> experienced this error and could help with a workaround? >> >> Thanks & Regards, >> >> Christian >> >> >> >> _______________________________________________ >> Backgroundrb-devel mailing list >> Backgroundrb-devel at rubyforge.org >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > As far as I know `sesid` is a unix call which sets the session id of a > "leader" process. All processes then spawned by that process belong to > the same session. Think a unix shell and any programs it starts up. > > I'm not sure what you need to do to get this work properly on windows > but that might help you turn up something. > > - Geoff Hi, by the way: I had it working on another WinXP machine. Could it be that it depends on the CPU architecture (Intel/AMD)? Regards, Christian From gethemant at gmail.com Sun Apr 8 19:07:04 2007 From: gethemant at gmail.com (hemant) Date: Mon, 9 Apr 2007 04:37:04 +0530 Subject: [Backgroundrb-devel] Error when starting Backgroundrb In-Reply-To: <46193155.1050009@wms-network.de> References: <4617EEFD.8050304@wms-network.de> <46193155.1050009@wms-network.de> Message-ID: On 4/8/07, Christian Schlaefcke wrote: > Geoff Garside schrieb: > > On 7 Apr 2007, at 20:20, Christian Schlaefcke wrote: > > > >> Hi Folks, > >> > >> I want to set up backgroundrb on my laptop to do some tests. But when I > >> run "scripts/background start" I get this error: > >> > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in > >> > >> `setsid': the setsid() function is unimplemented on this machine > >> (NotImplementedError) > >> from > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:160:in > >> > >> `call_as_daemon' > >> from > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:191:in > >> > >> `start_proc' > >> from > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in > >> > >> `start' > >> from > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:69:in > >> > >> `run' > >> from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in > >> `run_proc' > >> from > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in > >> `call' > >> from > >> ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in > >> `catch_exceptions' > >> from ruby/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in > >> `run_proc' > >> from > >> Test/vendor/plugins/backgroundrb-0.2.1/server/lib/backgroundrb_server.rb:301:in > >> > >> `run' > >> from Test/vendor/plugins/backgroundrb-0.2.1/script/backgroundrb:31 > >> > >> I had the fork-not-implemented-error before so I use "require > >> 'win32/process'" on my WinXP Notebook. This "setsid is unimplemented" is > >> new and even google could not tell me much about it. Anybody here who > >> experienced this error and could help with a workaround? > >> > >> Thanks & Regards, > >> > >> Christian > >> > >> > >> > >> _______________________________________________ > >> Backgroundrb-devel mailing list > >> Backgroundrb-devel at rubyforge.org > >> http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > > > As far as I know `sesid` is a unix call which sets the session id of a > > "leader" process. All processes then spawned by that process belong to > > the same session. Think a unix shell and any programs it starts up. > > > > I'm not sure what you need to do to get this work properly on windows > > but that might help you turn up something. > > > > - Geoff > > Hi, > > by the way: I had it working on another WinXP machine. Could it be that > it depends on the CPU architecture (Intel/AMD)? You had 0.2.1 working on Windows XP? thats wierd. -- gnufied ----------- There was only one Road; that it was like a great river: its springs were at every doorstep, and every path was its tributary. http://people.inxsasia.com/~hemant From pascal at rettigconsulting.com Mon Apr 9 14:01:44 2007 From: pascal at rettigconsulting.com (Pascal Rettig) Date: Mon, 09 Apr 2007 14:01:44 -0400 Subject: [Backgroundrb-devel] Drb Connection error on multiple dispatch.fcgi 's Message-ID: <1176141704.5950.48.camel@localhost> Hi All, I'm using Backgroundrb as a general purpose long-running-task back-end (upload processing, email sending, etc), and it's been a great solution. However I've recently run into some some intermittent connection issues that have me baffled. I'm running on apache2/fcgid and the problem occurs in both devlopment and production mode. The problem seems to only trigger when there's enough load on the server to create a second dispatch.fcgi instance. When this happens, all of a sudden 1/2 of my new workers fail. I haven't tried a mongrel setup but I found a message from January 30th from jacobatzen.dk that had the same backtrace error, so I don't think it's uniquely a fcgid issue. Anyone come across this before or have any ideas where to look to fix? The worker I'm using for testing right now just spits out a log message (code at the bottom). Thanks, -Pascal >From log/background_server.log : Connection reset by peer - (DRb::DRbConnError) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read' /usr/lib/ruby/1.8/drb/drb.rb:563:in `load' /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply' /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply' /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message' /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing' /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open' /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' /usr/lib/ruby/1.8/drb/drb.rb:1069:in `respond_to?' /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/slave.rb:454:in `initialize' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in `new_worker' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in `new_worker' /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:187:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:197:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in `start' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:72:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in `run_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in `catch_exceptions' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in `run_proc' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' ./script/backgroundrb:29 The output from ./script/backgroundrb run: undefined method `wait' for nil:NilClass - (NoMethodError) /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:217:in `new_worker' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in `new_worker' /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:187:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:197:in `start_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in `start' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:72:in `run' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in `run_proc' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in `catch_exceptions' /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in `run_proc' /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' ./script/backgroundrb:29 In lib/workers/other_worker.rb: class OtherWorker < BackgrounDRb::Rails def do_work(args) logger.warn("In Do Work:") @processed = true end def finished? @processed == true end end OtherWorker.register From ezmobius at gmail.com Mon Apr 9 15:19:26 2007 From: ezmobius at gmail.com (Ezra Zygmuntowicz) Date: Mon, 9 Apr 2007 12:19:26 -0700 Subject: [Backgroundrb-devel] Drb Connection error on multiple dispatch.fcgi 's In-Reply-To: <1176141704.5950.48.camel@localhost> References: <1176141704.5950.48.camel@localhost> Message-ID: I think what is happening is that when apache spawns a new dispatch.fcgi it forks the first one to do so. And the drb connection cannot survive across a fork. The quikc fix is to use a set number of dispatchers so they each start with their own drb connection. I'm sorry to say that I haven't used bdrb with fastcgi as I gave up on fcgi a while ago. But it does appear that apache-fcgid forking a new process from the first is what causes the second forked process to not have a good drb connection. -Ezra On Apr 9, 2007, at 11:01 AM, Pascal Rettig wrote: > Hi All, > > > I'm using Backgroundrb as a general purpose long-running-task back-end > (upload processing, email sending, etc), and it's been a great > solution. > However I've recently run into some some intermittent connection > issues > that have me baffled. I'm running on apache2/fcgid and the problem > occurs in both devlopment and production mode. The problem seems to > only > trigger when there's enough load on the server to create a second > dispatch.fcgi instance. When this happens, all of a sudden 1/2 of > my new > workers fail. I haven't tried a mongrel setup but I found a message > from > January 30th from jacobatzen.dk that had the same backtrace error, > so I > don't think it's uniquely a fcgid issue. > > Anyone come across this before or have any ideas where to look to fix? > The worker I'm using for testing right now just spits out a log > message > (code at the bottom). > > Thanks, > > -Pascal > > >> From log/background_server.log : > > Connection reset by peer - (DRb::DRbConnError) > /usr/lib/ruby/1.8/drb/drb.rb:563:in `read' > /usr/lib/ruby/1.8/drb/drb.rb:563:in `load' > /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply' > /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply' > /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message' > /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing' > /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open' > /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' > /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' > /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' > /usr/lib/ruby/1.8/drb/drb.rb:1069:in `respond_to?' > /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/slave.rb:454:in > `initialize' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/middleman.rb:210:in `new_worker' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/middleman.rb:199:in `new_worker' > /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' > /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' > /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' > /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' > /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' > /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' > /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' > /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' > /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb_server.rb:315:in `run' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:187:in `start_proc' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:197:in `start_proc' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:227:in `start' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb: > 72:in `run' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in > `run_proc' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb: > 105:in > `catch_exceptions' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in > `run_proc' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb_server.rb:301:in `run' > ./script/backgroundrb:29 > > The output from ./script/backgroundrb run: > > undefined method `wait' for nil:NilClass - (NoMethodError) > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/middleman.rb:217:in `new_worker' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/middleman.rb:199:in `new_worker' > /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' > /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' > /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' > /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' > /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' > /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' > /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' > /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' > /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb_server.rb:315:in `run' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:187:in `start_proc' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:197:in `start_proc' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/ > application.rb:227:in `start' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb: > 72:in `run' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in > `run_proc' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb: > 105:in > `catch_exceptions' > /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in > `run_proc' > /home/pascal/projects/work/sitenoodler/vendor/plugins/backgroundrb/ > server/lib/backgroundrb_server.rb:301:in `run' > ./script/backgroundrb:29 > > > In lib/workers/other_worker.rb: > > class OtherWorker < BackgrounDRb::Rails > > def do_work(args) > > logger.warn("In Do Work:") > @processed = true > end > > def finished? > @processed == true > end > > > end > OtherWorker.register > > > > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > -- Ezra Zygmuntowicz -- Lead Rails Evangelist -- ez at engineyard.com -- Engine Yard, Serious Rails Hosting -- (866) 518-YARD (9273) From simon.wilkinson at 434wireless.com Fri Apr 13 11:49:17 2007 From: simon.wilkinson at 434wireless.com (simon.wilkinson at 434wireless.com) Date: Fri, 13 Apr 2007 11:49:17 -0400 Subject: [Backgroundrb-devel] deleting a worker Message-ID: <1189209.159621176479357710.JavaMail.servlet@perfora> Hi, I have workers being created from within my main rails app when it receives XML documents. These workers then parse the XML and store the data that it contains. I would like the worker to delete itself at the end of its do_work method, so that the main rails app does not need to keep track of all the generated workers. I currently have the following lines at the end of the do_work method: ActiveRecord::Base.connection.disconnect! self.delete When I track usage, I see that the number of tasks that are running is equal to the number of workers, and that they disappear when the worker finishes it's parsing. However, I also noticed that the amount of memory used goes up every time a worker is created. At first it goes up by a lot, and then after the task completes the memory drops back down again, but remains higher than it was before the worker was created. This value doesn't seem to reduce at all over longer periods of inactivity. Is this a result of how I am deleting the worker, or are there known memory issues with using BackgrounDRb, or is there another step that I am missing to accomplish this? Any help would be appreciated. Thanks, Simon From gethemant at gmail.com Fri Apr 13 22:58:36 2007 From: gethemant at gmail.com (hemant) Date: Sat, 14 Apr 2007 08:28:36 +0530 Subject: [Backgroundrb-devel] deleting a worker In-Reply-To: <1189209.159621176479357710.JavaMail.servlet@perfora> References: <1189209.159621176479357710.JavaMail.servlet@perfora> Message-ID: On 4/13/07, simon.wilkinson at 434wireless.com wrote: > Hi, > > I have workers being created from within my main rails app when it receives XML documents. These workers then parse the XML and store the data that it contains. I would like the worker to delete itself at the end of its do_work method, so that the main rails app does not need to keep track of all the generated workers. > > I currently have the following lines at the end of the do_work method: > > ActiveRecord::Base.connection.disconnect! > self.delete > > When I track usage, I see that the number of tasks that are running is equal to the number of workers, and that they disappear when the worker finishes it's parsing. However, I also noticed that the amount of memory used goes up every time a worker is created. At first it goes up by a lot, and then after the task completes the memory drops back down again, but remains higher than it was before the worker was created. This value doesn't seem to reduce at all over longer periods of inactivity. > > Is this a result of how I am deleting the worker, or are there known memory issues with using BackgrounDRb, or is there another step that I am missing to accomplish this? Any help would be appreciated. > If you are using 0.2.xx version then: You can use self.delete ( search the list, this question has been asked many times) Another easy solution is, to simply put "exit 0", which works because all the workers are separate processes anyway. From gethemant at gmail.com Wed Apr 18 10:59:26 2007 From: gethemant at gmail.com (hemant) Date: Wed, 18 Apr 2007 20:29:26 +0530 Subject: [Backgroundrb-devel] [BUG] worker stopping abruptly Message-ID: I have been facing this damn ugly issue since quite sometime. So this worker reads RSS feed from Yahoo news and stores it in a DB. The worker is scheduled to run at every 40 minutes and my background_schedules.yml looks like this: feed_worker: :class: :feed_worker :job_key: :feed_worker_key :worker_method: :do_work :trigger_args: :start: <%= Time.now + 5 %> :repeat_interval: <%= 40*60 %> So story is, worker stops working suddenly. no exception thrown either in background.log or background_server.log. To also, to make sure that backgroundrb threads doesn't eat my excpetion my code looks like this: def do_work(args) begin logger.info "Starting the Yahoo feed worker on : #{Time.now}" yahoo_url = "http://finance.yahoo.com/rss/headline?s=" symbol_list = NasdaqSymbols.find_all() symbol_list.each do|sym| temp_sym = sym.symbol.strip temp_url = yahoo_url + temp_sym get_feeds(temp_url,temp_sym) rescue next end rescue logger.info $! logger.info $!.backtrace end end ^^^ could be naive, but the point is, why worker stops suddenly without any errors? Earlier I thought, may be logging has bug and worker is still running. But a quick check from database says otherwise. Following a full text of backgroundrb.log file: 20070417-15:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 15:51:01 +0530 2007 20070417-16:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 16:31:01 +0530 2007 20070417-17:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 17:11:01 +0530 2007 20070417-17:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 17:51:01 +0530 2007 20070417-18:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 18:31:01 +0530 2007 20070417-19:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 19:11:01 +0530 2007 20070417-19:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 19:51:01 +0530 2007 20070417-20:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 20:31:01 +0530 2007 20070417-21:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 21:11:01 +0530 2007 20070417-21:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 21:51:01 +0530 2007 20070417-22:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 22:31:01 +0530 2007 20070417-23:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 23:11:01 +0530 2007 20070417-23:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Tue Apr 17 23:51:01 +0530 2007 20070418-00:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 00:31:01 +0530 2007 20070418-01:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 01:11:01 +0530 2007 20070418-01:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 01:51:01 +0530 2007 20070418-02:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 02:31:01 +0530 2007 20070418-03:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 03:11:01 +0530 2007 20070418-03:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 03:51:01 +0530 2007 20070418-04:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 04:31:01 +0530 2007 20070418-05:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 05:11:01 +0530 2007 20070418-05:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 05:51:01 +0530 2007 20070418-06:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 06:31:01 +0530 2007 20070418-07:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 07:11:01 +0530 2007 20070418-07:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 07:51:01 +0530 2007 20070418-08:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 08:31:01 +0530 2007 20070418-09:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 09:11:01 +0530 2007 20070418-09:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 09:51:01 +0530 2007 20070418-10:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 10:31:01 +0530 2007 20070418-11:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 11:11:01 +0530 2007 20070418-11:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 11:51:01 +0530 2007 20070418-12:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 12:31:01 +0530 2007 20070418-13:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 13:11:01 +0530 2007 20070418-13:51:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 13:51:01 +0530 2007 20070418-14:31:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 14:31:01 +0530 2007 20070418-15:11:01 (22525) ***** Starting the Yahoo feed worker on Time : Wed Apr 18 15:11:01 +0530 2007 From gethemant at gmail.com Wed Apr 18 11:01:25 2007 From: gethemant at gmail.com (hemant) Date: Wed, 18 Apr 2007 20:31:25 +0530 Subject: [Backgroundrb-devel] [BUG] worker stopping abruptly In-Reply-To: References: Message-ID: On 4/18/07, hemant wrote: > I have been facing this damn ugly issue since quite sometime. > > So this worker reads RSS feed from Yahoo news and stores it in a DB. > The worker is scheduled to run at every 40 minutes and my > background_schedules.yml looks like this: > > feed_worker: > :class: :feed_worker > :job_key: :feed_worker_key > :worker_method: :do_work > :trigger_args: > :start: <%= Time.now + 5 %> > :repeat_interval: <%= 40*60 %> > > So story is, worker stops working suddenly. no exception thrown either > in background.log or background_server.log. > > To also, to make sure that backgroundrb threads doesn't eat my > excpetion my code looks like this: > > def do_work(args) > begin > logger.info "Starting the Yahoo feed worker on : #{Time.now}" > yahoo_url = "http://finance.yahoo.com/rss/headline?s=" > symbol_list = NasdaqSymbols.find_all() > > symbol_list.each do|sym| > temp_sym = sym.symbol.strip > temp_url = yahoo_url + temp_sym > get_feeds(temp_url,temp_sym) rescue next > end > rescue > logger.info $! > logger.info $!.backtrace > end > end > > ^^^ could be naive, but the point is, why worker stops suddenly > without any errors? Earlier I thought, may be logging has bug and > worker is still running. But a quick check from database says > otherwise. > > Following a full text of backgroundrb.log file: > > > 20070417-15:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 15:51:01 +0530 2007 > 20070417-16:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 16:31:01 +0530 2007 > 20070417-17:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 17:11:01 +0530 2007 > 20070417-17:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 17:51:01 +0530 2007 > 20070417-18:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 18:31:01 +0530 2007 > 20070417-19:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 19:11:01 +0530 2007 > 20070417-19:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 19:51:01 +0530 2007 > 20070417-20:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 20:31:01 +0530 2007 > 20070417-21:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 21:11:01 +0530 2007 > 20070417-21:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 21:51:01 +0530 2007 > 20070417-22:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 22:31:01 +0530 2007 > 20070417-23:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 23:11:01 +0530 2007 > 20070417-23:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Tue Apr 17 23:51:01 +0530 2007 > 20070418-00:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 00:31:01 +0530 2007 > 20070418-01:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 01:11:01 +0530 2007 > 20070418-01:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 01:51:01 +0530 2007 > 20070418-02:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 02:31:01 +0530 2007 > 20070418-03:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 03:11:01 +0530 2007 > 20070418-03:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 03:51:01 +0530 2007 > 20070418-04:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 04:31:01 +0530 2007 > 20070418-05:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 05:11:01 +0530 2007 > 20070418-05:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 05:51:01 +0530 2007 > 20070418-06:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 06:31:01 +0530 2007 > 20070418-07:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 07:11:01 +0530 2007 > 20070418-07:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 07:51:01 +0530 2007 > 20070418-08:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 08:31:01 +0530 2007 > 20070418-09:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 09:11:01 +0530 2007 > 20070418-09:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 09:51:01 +0530 2007 > 20070418-10:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 10:31:01 +0530 2007 > 20070418-11:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 11:11:01 +0530 2007 > 20070418-11:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 11:51:01 +0530 2007 > 20070418-12:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 12:31:01 +0530 2007 > 20070418-13:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 13:11:01 +0530 2007 > 20070418-13:51:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 13:51:01 +0530 2007 > 20070418-14:31:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 14:31:01 +0530 2007 > 20070418-15:11:01 (22525) ***** Starting the Yahoo feed worker on Time > : Wed Apr 18 15:11:01 +0530 2007 > Another quick check in form of "ps aux|grep back" shows backgroundrb running, but sadly feed worker didn't do its iteration after 3:11 PM. From jodi at nnovation.ca Wed Apr 18 12:56:16 2007 From: jodi at nnovation.ca (Jodi Showers) Date: Wed, 18 Apr 2007 12:56:16 -0400 Subject: [Backgroundrb-devel] [BUG] worker stopping abruptly In-Reply-To: References: Message-ID: > On 18-Apr-07, at 11:01 AM, hemant wrote: > >> On 4/18/07, hemant wrote: >>> I have been facing this damn ugly issue since quite sometime. >>> >>> So this worker reads RSS feed from Yahoo news and stores it in a DB. >>> The worker is scheduled to run at every 40 minutes and my >>> background_schedules.yml looks like this: >>> >>> feed_worker: >>> :class: :feed_worker >>> :job_key: :feed_worker_key >>> :worker_method: :do_work >>> :trigger_args: >>> :start: <%= Time.now + 5 %> >>> :repeat_interval: <%= 40*60 %> >>> >>> So story is, worker stops working suddenly. no exception thrown >>> either >>> in background.log or background_server.log. >>> >>> To also, to make sure that backgroundrb threads doesn't eat my >>> excpetion my code looks like this: >>> >>> def do_work(args) >>> begin >>> logger.info "Starting the Yahoo feed worker on : #{Time.now}" >>> yahoo_url = "http://finance.yahoo.com/rss/headline?s=" >>> symbol_list = NasdaqSymbols.find_all() >>> >>> symbol_list.each do|sym| >>> temp_sym = sym.symbol.strip >>> temp_url = yahoo_url + temp_sym >>> get_feeds(temp_url,temp_sym) rescue next >>> end >>> rescue >>> logger.info $! >>> logger.info $!.backtrace >>> end >>> end >>> >>> ^^^ could be naive, but the point is, why worker stops suddenly >>> without any errors? Earlier I thought, may be logging has bug and >>> worker is still running. But a quick check from database says >>> otherwise. >>> >>> Following a full text of backgroundrb.log file: >>> >>> >>> 20070417-15:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 15:51:01 +0530 2007 >>> 20070417-16:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 16:31:01 +0530 2007 >>> 20070417-17:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 17:11:01 +0530 2007 >>> 20070417-17:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 17:51:01 +0530 2007 >>> 20070417-18:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 18:31:01 +0530 2007 >>> 20070417-19:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 19:11:01 +0530 2007 >>> 20070417-19:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 19:51:01 +0530 2007 >>> 20070417-20:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 20:31:01 +0530 2007 >>> 20070417-21:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 21:11:01 +0530 2007 >>> 20070417-21:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 21:51:01 +0530 2007 >>> 20070417-22:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 22:31:01 +0530 2007 >>> 20070417-23:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 23:11:01 +0530 2007 >>> 20070417-23:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Tue Apr 17 23:51:01 +0530 2007 >>> 20070418-00:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 00:31:01 +0530 2007 >>> 20070418-01:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 01:11:01 +0530 2007 >>> 20070418-01:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 01:51:01 +0530 2007 >>> 20070418-02:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 02:31:01 +0530 2007 >>> 20070418-03:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 03:11:01 +0530 2007 >>> 20070418-03:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 03:51:01 +0530 2007 >>> 20070418-04:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 04:31:01 +0530 2007 >>> 20070418-05:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 05:11:01 +0530 2007 >>> 20070418-05:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 05:51:01 +0530 2007 >>> 20070418-06:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 06:31:01 +0530 2007 >>> 20070418-07:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 07:11:01 +0530 2007 >>> 20070418-07:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 07:51:01 +0530 2007 >>> 20070418-08:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 08:31:01 +0530 2007 >>> 20070418-09:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 09:11:01 +0530 2007 >>> 20070418-09:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 09:51:01 +0530 2007 >>> 20070418-10:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 10:31:01 +0530 2007 >>> 20070418-11:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 11:11:01 +0530 2007 >>> 20070418-11:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 11:51:01 +0530 2007 >>> 20070418-12:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 12:31:01 +0530 2007 >>> 20070418-13:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 13:11:01 +0530 2007 >>> 20070418-13:51:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 13:51:01 +0530 2007 >>> 20070418-14:31:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 14:31:01 +0530 2007 >>> 20070418-15:11:01 (22525) ***** Starting the Yahoo feed worker on >>> Time >>> : Wed Apr 18 15:11:01 +0530 2007 >>> >> >> Another quick check in form of "ps aux|grep back" shows backgroundrb >> running, but sadly feed worker didn't do its iteration after 3:11 PM. >> woops. >> >> the other log(s) are empty as well? >> >> are you killing the worker every time? (self.delete). If not you may >> be running out of resources. >> >> have you monitored ram (other resources) while it's running? >> >> > > I am not deleting them, because I thought, creating a new worker, each > time would be a overhead. But I am not having an issues with RAM and > all. I have couple of workers running all the time,in some i use > self.delete or exit and in some I don't. others with more fu than myself can help you with the implications of not deleting the worker. Why wouldn't you always terminate your workers in the same manner? My advice would be to be less loose and more particular when terminating. ++ you really should reply to the group. Others can benefit from documenting your challenge. cheers, Jodi > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel From masonhale at gmail.com Wed Apr 18 14:03:16 2007 From: masonhale at gmail.com (Mason Hale) Date: Wed, 18 Apr 2007 13:03:16 -0500 Subject: [Backgroundrb-devel] [BUG] worker stopping abruptly In-Reply-To: References: Message-ID: <8bca3aa10704181103s596ece8n99d44d8415b2dd0d@mail.gmail.com> Try running the backgroundrb server in the foreground, this may show some messages that are not making to the logs. Start it with: ./script/backgroundrb run instead of: ./script/backgroundrb start Mason On 4/18/07, Jodi Showers wrote: > > > > On 18-Apr-07, at 11:01 AM, hemant wrote: > > > >> On 4/18/07, hemant wrote: > >>> I have been facing this damn ugly issue since quite sometime. > >>> > >>> So this worker reads RSS feed from Yahoo news and stores it in a DB. > >>> The worker is scheduled to run at every 40 minutes and my > >>> background_schedules.yml looks like this: > >>> > >>> feed_worker: > >>> :class: :feed_worker > >>> :job_key: :feed_worker_key > >>> :worker_method: :do_work > >>> :trigger_args: > >>> :start: <%= Time.now + 5 %> > >>> :repeat_interval: <%= 40*60 %> > >>> > >>> So story is, worker stops working suddenly. no exception thrown > >>> either > >>> in background.log or background_server.log. > >>> > >>> To also, to make sure that backgroundrb threads doesn't eat my > >>> excpetion my code looks like this: > >>> > >>> def do_work(args) > >>> begin > >>> logger.info "Starting the Yahoo feed worker on : #{Time.now}" > >>> yahoo_url = "http://finance.yahoo.com/rss/headline?s=" > >>> symbol_list = NasdaqSymbols.find_all() > >>> > >>> symbol_list.each do|sym| > >>> temp_sym = sym.symbol.strip > >>> temp_url = yahoo_url + temp_sym > >>> get_feeds(temp_url,temp_sym) rescue next > >>> end > >>> rescue > >>> logger.info $! > >>> logger.info $!.backtrace > >>> end > >>> end > >>> > >>> ^^^ could be naive, but the point is, why worker stops suddenly > >>> without any errors? Earlier I thought, may be logging has bug and > >>> worker is still running. But a quick check from database says > >>> otherwise. > >>> > >>> Following a full text of backgroundrb.log file: > >>> > >>> > >>> 20070417-15:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 15:51:01 +0530 2007 > >>> 20070417-16:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 16:31:01 +0530 2007 > >>> 20070417-17:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 17:11:01 +0530 2007 > >>> 20070417-17:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 17:51:01 +0530 2007 > >>> 20070417-18:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 18:31:01 +0530 2007 > >>> 20070417-19:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 19:11:01 +0530 2007 > >>> 20070417-19:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 19:51:01 +0530 2007 > >>> 20070417-20:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 20:31:01 +0530 2007 > >>> 20070417-21:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 21:11:01 +0530 2007 > >>> 20070417-21:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 21:51:01 +0530 2007 > >>> 20070417-22:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 22:31:01 +0530 2007 > >>> 20070417-23:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 23:11:01 +0530 2007 > >>> 20070417-23:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Tue Apr 17 23:51:01 +0530 2007 > >>> 20070418-00:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 00:31:01 +0530 2007 > >>> 20070418-01:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 01:11:01 +0530 2007 > >>> 20070418-01:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 01:51:01 +0530 2007 > >>> 20070418-02:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 02:31:01 +0530 2007 > >>> 20070418-03:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 03:11:01 +0530 2007 > >>> 20070418-03:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 03:51:01 +0530 2007 > >>> 20070418-04:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 04:31:01 +0530 2007 > >>> 20070418-05:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 05:11:01 +0530 2007 > >>> 20070418-05:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 05:51:01 +0530 2007 > >>> 20070418-06:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 06:31:01 +0530 2007 > >>> 20070418-07:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 07:11:01 +0530 2007 > >>> 20070418-07:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 07:51:01 +0530 2007 > >>> 20070418-08:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 08:31:01 +0530 2007 > >>> 20070418-09:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 09:11:01 +0530 2007 > >>> 20070418-09:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 09:51:01 +0530 2007 > >>> 20070418-10:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 10:31:01 +0530 2007 > >>> 20070418-11:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 11:11:01 +0530 2007 > >>> 20070418-11:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 11:51:01 +0530 2007 > >>> 20070418-12:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 12:31:01 +0530 2007 > >>> 20070418-13:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 13:11:01 +0530 2007 > >>> 20070418-13:51:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 13:51:01 +0530 2007 > >>> 20070418-14:31:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 14:31:01 +0530 2007 > >>> 20070418-15:11:01 (22525) ***** Starting the Yahoo feed worker on > >>> Time > >>> : Wed Apr 18 15:11:01 +0530 2007 > >>> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070418/5e85cd1a/attachment.html From gethemant at gmail.com Wed Apr 18 18:03:38 2007 From: gethemant at gmail.com (hemant) Date: Thu, 19 Apr 2007 03:33:38 +0530 Subject: [Backgroundrb-devel] [BUG] worker stopping abruptly In-Reply-To: <8bca3aa10704181103s596ece8n99d44d8415b2dd0d@mail.gmail.com> References: <8bca3aa10704181103s596ece8n99d44d8415b2dd0d@mail.gmail.com> Message-ID: On 4/18/07, Mason Hale wrote: > Try running the backgroundrb server in the foreground, this may show some > messages that are not making to the logs. > > Start it with: ./script/backgroundrb run > instead of: ./script/backgroundrb start > > Mason Sorry Jodi, for hitting a reply button than reply all. But I would definitely, try to delete the worker after use. My initial doubts were with Mysql dropping connections and I am working on those lines, but if nothing comes out, then I think, I would have to switch from bdrb. :( Mason, how would that help? And as is turns out, I have remote access to the machine and bug appears only after running for a full day!! So, I can't possibly run a SSH session for that much time. -- gnufied ----------- There was only one Road; that it was like a great river: its springs were at every doorstep, and every path was its tributary. http://people.inxsasia.com/hemant From masonhale at gmail.com Wed Apr 18 19:29:25 2007 From: masonhale at gmail.com (Mason Hale) Date: Wed, 18 Apr 2007 18:29:25 -0500 Subject: [Backgroundrb-devel] [BUG] worker stopping abruptly In-Reply-To: References: <8bca3aa10704181103s596ece8n99d44d8415b2dd0d@mail.gmail.com> Message-ID: <8bca3aa10704181629u6206ab77p15f333fea231a0ae@mail.gmail.com> On 4/18/07, hemant wrote: > > On 4/18/07, Mason Hale wrote: > > Try running the backgroundrb server in the foreground, this may show > some > > messages that are not making to the logs. > > > > Start it with: ./script/backgroundrb run > > instead of: ./script/backgroundrb start > > > > Mason > > Mason, how would that help? And as is turns out, I have remote access > to the machine and bug appears only after running for a full day!! > > So, I can't possibly run a SSH session for that much time. What's with the extra exclamation points? I'm just making a suggestion that could help you see error messages that won't make it the error logs. This is how I found a bus error when tracking down a previous issue I was encountering in backgroundrb: http://rubyforge.org/pipermail/backgroundrb-devel/2006-December/000594.html What I saw was similar to what you are seeing (a script stopping mid-run with no logging and no exception thrown), but in my case it would happen immediately. (My issue by the way had to do with loading the postgres native driver on Mac OS X). You might be able to run backgroundrb in the 'foreground' using nohup, and then save the std output to a file, ex: nohup ./script/backgroundrb run > testrun.log &2>1 The nohup command should prevent your command from terminating when you disconnect from the ssh terminal. Mason -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070418/cc60476c/attachment.html From ruby at geoffgarside.co.uk Mon Apr 23 06:01:09 2007 From: ruby at geoffgarside.co.uk (Geoff Garside) Date: Mon, 23 Apr 2007 11:01:09 +0100 Subject: [Backgroundrb-devel] Using mapped database connections Message-ID: This morning I switched over my backgroundrb server from development mode to production mode. After that all I was getting was errors when attempting to connect to the backgroundrb server. I checked the backgroundrb_server.log file and noticed the following exception 20070423-10:30:00 (29673) development database is not configured - (ActiveRecord::AdapterNotSpecified) In my database.yml file I have the production database setup as production: development to map the production database back to the development one. Unfortunately BackgrounDRb::Worker::RailsBase doesn't know how to handle these types of database references. I also had a quick play with loading up irb and then requiring the config/boot.rb and config/ environment.rb files. Once I had done this I could access all my Rails models as I would expect. So I was wondering why the RailsBase class tries to do more than just require BACKGROUNDRB_ROOT + '/config/boot.rb' require BACKGROUNDRB_ROOT + '/config/environment.rb' if BackgrounDRb needs to set its own rails_env so that it can use a different database or something like that then ENV['RAILS_ENV'] would need to be set before requiring config/boot.rb but that still wouldn't require loading the database config manually. Alternatively, if it has to be done the way it is then the following code could be used if db_config[rails_environment].kind_of?(String) db_config[rails_environment] = db_config[db_config[rails_environment]] end backgrounDRb would then be able to handle instances when one rails environments database is mapped to that of another. Is this right or have I missed something somewhere? - Geoff From simon.wilkinson at 434wireless.com Mon Apr 23 13:42:06 2007 From: simon.wilkinson at 434wireless.com (simon.wilkinson at 434wireless.com) Date: Mon, 23 Apr 2007 13:42:06 -0400 Subject: [Backgroundrb-devel] backgroundrb and REXML issue Message-ID: <22597304.38251177350126780.JavaMail.servlet@perfora> Hi, I have an application that receives XML files, and then hands each one off to a backgroundrb worker to parse and store the info. Everything was working, until I needed to support a larger character set. So I added the following to the top of my XML files: Adding this worked fine when I tested it without using backgroundrb. That is, when parsing within the main application, I get the expected behaviour. However, when I try to hand it off to a worker, I get the following error in backgroundrb.log: 20070423-13:36:41 (8456) Connection reset by peer - (DRb::DRbConnError) 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `read' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:563:in `load' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:629:in `recv_reply' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:918:in `recv_reply' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1192:in `send_message' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1083:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1167:in `open' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1082:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1100:in `with_friend' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1081:in `method_missing' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1069:in `respond_to?' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 205:in `expr' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 203:in `expr' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb: 125:in `match' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath_parser.rb:56:in `parse' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/rexml/xpath.rb:53:in `each' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/lib/workers/ parser_worker.rb:376:in `do_work' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:55:in `work_thread' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:69:in `work_thread' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/worker.rb:67:in `work_thread' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:396:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/slave-1.2.0/lib/ slave.rb:391:in `initialize' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:210:in `new_worker' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:199:in `new_worker' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1552:in `perform_without_block' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1512:in `perform' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1586:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1582:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1578:in `main_loop' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1424:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1344:in `initialize' 20070423-13:36:41 (8456) /usr/lib/ruby/1.8/drb/drb.rb:1624:in `start_service' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:187:in `start_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/daemonize.rb:192:in `call_as_daemon' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:191:in `start_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/application.rb:227:in `start' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/controller.rb:69:in `run' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons.rb:182:in `run_proc' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons/cmdline.rb:105:in `catch_exceptions' 20070423-13:36:41 (8456) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/ daemons.rb:181:in `run_proc' 20070423-13:36:41 (8456) /usr/local/src/rails/434Wireless/vendor/ plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' 20070423-13:36:41 (8456) script/backgroundrb:29 I tried removing the line that gives the XML version from the top of the same test file, and then it crashes where I expect it to, when it encounters invalid characters. Any ideas as to what is happening? Thanks, Simon From mm at beatsteaks.de Tue Apr 24 10:44:15 2007 From: mm at beatsteaks.de (Mathias Meyer) Date: Tue, 24 Apr 2007 16:44:15 +0200 Subject: [Backgroundrb-devel] Bug in Rails worker creation through scheduler Message-ID: <207D4040-B1B9-444F-BDFE-9D8D83061A6B@beatsteaks.de> Hi Folks! A small bug in the Scheduler code caused a small nightmare for me over the last hours. The symptoms were that my scheduled worker (Rails-based) wouldn't start. Nothing in the log files, no errors on the console, no hints whatsoever about what went wrong. So I filled up the code with logging statements and tracked down the problem to line 48 of server/lib/backgroundrb/scheduler.rb. The following statement in that line is bogus: 47:rescue 48: BackgrounDRb::ServerLogger.log_exception('scheduler', e) 49:end Since there's no e defined in the rescue statement, the above statement fails badly and therefore nothing gets logged anywhere. The fix is to either use this style: 47:rescue 48: BackgrounDRb::ServerLogger.log_exception('scheduler', $!) 49:end or this one: 47:rescue Exception => e 48: BackgrounDRb::ServerLogger.log_exception('scheduler', e) 49:end This issue seems to hit when something went wrong in the initialization of the worker, in my case, an error in the Rails environment setup. So if you're finding yourself wondering why your workers aren't doing anything, you should check that there's no exception being swallowed. Cheers, Mathias -- // Mathias Meyer // www.paperplanes.de From brandon at opensoul.org Thu Apr 26 14:04:48 2007 From: brandon at opensoul.org (Brandon Keepers) Date: Thu, 26 Apr 2007 14:04:48 -0400 Subject: [Backgroundrb-devel] pool_size Message-ID: <845CB40C-013B-46BF-9E11-3189AC9DE9E9@opensoul.org> I just started using BackgrounDRb this week and am really impressed. Good work Ezra and everyone else that has contributed. I'm writing a web app to subscribe to feeds with video enclosures and then play them in a flash player, or export them to MythTV or something like it. I'm using BackgrounDRb for the downloading and video processing. The docs mention that the pool_size method can be used to limit the number of threads that are active at once, and the other jobs will just queue up, but it doesn't seem to be limiting the number of threads. I see there were a couple other discussion threads about this in December and March, but neither of them really had a resolution. If this is a bug, I would be happy to dig into, but I wanted to check here first to make sure I wasn't the only one still having this problem. Thanks, Brandon -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070426/7eae75d8/attachment.bin From brandon at opensoul.org Thu Apr 26 15:46:26 2007 From: brandon at opensoul.org (Brandon Keepers) Date: Thu, 26 Apr 2007 15:46:26 -0400 Subject: [Backgroundrb-devel] looking for a worker that's gone Message-ID: <491CA236-7F0D-47D2-B7B3-514680959191@opensoul.org> In my app, I store the job key when a worker is fired off and then check back in periodically to see how its doing. The workers delete themselves when they're done. I would expect when I try to get a worker that has been deleted, I would get nil or a proper exception, but instead I get: >> MiddleMan.worker('foobar') NoMethodError: undefined method `object' for nil:NilClass from (drbunix:///tmp/backgroundrbunix_localhost_2000) /Users/ brandon/projects/backgroundrb/vendor/plugins/backgroundrb/server/lib/ backgroundrb/middleman.rb:396:in `worker' I've modified MiddleMan#worker to return nil if the job doesn't exist: def worker(key) worker = ex { @jobs[key].object if @jobs[key] } WorkerProxy.new(worker) unless worker.nil? end What do you think? Brandon -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070426/862fca55/attachment.bin From ruby at geoffgarside.co.uk Thu Apr 26 17:00:28 2007 From: ruby at geoffgarside.co.uk (Geoff Garside) Date: Thu, 26 Apr 2007 22:00:28 +0100 Subject: [Backgroundrb-devel] looking for a worker that's gone In-Reply-To: <491CA236-7F0D-47D2-B7B3-514680959191@opensoul.org> References: <491CA236-7F0D-47D2-B7B3-514680959191@opensoul.org> Message-ID: <29591F8E-D44A-4490-B949-E784D272E088@geoffgarside.co.uk> On 26 Apr 2007, at 20:46, Brandon Keepers wrote: > In my app, I store the job key when a worker is fired off and then > check back in periodically to see how its doing. The workers > delete themselves when they're done. I would expect when I try to > get a worker that has been deleted, I would get nil or a proper > exception, but instead I get: > > >> MiddleMan.worker('foobar') > NoMethodError: undefined method `object' for nil:NilClass > from (drbunix:///tmp/backgroundrbunix_localhost_2000) / > Users/brandon/projects/backgroundrb/vendor/plugins/backgroundrb/ > server/lib/backgroundrb/middleman.rb:396:in `worker' > > I've modified MiddleMan#worker to return nil if the job doesn't exist: > > def worker(key) > worker = ex { @jobs[key].object if @jobs[key] } > WorkerProxy.new(worker) unless worker.nil? > end > > What do you think? > > Brandon > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel Certainly seems like a good idea to me. It is a little messy to be testing for a NoMethodError when checking to see if a worker is still alive. At the same time though you can currently do MiddleMan.jobs.has_key?(:my_worker_key) Geoff From eden at mojiti.com Thu Apr 26 22:21:52 2007 From: eden at mojiti.com (Eden Li) Date: Fri, 27 Apr 2007 10:21:52 +0800 Subject: [Backgroundrb-devel] looking for a worker that's gone In-Reply-To: <29591F8E-D44A-4490-B949-E784D272E088@geoffgarside.co.uk> References: <491CA236-7F0D-47D2-B7B3-514680959191@opensoul.org> <29591F8E-D44A-4490-B949-E784D272E088@geoffgarside.co.uk> Message-ID: <1dd361e10704261921x44f6f73aodeb70dc9d6199c94@mail.gmail.com> (resending to include list, sorry for the double email Geoff) This was the workaround I ended up using in my app. Maybe this check should be done in #worker before it tries to evaluate #object on nil? Looks like there's a ticket covering this: http://backgroundrb.devjavu.com/projects/backgroundrb/ticket/47 On 4/27/07, Geoff Garside wrote: > On 26 Apr 2007, at 20:46, Brandon Keepers wrote: > > > In my app, I store the job key when a worker is fired off and then > > check back in periodically to see how its doing. The workers > > delete themselves when they're done. I would expect when I try to > > get a worker that has been deleted, I would get nil or a proper > > exception, but instead I get: > > > > >> MiddleMan.worker('foobar') > > NoMethodError: undefined method `object' for nil:NilClass > > from (drbunix:///tmp/backgroundrbunix_localhost_2000) / > > Users/brandon/projects/backgroundrb/vendor/plugins/backgroundrb/ > > server/lib/backgroundrb/middleman.rb:396:in `worker' > > > > I've modified MiddleMan#worker to return nil if the job doesn't exist: > > > > def worker(key) > > worker = ex { @jobs[key].object if @jobs[key] } > > WorkerProxy.new(worker) unless worker.nil? > > end > > > > What do you think? > > > > Brandon > > _______________________________________________ > > Backgroundrb-devel mailing list > > Backgroundrb-devel at rubyforge.org > > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > > Certainly seems like a good idea to me. > > It is a little messy to be testing for a NoMethodError when checking > to see if a worker is still alive. At the same time though you can > currently do > > MiddleMan.jobs.has_key?(:my_worker_key) > > Geoff > _______________________________________________ > Backgroundrb-devel mailing list > Backgroundrb-devel at rubyforge.org > http://rubyforge.org/mailman/listinfo/backgroundrb-devel > From mcroeder at gmail.com Fri Apr 27 17:37:15 2007 From: mcroeder at gmail.com (Mike Roeder) Date: Fri, 27 Apr 2007 16:37:15 -0500 Subject: [Backgroundrb-devel] Backgroundrb running under cygwin - having issue with Railsbase workers Message-ID: Hello all, We're currently running version .1.x in our environment. We would like to move to .2.x to take advantage of some of the improvements made running in a multi-process environment. We deploy to a linux environment but develop locally in Windows. I'm aware that a number of people have reported the ability to run .2.x of backgroundrb inside cygwin, and for the most part, I've been able to do the same. My initial test worker runs fine as a non-RailsBase worker. Once I switch it to load rails as part of the worker, I run into a problem with fork, namely: Resource temporarily unavailable - fork(2) - (Errno::EAGAIN). My debug log statements within Middleman show that backgroundrb_results and backgroundrb_logger get loaded fine, but any rails worker dies in the new_worker method of middleman, when attempting to create the slave_obj. Any ideas? I've done a significant amount of poking around the cygwin lists but haven't found anything of import to relate. I'd love to hear from any of you guys running this under cygwin to see if perhaps I've just done something wrong or need my expectations adjusted. I've copied a stack trace to the end of this message. Thanks! 20070427-13:34:40 (3312) Resource temporarily unavailable - fork(2) - (Errno::EAGAIN) 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/slave-1.1.0/lib/slave.rb:103:in `fork' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/slave-1.1.0/lib/slave.rb:103:in `fork' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/slave-1.1.0/lib/slave.rb:353:in `initialize' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:220:in `new' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:220:in `new_worker' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:36:in `dispatch' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `initialize' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `new' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/thread_pool.rb:22:in `dispatch' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb/middleman.rb:206:in `new_worker' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1555:in `__send__' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1555:in `perform_without_block' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1515:in `perform' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1589:in `main_loop' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1585:in `loop' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1585:in `main_loop' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1581:in `start' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1581:in `main_loop' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1430:in `run' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `start' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1427:in `run' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1347:in `initialize' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1627:in `new' 20070427-13:34:40 (3312) /usr/lib/ruby/1.8/drb/drb.rb:1627:in `start_service' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:315:in `run' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:187:in `call' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:187:in `start_proc' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:192:in `call' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/daemonize.rb:192:in `call_as_daemon' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:191:in `start_proc' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/application.rb:227:in `start' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/controller.rb:69:in `run' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:182:in `run_proc' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in `call' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons/cmdline.rb:105:in `catch_exceptions' 20070427-13:34:40 (3312) /usr/lib/ruby/gems/1.8/gems/daemons-1.0.5/lib/daemons.rb:181:in `run_proc' 20070427-13:34:40 (3312) /var/www/apps/some_app/vendor/plugins/backgroundrb/server/lib/backgroundrb_server.rb:301:in `run' 20070427-13:34:40 (3312) ./script/backgroundrb:29 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070427/a8fbffcc/attachment.html From brandon at opensoul.org Sat Apr 28 00:05:29 2007 From: brandon at opensoul.org (Brandon Keepers) Date: Sat, 28 Apr 2007 00:05:29 -0400 Subject: [Backgroundrb-devel] backgroundrb and fastthread Message-ID: <881BFA98-DFE5-4711-AC44-C5BD80BB2753@opensoul.org> I was just curious if anyone noticed any significant performance improvements using fastthread (or Ruby 1.8.6) with BackgrounDRb Brandon -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20070428/f6599638/attachment-0001.bin From skaar at waste.org Sat Apr 28 00:23:08 2007 From: skaar at waste.org (skaar) Date: Fri, 27 Apr 2007 23:23:08 -0500 Subject: [Backgroundrb-devel] backgroundrb and fastthread In-Reply-To: <881BFA98-DFE5-4711-AC44-C5BD80BB2753@opensoul.org> References: <881BFA98-DFE5-4711-AC44-C5BD80BB2753@opensoul.org> Message-ID: <20070428042308.GA7867@waste.org> * Brandon Keepers (brandon at opensoul.org) [070427 22:53]: > I was just curious if anyone noticed any significant performance > improvements using fastthread (or Ruby 1.8.6) with BackgrounDRb I've been running with fastthread for a while now, although with 1.8.5 and the standalone library. I don't think it makes a huge difference for backgroundrb, but I hope to commit this soon together with some other thread related fixes. /skaar -- ---------------------------------------------------------------------- |\|\ where in the | s_u_b_s_t_r_u_c_t_i_o_n | | >=========== W.A.S.T.E. | genarratologies |/|/ (_) is the wisdom | skaar at waste.org ---------------------------------------------------------------------- From zackchandler at gmail.com Mon Apr 30 13:02:05 2007 From: zackchandler at gmail.com (Zack Chandler) Date: Mon, 30 Apr 2007 10:02:05 -0700 Subject: [Backgroundrb-devel] monit config? Message-ID: <33841ac70704301002u5d42ebbehd4160834c8890e98@mail.gmail.com> Anyone have a a good monit config section for bgrb? I'm thinking something like this: check process backgroundrb with pidfile /var/www/apps/foo/current/log/backgroundrb.pid start program = "/var/www/apps/foo/current/script/backgroundrb start" stop program = "/var/www/apps/foo/current/script/backgroundrb stop" if failed host 127.0.0.1 port 2000 then restart if 5 restarts within 5 cycles then timeout Problem is the port test fails - I thought 2000 is the default... Thanks, Zack