[Backgroundrb-devel] mysterious crash of a particular worker

hemant gethemant at gmail.com
Wed Jan 16 10:16:10 EST 2008


Hi Meng,

On Jan 16, 2008 1:57 PM, Meng Kuan <mengkuan at gmail.com> wrote:
> Hi,
>
> I am using the latest checkout from backgroundrb release 1.0.1. I
> have a worker called status_checker that periodically (every minute)
> checks the status of certain hosts over the network. It works fine at
> the beginning but after a while the worker will mysteriously
> disappear and stop working. I have other workers running but they do
> not disappear like this worker does.
>
> These are the logs generated from logger.info calls for this
> particular worker.
>
> 2008-01-16 15:55 StatusChecker: check method started
> 2008-01-16 15:55 StatusChecker: processing host head0
> 2008-01-16 15:55 StatusChecker: processing domain webconsole
> 2008-01-16 15:55 StatusChecker: processing host host1
> 2008-01-16 15:55 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:55 StatusChecker: check method completed
> 2008-01-16 15:56 StatusChecker: check method started
> 2008-01-16 15:56 StatusChecker: processing host head0
> 2008-01-16 15:56 StatusChecker: processing domain webconsole
> 2008-01-16 15:56 StatusChecker: processing host host1
> 2008-01-16 15:56 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:56 StatusChecker: check method completed
> 2008-01-16 15:57 StatusChecker: check method started
> 2008-01-16 15:57 StatusChecker: processing host head0
> 2008-01-16 15:57 StatusChecker: processing domain webconsole
> 2008-01-16 15:57 StatusChecker: processing host host1
> 2008-01-16 15:57 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:57 StatusChecker: check method completed
> 2008-01-16 15:58 StatusChecker: check method started
> 2008-01-16 15:58 StatusChecker: processing host head0
> 2008-01-16 15:58 StatusChecker: processing domain webconsole
> 2008-01-16 15:58 StatusChecker: processing host host1
> 2008-01-16 15:58 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:58 StatusChecker: check method completed
> 2008-01-16 15:59 StatusChecker: check method started
> 2008-01-16 15:59 StatusChecker: processing host head0
> 2008-01-16 15:59 StatusChecker: processing domain webconsole
> 2008-01-16 15:59 StatusChecker: processing host host1
> 2008-01-16 15:59 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 15:59 StatusChecker: check method completed
> 2008-01-16 16:00 StatusChecker: check method started
> 2008-01-16 16:00 StatusChecker: processing host head0
> 2008-01-16 16:00 StatusChecker: processing domain webconsole
> 2008-01-16 16:00 StatusChecker: processing host host1
> 2008-01-16 16:00 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:00 StatusChecker: check method completed
> 2008-01-16 16:01 StatusChecker: check method started
> 2008-01-16 16:01 StatusChecker: processing host head0
> 2008-01-16 16:01 StatusChecker: processing domain webconsole
> 2008-01-16 16:01 StatusChecker: processing host host1
> 2008-01-16 16:01 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:01 StatusChecker: check method completed
> 2008-01-16 16:02 StatusChecker: check method started
> 2008-01-16 16:02 StatusChecker: processing host head0
> 2008-01-16 16:02 StatusChecker: processing domain webconsole
> 2008-01-16 16:02 StatusChecker: processing host host1
> 2008-01-16 16:02 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:02 StatusChecker: check method completed
> 2008-01-16 16:03 StatusChecker: check method started
> 2008-01-16 16:03 StatusChecker: processing host head0
> 2008-01-16 16:03 StatusChecker: processing domain webconsole
> 2008-01-16 16:03 StatusChecker: processing host host1
> 2008-01-16 16:03 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:03 StatusChecker: check method completed
> 2008-01-16 16:04 StatusChecker: check method started
> 2008-01-16 16:04 StatusChecker: processing host head0
> 2008-01-16 16:04 StatusChecker: processing domain webconsole
> 2008-01-16 16:04 StatusChecker: processing host host1
> 2008-01-16 16:04 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:04 StatusChecker: check method completed
> 2008-01-16 16:05 StatusChecker: check method started
> 2008-01-16 16:05 StatusChecker: processing host head0
> 2008-01-16 16:05 StatusChecker: processing domain webconsole
> 2008-01-16 16:05 StatusChecker: processing host host1
> 2008-01-16 16:05 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:05 StatusChecker: check method completed
> 2008-01-16 16:06 StatusChecker: check method started
> 2008-01-16 16:06 StatusChecker: processing host head0
> 2008-01-16 16:06 StatusChecker: processing domain webconsole
> 2008-01-16 16:06 StatusChecker: processing host host1
> 2008-01-16 16:06 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:06 StatusChecker: check method completed
> 2008-01-16 16:07 StatusChecker: check method started
> 2008-01-16 16:07 StatusChecker: processing host head0
> 2008-01-16 16:07 StatusChecker: processing domain webconsole
> 2008-01-16 16:07 StatusChecker: processing host host1
> 2008-01-16 16:07 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:07 StatusChecker: check method completed
> 2008-01-16 16:08 StatusChecker: check method started
> 2008-01-16 16:08 StatusChecker: processing host head0
> 2008-01-16 16:08 StatusChecker: processing domain webconsole
> 2008-01-16 16:08 StatusChecker: processing host host1
> 2008-01-16 16:08 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:08 StatusChecker: check method completed
> 2008-01-16 16:09 StatusChecker: check method started
> 2008-01-16 16:09 StatusChecker: processing host head0
> 2008-01-16 16:09 StatusChecker: processing domain webconsole
> 2008-01-16 16:09 StatusChecker: processing host host1
> 2008-01-16 16:09 StatusChecker: error encountered for host host1:
> RuntimeError: Unable to ssh to 192.168.1.2 as root
> 2008-01-16 16:09 StatusChecker: check method completed
> 2008-01-16 16:10 StatusChecker: check method started
> 2008-01-16 16:10 StatusChecker: processing host head0
> 2008-01-16 16:10 StatusChecker: processing domain webconsole
> 2008-01-16 16:10 StatusChecker: processing host host1
>
> After 16:10 the worker seems to have disappeared. I found the
> following message in backgroundrb_debug.log:
>
> /home/webcon/apps/webconsole/releases/20080115103838/vendor/plugins/
> backgroundrb/server/meta_worker.rb:329:in `check_for
> _timer_events': undefined method `info' for nil:NilClass (NoMethodError)
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/core.rb:148:in `
> start_reactor'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/core.rb:147:in `
> loop'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/core.rb:147:in `
> start_reactor'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/worker.rb:21:in
> `start_worker'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :134:in `fork_and_load'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :98:in `load_workers'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :93:in `each'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :93:in `load_workers'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/framework/packet_master.rb
> :19:in `run'
>          from /home/webcon/apps/webconsole/releases/20080115103838/
> vendor/plugins/backgroundrb/server/master_worker.rb:16
> 3:in `initialize'
>          from script/backgroundrb:41:in `new'
>          from script/backgroundrb:41
>
>
> The code for this worker can be found here:
>
>    https://svn.sxven.com/webconsole/trunk/lib/workers/status_checker.rb
>
>
> Any pointers on how to debug this is much appreciated.
>

Can you sync your plugin with trunk and see if its fixed? Looks like
we had a bug in logger functionality.



-- 
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org


More information about the Backgroundrb-devel mailing list