[Backgroundrb-devel] Deploying to a staging server using Capistrano: how to start up BackgrounDRb?
Bobby Santiago
bobby.santiago at gmail.com
Mon May 5 20:27:42 EDT 2008
Hey, Stevie,
When I changed the port number in backgroundrb.yml to match the
mongrel ports in my conf file, the connection error went away. I
looked into /var/www/apps/my_app/shared/pids, and
backgroundrb_9201.pid is there. In the /var/www/apps/my_app/shared/log
dir, I have
backgroundrb_9201.log
backgroundrb_9201_debug.log
backgroundrb_server_9201.log
When I did ps aux | grep thumbnail_generator_worker, I got:
deploy 12279 0.0 0.1 1816 496 pts/0 R+ 20:07 0:00 grep
thumbnail_generator_worker
So I guess it's not running.
After setting :log: foreground in backgroundrb.yml and deploying, I
tailed the backgroundrb_9201.log, backgroundrb_9201_debug.log, and
backgroundrb_server_9201.log files. No output there.
In development, the backgroundrb_1106.log file has the correct output:
BillingWorker#create called...
billing_worker started
Schedules for worker loaded
ThumbnailGeneratorWorker#create called...
thumbnail_generator_worker started
Schedules for worker loaded
generate 37
There is no feedback from the ask_status method, I get nil when I
inspect the variable in staging.log.
I'm running out of ideas.
Bobby
On May 6, 2008, at 12:14 AM, Stevie Clifton wrote:
> Hey Bobby,
>
> I think you're dealing with two separate issues here. The first error
> you're seeing with capistrano is b/c capistrano doesn't auto-link the
> /RAILS_ROOT/tmp/pids folder, so backgroundrb is unable to create a pid
> file there. I do something like the following (untested):
>
> after "deploy:update_code", "deploy:symlink_pids"
>
> namespace :deploy
> desc "Symlink tmp/pids folder"
> task :symlink_pids do
> run "mkdir -p #{release_path}/tmp"
> run "ln -nfs #{shared_path}/pids #{release_path}/tmp/pids"
> end
> end
>
> For the second issue, it's a little difficult knowing why rails can't
> see your worker without seeing some bdrb log output. Have you
> verified that the worker is actually running? (ps aux | grep
> <worker_name>). Also, have you checked your backgroundrb_server.log
> to see if it's dying on instantiation because of some other
> dependencies? I'd recommend launching backgroundrb manually on
> staging and setting :log: foreground in backgroundrb.yml so you can
> see what's going on with your worker.
>
> stevie
>
>
> On Sun, May 4, 2008 at 11:02 PM, Bobby Santiago
> <bobby.santiago at gmail.com> wrote:
>> Hi.
>>
>> I am using BackgrounDRb to process thumbnails and upload to S3 -
>> things are
>> hunky-dory in development (thumbs are generated, these are uploaded
>> to S3,
>> the metadata is saved to trhe DB, and I get a nice status page
>> updated by
>> periodic calls via ask_status), but when I tried to deploy to our
>> staging
>> server and stop/start BackgrounDRb via Capistrano, things blew up -
>> well,
>> not exactly, but when the app tried to hand off the thumbnail
>> generation, I
>> got a "Could not connect to the BackgrounDRb server" error.
>>
>> Initially, when I didn't start the BDRb server manually, I got:
>>
>> * executing `after_update_code'
>> * executing `set_env_staging'
>> ENV['RAILS_ENV'] = staging
>> * executing `restart_backgroundrb'
>> * executing `stop_backgroundrb'
>> * executing "cd /var/www/apps/my_app/releases/20080501133415 &&
>> ./script/backgroundrb -e staging stop"
>> servers: ["staging.my_app.com"]
>> [staging.my_app.com] executing command
>> ** [out :: staging.my_app.com] ./script/backgroundrb:46:in
>> `initialize'
>> ** [out :: staging.my_app.com] :
>> ** [out :: staging.my_app.com] No such file or directory -
>> /var/www/apps/my_app/releases/20080501133415/tmp/pids/
>> backgroundrb_22222.pid
>> ** [out :: staging.my_app.com] (
>> ** [out :: staging.my_app.com] Errno::ENOENT
>> ** [out :: staging.my_app.com] )
>> ** [out :: staging.my_app.com] from ./script/backgroundrb:46:in
>> `open'
>> ** [out :: staging.my_app.com] from ./script/backgroundrb:46
>> command finished
>> *** [deploy:update_code] rolling back
>> * executing "rm -rf /var/www/apps/my_app/releases/20080501133415;
>> true"
>> servers: ["staging.my_app.com"]
>> [staging.my_app.com] executing command
>> command finished
>>
>> So I tried manually starting backgroundrb on the staging server,,
>> and the
>> deployment went through successfully. But when I tried generating
>> thumbs
>> with a background worker, I got the "Could not connect to the
>> BackgrounDRb
>> server" error.
>>
>> I check out from GitHub, and use Capistrano-ext for multistage
>> deployments:
>>
>> cap staging deploy
>>
>> Before I deploy, I SSH into the VPS and do
>> script/backgroundrb -e staging start
>>
>> in my config/deploy/staging.rb file:
>> ----------------
>> set :rails_env, 'staging'
>> ...
>>
>> task :after_update_code do
>> set_env_staging
>> copy_mongrel_cluster_config
>> restart_backgroundrb
>> end
>>
>> desc "Sets the environment variable RAILS_ENV='staging'."
>> task :set_env_staging do
>> ENV['RAILS_ENV'] = 'staging'
>> puts "ENV['RAILS_ENV'] = #{ENV['RAILS_ENV']}"
>> end
>>
>> desc "Copying the right mongrel cluster config for the current stage
>> environment."
>> task :copy_mongrel_cluster_config do
>> run "cp -f #{release_path}/config/deploy/staging/mongrel_cluster.yml
>> #{release_path}/config/mongrel_cluster.yml"
>> end
>>
>> desc "Start the backgroundrb server"
>> task :start_backgroundrb , :roles => :app do
>> run "cd #{current_path} && nohup ./script/backgroundrb start -- -r
>> staging > #{current_path}/log/backgroundrb-cap.log 2>&1"
>> end
>>
>>
>> desc "Stop the backgroundrb server"
>> task :stop_backgroundrb, :roles => :app do
>> run "cd #{release_path} && ./script/backgroundrb -e staging stop"
>> end
>>
>> desc "Restart the backgroundrb server"
>> task :restart_backgroundrb, :roles => :app do
>> stop_backgroundrb
>> start_backgroundrb
>> end
>>
>> -------
>> Here's the exception_notifier backtrace:
>>
>> [RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:81:in
>> `dump_object'
>> [RAILS_ROOT]/vendor/plugins/backgroundrb/lib/backgroundrb.rb:98:in
>> `new_worker'
>> [RAILS_ROOT]/app/models/uploader.rb:54:in `start_thumbnail_worker'
>> [RAILS_ROOT]/app/controllers/photos_controller.rb:49:in `create'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> mime_responds.rb:106:in
>> `call'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> mime_responds.rb:106:in
>> `respond_to'
>> [RAILS_ROOT]/app/controllers/photos_controller.rb:47:in `create'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> base.rb:1158:in
>> `send'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> base.rb:1158:in
>> `perform_action_without_filters'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> filters.rb:697:in
>> `call_filters'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> filters.rb:689:in
>> `perform_action_without_benchmark'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> benchmarking.rb:68:in
>> `perform_action_without_rescue'
>> /usr/lib/ruby/1.8/benchmark.rb:293:in `measure'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> benchmarking.rb:68:in
>> `perform_action_without_rescue'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> rescue.rb:199:in
>> `perform_action_without_caching'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> caching.rb:678:in
>> `perform_action'
>>
>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/
>> connection_adapters/abstract/query_cache.rb:33:in
>> `cache'
>>
>> /usr/lib/ruby/gems/1.8/gems/activerecord-2.0.2/lib/active_record/
>> query_cache.rb:8:in
>> `cache'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> caching.rb:677:in
>> `perform_action'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> base.rb:524:in
>> `send'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> base.rb:524:in
>> `process_without_filters'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> filters.rb:685:in
>> `process_without_session_management_support'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> session_management.rb:123:in
>> `sass_old_process'
>> /usr/lib/ruby/gems/1.8/gems/haml-1.8.1/lib/sass/plugin/rails.rb:
>> 15:in
>> `process'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> base.rb:388:in
>> `process'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> dispatcher.rb:171:in
>> `handle_request'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> dispatcher.rb:115:in
>> `dispatch'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> dispatcher.rb:126:in
>> `dispatch_cgi'
>>
>> /usr/lib/ruby/gems/1.8/gems/actionpack-2.0.2/lib/action_controller/
>> dispatcher.rb:9:in
>> `dispatch'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:76:in
>> `process'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in
>> `synchronize'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/rails.rb:74:in
>> `process'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:159:in
>> `process_client'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in
>> `each'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:158:in
>> `process_client'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
>> `run'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
>> `initialize'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
>> `new'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:285:in
>> `run'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
>> `initialize'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
>> `new'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel.rb:268:in
>> `run'
>>
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/
>> configurator.rb:282:in
>> `run'
>>
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/
>> configurator.rb:281:in
>> `each'
>>
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/
>> configurator.rb:281:in
>> `run'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:128:in
>> `run'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/lib/mongrel/command.rb:
>> 212:in
>> `run'
>> /usr/lib/ruby/gems/1.8/gems/mongrel-1.1.3/bin/mongrel_rails:281
>> /usr/bin/mongrel_rails:19:in `load'
>> /usr/bin/mongrel_rails:19
>>
>> My backgroundrb.yml file:
>> :backgroundrb:
>> :ip: 0.0.0.0
>>
>> :development:
>> :backgroundrb:
>> :port: 11006 # use port 11006
>> :log: foreground # foreground mode,print log messages on console
>> :debug_log: true
>>
>> :staging:
>> :backgroundrb:
>> :port: 22222 # use port 22222
>> :lazy_load: true # do not load models eagerly
>> :debug_log: false # disable log workers and other logging
>>
>> :production:
>> :backgroundrb:
>> :port: 33333 # use port 33333
>> :lazy_load: true # do not load models eagerly
>> :debug_log: false # disable log workers and other logging
>>
>> My worker:
>> class ThumbnailGeneratorWorker < BackgrounDRb::MetaWorker
>> set_worker_name :thumbnail_generator_worker
>> set_no_auto_load(true)
>>
>> def create(args = nil)
>> register_status(:percent_complete => 0)
>> args.each_with_index do |uploader_id, index|
>> @uploader = Uploader.find(uploader_id)
>> @uploader.generate_thumbnails
>> percent_complete = ((index + 1) * 100) / args.length
>> logger.info "Thumbnail generation is #{percent_complete}%
>> complete..."
>> register_status(:percent_complete => percent_complete)
>> end
>> exit
>>
>> end
>> end
>>
>> Any hints/help/ideas would be much appreciated.
>>
>> Bobby
>> _______________________________________________
>> Backgroundrb-devel mailing list
>> Backgroundrb-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>>
More information about the Backgroundrb-devel
mailing list