[Backgroundrb-devel] adding results from threads to a collection and returning it
neilmock at gmail.com
Tue Jun 10 19:56:27 EDT 2008
Forgive me if this has been addressed somewhere, but I have searched and
can't come up with anything.
I am basically trying to distribute several web page scraping tasks among
different threads, and have the results from each added to an Array which is
ultimately returned by the backgroundrb worker. Here is an example of what
I'm trying to do in a worker method:
pages = Array.new
pages_to_scrape.each do |url|
thread_pool.defer(url) do |url|
# model object performs the scraping
page = ScrapedPage.new(page.url)
pages << page
logger.info "page scrape failed"
>From monitoring the backgroundrb logs, it appears that all of the pages are
completed successfully in the threads. However, the array that is returned
is empty. This is to be expected I suppose because the threads don't
complete before the array is returned, but my question is: how can I make
the worker wait to return the array only when all of the threads are
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Backgroundrb-devel