[Backgroundrb-devel] Using Backgroundrb?

Paul Kmiec paul.kmiec at appfolio.com
Wed Apr 23 12:17:29 EDT 2008


You can use the built build thread pool to process more than one feed within
the same worker. So within the worker, you'd do,

def parse_feeds
  loop do
    feed = Feed.find_feed_to_process
    thread_pool.defer do
      feed.parse
    end
  end
end

I think the default pool size is 20. You can control the size of the thread
pool using a class level method, as I recall it is

pool_size x

Paul

On Wed, Apr 23, 2008 at 7:30 AM, Julien Genestoux <
julien.genestoux at gmail.com> wrote:

> Thanks Adam,
>
> That sounded weird to me as well to have one worker for each feed...
> However, if I only have one worker, that also means that I am parsing
> one feed only at any moment. An option, is maybe to have a few workers
> (denpending on the number of feeds)  that parse feeds concurrently?
>
> If I only have one worker, according to you what should be the
> winnning strategy to choose the "right" parse to feed? Obviously some
> feeds need to be parsed one every few minutes, while some other might
> no need to be parse more than every hour...
>
> Any idea/tip on this?
>
>
>
>
> On 4/23/08, Adam Williams <adam at thewilliams.ws> wrote:
> > On Apr 23, 2008, at 1:07 AM, Julien Genestoux wrote:
> >
> >  > I still have a few questions : shoud I have one worker for each feed
> >  > that is called periodically (add_periodic_timer) or rather one single
> >  > worker that calls every feed one by one?
> >  >
> >  > What is the best solution, perfomance-wise?
> >
> >
> > Good question... I don't suppose I know exactly. I would start by
> >  processing all the feeds in one worker invocation - that is what I
> >  have done for sending an unknown amount of email. It just seems wrong
> >  to me to invoke a worker for one email at a time.
> >
> >  The right answer likely lies in understanding the whole MasterWorker,
> >  Packet::Reactor/handler_instance.ask_work bits of the puzzle...
> >
> >
> >     adam
> >
> > _______________________________________________
> >  Backgroundrb-devel mailing list
> >  Backgroundrb-devel at rubyforge.org
> >  http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
>
>
> --
> --
> Julien Genestoux
> julien.genestoux at gmail.com
> http://www.ouvre-boite.com
> +1 (415) 254 7340
> +33 (0)8 70 44 76 29
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080423/2437376c/attachment.html 


More information about the Backgroundrb-devel mailing list