[Backgroundrb-devel] Best way to run hundreds of concurrent tasks?
gethemant at gmail.com
Mon Sep 15 07:37:29 EDT 2008
Can you paste the worker code? I am not sure, whats causing this.
Which OS btw?
On Sun, 2008-09-14 at 21:58 -0700, Robert Matei wrote:
> BDRB is the latest version from the SVN repository (supposed to mirror
> stable git releases)
> Packet 0.1.13
> using Facebooker to interact with Facebook.
> On Sun, Sep 14, 2008 at 9:30 PM, hemant kumar <gethemant at gmail.com>
> On Sun, 2008-09-14 at 19:34 -0700, Robert Matei wrote:
> > Hi all,
> > I'm trying to figure out to run some asynchronous tasks in a
> > app. I've got things working, but BRB crashes after a little
> > and I'm not sure if my setup is ideal.
> > Here's the scenario:
> > - whenever a user visits my app, I need to fire off a bunch
> of API
> > calls.
> > - these calls need to start right away, because the user
> sees a
> > 'loading' screen until they return.
> > - the job is not calculation or memory-intensive on my end,
> but it
> > does take 15-20 seconds to get all the responses from
> > I'm on a fairly small server, so it's important to keep
> memory use low
> > (and stability high), while still being able to handle
> thousands of
> > simultaneous visitors. Right now I'm using a single worker,
> and each
> > visitor generates a new thread in the thread pool. After a
> > while tasks only get a few calls in before failing with a
> > reset by peer" error. Eventually my worker just disappears
> > altogether.
> > What's going on? What kind of setup should I be running?
> What version of BDRB? What version of packet gem? Which OS?
> Are you using mechanize or something for interacting with
> facebook? or
> just normal RFacebook or something?
More information about the Backgroundrb-devel