[Backgroundrb-devel] Duplicate record weirdness ?
Charles Brian Quinn
me at seebq.com
Thu Sep 28 19:20:09 EDT 2006
Not to detract from backgroundRB because I love it myself, and am
excited about the rewrite, BTW, but we have a situation where we're
performing a "sync" operation with a salesforce database, and
sometimes it is quite slow (and has a lot to sync) -- as such, we had
to stop using backgroundrb and went to a regular cron job (ugh) using
script/runner and a custom class that just uses AR.
In doing so, we wrote a quick wrapper using ruby lockfile library to
make sure that when a task is running, that it creates and pings a
lockfile -- this way when a secondary cron set to update every 1 or 5
minutes runs, it waits a certain amount of time, and then just doesn't
run if one's already running. Simliarly, if a job has been "running"
for over an hour, it kills the lock and just steals it and runs. Is a
nice library and the wrapper was simple enough.
I have seen issues (on this list) with the timer doing strange things
(duplicate calls). In case you didn't know, there's a rewrite that
Ezra is squeezing in between his day job, writing a deployment book,
and launching a hosting company (!) -- it is fixing part of the
problems with the timer class, and he's outlined (search the archive)
the steps it needs to fix.
hope this helps, cheers,
Charles Brian Quinn
highgroove studios: www.highgroove.com
slingshot hosting: www.slingshothosting.com
On 9/28/06, Rowan Hick <ubermunkey at gmail.com> wrote:
> First off BackgroundRb rocks!!!!. Top marks Ezra, it's gone a long way to
> stopping us from doing a nasty curl/cron hack :)
> Okay so here's what we're using it for.
> We're synchronising data from a Filemaker database, via web service equest,
> into a mysql database via a rails model called 'Syncer' [How's that for an
> oddball use case!]. Now this model exposes a static method which goes away
> and does the business. When it starts synchronising it creates a log record
> for it, and when it finishes it updates that log record. Fair enough. All is
> working beautfiully and this is the perfect candidate for using with
> backgroundRb. First couple of runs through it worked perfectly. Then things
> started going wonky.
> My worker is as simple as this..
> class FilemakerSyncWorker < BackgrounDRb::Rails
> repeat_every 20.minutes
> first_run Time.now
> def do_work(args)
> What's really weird, and maybe is my complete lack of understanding of
> threads and such like is this, randomly, throughout the day the log records
> created in .synchronise are getting *almost* duplicated, it's like the
> method is getting invoked twice at the exact same time, and finishing just
> slightly off the previous time. Example from logs today..
> | ID | Status | Start Time | End Time | Status
> | 165 | Failed | 2006-09-28 10:28:23 | 2006-09-28 10:28:27 | Expected
> | 164 | Failed | 2006-09-28 10:08:23 | 2006-09-28 10:08:24 | Expected
> | 163 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:26 | Almost
> Duplicate (note end time)
> | 162 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:25 | Expected
> | 161 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:12 | Almost
> | 160 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:13 | Expected
> | 159 | Complete | 2006-09-28 09:08:23 | 2006-09-28 09:09:55 | Expected
> I've checked the running process, and there's just the one ruby process, so
> it's not like it's spawning any different process. Given the end times are
> fractionally different of the log records it's not like the record is
> getting duplicated. This is on a Dual G5 box, I haven't tested any other box
> yet. It's not killing us as the same records are getting updated when it
> duplicates so we don't really care that it is duplicating, it's just odd and
> I don't think should be happening...
> Can anyone shed any light on it ?
> Thanks again Ezra !!
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
More information about the Backgroundrb-devel