[Backgroundrb-devel] Duplicate record weirdness ?

[Charles Brian Quinn]

Not to detract from backgroundRB because I love it myself, and am
excited about the rewrite, BTW, but we have a situation where we're
performing a "sync" operation with a salesforce database, and
sometimes it is quite slow (and has a lot to sync) -- as such, we had
to stop using backgroundrb and went to a regular cron job (ugh) using
script/runner and a custom class that just uses AR.

In doing so, we wrote a quick wrapper using ruby lockfile library to
make sure that when a task is running, that it creates and pings a
lockfile -- this way when a secondary cron set to update every 1 or 5
minutes runs, it waits a certain amount of time, and then just doesn't
run if one's already running.  Simliarly, if a job has been "running"
for over an hour, it kills the lock and just steals it and runs.  Is a
nice library and the wrapper was simple enough.

I have seen issues (on this list) with the timer doing strange things
(duplicate calls).  In case you didn't know, there's a rewrite that
Ezra is squeezing in between his day job, writing a deployment book,
and launching a hosting company (!) -- it is fixing part of the
problems with the timer class, and he's outlined (search the archive)
the steps it needs to fix.

hope this helps, cheers,
-- 
Charles Brian Quinn
self-promotion: www.seebq.com
highgroove studios: www.highgroove.com
slingshot hosting: www.slingshothosting.com


On 9/28/06, Rowan Hick <ubermunkey at gmail.com> wrote:
> First off BackgroundRb rocks!!!!. Top marks Ezra, it's gone a long way to
> stopping us from doing a nasty curl/cron hack :)
>
> Okay so here's what we're using it for.
>
> We're synchronising data from a Filemaker database, via web service equest,
> into a mysql database via a rails model called 'Syncer'  [How's that for an
> oddball use case!]. Now this model exposes a static method which goes away
> and does the business. When it starts synchronising it creates a log record
> for it, and when it finishes it updates that log record. Fair enough. All is
> working beautfiully and this is the perfect candidate for using with
> backgroundRb. First couple of runs through it worked perfectly. Then things
> started going wonky.
>
> My worker is as simple as this..
>
> class FilemakerSyncWorker < BackgrounDRb::Rails
>   repeat_every 20.minutes
>   first_run Time.now
>   def do_work(args)
>     Syncer.new.synchronise
>   end
> end
>
> What's really weird, and maybe is my complete lack of understanding of
> threads and such like is this, randomly, throughout the day the log records
> created in .synchronise are getting *almost* duplicated, it's like the
> method is getting invoked twice at the exact same time, and finishing just
> slightly off the previous time. Example from logs today..
>
> | ID | Status | Start Time | End Time | Status
> | 165 | Failed   | 2006-09-28 10:28:23 | 2006-09-28 10:28:27 |  Expected
> | 164 | Failed   | 2006-09-28 10:08:23 | 2006-09-28 10:08:24 |   Expected
> | 163 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:26 |  Almost
> Duplicate  (note end time)
> | 162 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:25 |  Expected
> | 161 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:12 |  Almost
> Duplicate
> | 160 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:13 |  Expected
> | 159 | Complete | 2006-09-28 09:08:23 | 2006-09-28 09:09:55 |  Expected
>
> I've checked the running process, and there's just the one ruby process, so
> it's not like it's spawning any different process. Given the end times are
> fractionally different of the log records it's not like the record is
> getting duplicated. This is on a Dual G5 box, I haven't tested any other box
> yet. It's not killing us as the same records are getting updated when it
> duplicates so we don't really care that it is duplicating, it's just odd and
> I don't think should be happening...
>
> Can anyone shed any light on it ?
>
> Thanks again Ezra !!
>
>
> Rowan
> ---
> www.rowanhick.com
>
>
>
>
>
> _______________________________________________
> Backgroundrb-devel mailing list
> Backgroundrb-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
>