[Backgroundrb-devel] Duplicate record weirdness ?
ubermunkey at gmail.com
Fri Sep 29 10:14:52 EDT 2006
Hey Charles, thanks for the reply.
Interesting approach, I may just introduce a quick hack to keep things
ticking along for the moment. Just read the call for help email and will try
out the new version.
On 9/28/06, Charles Brian Quinn <me at seebq.com> wrote:
> Not to detract from backgroundRB because I love it myself, and am
> excited about the rewrite, BTW, but we have a situation where we're
> performing a "sync" operation with a salesforce database, and
> sometimes it is quite slow (and has a lot to sync) -- as such, we had
> to stop using backgroundrb and went to a regular cron job (ugh) using
> script/runner and a custom class that just uses AR.
> In doing so, we wrote a quick wrapper using ruby lockfile library to
> make sure that when a task is running, that it creates and pings a
> lockfile -- this way when a secondary cron set to update every 1 or 5
> minutes runs, it waits a certain amount of time, and then just doesn't
> run if one's already running. Simliarly, if a job has been "running"
> for over an hour, it kills the lock and just steals it and runs. Is a
> nice library and the wrapper was simple enough.
> I have seen issues (on this list) with the timer doing strange things
> (duplicate calls). In case you didn't know, there's a rewrite that
> Ezra is squeezing in between his day job, writing a deployment book,
> and launching a hosting company (!) -- it is fixing part of the
> problems with the timer class, and he's outlined (search the archive)
> the steps it needs to fix.
> hope this helps, cheers,
> Charles Brian Quinn
> self-promotion: www.seebq.com
> highgroove studios: www.highgroove.com
> slingshot hosting: www.slingshothosting.com
> On 9/28/06, Rowan Hick <ubermunkey at gmail.com> wrote:
> > First off BackgroundRb rocks!!!!. Top marks Ezra, it's gone a long way
> > stopping us from doing a nasty curl/cron hack :)
> > Okay so here's what we're using it for.
> > We're synchronising data from a Filemaker database, via web service
> > into a mysql database via a rails model called 'Syncer' [How's that for
> > oddball use case!]. Now this model exposes a static method which goes
> > and does the business. When it starts synchronising it creates a log
> > for it, and when it finishes it updates that log record. Fair enough.
> All is
> > working beautfiully and this is the perfect candidate for using with
> > backgroundRb. First couple of runs through it worked perfectly. Then
> > started going wonky.
> > My worker is as simple as this..
> > class FilemakerSyncWorker < BackgrounDRb::Rails
> > repeat_every 20.minutes
> > first_run Time.now
> > def do_work(args)
> > Syncer.new.synchronise
> > end
> > end
> > What's really weird, and maybe is my complete lack of understanding of
> > threads and such like is this, randomly, throughout the day the log
> > created in .synchronise are getting *almost* duplicated, it's like the
> > method is getting invoked twice at the exact same time, and finishing
> > slightly off the previous time. Example from logs today..
> > | ID | Status | Start Time | End Time | Status
> > | 165 | Failed | 2006-09-28 10:28:23 | 2006-09-28 10:28:27 | Expected
> > | 164 | Failed | 2006-09-28 10:08:23 | 2006-09-28 10:08:24 |
> > | 163 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:26 | Almost
> > Duplicate (note end time)
> > | 162 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:25 | Expected
> > | 161 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:12 | Almost
> > Duplicate
> > | 160 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:13 | Expected
> > | 159 | Complete | 2006-09-28 09:08:23 | 2006-09-28 09:09:55 | Expected
> > I've checked the running process, and there's just the one ruby process,
> > it's not like it's spawning any different process. Given the end times
> > fractionally different of the log records it's not like the record is
> > getting duplicated. This is on a Dual G5 box, I haven't tested any other
> > yet. It's not killing us as the same records are getting updated when it
> > duplicates so we don't really care that it is duplicating, it's just odd
> > I don't think should be happening...
> > Can anyone shed any light on it ?
> > Thanks again Ezra !!
> > Rowan
> > ---
> > www.rowanhick.com
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Backgroundrb-devel