[Backgroundrb-devel] Duplicate record weirdness ?

Rowan Hick ubermunkey at gmail.com
Fri Sep 29 10:14:52 EDT 2006


Hey Charles, thanks for the reply.

Interesting approach, I may just introduce a quick hack to keep things
ticking along for the moment. Just read the call for help email and will try
out the new version.

Cheers
R

On 9/28/06, Charles Brian Quinn <me at seebq.com> wrote:
>
> Not to detract from backgroundRB because I love it myself, and am
> excited about the rewrite, BTW, but we have a situation where we're
> performing a "sync" operation with a salesforce database, and
> sometimes it is quite slow (and has a lot to sync) -- as such, we had
> to stop using backgroundrb and went to a regular cron job (ugh) using
> script/runner and a custom class that just uses AR.
>
> In doing so, we wrote a quick wrapper using ruby lockfile library to
> make sure that when a task is running, that it creates and pings a
> lockfile -- this way when a secondary cron set to update every 1 or 5
> minutes runs, it waits a certain amount of time, and then just doesn't
> run if one's already running.  Simliarly, if a job has been "running"
> for over an hour, it kills the lock and just steals it and runs.  Is a
> nice library and the wrapper was simple enough.
>
> I have seen issues (on this list) with the timer doing strange things
> (duplicate calls).  In case you didn't know, there's a rewrite that
> Ezra is squeezing in between his day job, writing a deployment book,
> and launching a hosting company (!) -- it is fixing part of the
> problems with the timer class, and he's outlined (search the archive)
> the steps it needs to fix.
>
> hope this helps, cheers,
> --
> Charles Brian Quinn
> self-promotion: www.seebq.com
> highgroove studios: www.highgroove.com
> slingshot hosting: www.slingshothosting.com
>
>
> On 9/28/06, Rowan Hick <ubermunkey at gmail.com> wrote:
> > First off BackgroundRb rocks!!!!. Top marks Ezra, it's gone a long way
> to
> > stopping us from doing a nasty curl/cron hack :)
> >
> > Okay so here's what we're using it for.
> >
> > We're synchronising data from a Filemaker database, via web service
> equest,
> > into a mysql database via a rails model called 'Syncer'  [How's that for
> an
> > oddball use case!]. Now this model exposes a static method which goes
> away
> > and does the business. When it starts synchronising it creates a log
> record
> > for it, and when it finishes it updates that log record. Fair enough.
> All is
> > working beautfiully and this is the perfect candidate for using with
> > backgroundRb. First couple of runs through it worked perfectly. Then
> things
> > started going wonky.
> >
> > My worker is as simple as this..
> >
> > class FilemakerSyncWorker < BackgrounDRb::Rails
> >   repeat_every 20.minutes
> >   first_run Time.now
> >   def do_work(args)
> >     Syncer.new.synchronise
> >   end
> > end
> >
> > What's really weird, and maybe is my complete lack of understanding of
> > threads and such like is this, randomly, throughout the day the log
> records
> > created in .synchronise are getting *almost* duplicated, it's like the
> > method is getting invoked twice at the exact same time, and finishing
> just
> > slightly off the previous time. Example from logs today..
> >
> > | ID | Status | Start Time | End Time | Status
> > | 165 | Failed   | 2006-09-28 10:28:23 | 2006-09-28 10:28:27 |  Expected
> > | 164 | Failed   | 2006-09-28 10:08:23 | 2006-09-28 10:08:24 |
> Expected
> > | 163 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:26 |  Almost
> > Duplicate  (note end time)
> > | 162 | Complete | 2006-09-28 09:48:23 | 2006-09-28 09:49:25 |  Expected
> > | 161 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:12 |  Almost
> > Duplicate
> > | 160 | Complete | 2006-09-28 09:28:23 | 2006-09-28 09:29:13 |  Expected
> > | 159 | Complete | 2006-09-28 09:08:23 | 2006-09-28 09:09:55 |  Expected
> >
> > I've checked the running process, and there's just the one ruby process,
> so
> > it's not like it's spawning any different process. Given the end times
> are
> > fractionally different of the log records it's not like the record is
> > getting duplicated. This is on a Dual G5 box, I haven't tested any other
> box
> > yet. It's not killing us as the same records are getting updated when it
> > duplicates so we don't really care that it is duplicating, it's just odd
> and
> > I don't think should be happening...
> >
> > Can anyone shed any light on it ?
> >
> > Thanks again Ezra !!
> >
> >
> > Rowan
> > ---
> > www.rowanhick.com
> >
> >
> >
> >
> >
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > Backgroundrb-devel at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20060929/63ca2ba7/attachment-0001.html 


More information about the Backgroundrb-devel mailing list