Forums | Admin Discussion Forums: help Start New Thread Nested Flat Threaded Ultimate Show 25 Show 50 Show 75 Show 100   By: Adam Pisoni RE: how best to write a looping job [ reply ]   2008-07-09 17:32 Yeah, same idea. We use acts_as_state_machine for these types of things. Check it out http://agilewebdevelopment.com/plugins/acts_as_state_machine By: Nicolas Fouché RE: how best to write a looping job [ reply ]   2008-07-09 17:30 About the duplicated works: what about a "status" column in the model. Its value could be set from "pending" to "queued" by the daemon/cron script, then to "processing" when the worker takes the job, and finally to "pending" or "error" when the worker ends. So, if the poller daemon only takes pending database records, it won't recreate a job for already queued records. By: Bob Zoller RE: how best to write a looping job [ reply ]   2008-07-09 17:21 yup we're on the same page. I'll have to give this some more thought. I'd love to see the script - link to github is great too :) I've also been in the "daemons is overkill" boat a few times myself... interested to see what you've got. Thanks By: Bob Zoller RE: how best to write a looping job [ reply ]   2008-07-09 17:17 definitely a good idea. I was thinking about cron, but hesitated only because I'd have to code up another locking mechanism. Not like that's a big deal, but it left me wondering if there was a simple "find jobs - process jobs - find more jobs" looping mechanism I could use. Thanks! By: Adam Pisoni RE: how best to write a looping job [ reply ]   2008-07-09 17:15 In that case you'll have to write a daemon that polls the database periodically. I've used the ruby Daemons class before, but believe it was too complicated and does all sorts of funny things I don't like, so I wrote an ICanDaemonize class which makes making daemons trivial... and easy to integrate with your rails app. I was planning to release it to github any second now... and I guess I can, even if it's not done. The short answer is that you'll have to run a deamon that polls the database, though I don't like this kind of approach in general because of the possibility for duplicating work and the difficulty in distributing/making redundant the polling process itself. What I mean is... imagine you have a poller daemon and it does... how do you know which records it gave to skynet? I suppose in your case the poller would give records to Skynet WHEN they are marked as ready, but never give a record already marked as ready to skynet. I can just email you ICanDaemonize if you want to look at it. By: Nicolas Fouché RE: how best to write a looping job [ reply ]   2008-07-09 17:11 Personally, I made a cronned Ruby script which checks in my database for models ready to be updated, and creates a job for each one of them. (I do not use distributed_find, I create Skynet jobs "by hands") By: Bob Zoller RE: how best to write a looping job [ reply ]   2008-07-09 17:04 good question. "ready" is based on time - so if feeds.last_synced_at is > 1 hr ago, it's "ready" (which is why there's no event to trigger the insertion of the job) By: Adam Pisoni RE: how best to write a looping job [ reply ]   2008-07-09 16:22 Not a bad question. I'm working on a project that has a very similar issue. I guess the first question I have is how a record gets marked as "ready" and why you wouldn't just add that one record to the skynet queue when it's ready as opposed to polling the database? By: Bob Zoller how best to write a looping job [ reply ]   2008-07-09 06:09 I'm looking for recommendations on how to code up this situation using skynet: I have a rails app with a Feed model. I want to do a distributed_find with some conditions that indicate the feed is "ready", deal those feeds, and then do another distributed_find for "ready" feeds, etc.. etc.. infinitely. Basically the equivalent of the vigilant option in starfish. Apologies if this should be really obvious... I'm new to skynet. Thanks! --Bob