[Backgroundrb-devel] Good news: BackgrounDRb now works on Windows

Brian Morearty brian at morearty.org
Sun Apr 13 20:44:52 EDT 2008

Hello all,

I have some good news to announce. I have made some changes to a local copy
of BackgrounDRb 1.0.3 (actually most of the changes were in Packet 0.1.5)
and I've got it working on Windows. From what I've read, this is the first
time it's worked on Windows in a year and a half, since 0.2 was released.

 In my research I can see that a number of people have asked for Windows
support so I'm excited that I can help out.

I'm new to this mailing list and new to BackgrounDRb so I would appreciate
advice and help from you all on things like:
- how to test my changes. When I run rake it just exits without running any
tests. I'm not sure why. I also don't know how to test Packet.
- how to send a patch. I mostly do Windows development where textual patches
are not commonplace. But I did submit a patch to Rails that was accepted, so
at least I've done it once with success. :-)
- a code review to make sure I'm doing things in an approved way
- someone with *nix to make sure I didn't break anything by accident. I
tried hard to not but you can't be sure without testing.

Since I haven't figured out how to run automated tests I have created some
ad hoc tests of my own. These cases work:
 - ad hoc scheduling by calling a method on the worker
- passing parameters to the worker
- cron scheduling
- register_status and ask_status
- making a synchronous call and getting a result back kind of works, except
I get the whole hash back with the result in the :data key instead of just
getting the result alone. In other words I get a hash like this:
   {:type=>:response, :client_signature=>25, :result=>true, :data=>1}
  which is odd because it's calling the same "extract" function that
ask_status uses, so I'm not sure why this is any different.

There are some things I haven't tried yet but it's great that this many
cases work.

I definitely have to try to figure out why the return value of a synchronous
call is the whole hash, and I might need help on that because I'm mystified.

The primary obstacles to making BackgrounDRb work on Windows were:
1. UNIXSocket - it wasn't too hard to add code that uses TCPSocket unless
defined? UNIXSocket. And there were four lines of code that checked if a
socket was a UNIXSocket to decide whether it's an "internal" read or write,
but those lines were crashing because UNIXSocket isn't even defined on
Windows, much less used. My fix for that was to create a marker Module
called "Packet::InternalSocket." Each time I created an internal socket
(where the code used to create a UNIXSocket) I now call socket.extend
Packet::InternalSocket so that later on the code can distinguish the
internal ones from the others.
2. fork - well that's a harder matter. Windows doesn't support fork() and
probably never will. I saw a recent post by Hemant Kumar mentioning a fix
that uses fork and exec rather than just fork, but it still requires fork.
Simulating fork in a generic way is nearly impossible but replacing it in a
single application is conceivable, depending on what the app does. And in
this case I was able to do this:
 a) Leave the existing fork call there
 b) Fall back on IO.popen for operating systems where fork is not supported
 What I did is for each worker I launch (instead of fork) a new child
process, pass the read/write port numbers to the child on the command line,
and wait for it to connect to them.

Because Rails takes a long time to start up on Windows (11 seconds, ugh) you
can wait a pretty long time before all your workers are ready (11sec *
(number of workers+2)). But still, it's better than not being able to use it
at all. It's perfectly acceptable for a development machine, which is how I
use Rails on Windows. (My project is going to deploy on something that ends
in nix.)

On the other hand if you're going to be dynamically creating background
workers (e.g. reload_on_schedule true or set_no_auto_load true) you'll wait
a while for the worker to start up on Windows.

Will this slow startup time ever block a Rails server request? Not that I've
seen. From what I've seen BackgrounDRb never starts a new worker process
because of an API call. The only case I've seen where it starts a new worker
process on the fly is for a scheduled background task using the cron
feature. But I may have missed something.

Another thing that I didn't implement (at least not for now): support for
the "start" parameter to run the background processes as hidden processes.
For now on Windows you just have to run script/backgroundrb without the
start parameter and just leave it running in a console. (But I did make the
error message nicer.) The reason I didn't: Ruby does not seem to have
implemented a way to kill a process tree in Windows. Windows doesn't have a
single API to kill a process tree. You can add all the processes to a Job
and kill that, or you can enumerate the descendants of a process and kill
them all one by one. I just didn't want to bother adding the code to Ruby do
that, at least not now, because it works well enough in a console.

And the nice thing is Ctrl+C from the console does indeed kill the whole
process tree.

That's it for now. Please let me know how to proceed so us Windows users can
benefit from the wonderfulness of BackgrounDRb.

Brian Morearty
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/backgroundrb-devel/attachments/20080413/dd7c953d/attachment.html 

More information about the Backgroundrb-devel mailing list