[Rake-devel] Parallel tasks with Rake

[James M. Lawrence]

A few updates --

* The crashes I described previously with CompTree + 1.9 were all from
  the forking tests.  Drb + threads + fork is a very delicate matter,
  and there are likely some caveats for 1.9 and cygwin.  Fortunately
  this has nothing to do with Drake, as I don't use CompTree's fork
  option.

  So Drake runs well under 1.9.

* I removed the generated rb files.  It's better to deal with an
  occasional merge conflict from the CompTree --> Rake::CompTree
  rename.  For reasons given below, I think my initial inclination for
  using a Rake-ified branch of CompTree was right after all (as
  opposed to the gem).

* Fixed two curious bugs missed by the Rake unit tests:
  (a) I did not notice @prerequisites contained namespace-unqualifed
  names.

  (a) FileTask#needed? only looks at the immediate children.  In Drake
  this caused grandparents and above to not be executed (since all
  tasks are gathered with a dry run).

  If 'needed?' is officially supported for end-users, this might be
  considered a bug in Rake.  Otherwise there is no problem, of course,
  since Task#invoke propagates 'needed?' via recursion.

* Added a --rand[=SEED] flag to randomize prerequisites, a debugging
  tool for reaching j-safety.  SEED can be any string (String#hash is
  the real seed).

Ittay:
>    - btw,  this is possible with drake, right? so if one thread fails
> the other is signaled to quit execution when the current execution block
> finishes (cleanly)

It is possible, but CompTree doesn't work that way by default.  Since
in CompTree the purpose is to get an answer, there would be no reason
to finish any node's computation once an error is detected.

This is a fundamental difference between the goals of Rake (create
side-effects) and CompTree (compute a value), a good justification for
keeping the Rake::CompTree branch.  I think I would prefer not having
to support two antithetical "modes" at once (I already don't like
TaskNode in there).  Plus I get to remove drb in the Rake version,
which adds a measurable load time due to socket et al.

James M. Lawrence