[Rake-devel] Rake 0.9.3.beta.2 with -j option

Mark Watson watsonmw at gmail.com
Tue Oct 23 20:54:38 UTC 2012


What about having the old code called by default and if you specify -j
the new parallel code is executed?  That way old rakefiles still work,
and new ones can take advantage of the -j feature (after all that was
good enough for GNUmake).  This is what I've done with my own
parallelization patch (From the number of patches it seems -j is
certainly a much wanted rake feature! :)

https://github.com/watsonmw/rakecpp/blob/master/minusj/minusj.rb

On 23 October 2012 13:34, Jos Backus <jos at catnook.com> wrote:
> On Tue, Oct 23, 2012 at 9:18 AM, Jim Weirich <jim.weirich at gmail.com> wrote:
>>
>>
>> On Oct 22, 2012, at 4:04 PM, Hongli Lai <hongli at phusion.nl> wrote:
>>
>> > Conservative is one thing, but drake was written 2 years ago. There has
>> > been no response every time someone asks why drake was not merged.
>>
>> My main problem with drake is that it adds a second task execution engine
>> that is subtly different the mainline rake engine.  The difference isn't
>> critical and most projects won't even notice the difference, but having two
>> similar but different engines offends my sensibilities.
>
>
> It would trigger my OCD ;)
>
>>
>>
>> If drake were to be merge, I would want to either (a) discard the current
>> engine and use drake's engine exclusively, or (b) make the parallelization
>> mechanism work more closely with the current rake engine.
>>
>> I know drake uses a dry-run pass to compute the dependency tree, but I'm
>> not sure if the dry run pass uses the regular rake engine (which might
>> impact option (a)) or if it does its own thing.
>
>
> Is this something the drake author could help gain certainty about?
>
>>
>>
>> In any case, a drake merge won't happen in the 0.9.x series as I would
>> like to work out the current bug list and hit some simple features.  The
>> Thread pool looked like an easy win and is really needed for the multitask
>> stuff anyways. Michael has also proposed a -m option that implicitly turns
>> tasks into multitasks, and I'm considering that instead of a drake
>> integration.
>
>
> I like -m better, it avoids a future behavioral change conflict with -j.
>
>>
>>
>> However, if the -m flag is deemed inadequate, I will probably hold off on
>> the thread pool as well and reconsider a drake move a bit farther down the
>> line.
>>
>> Thoughts are welcome.
>>
>> (Postscript: I also have some concerns about turning on parallel execution
>> in arbitrary Rakefiles.  I suspect it will work fine in projects that most
>> shell out to compilers and linkers, but Rakefiles that run most Ruby code
>> will probably be broken in ways that are hard to detect and reproduce. If
>> anyone has any ideas on addressing that issue, I would love to hear them.)
>
>
> But would it not require users to specify some option? Iow, the default case
> would not be affected. And if someone specifies a new option, the
> documentation could point out that in the case of incomplete dependency
> specifications, recipes that depend on pure sequential operation for
> correctness could break, and the missing dependencies need to be specified.
>
> Jos
>>
>>
>> --
>> -- Jim Weirich
>> -- jim.weirich at gmail.com
>>
>>
>>
>>
>>
>> _______________________________________________
>> Rake-devel mailing list
>> Rake-devel at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rake-devel
>
>
>
>
> --
> Jos Backus
> jos at catnook.com
>
>
> _______________________________________________
> Rake-devel mailing list
> Rake-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rake-devel


More information about the Rake-devel mailing list