[Rake-devel] recursive task invocation

[Ittay Dror]

Ittay Dror wrote:
> Hi,
>
> I'm running a Buildr build and after 20 recursive prerequisites Ruby's 
> call stack is exhausted and I get a segmentation fault. The number of 
> modules and dependencies is large, so I can't decrease the prerequisites.
>
> How about doing an iterative DFS search on the prerequisites to order 
> them and then invoke them according to order?
>
> There are  a few other benefits, besides not needing recursion:
> 1. it allows to find circular dependencies before starting the invocation
> 2. it allows for rake to utilize threads to invoke the list of tasks 
> (assuming actions don't conflict)
Note that the current use of threads, in either MultiTask, or the 
JobTask I sent earlier is faulty. Imagine tasks A, B, C, D where  A => 
[B, C], B=>[D], C=>[D]. Now if B and C are invoked in parallel, C 
invokes D which fails. C promptly fails. B continues to run and to it D 
seems to have been already invoked, so it does its actions assuming D's 
actions were executed which isn't true. Now even if the 
MultiTask/JobTask were to cause all other threads to fail, there can be 
nested MultiTask/JobTask already running and the whole issue becomes messy.
>
> Note that currently tasks can "cheat" and add prerequisites lazily 
> when invoked by overriding invoke_prerequisites. This can be achieved 
> with the above suggestion by letting them override the prerequisites 
> accessor.
>
> Drawbacks:
> This doesn't allow for a task's action to add prerequisites to sibling 
> tasks during the invocation chain (unless an API is provided to allow 
> changing the invocation list during invocation)
>
> What do you think? I don't mind implementing this, but obviously it is 
> a big change.
> Ittay
>

-- 
--
Ittay Dror <ittay.dror at gmail.com>