[Rake-devel] Comments from a new rake user

James Tucker jftucker at gmail.com
Wed May 19 14:42:14 EDT 2010

On 19 May 2010, at 14:13, Heath Kehoe wrote:

> Hello all,
> I have recently implemented a build system based on Rake for a decently-sized project (~8000 source files) which builds code and data for several libraries and applications targeting four separate platforms.
> I was new to both rake and ruby when I started; and it's a testament to the awesomeness of both that I was able to learn them quickly to be able to implement a fairly complicated system in less than a month.
> Anyway, I wanted to pass along some comments.
> Firstly, when I first got stuff building with rake (replacing a build environment that used gnu make) I noticed that a null build (where everything was up to date) took a really long time. I used the ruby profiler and found that approx. half the run time was spent doing file stats (exist? and mtime), with an average of 70 calls *per file* during a rake run. As it runs under Cygwin on Windows those file stat operations are more expensive than they are on Linux. I should note that I'm using ruby 1.8.7 and the rake that gem installed (0.8.7).

Yeah, this is a problem for ruby on windows generally, and somewhat hard to overcome in the generic case.

> If you look at the FileTask code, the needed? method calls File.exist? then timestamp, which calls File.exist? then File.mtime. That's three stats in a row right there for the file itself; then each prerequisite is asked for its timestamp which generates two stats for each (for FileTasks that is). My approach was to create a simple global cache that uses the filename as the key and stores the file's mtime.
> Now, I'm not saying rake should adopt this specific optimization; however I think you should consider some type of caching to reduce the quantity of exist?/mtime calls. Perhaps the FileTask could simply cache its own exist?/mtime results (invalidated when execute runs).

It's a bit hacky and incomplete, but here's an intro implementation...

  class FileTask < Task
    NOFILESTAT = File::Stat.allocate
    class << NOFILESTAT
      def mtime
      # TODO fixup other methods that are busted by the fact we had to hack
      # with allocate.

    # Is this file task needed?  Yes if it doesn't exist, or if its time stamp
    # is out of date.
    def needed?
      stat == NOFILESTAT || out_of_date?(timestamp)

    # Time stamp for file task.
    def timestamp

    def execute(*a)
      super.tap { @stat = nil }

    def stat(refresh = false)
      return @stat if @stat && !refresh
      @stat = begin
      rescue Errno::ENOENT

More information about the Rake-devel mailing list