[Rake-devel] timestamp checking is slow?

Mark Watson watsonmw at gmail.com
Wed Jul 23 04:15:19 EDT 2008

I ran into a similar issue, the way i got around it was by caching the
timestamp the first time File.mtime is called.  This brings about its
own problems because you don't necessarily know that the file won't be
modified outside the file task.  If you have a task that generates
several files the cached timestamps won't be updated.

I tried using Windows API directly instead of File.mtime but this
wasn't much faster.  I didn't get a chance to look into what make is
doing, but i suspect it is caching the timestamps too.  Make does
allow you to specify several files as the output of one task, possibly
as a way around the problems with caching the timestamp.

# File task with timestamp speedup.  The regular file task will query
# the File.mtime for every task that depends on this file.  Typically
# when compiling c/c++ some header files are included very often, and
# if a a regular file task is used to represent header files then the
# File.mtime function will be called each time a source file depends
# on the header.  By caching the timestamp we insure that this
# operation occurs only once for each header file.  On typical c/c++
# projects where most files depend on a core set of headers, a ~3x
# speedup on rake startup time can be expected with this patch.
# WARNING: Caching the timestamp may cause problems if the file is
# generated/updated by another task as a side effect.
class Rake::FastFileTask < Rake::FileTask
  # Time stamp for file task.
  def timestamp
    # Cache the timestamp since it is accessed often
    if instance_variable_defined?(:@timestamp)
      # exist? and mtime can be done together with one call to File.stat, but
      # for some reason this takes longer on Windows.  Ruby is probably making
      # several Win32 calls.  Opimization would be to call GetFileAttributesEx()
      # directly.  Using Win32API module this is slightly faster, making the
      # call via a c extension would probably be faster again.
      if File.exist?(name)
        @timestamp = File.mtime(name.to_s)
        # Update the time stamp when the task is executed
        enhance do
          @timestamp = Time.now
        # Build me!

2008/7/22 Ittay Dror <ittay.dror at gmail.com>:
> Hi,
> I'm using Rake (actually Buildr) to compile C++ files. As part of that I'm
> loading dependencies on header files. The total amount of  dependencies is
> 869 on 142 obj files.
> The issue I'm seeing is that it takes a lot of time for Rake to timestamp
> these files to reach a conclusion that it has nothing to do. Compared to
> make that takes 0.7 seconds to run, Rake takes 3 seconds. If I remove the
> creation of the dependencies (but still leave the loading and parsing of the
> files) rake takes 1 second.
> I understand that 2 seconds seem like not a long time, but the project I'm
> working on has over 1000 of source files, so I expect the time to be much
> larger. Do you think anything can be done to make Rake closer to make? note
> that without the timestamp checking, the rest of the work doesn't amount to
> a lot of overhead, so I am hoping it is not the dynamic nature of Ruby that
> is the cause of the problem (but maybe poor implementation of
> File(file).mtime?)
> Thank you,
> Ittay
> --
> --
> Ittay Dror <ittay.dror at gmail.com>
> _______________________________________________
> Rake-devel mailing list
> Rake-devel at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rake-devel

More information about the Rake-devel mailing list