[Rubygems-developers] I've replaced LoadPathManager; good performance results

Gavin Sinclair gsinclair at soyabean.com.au
Sat Dec 11 11:13:55 EST 2004


I've made a risky change to the CVS: reimplemented loadpath_manager.rb
to speed up the custom "require".  The new implementation is in
"custom_require.rb".  In rubygems.rb you'll see this:

  #require 'rubygems/loadpath_manager'
  require 'rubygems/custom_require'

You can invert the comment to access the old behaviour.  This change
is certainly not ready for release -- it needs testing, of course --
but it looks promising.  Here are my timing results:

                                         old        new        old        new
  LIBRARY                 |  DIRECT  |   GEM    |   GEM    |  COMBO   |  COMBO
  'rake'                  |  0.203s  |  3.772s  |  0.757s  |  0.481s  |  0.426s
  'redcloth'              |  0.477s  |  0.577s  |  0.089s  |  1.355s  |  0.433s
  'log4r'                 |  0.430s  |  0.908s  |  0.767s  |  2.545s  |  0.476s
  'extensions/all'        |  0.536s  |  5.594s  |  0.795s  |  2.741s  |  0.643s
  'dev-utils/debug'       |  0.640s  |  5.697s  |  1.879s  |  2.056s  |  0.820s
  'celsoft.com/template'  |  0.075s  |  0.362s  |  0.174s  |  0.096s  |  0.019s

A reminder of the meaning of the terms:
 * DIRECT: built-in require used; no RubyGems involvement
 * GEM: custom require used to load a library that's only
        available as a gem
 * COMBO: custom require used to load a library that's
        directly installed
 * old: using loadpath_manager.rb
 * new: using custom_require.rb

There are a couple of shinky-looking figures there:
 * redcloth with the new gem
 * celsoft.com/template with the new combo

However, there's a significant improvement in speed more or less
across the board.  I'll be running all the speed tests again, instead
of collating data taken from a 12-hour period, but the results gave me
confidence enough to commit the code.

More important than speed is correctness.  I'll be dreaming up ways to
test it for correctness.  Any ideas would be welcomed.

The new implementation is simpler than the old one, better factored,
and well commented. Basically, it builds the list of specs upfront
(SourceIndex.from_installed_gems) and searches them when asked.  I
think LoadPathManager went to great lengths to avoid this, but it
doesn't seem necessary.  Of course, if you have 500 gems installed
there could be trouble...

Anyway, a few other classes have been embellished to support this
change; here's the ChangeLog:

    2004-12-12  Gavin Sinclair  <gsinclair at soyabean.com.au>

        * lib/rubygems/custom_require.rb: new file to replace
        loadpath_manager.rb and speed up the custom require functionality.

        * lib/rubygems/timer.rb: new file; simple benchmarking tool that's
        useful at the moment but needn't stay in RubyGems permanently.

        * lib/rubygems/source_index.rb: include Enumerable and implement #size
        and #length.

        * lib/rubygems/specification.rb: removed lazy initialisation of
        attributes to reduce #copy_of invocations; folded
        Specification.copy_of into Specification#copy_of.

        * lib/rubygems.rb: require 'rubygems/custom_require' instead of
        'rubygems/loadpath_manager'; removed out of date documentation; added
        "require 'rubygems/timer'" to temporarily assist with benchmarks.

Please be on the lookout for bugs :)


More information about the Rubygems-developers mailing list