[Rubygems-developers] I've replaced LoadPathManager;
good performance results
gsinclair at soyabean.com.au
Sat Dec 11 11:13:55 EST 2004
I've made a risky change to the CVS: reimplemented loadpath_manager.rb
to speed up the custom "require". The new implementation is in
"custom_require.rb". In rubygems.rb you'll see this:
You can invert the comment to access the old behaviour. This change
is certainly not ready for release -- it needs testing, of course --
but it looks promising. Here are my timing results:
old new old new
LIBRARY | DIRECT | GEM | GEM | COMBO | COMBO
'rake' | 0.203s | 3.772s | 0.757s | 0.481s | 0.426s
'redcloth' | 0.477s | 0.577s | 0.089s | 1.355s | 0.433s
'log4r' | 0.430s | 0.908s | 0.767s | 2.545s | 0.476s
'extensions/all' | 0.536s | 5.594s | 0.795s | 2.741s | 0.643s
'dev-utils/debug' | 0.640s | 5.697s | 1.879s | 2.056s | 0.820s
'celsoft.com/template' | 0.075s | 0.362s | 0.174s | 0.096s | 0.019s
A reminder of the meaning of the terms:
* DIRECT: built-in require used; no RubyGems involvement
* GEM: custom require used to load a library that's only
available as a gem
* COMBO: custom require used to load a library that's
* old: using loadpath_manager.rb
* new: using custom_require.rb
There are a couple of shinky-looking figures there:
* redcloth with the new gem
* celsoft.com/template with the new combo
However, there's a significant improvement in speed more or less
across the board. I'll be running all the speed tests again, instead
of collating data taken from a 12-hour period, but the results gave me
confidence enough to commit the code.
More important than speed is correctness. I'll be dreaming up ways to
test it for correctness. Any ideas would be welcomed.
The new implementation is simpler than the old one, better factored,
and well commented. Basically, it builds the list of specs upfront
(SourceIndex.from_installed_gems) and searches them when asked. I
think LoadPathManager went to great lengths to avoid this, but it
doesn't seem necessary. Of course, if you have 500 gems installed
there could be trouble...
Anyway, a few other classes have been embellished to support this
change; here's the ChangeLog:
2004-12-12 Gavin Sinclair <gsinclair at soyabean.com.au>
* lib/rubygems/custom_require.rb: new file to replace
loadpath_manager.rb and speed up the custom require functionality.
* lib/rubygems/timer.rb: new file; simple benchmarking tool that's
useful at the moment but needn't stay in RubyGems permanently.
* lib/rubygems/source_index.rb: include Enumerable and implement #size
* lib/rubygems/specification.rb: removed lazy initialisation of
attributes to reduce #copy_of invocations; folded
Specification.copy_of into Specification#copy_of.
* lib/rubygems.rb: require 'rubygems/custom_require' instead of
'rubygems/loadpath_manager'; removed out of date documentation; added
"require 'rubygems/timer'" to temporarily assist with benchmarks.
Please be on the lookout for bugs :)
More information about the Rubygems-developers