[Rubygems-developers] Current bugs and loadpath_manager.rb

Richard Kilmer rich at infoether.com
Fri Aug 27 23:44:48 EDT 2004

On 8/27/04 7:40 PM, "Chad Fowler" <chad at chadfowler.com> wrote:

> preferably to _any_ of our library stubs.  If we were to discuss and
> agree, perhaps we could just rip all of the stub stuff back out?  Rich,
> would you like to introduce how this works?
> Chad

OK...here goes.  The stub stuff has been bothering me for a while.  The main
issue is we are 'dirtying up' the site_ruby directory with all these stubs,
which was one thing we wanted to avoid when doing gems to begin with.
Anyway, the problem was managing how to 'fault in' gems when they are
needed.  There were various stabs at a 'require hack' that attempted to do
this, but I realized for the require hack to work right, it would need to be
fast, efficient, and as transparent as possible.  It would need to allow you
to require ANY file in ANY gem (with the highest version first).  What we
effectively needed was something that scanned all the paths in all the gems
(sorted in highest-version-first gem order) for a file match, and then
require_gem'd the gem the file was in, then required that file.

This is what the load_path_manager.rb does.  It contains a require hack, and
a module that quickly searches for a required file first in the LOAD_PATH,
then in all the gems.  To achieve speed it builds Dir.glob pattern that
contains all the directories to search (both for the LOAD_PATH and

Here is an example of the jabber4r gem dir:


In the .gemspec it has:

  s.require_paths = ["lib"]

So, once you require 'rubygems' and 'rubygems/load_path_manager.rb'...

require 'jabber4r/jid'

The LoadPathManager class searches the LOAD_PATH for a matching file, and
fines none.  It then searches all the require_paths of all the gems for a
matching file.  I actually have two versions of jabber4r installed, so it
ends up finding jabber4r/jid in two places.  It picks the first, then from
the list and from there determines which gem it is (jabber4r version 0.8.0)
and does a:

require_gem 'jabber4r', '= 0.8.0'

Once that gem is 'required' the LOAD_PATH has been modified, and so the
original require (aliased as require__) is called with 'jabber4r/jid' which
it now has in the LOAD_PATH.  One consequence of this is autorequire is no
longer needed because files are required after gem_require.

Performance wise, this is really fast.  I built an optimized way of loading
all the gemspec files (I created a 'mock' Gem::Specification to speed things
up because I only need certain data).  From there, I build a Dir.glob
pattern, which when called is executed in C...and is quite fast.  The only
performance hit we get right now is loading the lib/specification.rb file
because that file has a lot of cool ruby hackery in it (clever, but a tad

So again, this is a generally useful technique for allowing require to work
with rubygems transparently, and if you want a particular version, you can
still do a require_gem on whatever you like.  The problem, of course, is
getting the rubygems require in there to begin with.  As with the original
require hacks, we could have folks use the ENV variable RUBYOPT and specify:

set RUBYOPT='rrubygems'

Then when you run ruby the reuire is already overridden.  Anyway, that is
the way is stands now.  It is not currently enabled until I check in a
change in the rubygems.rb file that requires the
rubygems/load_path_manager.rb file.

Let me know comments,


More information about the Rubygems-developers mailing list