[Rubygems-developers] faster rubygems.rb loading by way of deferred plugin loads, and other resource usage enhancements

James Tucker jftucker at gmail.com
Wed Jul 7 11:27:46 EDT 2010

Hey all,

I've started work on a branch to reduce the runtime resource usage by loading rubygems.rb. Some of you are already aware of this [1].

The main 'cheat' in the branch so far is to move plugin loads into the command manager, but I'm not entirely satisfied with this, and so I'm polling for interest in what features plugin authors actually need / want, and where those can be injected. Most specifically, I want to see if they can be done so more lightly.

Initially, probably the most commonly used plugin is the gemcutter plugin, which provides commands. At present it deliberately loads the CommandManager, which results in loading in about 37 files and the spec cache into memory whenever rubygems.rb is loaded. The difference in startup performance is significantly over an order of magnitude on common developer systems (with largish numbers of gems installed).

A solution for command plugins would be to have plugins simply register a list of files that should be loaded when commands are going to be invoked. A continuation of this idea leads to an API something along the lines of:

Gem.register_plugin(:install, 'some_plugin/gem/installer')
Gem.register_plugin(:command, 'some_plugin/gem/command')

Such that rubygems could contain simple internal hooks to load these on demand, rather than ahead of time, avoiding pre-activation of plugins and the current increased cost of loading rubygems.

My end goal with this is to aim to get rubygems.rb fast and light enough that it could be used for the core integration in 1.9, to avoid the remaining and otherwise unavoidable bugs in gem_prelude.rb.

Other work that will be included in this, is indexing the paths to rubygems_plugin.rb files, and reducing the resource requirements for activation, by having a lightweight version and index set that can be used by Gem.activate in place of the current heavyweight Gem.spec.

If anyone has any comments on any of the above, I would really like to hear them before I embark on these various enhancements.


James Tucker

[1] http://github.com/rubygems/rubygems/compare/master...perflude

More information about the Rubygems-developers mailing list