[Rubygems-developers] faster rubygems.rb loading by way of deferred plugin loads, and other resource usage enhancements

Luis Lavena luislavena at gmail.com
Fri Jul 9 12:25:05 EDT 2010

On Wed, Jul 7, 2010 at 12:27 PM, James Tucker <jftucker at gmail.com> wrote:
> Hey all,
> I've started work on a branch to reduce the runtime resource usage by loading rubygems.rb. Some of you are already aware of this [1].
> The main 'cheat' in the branch so far is to move plugin loads into the command manager, but I'm not entirely satisfied with this, and so I'm polling for interest in what features plugin authors actually need / want, and where those can be injected. Most specifically, I want to see if they can be done so more lightly.
> Initially, probably the most commonly used plugin is the gemcutter plugin, which provides commands. At present it deliberately loads the CommandManager, which results in loading in about 37 files and the spec cache into memory whenever rubygems.rb is loaded. The difference in startup performance is significantly over an order of magnitude on common developer systems (with largish numbers of gems installed).
> A solution for command plugins would be to have plugins simply register a list of files that should be loaded when commands are going to be invoked. A continuation of this idea leads to an API something along the lines of:
> Gem.register_plugin(:install, 'some_plugin/gem/installer')
> Gem.register_plugin(:command, 'some_plugin/gem/command')

Wouldn't be more appropriate Gem.register_command(:push,
'path/to/gemcutter/plugin') ?

After all, you're registering commands, since the plugin loader has
already been started.

> Such that rubygems could contain simple internal hooks to load these on demand, rather than ahead of time, avoiding pre-activation of plugins and the current increased cost of loading rubygems.

I like this, a lot.

> My end goal with this is to aim to get rubygems.rb fast and light enough that it could be used for the core integration in 1.9, to avoid the remaining and otherwise unavoidable bugs in gem_prelude.rb.
> Other work that will be included in this, is indexing the paths to rubygems_plugin.rb files, and reducing the resource requirements for activation, by having a lightweight version and index set that can be used by Gem.activate in place of the current heavyweight Gem.spec.

Wouldn't rubygems_plugin.rb do the Gem.register* command and deffer
the actual plugin loading script for the command invitation?

What is the benefit on indexing those, since these were already loaded?

Luis Lavena
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

More information about the Rubygems-developers mailing list