[Rubygems-developers] Status of disabling plugins except for command line?

James Tucker jftucker at gmail.com
Thu Dec 23 11:48:50 EST 2010

On Dec 23, 2010, at 10:55 AM, Trans wrote:

> On Dec 23, 4:24 am, Charles Oliver Nutter <head... at headius.com> wrote:
>> On Thu, Dec 23, 2010 at 3:08 AM, Trans <transf... at gmail.com> wrote:
>>> I know of at least two gems it will break, and I suspect they are not
>>> your four.
>> You guys with your mystery lists. Spill it, man!
> The two I know of are gem-wedge and ore.
> Ore is especially ironic, b/c I was recently told that RubyGems did
> not need to support plain YAML specs itself b/c a plugin, like Ore,
> could do the job.

Ore doesn't contain a rubygems_plugin.rb. Dude, come on.

When I looked at wedge, I figured it was largely experimental and not widely used. It's hooks can also be installed elsehow (that is, by having the application load it - the same way bundler and isolate do). If you want to avoid that, you could add -rwedge to RUBYOPT.

>>> I thought RubyGems was supposed to be so *conservative* about changes,
>>> and here a change is just going to be pushed that breaks people's
>>> plugins for the sake of a speed up for those who have 500+ gems
>>> installed?
>> It's not just a problem for 500+, it's a problem for anyone who
>> installs, say, Rails 3's several dozen dependencies. Globbing through
>> all those directories gets linearly slower, and even people with just
>> a few gems will notice a startup improvement from this.
>> The right answer is that plugins should NEVER have been
>> auto-discovered with a filesystem scan on every startup since that's
>> inherently O(N) as more gems are introduced (and especially bad on
>> implementations that already have startup concerns like JRuby or
>> Rubinius).
> Doesn't the file system itself cache the file access, so after the
> first run it speeds up for every subsequent use? So it's not quite
> that bad. But to be sure I am not against fixing this, I just don't
> want to see non-command plugins broken to do it.

No, because the search loads the entire source index and scans through it in ruby space.

>> So I feel more like we're righting an over-eager change
>> (plugins scanned on every load of 'rubygems') by backing off the most
>> egregious aspect of that decision. We're not breaking good behavior,
>> we're rolling back bad behavior. I call that a bug fix.
> One man's bug is another man's feature ;)

Actually, I've seen more bug reports from broken plugins forcing people to manually remove gems and/or reinstall everything than I have seen usage of non-command plugins. Bear in mind that I've been both following the tracker, and diving through /all gems/ as my research.

>>> Overlooking the fact that there other solutions to deal with large gem
>>> collections, like rvm's gemsets. Why wouldn't the prudent course of
>>> action be to actually fix this _correctly now_, rather than implement
>>> a "phase 1" change that will leave certain types of plugins out in the
>>> cold for who knows how long?
>>> If that's asking too much, then at the very least, make this patch a
>>> configurable option in the .gemrc file. Power users who aren't using
>>> any of the above mentioned plugins could then easily flip the switch
>>> to get the speed up.
>> No plugins have been mentioned :) All I see are shadows and spectres.
>> I want to know what these plugins are and why this change breaks them.
>> If there's additional changes required to keep them working, we can
>> discuss that. But having a simple load of 'rubygems' scan the
>> filesystem every freaking time is a nonstarter (and should always have
>> been).
> No, it was a shortcut then, and now another shortcut is being made to
> remedy the first.
> A real solution would check for a plugins at install time, if one is
> found copy it to a special location, e.g. gems/plugins, in much the
> same way that a gem's specification is copied to a special location.
> Then when Rubygems is loaded there is no need to search the file
> system, all the plugins are in one place.

That's actually not quite enough. These problems have nasty edge cases:

Many plugins use File.expand_path to load their other parts, because we don't activate plugins.
We should not be loading multiple versions of a single plugin.
What should be the policy when a newer gem is installed that does not contain a plugin, but an older version does contain a plugin? This can lead to unresolvable problems.
We need options to turn the features off in future.

This is all in my long term plan. We're not alienating anyone by this change, and people have been trying it out in the wild for a good deal of time too, as well as I've been watching and researching it.

More information about the Rubygems-developers mailing list