Gem file size limits
Charles Oliver Nutter
headius at headius.com
Wed Jan 18 17:36:35 EST 2012
Checking in on this one...
On Tue, Jan 17, 2012 at 5:10 PM, Evan Phoenix <evan at phx.io> wrote:
> I believe that rubygems.org needs to limit the max size of a .gem file which will be allowed.
I agree, but having a fairly important gem that's over the 10MB limit
(jruby-jars, fetched by anyone deploying JRuby to a Java webapp
server) I have concerns :)
> This serves two purposes:
> 1) It protects users from themselves. The top 19 of 20 gems sorted by size are all huge because they accidentally packaged all previous versions within themselves. This issue needs to be fixed on the gem build side also, but there is no reason to allow these gems.
> 2) Cost. Rubygems.org is becoming increasingly expensive to run and thus we need to begin thinking of ways to keep it mean and lean.
Limiting gem sizes may stall the cost issue, but it's not going to
eliminate it. Eventually you'll be right back up there with "well
behaved" gems that have released many versions over time.
> To start the decision, let me throw out a starting point: 10 megs.
> Looking at the biggest non-accidental gems, they're almost all jruby related and contain huge .jar files. We've pinged others about removing the impediment to pushing gems with maven deps and thusly devs would use that functionality rather than packaging the jars within the gems themselves.
What about a Github approach? Everyone can get an account with some
amount of free space, and if you go beyond that you have to pay or
move stuff off to your own server (I will get to the "federated"
thread in a minute).
The problem with choosing a limit based on individual gem size is that
the worst offenders may be below that limit but push lots of
revisions, compared to JRuby which has maybe a dozen jruby-jars
versions around 10MB each. Basing the limit on a per-account "free"
tier makes more sense to me.
It's also possible (if not likely) that people interested in pushing
big gems and not hosting them would happily pay for larger tiers of
service. That would immediately start blunting costs without any
complicated multi-home or federation work.
Another mad-cap idea: multihome *using* github. Make it possible for
people to host their gems on github and use that as the store for gems
associated with their account. Then you don't even have to manage the
paid tiers; they just do that themselves.
There's many better options than just shutting the door at 10MB.
More information about the RubyGems-Developers