[Rubygems-developers] Patch and scripts for offline rubygems usage

David Glasser glasser at mit.edu
Tue Jun 7 11:54:32 EDT 2005

[ Just sent this to ruby-talk, but perhaps this is a better place to  
send it. ]

See http://web.mit.edu/glasser/www/ruby/ for the patch and scripts.   
I suspect I should be contributing at least the patch to the rubygems  
project, but I'm not sure the best way to do this -- hopefully  
somebody from the project will read this and tell me.


(Below is just a w3m of the above web page.)

Patch and scripts for offline rubygems usage

I would like to be able to use rubygems to install new gems when not  
connected to the internet. I don't want to just automatically install  
every gem, though. Specifically, I'd like to be able to locally
mirror any gem server. In general, I'd also like to make it easier  
for people to run mirrors of gem servers. To accomplish this, I've  
written three things: a patch to rubygems/remote_installer.rb to  
allow the
use of file:// URLs as sources; a script called gem_mirror to mirror  
from an arbitrary gem server to local disk; and a script called  
gem_server_dumb to run a brainless gem server serving a gem_mirrored

As a reminder, installed gems live in a gem directory with  
subdirectories such as cache and specifications; I will refer to this  
as installed-layout. The gem remote installer, on the other hand,  
looks for a
file called yaml (optionally first compressed as yaml.Z) and a  
subdirectory called gems; I will refer to this as server-layout. The  
standard gem_server script creates a web server which serves sites in
server-layout (plus some other files which are not used by the remote  
installer) based on the contents of an installed-layout gem directory.

rubygems/remote_installer.rb file:// support patch

rubygems_remote_installer_file_uri.patch adds support for rubygems  
sources to be file:// URIs, either by an explicit --sources argument  
to gem or by inclusion in .gemrc. While it would seem more clean to make
this a patch to open-uri, the fact that RemoteSourceFetcher#read_size  
doesn't go through the open-uri abstraction would have meant that  
that method would have still needed to be special-cased; so instead I
simply put the file:// handling inside RemoteSourceFetcher. Note that  
the directory specified by the file:// URL should be in server-layout  

I hope that the rubygems maintainers integrate this patch into the  


The above patch raises the question of where a local server-layout  
directory would come from! Using gem_mirror, you can easily mirror  
gem servers (which can be specified as any URL that open-uri accepts  
or as
paths, but ironically not as file:// URLs) to local directories.  
gem_mirror fetches the yaml file and checks to see which gems  
specified in it it does not have; it saves them to the specified  
directory, along
with the yaml file itself. You can specify as many mirroring  
operations as you would like in a YAML file called .gemmirrorrc; mine  
looks like:

   from: http://gems.rubyforge.org/
   to: /Users/glasser/MyGEMS/gems.rubyforge.org


Just to complete the loop, gem_server_dumb is a simple WEBrick web  
server which serves server-layout directories to the world. So if you  
run gem_mirror periodically and also run gem_server_dumb, you will be
an http mirror of whatever server your are mirroring from, at least  
as far as the remote installer is concerned. (That is, you won't be  
mirroring RDoc or the top page of the gem server, and so on.)

This really doesn't do anything that you couldn't to by just pointing  
any other web server at your server-layout directory.

   Code Monkey, Best Practical Solutions
David Glasser | glasser at bestpractical.com

More information about the Rubygems-developers mailing list