From jim at weirichhouse.org Wed Dec 1 09:01:23 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Wed Dec 1 09:00:29 2004 Subject: [Rubygems-developers] More work on the remote installer Message-ID: <200412010901.24161.jim@weirichhouse.org> I am continuing my work on the remote installer. Recent changes include : (1) proxy support added back to the RemoteSourceFetcher. We lost proxy support when I switched to using an HTTP HEAD request. We now have that back. In addition, there are several unit tests covering proxy support in RemoteSourceFetcher. The unit tests involve firing up two web services during the unit test. I would appreciate if people try it and and report any problems. (2) The YAML file retrieved from the remote source is "reduced" before passing it to YAML.load. I was having a terrible time with large YAML files and the YAML loader crashing. Reducing the size of the YAML works around this bug, but does not eliminate the root case. The YAML file is reduced by removing the files list from all the gemspecs. This reduced the size of the YAML file by 60% (IIRC). Since the files list is not used in the source info cache, I don't anticipate any problems. We might consider reducing the YAML file /before/ sending it over the wire to reduce bandwith usage. Next on my hit list ... (A) Fix the test_install method in test_remote_installer, which was broken (and currently disabled) by the RemoteSourceFetcher changes. (B) Add support for a writable per-user source info cache for use when the global cache cannot be written.j (C) Continued refactoring of the remote installer. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From chad at chadfowler.com Wed Dec 1 09:09:56 2004 From: chad at chadfowler.com (Chad Fowler) Date: Wed Dec 1 09:09:12 2004 Subject: [Rubygems-developers] More work on the remote installer In-Reply-To: <200412010901.24161.jim@weirichhouse.org> References: <200412010901.24161.jim@weirichhouse.org> Message-ID: On 01-Dec-04, at 9:01 AM, Jim Weirich wrote: > I am continuing my work on the remote installer. Recent changes > include : > > (1) proxy support added back to the RemoteSourceFetcher. We lost proxy > support when I switched to using an HTTP HEAD request. We now have > that > back. In addition, there are several unit tests covering proxy > support in > RemoteSourceFetcher. The unit tests involve firing up two web services > during the unit test. I would appreciate if people try it and and > report any > problems. > Worked great for me! > (2) The YAML file retrieved from the remote source is "reduced" before > passing > it to YAML.load. I was having a terrible time with large YAML files > and the > YAML loader crashing. Reducing the size of the YAML works around this > bug, > but does not eliminate the root case. The YAML file is reduced by > removing > the files list from all the gemspecs. This reduced the size of the > YAML file > by 60% (IIRC). Since the files list is not used in the source info > cache, I > don't anticipate any problems. We might consider reducing the YAML > file /before/ sending it over the wire to reduce bandwith usage. > Not a bad idea. We're not currently serving the yaml dynamically, but I bet it wouldn't take much to change the reaper script. Rich has so much free time these days, he could probably get it done today ;) (just kidding). I'm thinking we can actually make this change on the server without breaking existing code. Is that right as per your understanding? > Next on my hit list ... > > (A) Fix the test_install method in test_remote_installer, which was > broken > (and currently disabled) by the RemoteSourceFetcher changes. > > (B) Add support for a writable per-user source info cache for use when > the > global cache cannot be written.j > > (C) Continued refactoring of the remote installer. > Great stuff, Jim. This is definitely the area that needs the most help. Chad From hgs at dmu.ac.uk Wed Dec 1 09:25:46 2004 From: hgs at dmu.ac.uk (Hugh Sasse Staff Elec Eng) Date: Wed Dec 1 09:25:07 2004 Subject: [Rubygems-developers] More work on the remote installer In-Reply-To: <200412010901.24161.jim@weirichhouse.org> References: <200412010901.24161.jim@weirichhouse.org> Message-ID: On Wed, 1 Dec 2004, Jim Weirich wrote: > I am continuing my work on the remote installer. Recent changes include : [...] > (B) Add support for a writable per-user source info cache for use when the > global cache cannot be written.j Nobody has responded to my remarks about possible denial of service attacks by users if data from this is merged with the system cache. Have I misunderstood this? Is there no way a user can damage the site-wide data if they choose? http://www.theregister.co.uk/2004/11/29/ie_security_holes/ I'm trying to think of what is possible when you can only really trust some users on a system. There's a reason why ruby and other languages need a root install.... Hugh From batsman.geo at yahoo.com Wed Dec 1 09:32:28 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Wed Dec 1 09:31:47 2004 Subject: [Rubygems-developers] More work on the remote installer In-Reply-To: References: <200412010901.24161.jim@weirichhouse.org> Message-ID: <20041201143228.GA13209@student.ei.uni-stuttgart.de> On Wed, Dec 01, 2004 at 02:25:46PM +0000, Hugh Sasse Staff Elec Eng wrote: > On Wed, 1 Dec 2004, Jim Weirich wrote: > > >I am continuing my work on the remote installer. Recent changes include : > [...] > >(B) Add support for a writable per-user source info cache for use when the > >global cache cannot be written.j > > Nobody has responded to my remarks about possible denial of service > attacks by users if data from this is merged with the system cache. > Have I misunderstood this? Is there no way a user can damage the > site-wide data if they choose? If I've understood it correctly, the per-user cached data would never be merged into the system cache. -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From jim at weirichhouse.org Wed Dec 1 09:49:09 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Wed Dec 1 09:48:04 2004 Subject: [Rubygems-developers] More work on the remote installer In-Reply-To: References: <200412010901.24161.jim@weirichhouse.org> Message-ID: <200412010949.09477.jim@weirichhouse.org> On Wednesday 01 December 2004 09:25 am, Hugh Sasse Staff Elec Eng wrote: > > (B) Add support for a writable per-user source info cache for use when > > the global cache cannot be written.j > > Nobody has responded to my remarks about possible denial of service > attacks by users if data from this is merged with the system cache. > Have I misunderstood this? Is there no way a user can damage the > site-wide data if they choose? My plan is to update the site-wide file if the current user has write access to it. Otherwise update the user specific file. Both of these updates would be from the downloaded information. I would not update the site-wide file with info from the user. However, the user could specifically update his own copy manually. I'm not sure I understand your concern over a DOS (Denial of Service ... right?) attack based on a hacked cache file. These cache files are merely hashes that map source and gem names into gemspecs for searching purposes. If the cache file is incorrect, it would mean that the gem command would attempt to grab a version of a gem from the server that might not be there (or tell you it can't find a gem that really is there) Could you expand on your concerns about DOS attacks. Thanks. > http://www.theregister.co.uk/2004/11/29/ie_security_holes/ > > I'm trying to think of what is possible when you can only really > trust some users on a system. There's a reason why ruby and other > languages need a root install.... -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Wed Dec 1 09:53:29 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Wed Dec 1 09:52:23 2004 Subject: [Rubygems-developers] Extension/Binary Install Message-ID: <200412010953.29076.jim@weirichhouse.org> I'm about to try my hand at a gem with a C extension (and subsequently, a windows-binary gem). Any hints or advice before I go down this path. I'm particularly interested in directory structure issues, especially to interoperate with RPA (and other) packagers. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From hgs at dmu.ac.uk Wed Dec 1 09:56:04 2004 From: hgs at dmu.ac.uk (Hugh Sasse Staff Elec Eng) Date: Wed Dec 1 09:55:23 2004 Subject: [Rubygems-developers] More work on the remote installer In-Reply-To: <20041201143228.GA13209@student.ei.uni-stuttgart.de> References: <200412010901.24161.jim@weirichhouse.org> <20041201143228.GA13209@student.ei.uni-stuttgart.de> Message-ID: On Wed, 1 Dec 2004, Mauricio [iso-8859-1] Fernández wrote: > On Wed, Dec 01, 2004 at 02:25:46PM +0000, Hugh Sasse Staff Elec Eng wrote: >> >> Nobody has responded to my remarks about possible denial of service > > If I've understood it correctly, the per-user cached data would never > be merged into the system cache. OK, rereading I see that now. I was assuming that the central system would attempt to [get] leverage [from] the user updates when the time came to update as root, but this was never stated. To implement it would probably be useless anyway, since the user files could now be out of date, as well. Thanks, sorry for the noise :-) > Hugh From chad at chadfowler.com Wed Dec 1 09:56:34 2004 From: chad at chadfowler.com (Chad Fowler) Date: Wed Dec 1 09:55:49 2004 Subject: [Rubygems-developers] Extension/Binary Install In-Reply-To: <200412010953.29076.jim@weirichhouse.org> References: <200412010953.29076.jim@weirichhouse.org> Message-ID: <31C47102-43A9-11D9-BBCD-000D9337C01C@chadfowler.com> On 01-Dec-04, at 9:53 AM, Jim Weirich wrote: > I'm about to try my hand at a gem with a C extension (and > subsequently, a > windows-binary gem). Any hints or advice before I go down this path. > I'm > particularly interested in directory structure issues, especially to > interoperate with RPA (and other) packagers. > The best thing to do is to get setup.rb and use its structure as a guide. We also have some extension issues that you might come across and fix as you go ;) For example, pay attention to where the compiled extension goes during install time. Chad From hgs at dmu.ac.uk Wed Dec 1 10:03:24 2004 From: hgs at dmu.ac.uk (Hugh Sasse Staff Elec Eng) Date: Wed Dec 1 10:02:41 2004 Subject: [Rubygems-developers] More work on the remote installer In-Reply-To: <200412010949.09477.jim@weirichhouse.org> References: <200412010901.24161.jim@weirichhouse.org> <200412010949.09477.jim@weirichhouse.org> Message-ID: On Wed, 1 Dec 2004, Jim Weirich wrote: > > My plan is to update the site-wide file if the current user has write access > to it. Otherwise update the user specific file. Both of these updates would > be from the downloaded information. I would not update the site-wide file > with info from the user. OK. > > However, the user could specifically update his own copy manually. I'm not > sure I understand your concern over a DOS (Denial of Service ... right?) > attack based on a hacked cache file. These cache files are merely hashes > that map source and gem names into gemspecs for searching purposes. If the > cache file is incorrect, it would mean that the gem command would attempt to > grab a version of a gem from the server that might not be there (or tell you Which would be an inconvenience but, ... > it can't find a gem that really is there) Could you expand on your concerns ... that is the kind of thing I was thinking of. One student could disrupt another's work if he knew her work depended on a gem he wasn't using. She'd not be able to get the updates she neeeds, etc. But since you aren't doing this it is irrelevant now. > about DOS attacks. Thanks. > Thank you, Hugh From batsman.geo at yahoo.com Wed Dec 1 11:10:04 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Wed Dec 1 11:09:15 2004 Subject: [Rubygems-developers] Extension/Binary Install In-Reply-To: <31C47102-43A9-11D9-BBCD-000D9337C01C@chadfowler.com> References: <200412010953.29076.jim@weirichhouse.org> <31C47102-43A9-11D9-BBCD-000D9337C01C@chadfowler.com> Message-ID: <20041201161004.GA19964@student.ei.uni-stuttgart.de> On Wed, Dec 01, 2004 at 09:56:34AM -0500, Chad Fowler wrote: > >I'm about to try my hand at a gem with a C extension (and > >subsequently, a > >windows-binary gem). Any hints or advice before I go down this path. > >I'm > >particularly interested in directory structure issues, especially to > >interoperate with RPA (and other) packagers. > > > > The best thing to do is to get setup.rb and use its structure as a > guide. That's good advice. Some other things to take into account: * avoid complex build phases if possible -- extconf.rb only is best * if you use your own build scripts (for instance Rakefiles), make sure they don't assume too much about the environment (unfortunately, require 'rubygems' is, by design, required, if you're using it to build the .gem package) * make sure the RubyGems package contains all the relevant information, _especially_ if it's going to be a gem-only release * verify that your gem-build scripts are idempotent and/or provide the pristine sources separately In general ask yourself: would a user be able to rebuild the .gem package from mine? That's a basic requirement for an upstream release. -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From drbrain at segment7.net Thu Dec 2 01:13:37 2004 From: drbrain at segment7.net (Eric Hodel) Date: Thu Dec 2 01:12:55 2004 Subject: [Rubygems-developers] Gem::Specification#copy_of and Rails makes tests startup slow Message-ID: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> I have a small testcase in rails that has 7 tests in it (4 of them raise NotImplementedError), but simply starting up the testcase takes 3 seconds: $ ruby -r profile -Ilib:test test/unit/emailer_test.rb % cumulative self self total time seconds seconds calls ms/call ms/call name 18.37 5.57 5.57 462 12.06 19.42 Integer#gcd 10.26 8.68 3.11 4008 0.78 2.37 Gem::Specification#copy_of The tests themselves complete in less time than it takes to perform all the copying: $ ruby -Ilib:test test/unit/emailer_test.rb | unit_diff Loaded suite test/unit/emailer_test Started ..E.EEE Finished in 2.526909 seconds. [...] 7 tests, 18 assertions, 0 failures, 4 errors When running the entire test suite this startup cost is minimal compared to the run time of the tests, but it rarely makes sense to run the entire test suite at once. Is all the copying done in #copy_of completely necessary? BTW: These were run on a PowerBook G4 1.5GHz From jim at weirichhouse.org Fri Dec 3 02:37:32 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Fri Dec 3 02:36:11 2004 Subject: [Rubygems-developers] Remote Installer Progress Message-ID: <200412030237.32642.jim@weirichhouse.org> Recent changes include: (1) Support for a user writable cache file if the system wide file is not writable. The user writable file will be $HOME/.gem/source_cache (overridable via GEMCACHE env variable). I do attempt to find a reasonable home directory for windows machines (see the find_home method in rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when in windows, I'm open to suggestions. (2) Included a patch provided by Austin Ziegler to work around problems with the 1.1.4 zlib on windows. The 1.1.4 zlib problem caused gem to crash with a 'buffer error' when installing ruwiki. I've tested (2) on windows XP and linux (Debian), but would like to hear feedback from a few other platforms as well. There is still a bit of work in the remote install portion. I need to tweak the cache file writing a bit and Chad has suggested looking at some dependency install issues. However, given that the source base now includes several fixes (zlib as described here, a large yaml file workaround and the null byte workaround), I would like to do a point release soon. I'm suggesting this weekend. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From hgs at dmu.ac.uk Fri Dec 3 06:56:26 2004 From: hgs at dmu.ac.uk (Hugh Sasse Staff Elec Eng) Date: Fri Dec 3 06:56:13 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <200412030237.32642.jim@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org> Message-ID: On Fri, 3 Dec 2004, Jim Weirich wrote: > rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when in > windows, I'm open to suggestions. Seems OK to me -- these are rather old: http://filext.com/detaillist.php?extdetail=gem&goButton=Go Thank you, Hugh From jim at weirichhouse.org Fri Dec 3 08:55:34 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Fri Dec 3 08:54:13 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: References: <200412030237.32642.jim@weirichhouse.org> Message-ID: <200412030855.35161.jim@weirichhouse.org> On Friday 03 December 2004 06:56 am, Hugh Sasse Staff Elec Eng wrote: > On Fri, 3 Dec 2004, Jim Weirich wrote: > > rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when > > in windows, I'm open to suggestions. > > Seems OK to me -- these are rather old: > > http://filext.com/detaillist.php?extdetail=gem&goButton=Go Ahhh ... just to be clear, ".gem" is not the file extension, but the entire file name. It is a convention on unix that file beginnnig with "." are hidden. Lots of apps in unix put a ".appname" file in a users home directory to store per user information. So that's what I was asking, is a file name (not extension) ".gem" too weird for windows users? -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Fri Dec 3 08:57:28 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Fri Dec 3 08:56:03 2004 Subject: [Rubygems-developers] RubyGems API Docs Message-ID: <200412030857.28639.jim@weirichhouse.org> I just realized that we have not published the RubyGems API docs anywhere. There wasn't even a Rakefile task to generate the rdocs. So I took the liberty of generating them and putting them out at http://rubygems.rubyforge.org/rdoc (and added pointers on the RubyGems main page). -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From halostatue at gmail.com Fri Dec 3 08:57:44 2004 From: halostatue at gmail.com (Austin Ziegler) Date: Fri Dec 3 08:56:50 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <200412030855.35161.jim@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org> <200412030855.35161.jim@weirichhouse.org> Message-ID: <9e7db911041203055751f7c5aa@mail.gmail.com> On Fri, 3 Dec 2004 08:55:34 -0500, Jim Weirich wrote: > On Friday 03 December 2004 06:56 am, Hugh Sasse Staff Elec Eng wrote: > > On Fri, 3 Dec 2004, Jim Weirich wrote: > > > rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when > > > in windows, I'm open to suggestions. > > Seems OK to me -- these are rather old: > > > > http://filext.com/detaillist.php?extdetail=gem&goButton=Go > Ahhh ... just to be clear, ".gem" is not the file extension, but the entire > file name. It is a convention on unix that file beginnnig with "." are > hidden. Lots of apps in unix put a ".appname" file in a users home directory > to store per user information. > > So that's what I was asking, is a file name (not extension) ".gem" too weird > for windows users? If there's a chance that people using Windows 9x will be using this, then use _gem. Otherwise, .gem is okay. -austin -- Austin Ziegler * halostatue@gmail.com * Alternate: austin@halostatue.ca From batsman.geo at yahoo.com Fri Dec 3 09:36:53 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Fri Dec 3 09:36:00 2004 Subject: [Rubygems-developers] RubyGems API Docs In-Reply-To: <200412030857.28639.jim@weirichhouse.org> References: <200412030857.28639.jim@weirichhouse.org> Message-ID: <20041203143653.GA10686@student.ei.uni-stuttgart.de> On Fri, Dec 03, 2004 at 08:57:28AM -0500, Jim Weirich wrote: > I just realized that we have not published the RubyGems API docs anywhere. > There wasn't even a Rakefile task to generate the rdocs. > > So I took the liberty of generating them and putting them out at > http://rubygems.rubyforge.org/rdoc (and added pointers on the RubyGems main > page). Is the API officially considered stable, i.o.w. are you going to commit to it? I thought that only a very small part of RubyGems was to be exposed through a public API: Gem::Cache, Gem::Specification, Gem::RemoteInstaller... -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From hgs at dmu.ac.uk Fri Dec 3 09:58:35 2004 From: hgs at dmu.ac.uk (Hugh Sasse Staff Elec Eng) Date: Fri Dec 3 09:57:49 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <200412030855.35161.jim@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org> <200412030855.35161.jim@weirichhouse.org> Message-ID: On Fri, 3 Dec 2004, Jim Weirich wrote: > On Friday 03 December 2004 06:56 am, Hugh Sasse Staff Elec Eng wrote: >> On Fri, 3 Dec 2004, Jim Weirich wrote: >>> rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when >>> in windows, I'm open to suggestions. >> >> Seems OK to me -- these are rather old: >> >> http://filext.com/detaillist.php?extdetail=gem&goButton=Go > > Ahhh ... just to be clear, ".gem" is not the file extension, but the entire > file name. It is a convention on unix that file beginnnig with "." are > hidden. Lots of apps in unix put a ".appname" file in a users home directory > to store per user information. Oh, sorry, I wasn't thinking clearly when I wrote that. I think I've had this conversation before in the context of some ruby thing when I mentioned that vim allows an _vimrc file on Windows in case the .vimrc is a problem. People asserted quite firmly that on windows .vimrc should never be a problem and thus . would not be a problem either. [In case anyone is as confused as I was above: that's in XML style, not redirections and a directory reference! :-)] > > So that's what I was asking, is a file name (not extension) ".gem" > too weird for windows users? > So I think the answer to that is "It should be fine." But it's proprietry, so Windows may well do whatever it likes :-) Thank you, Hugh From halostatue at gmail.com Fri Dec 3 10:48:05 2004 From: halostatue at gmail.com (Austin Ziegler) Date: Fri Dec 3 10:47:11 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <200412030237.32642.jim@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org> Message-ID: <9e7db911041203074838556502@mail.gmail.com> On Fri, 3 Dec 2004 02:37:32 -0500, Jim Weirich wrote: > (2) Included a patch provided by Austin Ziegler to work around problems with > the 1.1.4 zlib on windows. The 1.1.4 zlib problem caused gem to crash with a > 'buffer error' when installing ruwiki. > > I've tested (2) on windows XP and linux (Debian), but would like to hear > feedback from a few other platforms as well. How quickly will this next release be coming? If it isn't very very quickly (e.g., by Monday), would it be possible to have a 0.8.x release that includes this patch? This is preventing Ruwiki from being installed as a gem at all on Windows, and may present for other gems as well. Also, Mauricio was experiencing an apparent YAML/Syck error when trying to reproduce on his Linux box with an 11/26 build of Ruby, so there may be some issues that need to be dealt with there. -austin -- Austin Ziegler * halostatue@gmail.com * Alternate: austin@halostatue.ca From rich at infoether.com Fri Dec 3 10:58:08 2004 From: rich at infoether.com (Richard Kilmer) Date: Fri Dec 3 10:57:17 2004 Subject: [Rubygems-developers] RubyGems API Docs In-Reply-To: <20041203143653.GA10686@student.ei.uni-stuttgart.de> Message-ID: The whole API is public in that it is open source. All the classes can be manipulated by any person that wants (on their own machine, etc), and we should publish it all so that someone who wants to fully understand/ contribute to the API can do so. But you are correct that there is a subset of the API that library developers that package stuff as gems need to understand to be functional. We will change the API over time as necessary, attempt to always be backward compatible where possible, and provide clear direction on how to migrate in the event that compatibility is not achieved. -rich On 12/3/04 9:36 AM, "Mauricio Fern?ndez" wrote: > On Fri, Dec 03, 2004 at 08:57:28AM -0500, Jim Weirich wrote: >> I just realized that we have not published the RubyGems API docs anywhere. >> There wasn't even a Rakefile task to generate the rdocs. >> >> So I took the liberty of generating them and putting them out at >> http://rubygems.rubyforge.org/rdoc (and added pointers on the RubyGems main >> page). > > Is the API officially considered stable, i.o.w. are you going to commit > to it? > > I thought that only a very small part of RubyGems was to be > exposed through a public API: Gem::Cache, Gem::Specification, > Gem::RemoteInstaller... From gsinclair at soyabean.com.au Fri Dec 3 20:18:23 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Fri Dec 3 20:18:29 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <9e7db911041203074838556502@mail.gmail.com> References: <200412030237.32642.jim@weirichhouse.org> <9e7db911041203074838556502@mail.gmail.com> Message-ID: <199753264787.20041204121823@soyabean.com.au> On Saturday, December 4, 2004, 2:48:05 AM, Austin wrote: > On Fri, 3 Dec 2004 02:37:32 -0500, Jim Weirich wrote: >> (2) Included a patch provided by Austin Ziegler to work around problems with >> the 1.1.4 zlib on windows. The 1.1.4 zlib problem caused gem to crash with a >> 'buffer error' when installing ruwiki. >> >> I've tested (2) on windows XP and linux (Debian), but would like to hear >> feedback from a few other platforms as well. > How quickly will this next release be coming? If it isn't very very > quickly (e.g., by Monday), would it be possible to have a 0.8.x > release that includes this patch? This is preventing Ruwiki from being > installed as a gem at all on Windows, and may present for other gems > as well. > Also, Mauricio was experiencing an apparent YAML/Syck error when > trying to reproduce on his Linux box with an 11/26 build of Ruby, so > there may be some issues that need to be dealt with there. 0.8.2 would be more appropriate than 0.9.0 at this point anyway, wouldn't it? Gavin From gsinclair at soyabean.com.au Fri Dec 3 20:20:04 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Fri Dec 3 20:20:09 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <9e7db911041203055751f7c5aa@mail.gmail.com> References: <200412030237.32642.jim@weirichhouse.org> <200412030855.35161.jim@weirichhouse.org> <9e7db911041203055751f7c5aa@mail.gmail.com> Message-ID: <52753365722.20041204122004@soyabean.com.au> On Saturday, December 4, 2004, 12:57:44 AM, Austin wrote: > On Fri, 3 Dec 2004 08:55:34 -0500, Jim Weirich wrote: >> On Friday 03 December 2004 06:56 am, Hugh Sasse Staff Elec Eng wrote: >> > On Fri, 3 Dec 2004, Jim Weirich wrote: >> > > rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when >> > > in windows, I'm open to suggestions. >> > Seems OK to me -- these are rather old: >> > >> > http://filext.com/detaillist.php?extdetail=gem&goButton=Go >> Ahhh ... just to be clear, ".gem" is not the file extension, but the entire >> file name. It is a convention on unix that file beginnnig with "." are >> hidden. Lots of apps in unix put a ".appname" file in a users home directory >> to store per user information. >> >> So that's what I was asking, is a file name (not extension) ".gem" too weird >> for windows users? > If there's a chance that people using Windows 9x will be using this, > then use _gem. Otherwise, .gem is okay. But only if absolutely necessary. The .vimrc/_vimrc difference is a major PITA, AFAIC. Gavin From gsinclair at soyabean.com.au Fri Dec 3 20:22:30 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Fri Dec 3 20:22:34 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <200412030237.32642.jim@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org> Message-ID: <120753511241.20041204122230@soyabean.com.au> On Friday, December 3, 2004, 6:37:32 PM, Jim wrote: > (1) Support for a user writable cache file if the system wide file is not > writable. The user writable file will be $HOME/.gem/source_cache > (overridable via GEMCACHE env variable). I do attempt to find a reasonable > home directory for windows machines (see the find_home method in > rubygems.rb). Also, I'm not sure if ".gem" is an appropriate name when in > windows, I'm open to suggestions. Wouldn't a more descriptive name be better, like .gem-cache? Gavin From jim at weirichhouse.org Fri Dec 3 22:32:34 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Fri Dec 3 22:31:03 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <199753264787.20041204121823@soyabean.com.au> References: <200412030237.32642.jim@weirichhouse.org> <9e7db911041203074838556502@mail.gmail.com> <199753264787.20041204121823@soyabean.com.au> Message-ID: <200412032232.34450.jim@weirichhouse.org> On Friday 03 December 2004 08:18 pm, Gavin Sinclair wrote: > On Saturday, December 4, 2004, 2:48:05 AM, Austin wrote: > > On Fri, 3 Dec 2004 02:37:32 -0500, Jim Weirich wrote: > >> (2) Included a patch provided by Austin Ziegler to work around problems > >> with the 1.1.4 zlib on windows. The 1.1.4 zlib problem caused gem to > >> crash with a 'buffer error' when installing ruwiki. > >> > >> I've tested (2) on windows XP and linux (Debian), but would like to hear > >> feedback from a few other platforms as well. > > > > How quickly will this next release be coming? If it isn't very very > > quickly (e.g., by Monday), would it be possible to have a 0.8.x > > release that includes this patch? > 0.8.2 would be more appropriate than 0.9.0 at this point anyway, > wouldn't it? I intend for the release this weekend to be a 0.8.2 release. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim.weirich at gmail.com Fri Dec 3 22:45:35 2004 From: jim.weirich at gmail.com (Jim Weirich) Date: Fri Dec 3 22:44:41 2004 Subject: [Rubygems-developers] Remote Installer Progress Message-ID: On Friday 03 December 2004 08:18 pm, Gavin Sinclair wrote: > On Saturday, December 4, 2004, 2:48:05 AM, Austin wrote: > > How quickly will this next release be coming? If it isn't very very > > quickly (e.g., by Monday), would it be possible to have a 0.8.x > > release that includes this patch? > 0.8.2 would be more appropriate than 0.9.0 at this point anyway, > wouldn't it? I intend for the release this weekend to be a 0.8.2 release. -- -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Fri Dec 3 11:55:34 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Fri Dec 3 22:59:46 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: References: <200412030237.32642.jim@weirichhouse.org><200412030855.35161.jim@weirichhouse.org> Message-ID: <51422.192.223.163.6.1102092934.squirrel@weirichhouse.org> Hugh Sasse Staff Elec Eng said: >> So that's what I was asking, is a file name (not extension) ".gem" >> too weird for windows users? >> > So I think the answer to that is "It should be fine." But it's > proprietry, so Windows may well do whatever it likes :-) I'm at work now and can check a windows machine. I looked in the directory specified by %USERPROFILE% and found numerous files and directories beginning with a "." dot. Most were from programs developed originally on unix systems but the precedent is there. BTW, I use the following algorithm to determine a home directory. This probably fails on older windows systems that don't set USERFPROFILE, HOMEDRIVE or HOMEPATH. And expanding "~" fails on anything without HOME defined, so I need a better "last resort". Suggestions welcome. def find_home ['HOME', 'USERPROFILE'].each do |homekey| return ENV[homekey] if ENV[homekey] end if ENV['HOMEDRIVE'] && ENV['HOMEPATH'] return "#{ENV['HOMEDRIVE']}:#{ENV['HOMEPATH']}" end File.expand_path("~") end -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Fri Dec 3 12:02:43 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Fri Dec 3 22:59:47 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <9e7db911041203074838556502@mail.gmail.com> References: <200412030237.32642.jim@weirichhouse.org> <9e7db911041203074838556502@mail.gmail.com> Message-ID: <20492.192.223.163.6.1102093363.squirrel@weirichhouse.org> Austin Ziegler said: >> I've tested (2) on windows XP and linux (Debian), but would like to hear >> feedback from a few other platforms as well. > > How quickly will this next release be coming? [...] I'm planning on a point release this weekend unless someone objects strongly. There are a couple of things that probably won't make it be then (e.g. remote dependecy cleanup), but I don't think those should stop a point release. > Also, Mauricio was experiencing an apparent YAML/Syck error when > trying to reproduce on his Linux box with an 11/26 build of Ruby, so > there may be some issues that need to be dealt with there. I was having great problems with the YAML supplied by the gem servers. The current code base arbitrarily cuts down the size of the YAML stuff, but that's just a workaround for a deeper problem. But, I haven't been able to isolate the problem in an easy to reproduce form yet. If someone can do this, we really need to feed the problem to Why (if indeed it is a YAML/SYCK problem). -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From chad at chadfowler.com Sat Dec 4 08:31:45 2004 From: chad at chadfowler.com (Chad Fowler) Date: Sat Dec 4 08:30:50 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <20492.192.223.163.6.1102093363.squirrel@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org> <9e7db911041203074838556502@mail.gmail.com> <20492.192.223.163.6.1102093363.squirrel@weirichhouse.org> Message-ID: On 03-Dec-04, at 12:02 PM, Jim Weirich wrote: > > Austin Ziegler said: >>> I've tested (2) on windows XP and linux (Debian), but would like to >>> hear >>> feedback from a few other platforms as well. >> >> How quickly will this next release be coming? [...] > > I'm planning on a point release this weekend unless someone objects > strongly. There are a couple of things that probably won't make it be > then (e.g. remote dependecy cleanup), but I don't think those should > stop > a point release. > I agree. >> Also, Mauricio was experiencing an apparent YAML/Syck error when >> trying to reproduce on his Linux box with an 11/26 build of Ruby, so >> there may be some issues that need to be dealt with there. > > I was having great problems with the YAML supplied by the gem servers. > The current code base arbitrarily cuts down the size of the YAML stuff, > but that's just a workaround for a deeper problem. But, I haven't been > able to isolate the problem in an easy to reproduce form yet. If > someone > can do this, we really need to feed the problem to Why (if indeed it > is a > YAML/SYCK problem). > I tried to reproduce it myself and couldn't. I think it happened once for me. I tried setting up some stress programs to force the situation but never could. Thanks for all of your recent work! Chad From gsinclair at soyabean.com.au Sat Dec 4 10:49:34 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Sat Dec 4 10:50:06 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <51422.192.223.163.6.1102092934.squirrel@weirichhouse.org> References: <200412030237.32642.jim@weirichhouse.org><200412030855.35161.jim@weirichhouse.org> <51422.192.223.163.6.1102092934.squirrel@weirichhouse.org> Message-ID: <139805535568.20041205024934@soyabean.com.au> On Saturday, December 4, 2004, 3:55:34 AM, Jim wrote: > BTW, I use the following algorithm to determine a home directory. This > probably fails on older windows systems that don't set USERFPROFILE, > HOMEDRIVE or HOMEPATH. And expanding "~" fails on anything without HOME > defined, so I need a better "last resort". Suggestions welcome. A good last resort is probably C:\. Perhaps there's an environment variable that points to the drive where the OS is installed. I doubt anyone would be using Win98 etc. now anyway. Hal Fulton may have been the last of the old guard... Gavin From jim at weirichhouse.org Mon Dec 6 08:03:10 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Mon Dec 6 08:01:20 2004 Subject: [Rubygems-developers] Release 0.8.2 Uploaded Message-ID: <200412060803.10388.jim@weirichhouse.org> I've uploaded the 0.8.2 tgz and zip files to Rubyforge. I plan to double check the files my windows box at work today and (assuming it looks good) make a release announcement today. If anybody wishes to download the files and do any last minute checking before the announcement, that would be appreciated. Thanks. If everything goes well, I will update the rubygems-update file this evening and make that available as a gem. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Mon Dec 6 08:08:14 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Mon Dec 6 08:06:24 2004 Subject: [Rubygems-developers] RubyGems Build Monitor Message-ID: <200412060808.14409.jim@weirichhouse.org> You may also be interested in this: http://onestepback.org/cgi-bin/rubygems_build.cgi?action=show_history -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From chad at chadfowler.com Mon Dec 6 08:13:04 2004 From: chad at chadfowler.com (Chad Fowler) Date: Mon Dec 6 08:12:03 2004 Subject: [Rubygems-developers] Release 0.8.2 Uploaded In-Reply-To: <200412060803.10388.jim@weirichhouse.org> References: <200412060803.10388.jim@weirichhouse.org> Message-ID: <90135F55-4788-11D9-B545-000D9337C01C@chadfowler.com> On 06-Dec-04, at 8:03 AM, Jim Weirich wrote: > I've uploaded the 0.8.2 tgz and zip files to Rubyforge. I plan to > double > check the files my windows box at work today and (assuming it looks > good) > make a release announcement today. If anybody wishes to download the > files > and do any last minute checking before the announcement, that would be > appreciated. Thanks. > > If everything goes well, I will update the rubygems-update file this > evening > and make that available as a gem. > It looks like the usually-included sources gem isn't in this tarball. Does that seem true to you? Chad From jim at weirichhouse.org Mon Dec 6 08:21:20 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Mon Dec 6 08:19:29 2004 Subject: [Rubygems-developers] Release 0.8.2 Uploaded In-Reply-To: <200412060803.10388.jim@weirichhouse.org> References: <200412060803.10388.jim@weirichhouse.org> Message-ID: <200412060821.20125.jim@weirichhouse.org> On Monday 06 December 2004 08:03 am, Jim Weirich wrote: > I've uploaded the 0.8.2 tgz and zip files to Rubyforge. I plan to double > check the files my windows box at work today and (assuming it looks good) > make a release announcement today. If anybody wishes to download the files > and do any last minute checking before the announcement, that would be > appreciated. Thanks. > > If everything goes well, I will update the rubygems-update file this > evening and make that available as a gem. Hold off on this a bit. Within minutes of posting this, I discovered a problem on systems that might not have an initialized user cache (which would be everybody but my test box). Correction will follow soon. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Mon Dec 6 08:46:05 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Mon Dec 6 08:44:18 2004 Subject: [Rubygems-developers] Release 0.8.2 Uploaded In-Reply-To: <90135F55-4788-11D9-B545-000D9337C01C@chadfowler.com> References: <200412060803.10388.jim@weirichhouse.org> <90135F55-4788-11D9-B545-000D9337C01C@chadfowler.com> Message-ID: <200412060846.05102.jim@weirichhouse.org> On Monday 06 December 2004 08:13 am, Chad Fowler wrote: > On 06-Dec-04, at 8:03 AM, Jim Weirich wrote: > > I've uploaded the 0.8.2 tgz and zip files to Rubyforge. I plan to > > double > > check the files my windows box at work today and (assuming it looks > > good) > > make a release announcement today. If anybody wishes to download the > > files > > and do any last minute checking before the announcement, that would be > > appreciated. Thanks. > > > > If everything goes well, I will update the rubygems-update file this > > evening > > and make that available as a gem. > > It looks like the usually-included sources gem isn't in this tarball. > Does that seem true to you? I don't see it either. There is some confusion in the whether the sources gem stuff should be in packages or pkgs (I think pkgs is the latest, right?). Unfortunately I have to get to work this morning and don't have time to continue with this right now. I've withdrawn 0.8.2 from rubyforge because this and the other problem. I'll try again this evening. Checkout the CVS version if anyone wants to take a look. Sorry for the false alarm. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From chad at chadfowler.com Mon Dec 6 12:12:26 2004 From: chad at chadfowler.com (chad@chadfowler.com) Date: Mon Dec 6 11:07:17 2004 Subject: [Rubygems-developers] Release 0.8.2 Uploaded In-Reply-To: <200412060846.05102.jim@weirichhouse.org> References: <200412060803.10388.jim@weirichhouse.org> <90135F55-4788-11D9-B545-000D9337C01C@chadfowler.com> <200412060846.05102.jim@weirichhouse.org> Message-ID: <16289.207.40.151.167.1102353146.squirrel@207.40.151.167> > On Monday 06 December 2004 08:13 am, Chad Fowler wrote: >> On 06-Dec-04, at 8:03 AM, Jim Weirich wrote: >> > I've uploaded the 0.8.2 tgz and zip files to Rubyforge. I plan to >> > double >> > check the files my windows box at work today and (assuming it looks >> > good) >> > make a release announcement today. If anybody wishes to download the >> > files >> > and do any last minute checking before the announcement, that would be >> > appreciated. Thanks. >> > >> > If everything goes well, I will update the rubygems-update file this >> > evening >> > and make that available as a gem. >> >> It looks like the usually-included sources gem isn't in this tarball. >> Does that seem true to you? > > I don't see it either. There is some confusion in the whether the sources > gem > stuff should be in packages or pkgs (I think pkgs is the latest, right?). > Unfortunately I have to get to work this morning and don't have time to > continue with this right now. I've withdrawn 0.8.2 from rubyforge because > this and the other problem. I'll try again this evening. Checkout the > CVS > version if anyone wants to take a look. > Yea, it's supposed to be "pkgs". setup.rb silently barfs if you have a "packages" directory (unless you intended to have a setup.rb compliant one). Chad From jim at weirichhouse.org Tue Dec 7 00:47:45 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Tue Dec 7 00:45:48 2004 Subject: [Rubygems-developers] We try again: RubyGem Release 0.8.3 Message-ID: <200412070047.45829.jim@weirichhouse.org> Well, I'm ready to try again. This is release 0.8.3 and I think I've worked the bugs out. I was able to try this on a windows machine today too, so that is good (and Ruwiki installs fine with this version). The download files are in the files area of RubyForge. I'm going to spend a little time tonight yet to put together a release announcement for tommorrow. Any feedback between now and then would be appreciated. (I'll upload the gem file when I make the announcement). By the way, there is now a release target in the RubyGems rakefile. Just say, "rake release REL=0.8.3", and it will make sure you have checked in all your files, clean out all your transient files, run all the unit tests, update the version in the source file, check the updated source file into CVS, build all the package deliverables and finally ... tag CVS with the release number. All you have to do is upload the files to RubyForge and write a release notice. Thanks to everyone for their feedback. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Tue Dec 7 01:50:36 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Tue Dec 7 01:48:38 2004 Subject: [Rubygems-developers] We try again: RubyGem Release 0.8.3 In-Reply-To: <200412070047.45829.jim@weirichhouse.org> References: <200412070047.45829.jim@weirichhouse.org> Message-ID: <200412070150.36425.jim@weirichhouse.org> On Tuesday 07 December 2004 12:47 am, Jim Weirich wrote: > [...] I'm going to spend > a little time tonight yet to put together a release announcement for > tommorrow. Here's the release announce I'm planning on for tomorrow. Let me know if I have misstated anything. I'll give this until tommorrow evening for feedback before I upload the gem and make the announcement official. --------------------------------------------------------------------------------------- Wow, we are nearing the 8000 mark for downloads of RubyGems. I am please to announce that a new point release of RubyGems is now available on RubyForge. This release is mostly a bug fix and code cleanup release. The good news is that there are several really annoying issues that have been addressed in 0.8.3. Here's the important stuff ... To download and install: (1) DOWNLOAD FROM: http://rubyforge.org/frs/?group_id=126 (2) UNPACK INTO A DIRECTORY AND CD THERE (3) INSTALL WITH: ruby setup.rb all (you may need admin/root privilege) ... or, if you have an existing RubyGem installation .... $ gem install rubygems-update (again, might need to be admin/root) $ update_rubygems (... here too) What's Changed? ====================================================================== No real earth shattering news here, but there were a number of really annoying issues involving other libraries that RubyGems depends upon. 0.8.3 contains some workarounds for these issues. In particular: * Added workaround for the null byte in Dir string issue. (see http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121702). * Added workaround for old version of Zlib on windows that caused Ruwiki to fail to install. (see http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121770) * Added workaround for large YAML file issues. (We dynamically cut down the size of the source index YAML file and seem to have worked around immediate issues. There has been some minor usability enhancements and changes ... * A user specific source index cache can be used when the site-wide cache is unwritable (i.e. because you are running as a non-admin). This *greatly* speeds up gem commands run in non-admin mode when the site-wide cache is out of date. * The gem command now used an HTTP HEAD command to detect if the server's source index needs to be downloaed. * gem check gemname --test will run unit tests on installed gems that have unit tests. * Multiple gem names are allowed on the gem install command line. * Multiple authors my be specified in a Gem spec. * Switch to using setup.rb (rather than a custom install script). And finally there have been a number of internal changes, enhancements and just general cleanup. A lot of work has been done in the remote installer area (some of which you can see in the caching changes noted above) in preparation for unifying the local and remote logic so that there is less difference in installing local gems and remote gems. In particular, we want to be able to satisfy dependencies from locally available gem files. We are not there yet, but we are working on it. Summary ====================================================================== Thanks to everyone for the great feedback we have been getting. Please continue to let us know what you think and what you want. Many minor changes in this release were initiated/requested by folks other than the core gems team. That's great. And most of all, keep making and releasing those gems! -- Jim (for the RubyGems team) -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From batsman.geo at yahoo.com Tue Dec 7 04:37:35 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Tue Dec 7 04:36:43 2004 Subject: [Rubygems-developers] RubyGems API Docs In-Reply-To: References: <20041203143653.GA10686@student.ei.uni-stuttgart.de> Message-ID: <20041207093735.GA5949@student.ei.uni-stuttgart.de> On Fri, Dec 03, 2004 at 10:58:08AM -0500, Richard Kilmer wrote: > The whole API is public in that it is open source. All the classes can be > manipulated by any person that wants (on their own machine, etc), and we There is a distinction between API and mere source code, though. > should publish it all so that someone who wants to fully understand/ > contribute to the API can do so. Anybody willing to fully understand/contribute to RubyGems would be able to run rdoc on the most recent source code, taken directly from CVS. I'm not sure I got the message across: the question was intended in the sense of "are you sure you're satisfied with all of the API and want to commit to keeping backwards compatibility for all the interfaces shown in the RDoc documentation?". The effect of publishing the RDoc documentation is that people will think the whole API is considered stable and might use it in ways you didn't anticipate. Once that happens, it becomes more difficult for you to change/fix any interface... > But you are correct that there is a subset > of the API that library developers that package stuff as gems need to ===================================== Not only packagers, also ppl building things atop RubyGems, such as alternative interfaces... The subset one needs to package is better documented elsewhere. > understand to be functional. We will change the API over time as necessary, > attempt to always be backward compatible where possible, and provide clear > direction on how to migrate in the event that compatibility is not achieved. That's an admirable, albeit too generic declaration of intentions. Trying to translate that into something more specific, are you sure you're satisfied with Gem::RemoteInstaller and Gem::Installer, for instance? And Gem::RemoteSourceFetcher, etc? >On 12/3/04 9:36 AM, "Mauricio Fern?ndez" wrote: > > On Fri, Dec 03, 2004 at 08:57:28AM -0500, Jim Weirich wrote: > >> So I took the liberty of generating them and putting them out at > >> http://rubygems.rubyforge.org/rdoc (and added pointers on the RubyGems main > >> page). > > > > Is the API officially considered stable, i.o.w. are you going to commit > > to it? > > > > I thought that only a very small part of RubyGems was to be > > exposed through a public API: Gem::Cache, Gem::Specification, > > Gem::RemoteInstaller... PS: top-posting :-# -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From batsman.geo at yahoo.com Tue Dec 7 04:40:13 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Tue Dec 7 04:39:12 2004 Subject: [Rubygems-developers] We try again: RubyGem Release 0.8.3 In-Reply-To: <200412070047.45829.jim@weirichhouse.org> References: <200412070047.45829.jim@weirichhouse.org> Message-ID: <20041207094013.GB5949@student.ei.uni-stuttgart.de> On Tue, Dec 07, 2004 at 12:47:45AM -0500, Jim Weirich wrote: > Well, I'm ready to try again. This is release 0.8.3 and I think I've worked > the bugs out. I was able to try this on a windows machine today too, so that > is good (and Ruwiki installs fine with this version). Wouldn't it make more sense to release as 0.8.2? AFAIK there's no reason not to reuse REL-0.8.2. -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From jim at weirichhouse.org Tue Dec 7 07:02:13 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Tue Dec 7 07:00:10 2004 Subject: [Rubygems-developers] We try again: RubyGem Release 0.8.3 In-Reply-To: <20041207094013.GB5949@student.ei.uni-stuttgart.de> References: <200412070047.45829.jim@weirichhouse.org> <20041207094013.GB5949@student.ei.uni-stuttgart.de> Message-ID: <200412070702.13204.jim@weirichhouse.org> On Tuesday 07 December 2004 04:40 am, Mauricio Fern?ndez wrote: > Wouldn't it make more sense to release as 0.8.2? AFAIK there's no reason > not to reuse REL-0.8.2. There were two downloads of 0.8.2 before I removed them from the server. Most likely it was folks on this list so there would be little confussion (although there are 15 people monitoring the RubyGems file area ... not sure what that means). But just to be on the safe side, I incremented the release number. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Tue Dec 7 07:11:09 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Tue Dec 7 07:09:07 2004 Subject: [Rubygems-developers] RubyGems API Docs In-Reply-To: <20041207093735.GA5949@student.ei.uni-stuttgart.de> References: <20041203143653.GA10686@student.ei.uni-stuttgart.de> <20041207093735.GA5949@student.ei.uni-stuttgart.de> Message-ID: <200412070711.09556.jim@weirichhouse.org> On Tuesday 07 December 2004 04:37 am, Mauricio Fern?ndez wrote: > On Fri, Dec 03, 2004 at 10:58:08AM -0500, Richard Kilmer wrote: > > The whole API is public in that it is open source. All the classes can > > be manipulated by any person that wants (on their own machine, etc), and > > we > > There is a distinction between API and mere source code, though. No, you are right, there is a distinct difference. When I used the term, I was differentiating between design documentation and documentation about how to call and use the existing objects in the system. In other words, the object level interfaces, as opposed to the system level API. Perhaps it would be better to just call it the "rdocs" (just as java programmers refer to their "javadocs"), thereby leaving the confussing "A" out of the PI. That's all I meant by it. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From chad at chadfowler.com Tue Dec 7 07:24:03 2004 From: chad at chadfowler.com (Chad Fowler) Date: Tue Dec 7 07:23:05 2004 Subject: [Rubygems-developers] We try again: RubyGem Release 0.8.3 In-Reply-To: <200412070150.36425.jim@weirichhouse.org> References: <200412070047.45829.jim@weirichhouse.org> <200412070150.36425.jim@weirichhouse.org> Message-ID: Some comments/additions below. On 07-Dec-04, at 1:50 AM, Jim Weirich wrote: > On Tuesday 07 December 2004 12:47 am, Jim Weirich wrote: >> [...] I'm going to spend >> a little time tonight yet to put together a release announcement for >> tommorrow. > > Here's the release announce I'm planning on for tomorrow. Let me know > if I > have misstated anything. > > I'll give this until tommorrow evening for feedback before I upload > the gem > and make the announcement official. > > ----------------------------------------------------------------------- > ---------------- > Wow, we are nearing the 8000 mark for downloads of RubyGems. Technically, we passed 8,000 a while back if you count the "gem install rubygems-update". If you count the Windows installer, it's more like 10,000 but let's not go there :) > I am > please to announce that a new point release of RubyGems is now > available on RubyForge. This release is mostly a bug fix and code > cleanup release. The good news is that there are several really > annoying issues that have been addressed in 0.8.3. > > Here's the important stuff ... > > To download and install: > > (1) DOWNLOAD FROM: http://rubyforge.org/frs/?group_id=126 > (2) UNPACK INTO A DIRECTORY AND CD THERE > (3) INSTALL WITH: ruby setup.rb all (you may need admin/root > privilege) > > ... or, if you have an existing RubyGem installation .... > > $ gem install rubygems-update (again, might need to be admin/root) > $ update_rubygems (... here too) > > > What's Changed? > ====================================================================== > > No real earth shattering news here, but there were a number of really > annoying issues involving other libraries that RubyGems depends upon. > 0.8.3 contains some workarounds for these issues. In particular: > > * Added workaround for the null byte in Dir string issue. (see > http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121702). (credit mauricio) > > * Added workaround for old version of Zlib on windows that caused > Ruwiki to fail to install. (see > http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121770) > > * Added workaround for large YAML file issues. (We dynamically cut > down the size of the source index YAML file and seem to have worked > around immediate issues. > > There has been some minor usability enhancements and changes ... > > * A user specific source index cache can be used when the site-wide > cache is unwritable (i.e. because you are running as a non-admin). > This *greatly* speeds up gem commands run in non-admin mode when the > site-wide cache is out of date. > > * The gem command now used an HTTP HEAD command to detect if the > server's source index needs to be downloaed. > > * gem check gemname --test will run unit tests on installed gems that > have unit tests. > > * Multiple gem names are allowed on the gem install command line. > > * Multiple authors my be specified in a Gem spec. > > * Switch to using setup.rb (rather than a custom install script). > I would be sure to say that we switched to setup.rb for installing the rubygems framework itself. Also, Ryan Davis made a patch so you can use an env variable, GEM_SKIP, to tell loadpath_manager not to load gems of those names. This was useful for him while testing libs that he had in development. It's also possible to install multiple gems at once now like: gem install *gem or gem install bluecloth rake some_gems/*gem You've done some great stuff lately, Jim. Thanks for keeping things moving when a lot of us are unable to find the time! Chad > > > -- > -- Jim Weirich jim@weirichhouse.org http://onestepback.org > ----------------------------------------------------------------- > "Beware of bugs in the above code; I have only proved it correct, > not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) > _______________________________________________ > Rubygems-developers mailing list > Rubygems-developers@rubyforge.org > http://rubyforge.org/mailman/listinfo/rubygems-developers From batsman.geo at yahoo.com Tue Dec 7 07:49:29 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Tue Dec 7 07:48:27 2004 Subject: [Rubygems-developers] RubyGems API Docs In-Reply-To: <200412070711.09556.jim@weirichhouse.org> References: <20041203143653.GA10686@student.ei.uni-stuttgart.de> <20041207093735.GA5949@student.ei.uni-stuttgart.de> <200412070711.09556.jim@weirichhouse.org> Message-ID: <20041207124928.GA18913@student.ei.uni-stuttgart.de> On Tue, Dec 07, 2004 at 07:11:09AM -0500, Jim Weirich wrote: > On Tuesday 07 December 2004 04:37 am, Mauricio Fern?ndez wrote: > > On Fri, Dec 03, 2004 at 10:58:08AM -0500, Richard Kilmer wrote: > > > The whole API is public in that it is open source. All the classes can > > > be manipulated by any person that wants (on their own machine, etc), and > > > we > > > > There is a distinction between API and mere source code, though. > > No, you are right, there is a distinct difference. When I used the term, I > was differentiating between design documentation and documentation about how > to call and use the existing objects in the system. In other words, the > object level interfaces, as opposed to the system level API. Perhaps it > would be better to just call it the "rdocs" (just as java programmers refer > to their "javadocs"), thereby leaving the confussing "A" out of the PI. > > That's all I meant by it. I see. Would it make sense to work on the interface and come up with something that can rightfully be called *A*PI? I think this would be good because: * since RubyGems is meant to become the Ruby standard, you'll want to ensure backwards compatibility, as Rich pointed out. This is much easier if you explicitly expose some parts of RubyGems through a public API. * it is in RubyGems' best interest to make people view it not only as a tool but also as a codebase to build upon through well-defined interfaces. This opens the door to more intelligent tools that leverage RubyGems' infrastructure. * the creation of such tools would be easier if you offer a stable target * it is better to do it now before code dependent on RubyGems can be broken by the ongoing refactoring. * the creation of an API can happen in parallel with the current refactoring (re: RemoteInstaller) and makes the code arguably cleaner and easier to document. What do you think? -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From hgs at dmu.ac.uk Tue Dec 7 08:13:16 2004 From: hgs at dmu.ac.uk (Hugh Sasse Staff Elec Eng) Date: Tue Dec 7 08:12:53 2004 Subject: [Rubygems-developers] Remote Installer Progress In-Reply-To: <139805535568.20041205024934@soyabean.com.au> References: <200412030237.32642.jim@weirichhouse.org><200412030855.35161.jim@weirichhouse.org> <51422.192.223.163.6.1102092934.squirrel@weirichhouse.org> <139805535568.20041205024934@soyabean.com.au> Message-ID: On Sun, 5 Dec 2004, Gavin Sinclair wrote: > I doubt anyone would be using Win98 etc. now anyway. Hal Fulton may > have been the last of the old guard... One of my machines at work is win98. No time to upgrade it for the forseeable future.... > > Gavin > Hugh From jim at weirichhouse.org Tue Dec 7 10:18:40 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Tue Dec 7 10:17:38 2004 Subject: [Rubygems-developers] We try again: RubyGem Release 0.8.3 In-Reply-To: References: <200412070047.45829.jim@weirichhouse.org><200412070150.36425.jim@weirichhouse.org> Message-ID: <48227.192.223.163.6.1102432720.squirrel@weirichhouse.org> Chad Fowler said: > Some comments/additions below. >> * Added workaround for the null byte in Dir string issue. (see >> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121702). > > (credit mauricio) Oh, sorry. Of course! (Thanks Mauricio). [...] >> * Multiple gem names are allowed on the gem install command line. [...] >> * Switch to using setup.rb (rather than a custom install script). > I would be sure to say that we switched to setup.rb for installing the > rubygems framework itself. Can you fill me in on the driving reasons we switched. I was a bit out of the loop on that one. > Also, Ryan Davis made a patch so you can use an env variable, GEM_SKIP, > to tell loadpath_manager not to load gems of those names. This was > useful for him while testing libs that he had in development. Do you have details on how this one works? I'm going to try to review the wiki docs and bring them up to date before the announcement. > It's also possible to install multiple gems at once now like: Got that one covered! Thanks for the feedback. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From drbrain at segment7.net Tue Dec 7 12:26:02 2004 From: drbrain at segment7.net (Eric Hodel) Date: Tue Dec 7 12:25:09 2004 Subject: [Rubygems-developers] Gem::Specification#copy_of and Rails makes tests startup slow In-Reply-To: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> References: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> Message-ID: <1191B713-4875-11D9-A5CC-000D93436DA0@segment7.net> Bump On 01 Dec 2004, at 22:13, Eric Hodel wrote: > I have a small testcase in rails that has 7 tests in it (4 of them > raise NotImplementedError), but simply starting up the testcase takes > 3 seconds: > > $ ruby -r profile -Ilib:test test/unit/emailer_test.rb > % cumulative self self total > time seconds seconds calls ms/call ms/call name > 18.37 5.57 5.57 462 12.06 19.42 Integer#gcd > 10.26 8.68 3.11 4008 0.78 2.37 > Gem::Specification#copy_of > > The tests themselves complete in less time than it takes to perform > all the copying: > > $ ruby -Ilib:test test/unit/emailer_test.rb | unit_diff > Loaded suite test/unit/emailer_test > Started > ..E.EEE > Finished in 2.526909 seconds. > [...] > 7 tests, 18 assertions, 0 failures, 4 errors > > When running the entire test suite this startup cost is minimal > compared to the run time of the tests, but it rarely makes sense to > run the entire test suite at once. Is all the copying done in > #copy_of completely necessary? > > BTW: These were run on a PowerBook G4 1.5GHz From gsinclair at soyabean.com.au Tue Dec 7 17:47:01 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Tue Dec 7 17:48:26 2004 Subject: [Rubygems-developers] RubyGems API Docs In-Reply-To: <20041207124928.GA18913@student.ei.uni-stuttgart.de> References: <20041203143653.GA10686@student.ei.uni-stuttgart.de> <20041207093735.GA5949@student.ei.uni-stuttgart.de> <200412070711.09556.jim@weirichhouse.org> <20041207124928.GA18913@student.ei.uni-stuttgart.de> Message-ID: <1111089782614.20041208094701@soyabean.com.au> On Tuesday, December 7, 2004, 11:49:29 PM, Mauricio wrote: > I see. Would it make sense to work on the interface and come up with > something that can rightfully be called *A*PI? > I think this would be good because: > * since RubyGems is meant to become the Ruby standard, you'll want to > ensure backwards compatibility, as Rich pointed out. This is much > easier if you explicitly expose some parts of RubyGems through a > public API. > * it is in RubyGems' best interest to make people view it not only as a > tool but also as a codebase to build upon through well-defined > interfaces. This opens the door to more intelligent tools that > leverage RubyGems' infrastructure. > * the creation of such tools would be easier if you offer a stable > target > * it is better to do it now before code dependent on RubyGems can be > broken by the ongoing refactoring. > * the creation of an API can happen in parallel with the current > refactoring (re: RemoteInstaller) and makes the code arguably cleaner > and easier to document. > What do you think? Agreed on all points. Gavin From jim at weirichhouse.org Tue Dec 7 22:28:31 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Tue Dec 7 22:26:30 2004 Subject: [Rubygems-developers] [ANN] RubyGems 0.8.3 Released! Message-ID: <200412072228.31341.jim@weirichhouse.org> = Announce: RubyGems Release 0.8.3 Wow, over 8000 download of either the RubyGems package or the RubyGems-update gem! And over 150 different packages available as a gem. Things have really moved foreward. I am pleased to announce that a new point release of RubyGems is now available on RubyForge. This release is mostly a bug fix and code cleanup release. But the good news is that there are several really annoying issues that have been addressed in 0.8.3. But first ... == What is RubyGems? RubyGems is a package management system for Ruby applications and libraries. RubyGems one command download makes installing Ruby software fun and enjoyable again. (Ok, not really.) Many gems are available for download from the RubyForge site. Browse the list of gems with a "gem list --remote" command and download what you need with a simple "gem install ". RubyGems takes care of the details of installing, not only the gem you requested, but also any gems needed by the software you selected. So now you are asking ... == How can I get all this great stuff? Well, here's how ... To download and install: 1. DOWNLOAD FROM: http://rubyforge.org/frs/?group_id=126 2. UNPACK INTO A DIRECTORY AND CD THERE 3. INSTALL WITH: ruby setup.rb all (you may need admin/root privilege) ... or, if you have an existing RubyGem installation .... $ gem install rubygems-update (again, might need to be admin/root) $ update_rubygems (... here too) I bet you are wondering... == So What's Changed in this Release? No real earth shattering news here, but there were a number of really annoying issues involving other libraries that RubyGems depends upon. 0.8.3 contains some workarounds for these issues. In particular: * Added workaround for the null byte in Dir string issue. (see http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121702). (Thanks to Mauricio Fern?ndez for the quick response on this one). * Added workaround for old version of Zlib on windows that caused Ruwiki to fail to install. (see http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/121770) * Added workaround for large YAML file issues. (We dynamically cut down the size of the source index YAML file and seem to have worked around immediate issues. There has been some minor usability enhancements and changes ... * A user specific source index cache can be used when the site-wide cache is unwritable (i.e. because you are running as a non-admin). This *greatly* speeds up gem commands run in non-admin mode when the site-wide cache is out of date. * The gem command now used an HTTP HEAD command to detect if the server's source index needs to be downloaed. * gem check gemname --test will run unit tests on installed gems that have unit tests. * Multiple gem names are allowed on the gem install command line. This means you can do: gem install rake rails needle postgres-pr pimki (Ok, you get the idea) * Multiple authors my be specified in a Gem spec. * Switched to using setup.rb (rather than a custom install script) for the installation of RubyGems itself. If you have installed RubyGems before, double check the installation instructions and make sure you use setup.rb instead of install.rb. * Ryan Davis has provided a patch so you can use an env variable (GEM_SKIP), to tell loadpath_manager not to load gems of those names. This was useful for him while testing libs that he had in development. And finally there have been a number of internal changes, enhancements and just general cleanup. A lot of work has been done in the remote installer area (some of which you can see in the caching changes noted above) in preparation for unifying the local and remote logic so that there is less difference in installing local gems and remote gems. In particular, we want to be able to satisfy dependencies from locally available gem files. We are not there yet, but we are working on it. == Some Random Information about RubyGems Home Page:: http://rubygems.rubyforge.org Downloads:: http://rubyforge.org/project/showfiles.php?group_id=126 User Guide:: http://rubygems.rubyforge.org/wiki/wiki.pl?UserGuide == Summary Thanks to everyone for the great feedback we have been getting. Please continue to let us know what you think and what you want. Many minor changes in this release were initiated/requested by folks other than the core gems team. That's great. And most of all, keep making and releasing those gems! -- Jim (for the RubyGems team) -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From batsman.geo at yahoo.com Wed Dec 8 09:17:20 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Wed Dec 8 09:16:22 2004 Subject: [Rubygems-developers] Non-upstream packages Message-ID: <20041208141720.GA19183@student.ei.uni-stuttgart.de> I was surprised to see a libbz2-0.4.gem in your repository; surprised because the last upstream version is 0.2.1: that's the one listed in RAA and what one can download from the primary source. On closer inspection, I found a few interesting things: * in addition to representing a non-existent (?) upstream version, the .gem is in a deprecated format * it doesn't compile the extension automatically * the .gem includes a copy of another .gem inside * the description is description: pretty cool stuff, the email field is email: "?" ... In a word, the package doesn't look very good. The author field has the value "Guy Decoux", but I'm inclined to think that he didn't create that mess... So the general problem is: how can I know who is responsible for a given .gem? Where are bug reports to be sent? Where does it come from to begin with: from Rubyforge, direct email submission...?? I know several libraries have been packaged by third parties on behalf of the original authors, but the .gem always point to the original developers, which doesn't feel right, especially when a .gem is as unsatisfactory as the one at hand. -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From jim at weirichhouse.org Wed Dec 8 10:13:12 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Wed Dec 8 10:12:09 2004 Subject: [Rubygems-developers] Non-upstream packages In-Reply-To: <20041208141720.GA19183@student.ei.uni-stuttgart.de> References: <20041208141720.GA19183@student.ei.uni-stuttgart.de> Message-ID: <47832.192.223.163.6.1102518792.squirrel@weirichhouse.org> Mauricio Fern?ndez said: > > I was surprised to see a libbz2-0.4.gem in your repository; surprised > because the last upstream version is 0.2.1: that's the one listed in > RAA and what one can download from the primary source. The gem showed up on the same day that Asami, a GNOME desktop app, was released. Sure enough, Asami has a dependency on libbz2 (>= 0.4). Checking the Asami download page (http://rubyforge.org/frs/?group_id=359) show that the libbz2 gem came from there. I would suggest sending a query to Darren Willis at his gmail account. Ya'know, it would be nice if there was a direct way to go from browsing the gems back to the download page where a particular gem originated from. > On closer inspection, I found a few interesting things: > * in addition to representing a non-existent (?) upstream version, > the .gem is in a deprecated format > * it doesn't compile the extension automatically > * the .gem includes a copy of another .gem inside His gem also includes a number of emacs backup files. I imagine his gem generation consists of grabbing everything in the current directory. The second time you run it, the gem includes the original generated version of his gem. The Nth time you run it, it will include a gem with N-1 nested copies. It might be fun to download the gem and figure out how many times he built it ... on second thought, that wouldn't be fun. > So the general problem is: how can I know who is responsible for a > given .gem? Where are bug reports to be sent? Where does it come from > to begin with: from Rubyforge, direct email submission...?? Gems now support multiple author fields. If you are repackaging someone else's software as a gem, you should include both the original author and your own contact information. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From batsman.geo at yahoo.com Wed Dec 8 10:39:38 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Wed Dec 8 10:38:35 2004 Subject: [Rubygems-developers] Non-upstream packages In-Reply-To: <47832.192.223.163.6.1102518792.squirrel@weirichhouse.org> References: <20041208141720.GA19183@student.ei.uni-stuttgart.de> <47832.192.223.163.6.1102518792.squirrel@weirichhouse.org> Message-ID: <20041208153938.GA24754@student.ei.uni-stuttgart.de> On Wed, Dec 08, 2004 at 03:13:12PM -0000, Jim Weirich wrote: > > So the general problem is: how can I know who is responsible for a > > given .gem? Where are bug reports to be sent? Where does it come from > > to begin with: from Rubyforge, direct email submission...?? > > Gems now support multiple author fields. If you are repackaging someone > else's software as a gem, you should include both the original author and > your own contact information. Sounds reasonable; I think this should be mentioned in RubyGems' documentation. It might even make sense to issue a warning if the rubyforge_project field is empty and author: doesn't contain at least 2 authors. But what about the deeper problem of "ownership". Is it OK to release packages for software you didn't write? Does that make you responsible for the package? The problem is that the RubyGems repository works as a whole, and the mistakes from one devel. can end up affecting many others. For instance, what would happen if we hadn't noticed this particular case and many .gems depending on libbz2 >= 0.4 had been released? Correcting that situation would require many coordinated modifications. -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From gsinclair at soyabean.com.au Wed Dec 8 10:53:17 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Wed Dec 8 10:52:22 2004 Subject: [Rubygems-developers] Non-upstream packages In-Reply-To: <20041208153938.GA24754@student.ei.uni-stuttgart.de> References: <20041208141720.GA19183@student.ei.uni-stuttgart.de> <47832.192.223.163.6.1102518792.squirrel@weirichhouse.org> <20041208153938.GA24754@student.ei.uni-stuttgart.de> Message-ID: <1111151211754.20041209025317@soyabean.com.au> On Thursday, December 9, 2004, 2:39:38 AM, Mauricio wrote: > On Wed, Dec 08, 2004 at 03:13:12PM -0000, Jim Weirich wrote: >> > So the general problem is: how can I know who is responsible for a >> > given .gem? Where are bug reports to be sent? Where does it come from >> > to begin with: from Rubyforge, direct email submission...?? >> >> Gems now support multiple author fields. If you are repackaging someone >> else's software as a gem, you should include both the original author and >> your own contact information. > Sounds reasonable; I think this should be mentioned in RubyGems' > documentation. It might even make sense to issue a warning if the > rubyforge_project field is empty and author: doesn't contain at least 2 > authors. ISTM that using the "author" field for that packager's details is a bad idea. At the least, it's very confusing. "packager" !~ "author" in any dictionary I know. > But what about the deeper problem of "ownership". Is it OK to release > packages for software you didn't write? Yes, license notwithstanding. Although it's "nice" to hand over the packaging responsibility back to the author, or at least offer. > Does that make you responsible for the package? No more than (re-)releasing a tarball of someone else's software. It makes you responsible for the _packaging_, I guess, insofar as anyone is "responsible" for anything in an open-source world. > The problem is that the RubyGems repository works as a whole, and the > mistakes from one devel. can end up affecting many others. For instance, > what would happen if we hadn't noticed this particular case and many > .gems depending on libbz2 >= 0.4 had been released? Correcting that > situation would require many coordinated modifications. That's a contrived problem. Any software depending on 'libbz2 >= 0.4' would obviously be tested against that gem, so the problems with that gem would be uncovered pretty quickly. In other words, either that gem would be fixed, or no software of any value would actually be written to depend on it. If RubyGems is just an open packaging format, plus a public server, then we should expect the average quality of gems to be somewhere near the average quality of tarballs on rubyforge, or sourceforge. Ain't much we can do about it. Gavin From gsinclair at soyabean.com.au Thu Dec 9 20:52:15 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Thu Dec 9 20:51:46 2004 Subject: [Rubygems-developers] Gem::Specification#copy_of and Rails makes tests startup slow In-Reply-To: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> References: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> Message-ID: <651273549026.20041210125215@soyabean.com.au> On Thursday, December 2, 2004, 5:13:37 PM, Eric wrote: > I have a small testcase in rails that has 7 tests in it (4 of them > raise NotImplementedError), but simply starting up the testcase takes 3 > seconds: > $ ruby -r profile -Ilib:test test/unit/emailer_test.rb > % cumulative self self total > time seconds seconds calls ms/call ms/call name > 18.37 5.57 5.57 462 12.06 19.42 Integer#gcd > 10.26 8.68 3.11 4008 0.78 2.37 Gem::Specification#copy_of I'm looking into this now, Eric. 4008 is a lot of calls. See bottom of email for a patch you can try to see if it gets reduced. > [...] Is all the copying done in #copy_of completely necessary? Well, that is the question. find, xargs, and grep tell me that copy_of appears in the following places only: specification.rb:124: @#{name} ||= copy_of(@@default_value[:#{name}]) specification.rb:350: self.send "#{name}=", copy_of(default) specification.rb:567: def self.copy_of(obj) specification.rb:575: def copy_of(obj) specification.rb:576: self.class.copy_of(obj) The first two are invocations; the rest are defintions, or part thereof. The first one appears in this context: def self.attribute(name, default=nil) @@attributes << [name, default] @@default_value[name] = default attr_writer(name) class_eval %{ def #{name} @#{name} ||= copy_of(@@default_value[:#{name}]) end } end This is to allow the specification of default attribute values. class Specification attribute :bindir, 'bin' attribute :files, [] ... end The comment above the method is: ## # Used to specify the name and default value of a specification attribute. The side # effects are: # * the name and default value are added to the @@attributes list and # @@default_value map # * a standard _writer_ method (attribute=) is created # * a non-standard _reader method (attribute) is created # # The reader method behaves like this: # def attribute # @attribute ||= (copy of default value) # end # # This allows lazy initialization of attributes to their default values. # So attribute reader methods are generated. The code corresponding to our two attributes 'bindir' and 'files' above is: def bindir @bindir ||= copy_of(@@default_value[:bindir]) # @@default_value[:bindir] == 'bin' end def files @files ||= copy_of(@@default_value[:files]) # @@default_value[:files] == [] end Do you see the need for the copy_of? The default value for the attributes is held in a class variable. The reader lazily sets the instance variable to the default value. We must take a copy of the default value, otherwise every Specification object will be pointing to the same array of files, for instance. Now, let's challenge some of the thinking behind all this. Q. Why all this metaprogramming? A. To allow Gem::Specification to be defined in a non-redundant manner, avoiding inconsistencies which are ugly and often lead to bugs. Also, it exposes the names and defaults of attributes programmatically, which can help with documentation. Q. But doesn't it slow things down? A. The impact of the metaprogramming should only be felt when the class is defined. This performance concern with #copy_of demonstrates that it impacts attribute readers as well, which we should try to fix. Q. Why perform lazy initialisation of attributes? Why not just initialize all instance variables up front? A. Good question. It's to avoid creating lots of instance variables that aren't necessarily used. The motive for cutting down on instance variables is to reduce the amount of YAML output. YAML reduction can be achieved in other -- probably better -- ways, so we should see if we can get a performance boost from up-front initialisation. It so happens that up-front initialisation _does_ occur with this bit of #initialize: @@attributes.each do |name, default| self.send "#{name}=", copy_of(default) end I'll investigate whether this apparent duplication is an oversight. It seems that way. If so, the "smart" attribute reader methods can become dumb ones, since we're creating all the instance variables anyway. That should reduce the number of calls to copy_of. Eric, if you're still reading, you can help investigate this issue by applying a simple patch (at bottom of email) against the latest specification.rb (which shipped with RubyGems 0.8.3). Run your tests again, and see how many calls are made to #copy_of, and see if anything breaks. This is an investigative patch only, so you should be careful to restore the original afterwards. Cheers, Gavin Index: specification.rb =================================================================== RCS file: /var/cvs/rubygems/rubygems/lib/rubygems/specification.rb,v retrieving revision 1.65 diff -u -r1.65 specification.rb --- specification.rb 7 Dec 2004 04:01:05 -0000 1.65 +++ specification.rb 10 Dec 2004 01:42:03 -0000 @@ -124,12 +124,7 @@ def self.attribute(name, default=nil) @@attributes << [name, default] @@default_value[name] = default - attr_writer(name) - class_eval %{ - def #{name} - @#{name} ||= copy_of(@@default_value[:#{name}]) - end - } + attr_accessor(name) end # Same as attribute above, but also records this attribute as mandatory. From drbrain at segment7.net Fri Dec 10 02:07:18 2004 From: drbrain at segment7.net (Eric Hodel) Date: Fri Dec 10 02:06:27 2004 Subject: [Rubygems-developers] Gem::Specification#copy_of and Rails makes tests startup slow In-Reply-To: <651273549026.20041210125215@soyabean.com.au> References: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> <651273549026.20041210125215@soyabean.com.au> Message-ID: <212E8B5D-4A7A-11D9-9E28-000D93436DA0@segment7.net> On 09 Dec 2004, at 17:52, Gavin Sinclair wrote: > On Thursday, December 2, 2004, 5:13:37 PM, Eric wrote: > >> I have a small testcase in rails that has 7 tests in it (4 of them >> raise NotImplementedError), but simply starting up the testcase takes >> 3 >> seconds: > >> $ ruby -r profile -Ilib:test test/unit/emailer_test.rb >> % cumulative self self total >> time seconds seconds calls ms/call ms/call name >> 18.37 5.57 5.57 462 12.06 19.42 Integer#gcd >> 10.26 8.68 3.11 4008 0.78 2.37 >> Gem::Specification#copy_of > > I'm looking into this now, Eric. 4008 is a lot of calls. See bottom > of email for a patch you can try to see if it gets reduced. > >> [...] Is all the copying done in #copy_of completely necessary? [snip lots of good info] > Do you see the need for the copy_of? The default value for the > attributes is held in a class variable. The reader lazily sets the > instance variable to the default value. We must take a copy of the > default value, otherwise every Specification object will be pointing > to the same array of files, for instance. I was quite sure there was a valid need for the copying, but I was unsure if anybody realized the extent of its consequences. If I come across #dup in someone's code, I'm pretty sure that removing it is going to break things catastrophically, subtly, or both. > Now, let's challenge some of the thinking behind all this. [snip challenged thinking] > Eric, if you're still reading, you can help investigate this issue by > applying a simple patch (at bottom of email) against the latest > specification.rb (which shipped with RubyGems 0.8.3). Run your tests > again, and see how many calls are made to #copy_of, and see if > anything breaks. > > This is an investigative patch only, so you should be careful to > restore the original afterwards. Cool, I'll apply this tomorrow and have a report its effectiveness. From drbrain at segment7.net Fri Dec 10 16:38:36 2004 From: drbrain at segment7.net (Eric Hodel) Date: Fri Dec 10 16:37:30 2004 Subject: [Rubygems-developers] Gem::Specification#copy_of and Rails makes tests startup slow In-Reply-To: <651273549026.20041210125215@soyabean.com.au> References: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> <651273549026.20041210125215@soyabean.com.au> Message-ID: On 09 Dec 2004, at 17:52, Gavin Sinclair wrote: > Eric, if you're still reading, you can help investigate this issue by > applying a simple patch (at bottom of email) against the latest > specification.rb (which shipped with RubyGems 0.8.3). Run your tests > again, and see how many calls are made to #copy_of, and see if > anything breaks. > > This is an investigative patch only, so you should be careful to > restore the original afterwards. Much better on the copy_of calls: % cumulative self self total time seconds seconds calls ms/call ms/call name [...] 0.31 67.62 0.29 416 0.70 2.72 Gem::Specification#copy_of $ gem -v 0.8.3 Test suite startup is no faster, but now I can no longer be sure rubygems is the problem. (Something in Rails gives widely varying test run times, sometimes 6s, sometimes as much as 9s.) From gsinclair at soyabean.com.au Fri Dec 10 20:30:41 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Fri Dec 10 20:30:18 2004 Subject: [Rubygems-developers] Gem::Specification#copy_of and Rails makes tests startup slow In-Reply-To: <212E8B5D-4A7A-11D9-9E28-000D93436DA0@segment7.net> References: <4D959574-4429-11D9-9433-000D93436DA0@segment7.net> <651273549026.20041210125215@soyabean.com.au> <212E8B5D-4A7A-11D9-9E28-000D93436DA0@segment7.net> Message-ID: <1931358655122.20041211123041@soyabean.com.au> On Friday, December 10, 2004, 6:07:18 PM, Eric wrote: >> This is an investigative patch only, so you should be careful to >> restore the original afterwards. > Cool, I'll apply this tomorrow and have a report its effectiveness. I don't think it will make much difference. Running a small test myself, I'm getting 520 calls to #copy_of. Note there's a useless delegation from Spec#copy_of to Spec.copy_of which I've removed, so that used to be 1440 calls (the profiler doesn't distinguish the class and instance methods). That number 520 == (20 locally installed gems) x (26 attributes per gem) Your number of #copy_of calls (4008) is inflated by: - the douple dipping from delegation - lazy initialization of attributes) * (yes, attrs get initialized up front, but some are nil, so they'll get initialized again and again) Bottom line: if you have a lot of gems installed on your system, you'll get a lot of calls to #copy_of. I'm now looking at what can be done to avoid this. I think #copy_of is not the only problem, though. Each "require" is expensive on my system, it seems. I'm running a program with some timing logic: require 'rubygems': 0.317000150680542s Gem.manage_gems: 4.154999971389771s require 'yaml': 0.070000171661377s require 'dev-utils/debug': 3.365999937057495s require 'log4r': 4.733000040054321s Anyway, I'm looking into it. Gavin From gsinclair at soyabean.com.au Fri Dec 10 21:34:03 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Fri Dec 10 21:33:41 2004 Subject: [Rubygems-developers] RubyGems performance Message-ID: <1021362457740.20041211133403@soyabean.com.au> As I mentioned in the other thread ("Gem::Specification#copy_of and Rails makes tests startup slow"), I'm doing some performance investigations of RubyGems. Just FYI I'm posting results here as I go. Source code (rgperf.rb): $:.unshift 'lib' require 'rubygems/logger' time("require 'rubygems'") { require 'rubygems' } time("Gem.manage_gems") { Gem.manage_gems } time("require_gem 'log4r'") { require_gem 'log4r' } Source code(rubygems/logger.rb): $log = Object.new def $log.debug(str) STDERR.puts str end def time(msg, width=25) t = Time.now yield elapsed = Time.now.to_f - t.to_f elapsed = sprintf("%3.3f", elapsed) $log.debug "#{msg.ljust(width)}: #{elapsed}s" end Source code (extract from rubygems.rb): def manage_gems $log.debug "Timing manage_gems" time("manage_gems") { time(" require 'rubygems/user_interaction'", 40) { require 'rubygems/user_interaction' } time(" require 'rubygems/builder'", 40) { require 'rubygems/builder' } time(" require 'rubygems/format'", 40) { require 'rubygems/format' } time(" require 'rubygems/remote_installer'", 40) { require 'rubygems/remote_installer' } time(" require 'rubygems/installer'", 40) { require 'rubygems/installer' } time(" require 'rubygems/validator'", 40) { require 'rubygems/validator' } time(" require 'rubygems/doc_manager'", 40) { require 'rubygems/doc_manager' } time(" require 'rubygems/cmd_manager'", 40) { require 'rubygems/cmd_manager' } time(" require 'rubygems/gem_runner'", 40) { require 'rubygems/gem_runner' } time(" require 'rubygems/config_file'", 40) { require 'rubygems/config_file' } } end Results (ruby rgperf.rb): require 'rubygems' : 0.323s Timing manage_gems require 'rubygems/user_interaction' : 0.060s require 'rubygems/builder' : 2.304s require 'rubygems/format' : 0.152s require 'rubygems/remote_installer' : 0.308s require 'rubygems/installer' : 0.339s require 'rubygems/validator' : 0.140s require 'rubygems/doc_manager' : 0.085s require 'rubygems/cmd_manager' : 0.509s require 'rubygems/gem_runner' : 0.082s require 'rubygems/config_file' : 0.158s manage_gems : 4.146s Gem.manage_gems : 4.146s require_gem 'log4r' : 3.434s Comments: * manage_gems and require_gem both take >3s to complete, whereas a regular require takes 0.3s * obviously not all requires and require_gems are alike, but 3.4s for require_gem 'log4r' is too long * manage_gem isn't such an issue because it's not used in runtime situations, EXCEPT when using the gem command (so it would be nice to improve somehow) * a very disproportionate amount of time in manage_gem is spent loading 'rubygems/builder' * the culprit is obviously 'rubygems/package', which I doubt has any glaring inefficiencies -- it's just the volume of Ruby code * perhaps we should only load it when we need it? * my investigation will now focus on require_gem Gavin P.S. Loading 'rubygems/specification' FIRST (in a separate test) took 0.386s. By comparison, stdlib's 'logger' took 0.034s and 'yaml' took 0.600s. There's probably room for improvement, but it's not the pressing issue. From gsinclair at soyabean.com.au Fri Dec 10 23:50:26 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Fri Dec 10 23:50:06 2004 Subject: [Rubygems-developers] RubyGems performance In-Reply-To: <1021362457740.20041211133403@soyabean.com.au> References: <1021362457740.20041211133403@soyabean.com.au> Message-ID: <331370640356.20041211155026@soyabean.com.au> On Saturday, December 11, 2004, 1:34:03 PM, Gavin wrote: > As I mentioned in the other thread ("Gem::Specification#copy_of and > Rails makes tests startup slow"), I'm doing some performance > investigations of RubyGems. Just FYI I'm posting results here as I > go. More results: require 'rubygems' : 0.321s require_gem 'log4r' : 3.601s require_gem 'RedCloth' : 0.119s require_gem 'rake' : 0.013s require_gem 'extensions' : 0.009s require_gem 'cstemplate' : 0.009s require_gem 'dev-utils' : 0.014s The first require_gem is slow; subsequent ones are fast. That's good. Using require instead of require_gem (with none of these libraries installed directly): require 'rubygems' : 0.305s require 'log4r' : 3.772s require 'redcloth' : 0.577s require 'rake' : 0.908s require 'extensions/all' : 5.594s require 'celsoft.com/template': 0.362s require 'dev-utils/debug': 5.697s Log4r is probably taking a hit as an index is being built up. RedCloth is comparable to require_gem. Rake and cstemplate both take a significant hit, explained by the lack of autorequire in their gemspecs. 'extensions/all' and 'dev-utils/debug' -- my libraries -- are embarassingly slow. Again, it's just that require_gem is fast because they don't use autorequire. Just to check, I rearranged the order to see if the first use of require (after rubygems) takes a hit: require 'rubygems' : 0.347s require 'redcloth' : 0.633s require 'log4r' : 3.961s require 'rake' : 0.924s It's inconclusive. RedCloth is consistently taking longer than before (0.6 instead of 0.2), but Log4r is much the same. One more time, with Rake leading the way: require 'rubygems' : 0.304s require 'rake' : 1.135s require 'redcloth' : 0.597s require 'log4r' : 4.547s Yes, there's definitely an effect. RedCloth is back to normal, and Rake is taking a bit longer. Anyway, the REAL test is to pit directly-installed libraries against gem-installed libraries. So I'll install those six libraries manually and time their loading. The results are interesting: require 'rake' : 0.203s require 'redcloth' : 0.477s require 'log4r' : 0.430s require 'extensions/all' : 0.536s require 'dev-utils/debug': 0.640s require 'celsoft.com/template': 0.075s I'll summarise the results in a table. * "DIRECT" means calling require 'X' where 'X' is installed directly (via RPA, as it happens). * "GEM" means calling require 'X' where X is installed as a gem. * "COMBO" means calling require 'X' _after_ having loaded 'rubygems' but where the library is also installed directly. LIBRARY | DIRECT | GEM | COMBO ------------------------+----------+----------+--------- 'rake' | 0.203s | 3.772s | 0.481s 'redcloth' | 0.477s | 0.577s | 1.355s 'log4r' | 0.430s | 0.908s | 2.545s 'extensions/all' | 0.536s | 5.594s | 2.741s 'dev-utils/debug' | 0.640s | 5.697s | 2.056s 'celsoft.com/template' | 0.075s | 0.362s | 0.096s *** CONCLUSION *** There's a startling difference between loading certain libraries directly vs loading them as a gem. And the difference is inconsistent between libraries. In all cases, it's slower to load gems than load direct libraries. That's not surprising. It's worth trying to speed the gem loading up, because it becomes noticable in the startup time of many applications, especially small applications that get run a lot (e.g. unit tests, rake). My next study will look into those libraries to find the reason for the discrepancies and the inconsistencies. From jim at weirichhouse.org Sat Dec 11 01:01:55 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Sat Dec 11 00:59:23 2004 Subject: [Rubygems-developers] 0.8.3 Documentation Message-ID: <200412110101.55488.jim@weirichhouse.org> I've been going through the RubyGems wiki site to update some of the documentation. Here is a couple of notes in that regard. Wiki Issues --------------- We have been hit *bad* with wiki spam. In some causes actual page content was lost because we did not uncover the spam in time. In particular, the Environment variable page was trashed and the original contest was no longer in the history. I've rewritten that page from scratch (and if someone would like to review, I would appreciate it). We finally have the wiki spam at RubyGarden under control, but the RubyForge wikis are particularlly troublesome. We have no RSS feed (to make spammed pages easier to detect). The version of UseMod is not the latest. I'm not sure who (if anyone) as admin writes to the wiki (to maintain a banned list and delete non-relevent pages). I'm really concerned about possible documentation loss. What we have suffered up to date is not serious, but it could easily become serious. I have barely enough time to write the docs once, let alone multiple times. I would welcome discussion on possible alternatives (e.g. a WIki backup plan, moving basic docs off the wiki, or something else). Gem Specification Reference -------------------------------------- The GemSpecification Reference page was one that was lost. Fortunately, it is one of two pages that are generated outside the wiki. I regenerated the page and updated it, but there are some spec fields that need some writeup. I'm asking that the folks who added the spec fields take a few moments and update the docs. You can find them in the scripts area of the CVS working directory. (specdoc.rb/yaml/data). Thanks. The gem command line reference is a bit out of date as well, I can get this one (unless someone beats me to it). Thanks to everyone for their time. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Sat Dec 11 01:10:04 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Sat Dec 11 01:07:26 2004 Subject: [Rubygems-developers] Rakefile change regarding RDoc template Message-ID: <200412110110.04019.jim@weirichhouse.org> > Update of /var/cvs/rubygems/rubygems > In directory rubyforge.org:/tmp/cvs-serv25647 > Modified Files: > ????????Rakefile > Log Message: > Removed seemingly invalid RDoc template The jamis template comes from Jamis Buck. You can find it in the doc/rdoc_template directory of the CVS package. You need to install it before you can use it. It's one of the nicer RDoc templates that I have seen and have used it to generate the docs at http://rubygems.rubyforge.org/rdoc. I'll fix it so that it will default to something else if you don't have it installed. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Sat Dec 11 01:18:31 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Sat Dec 11 01:15:54 2004 Subject: [Rubygems-developers] GEM_SKIP Questions Message-ID: <200412110118.31601.jim@weirichhouse.org> Sorry to put out so many message on the list at once. But not so sorry that I will stop. I know that Ryan Davis has submitted the GEM_SKIP patch that prevents certain gems from being loaded. However, I haven't seen the need for this. It seems that RubyGems is very polite about not activating gems that already exist in the load path. I have have had great success in testing rake from a non-gem development area (and I always install rake as a gem). So, I guess I need some enlightenment on exactly what the conditions are that give rise to the need for GEM_SKIP. I'm probably just missing something fundamental. Thanks! Is Ryan on the list? I think I will include him as a CC so he can respond if he wished. Thanks. I'm probably just being slow tonight. (And I think this is the last of the messages I needed to send to the RubyGems list tonight. So you can all rest until I find more issues.) -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From drbrain at segment7.net Sat Dec 11 01:34:18 2004 From: drbrain at segment7.net (Eric Hodel) Date: Sat Dec 11 01:33:18 2004 Subject: [Rubygems-developers] RubyGems performance In-Reply-To: <331370640356.20041211155026@soyabean.com.au> References: <1021362457740.20041211133403@soyabean.com.au> <331370640356.20041211155026@soyabean.com.au> Message-ID: On 10 Dec 2004, at 20:50, Gavin Sinclair wrote: > I'll summarise the results in a table. > > * "DIRECT" means calling require 'X' where 'X' is installed > directly (via RPA, as it happens). > * "GEM" means calling require 'X' where X is installed as a gem. > * "COMBO" means calling require 'X' _after_ having loaded 'rubygems' > but where the library is also installed directly. > > LIBRARY | DIRECT | GEM | COMBO | GEMSPEC > SIZE > > ------------------------+----------+----------+--------- > +-------------- > 'rake' | 0.203s | 3.772s | 0.481s | 1763 bytes > 'redcloth' | 0.477s | 0.577s | 1.355s | 1762 bytes > 'log4r' | 0.430s | 0.908s | 2.545s | 10117 bytes > 'extensions/all' | 0.536s | 5.594s | 2.741s | > 'dev-utils/debug' | 0.640s | 5.697s | 2.056s | > 'celsoft.com/template' | 0.075s | 0.362s | 0.096s | For the record, rails is: 2018 bytes | actionmailer 5406 bytes | actionpack 5501 bytes | activerecord 2109 bytes | rails ------------+-------------- 15034 bytes | total Also, log4r includes all of its rdoc in the gem, it doesn't get auto-generated. From curt at hibbs.com Sat Dec 11 01:49:15 2004 From: curt at hibbs.com (Curt Hibbs) Date: Sat Dec 11 01:48:04 2004 Subject: [Rubygems-developers] 0.8.3 Documentation In-Reply-To: <200412110101.55488.jim@weirichhouse.org> Message-ID: Jim Weirich wrote: > > We have been hit *bad* with wiki spam. In some causes actual > page content was > lost because we did not uncover the spam in time. In particular, the > Environment variable page was trashed and the original contest > was no longer > in the history. I had the same thing happen to me. After that I spent 1+ hours a day cleaning up the spam from multiple wiki sites. But since I had Tom Copeland install the patch that disallows external links unless the http part is upper case, I have not had a single spam attack (aren't you the one who created this patch??). You should ask Tom to install this patch on the RubyGems wiki so that this does not happen anymore (later we can all move to Ruwiki once its ready to go with RubyForge authentication). Curt From gsinclair at soyabean.com.au Sat Dec 11 02:16:28 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Sat Dec 11 02:16:08 2004 Subject: [Rubygems-developers] Oddity in Gem.activate? Message-ID: <1661379401865.20041211181628@soyabean.com.au> My informal test shows something funny: $ irb --simple-prompt >> require 'rubygems' => true >> Gem.activate('rake', true) => true >> puts $LOAD_PATH /usr/local/lib/ruby/gems/1.8/gems/rake-0.4.10/bin /usr/local/lib/ruby/gems/1.8/gems/rake-0.4.10/lib /usr/local/lib/ruby/site_ruby/1.8 /usr/local/lib/ruby/site_ruby/1.8/i386-cygwin /usr/local/lib/ruby/site_ruby /usr/local/lib/ruby/1.8 /usr/local/lib/ruby/1.8/i386-cygwin Why is the _bin_ path of the Rake gem being added to the $LOAD_PATH? Shouldn't it just be the _lib_ directory? Gavin From gsinclair at soyabean.com.au Sat Dec 11 06:48:05 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Sat Dec 11 06:47:49 2004 Subject: [Rubygems-developers] Rakefile change regarding RDoc template In-Reply-To: <200412110110.04019.jim@weirichhouse.org> References: <200412110110.04019.jim@weirichhouse.org> Message-ID: <1301395699149.20041211224805@soyabean.com.au> On Saturday, December 11, 2004, 5:10:04 PM, Jim wrote: >> Update of /var/cvs/rubygems/rubygems >> In directory rubyforge.org:/tmp/cvs-serv25647 >> Modified Files: >> ????????Rakefile >> Log Message: >> Removed seemingly invalid RDoc template > The jamis template comes from Jamis Buck. You can find it in the > doc/rdoc_template directory of the CVS package. You need to install it > before you can use it. It's one of the nicer RDoc templates that I have seen > and have used it to generate the docs at > http://rubygems.rubyforge.org/rdoc. Ah, I see. That ought to get distributed with RDoc. It's nice. The code is too small for me, but I've had (and solved) that problem before. For whatever reason, that template works much better in Firefox than Avant (IE-based). Gavin From halostatue at gmail.com Sat Dec 11 09:11:58 2004 From: halostatue at gmail.com (Austin Ziegler) Date: Sat Dec 11 09:10:47 2004 Subject: [Rubygems-developers] 0.8.3 Documentation In-Reply-To: References: <200412110101.55488.jim@weirichhouse.org> Message-ID: <9e7db91104121106112e7dfcca@mail.gmail.com> On Sat, 11 Dec 2004 00:49:15 -0600, Curt Hibbs wrote: > You should ask Tom to install this patch on the RubyGems wiki so that this > does not happen anymore (later we can all move to Ruwiki once its ready to > go with RubyForge authentication). This has been ready for a while, as you can see with http://ruwiki.rubyforge.org -- it's just that there are other features missing (e.g., RSS, revision history, etc.) that won't be done for a little while as the holidays are upon us and I need a break from Ruwiki before tackling another big update. -austin -- Austin Ziegler * halostatue@gmail.com * Alternate: austin@halostatue.ca From jim at weirichhouse.org Sat Dec 11 09:55:34 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Sat Dec 11 09:52:54 2004 Subject: [Rubygems-developers] Oddity in Gem.activate? In-Reply-To: <1661379401865.20041211181628@soyabean.com.au> References: <1661379401865.20041211181628@soyabean.com.au> Message-ID: <200412110955.34627.jim@weirichhouse.org> On Saturday 11 December 2004 02:16 am, Gavin Sinclair wrote: > puts $LOAD_PATH > > /usr/local/lib/ruby/gems/1.8/gems/rake-0.4.10/bin > /usr/local/lib/ruby/gems/1.8/gems/rake-0.4.10/lib [...] > Why is the _bin_ path of the Rake gem being added to the $LOAD_PATH? > Shouldn't it just be the _lib_ directory? RubyGems has been doing this since forever, but I agree that it is not appropriate to include the bin directory in the load path. I don't /know/ of anything that depends on this, but have been reluctant to remove it without some kind of testing to see if any gems are affected. It certainly /shouldn't/ bother rake to remove bin from the load path. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From gsinclair at soyabean.com.au Sat Dec 11 11:13:55 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Sat Dec 11 11:13:43 2004 Subject: [Rubygems-developers] I've replaced LoadPathManager; good performance results Message-ID: <921411649614.20041212031355@soyabean.com.au> Folks, I've made a risky change to the CVS: reimplemented loadpath_manager.rb to speed up the custom "require". The new implementation is in "custom_require.rb". In rubygems.rb you'll see this: #require 'rubygems/loadpath_manager' require 'rubygems/custom_require' You can invert the comment to access the old behaviour. This change is certainly not ready for release -- it needs testing, of course -- but it looks promising. Here are my timing results: old new old new LIBRARY | DIRECT | GEM | GEM | COMBO | COMBO ------------------------+----------+----------+----------+----------+---------- 'rake' | 0.203s | 3.772s | 0.757s | 0.481s | 0.426s 'redcloth' | 0.477s | 0.577s | 0.089s | 1.355s | 0.433s 'log4r' | 0.430s | 0.908s | 0.767s | 2.545s | 0.476s 'extensions/all' | 0.536s | 5.594s | 0.795s | 2.741s | 0.643s 'dev-utils/debug' | 0.640s | 5.697s | 1.879s | 2.056s | 0.820s 'celsoft.com/template' | 0.075s | 0.362s | 0.174s | 0.096s | 0.019s A reminder of the meaning of the terms: * DIRECT: built-in require used; no RubyGems involvement * GEM: custom require used to load a library that's only available as a gem * COMBO: custom require used to load a library that's directly installed * old: using loadpath_manager.rb * new: using custom_require.rb There are a couple of shinky-looking figures there: * redcloth with the new gem * celsoft.com/template with the new combo However, there's a significant improvement in speed more or less across the board. I'll be running all the speed tests again, instead of collating data taken from a 12-hour period, but the results gave me confidence enough to commit the code. More important than speed is correctness. I'll be dreaming up ways to test it for correctness. Any ideas would be welcomed. The new implementation is simpler than the old one, better factored, and well commented. Basically, it builds the list of specs upfront (SourceIndex.from_installed_gems) and searches them when asked. I think LoadPathManager went to great lengths to avoid this, but it doesn't seem necessary. Of course, if you have 500 gems installed there could be trouble... Anyway, a few other classes have been embellished to support this change; here's the ChangeLog: 2004-12-12 Gavin Sinclair * lib/rubygems/custom_require.rb: new file to replace loadpath_manager.rb and speed up the custom require functionality. * lib/rubygems/timer.rb: new file; simple benchmarking tool that's useful at the moment but needn't stay in RubyGems permanently. * lib/rubygems/source_index.rb: include Enumerable and implement #size and #length. * lib/rubygems/specification.rb: removed lazy initialisation of attributes to reduce #copy_of invocations; folded Specification.copy_of into Specification#copy_of. * lib/rubygems.rb: require 'rubygems/custom_require' instead of 'rubygems/loadpath_manager'; removed out of date documentation; added "require 'rubygems/timer'" to temporarily assist with benchmarks. Please be on the lookout for bugs :) Cheers, Gavin From batsman.geo at yahoo.com Sat Dec 11 12:58:18 2004 From: batsman.geo at yahoo.com (Mauricio =?iso-8859-1?Q?Fern=E1ndez?=) Date: Sat Dec 11 12:57:12 2004 Subject: [Rubygems-developers] I've replaced LoadPathManager; good performance results In-Reply-To: <921411649614.20041212031355@soyabean.com.au> References: <921411649614.20041212031355@soyabean.com.au> Message-ID: <20041211175818.GA7992@student.ei.uni-stuttgart.de> On Sun, Dec 12, 2004 at 03:13:55AM +1100, Gavin Sinclair wrote: > old new old new > LIBRARY | DIRECT | GEM | GEM | COMBO | COMBO > ------------------------+----------+----------+----------+----------+---------- > 'rake' | 0.203s | 3.772s | 0.757s | 0.481s | 0.426s > 'redcloth' | 0.477s | 0.577s | 0.089s | 1.355s | 0.433s > 'log4r' | 0.430s | 0.908s | 0.767s | 2.545s | 0.476s > 'extensions/all' | 0.536s | 5.594s | 0.795s | 2.741s | 0.643s > 'dev-utils/debug' | 0.640s | 5.697s | 1.879s | 2.056s | 0.820s > 'celsoft.com/template' | 0.075s | 0.362s | 0.174s | 0.096s | 0.019s [...] > However, there's a significant improvement in speed more or less > across the board. I'll be running all the speed tests again, instead > of collating data taken from a 12-hour period, but the results gave me > confidence enough to commit the code. Could you also provide the following info? * number of installed packages (whole list if possible) * value of $: after requiring the lib I believe they could prove fairly useful, especially the former. Regarding your methodology, are all these values obtained in the same conditions regarding FS caching? Is the require 'rubygems' overhead included in the above figures or accounted separately? -- Hassle-free packages for Ruby? RPA is available from http://www.rubyarchive.org/ From gsinclair at soyabean.com.au Sat Dec 11 20:08:58 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Sat Dec 11 20:08:49 2004 Subject: [Rubygems-developers] I've replaced LoadPathManager; good performance results In-Reply-To: <20041211175818.GA7992@student.ei.uni-stuttgart.de> References: <921411649614.20041212031355@soyabean.com.au> <20041211175818.GA7992@student.ei.uni-stuttgart.de> Message-ID: <1421443752456.20041212120858@soyabean.com.au> On Sunday, December 12, 2004, 4:58:18 AM, Mauricio wrote: > On Sun, Dec 12, 2004 at 03:13:55AM +1100, Gavin Sinclair wrote: >> old new old new >> LIBRARY | DIRECT | GEM | GEM | COMBO | COMBO >> >> ------------------------+----------+----------+----------+----------+---------- >> 'rake' | 0.203s | 3.772s | 0.757s | 0.481s | 0.426s >> 'redcloth' | 0.477s | 0.577s | 0.089s | 1.355s | 0.433s >> 'log4r' | 0.430s | 0.908s | 0.767s | 2.545s | 0.476s >> 'extensions/all' | 0.536s | 5.594s | 0.795s | 2.741s | 0.643s >> 'dev-utils/debug' | 0.640s | 5.697s | 1.879s | 2.056s | 0.820s >> 'celsoft.com/template' | 0.075s | 0.362s | 0.174s | 0.096s | 0.019s > [...] >> However, there's a significant improvement in speed more or less >> across the board. I'll be running all the speed tests again, instead >> of collating data taken from a 12-hour period, but the results gave me >> confidence enough to commit the code. > Could you also provide the following info? > * number of installed packages (whole list if possible) 20 packages (listed at end of email) > * value of $: after requiring the lib Again, at end of email. > I believe they could prove fairly useful, especially the former. > Regarding your methodology, are all these values obtained in the same > conditions regarding FS caching? Is the require 'rubygems' overhead > included in the above figures or accounted separately? See test script (rgperf.rb) below. You should be able to run this against the latest CVS. Of course, you have to install/uninstall the gems/rpas as appropriate. And for some tests, you don't want "require 'rubygems'" in the script. All libraries are loaded in a single test. For me, the time taken to load 'rubygems' is currently 0.417s. The first time you search for a gem (require 'rake' in this case) pays a penalty as that's when the source index is built up. I'm not sure how to answer your specific questions, though. Gavin ** rgperf.rb: $:.unshift 'lib' require 'rubygems/timer' time("require 'rubygems'") { require 'rubygems' } time("require 'logger'") { require 'logger' } time("require 'rake'") { require 'rake' } time("require 'redcloth'") { require 'redcloth' } time("require 'log4r'") { require 'log4r' } time("require 'extensions/all'") { require 'extensions/all' } time("require 'dev-utils/debug'") { require 'dev-utils/debug' } time("require 'celsoft.com/template'") { require 'celsoft.com/template' } puts $LOAD_PATH ** Packages installed: RedCloth-2.0.11 cstemplate-0.5.1 extensions-0.5.0 rails-0.8.0 rubygems-update-0.8.3 actionmailer-0.3.0 dev-utils-1.0 extensions-0.6.0 rake-0.4.10 sources-0.0.1 actionpack-0.9.0 dev-utils-1.0.1 lafcadio-0.5.2 rake-0.4.4 sqlite-ruby-2.2.1 activerecord-1.0.0 extensions-0.4.0 log4r-1.0.5 rake-0.4.8 vim-ruby-2004.09.20 ** $LOAD_PATH after test is complete: /usr/local/lib/ruby/gems/1.8/gems/cstemplate-0.5.1/bin /usr/local/lib/ruby/gems/1.8/gems/cstemplate-0.5.1/ext /usr/local/lib/ruby/gems/1.8/gems/dev-utils-1.0/bin /usr/local/lib/ruby/gems/1.8/gems/dev-utils-1.0/lib /usr/local/lib/ruby/gems/1.8/gems/extensions-0.6.0/bin /usr/local/lib/ruby/gems/1.8/gems/extensions-0.6.0/lib /usr/local/lib/ruby/gems/1.8/gems/log4r-1.0.5/bin /usr/local/lib/ruby/gems/1.8/gems/log4r-1.0.5/src /usr/local/lib/ruby/gems/1.8/gems/RedCloth-2.0.11/bin /usr/local/lib/ruby/gems/1.8/gems/RedCloth-2.0.11/lib /usr/local/lib/ruby/gems/1.8/gems/rake-0.4.10/bin /usr/local/lib/ruby/gems/1.8/gems/rake-0.4.10/lib lib /usr/local/lib/ruby/site_ruby/1.8 /usr/local/lib/ruby/site_ruby/1.8/i386-cygwin /usr/local/lib/ruby/site_ruby /usr/local/lib/ruby/1.8 /usr/local/lib/ruby/1.8/i386-cygwin From patrick at hexane.org Sun Dec 12 15:56:07 2004 From: patrick at hexane.org (Patrick May) Date: Sun Dec 12 15:55:01 2004 Subject: [Rubygems-developers] 0.8.3 Documentation In-Reply-To: <200412110101.55488.jim@weirichhouse.org> Message-ID: <3EEE0180-4C80-11D9-8C6F-000A95848050@hexane.org> On Saturday, December 11, 2004, at 01:01 AM, Jim Weirich wrote: > We have been hit *bad* with wiki spam. In some causes actual page > content was > lost because we did not uncover the spam in time. In particular, the > Environment variable page was trashed and the original contest was no > longer > in the history. I've rewritten that page from scratch (and if someone > would > like to review, I would appreciate it). It might be good to allow project managers to lock their wikis down. Maybe the default should only allow project developers to edit the wiki. Much of the documentation is broadcast from the project developer. Ruby-forge provides functionality that is good for spammers, but isn't really used by many developers yet. The ruby community is concentrated on ruby-talk on ruby-garden. The ruby-forge wikis spread the focus out a bit. Another tactic would be to setup the ruby-forge wiki's to have properly nested home pages on the ruby-garden wiki. That way, the admin attention can be more focused. I dunno. I feel like I've spent more time fighting spammers, and less time working on narf (which doesn't use the ruby-forge wiki, but which did have a project wiki, hence similar spam problems). That feels wrong to me. ~ patrick From halostatue at gmail.com Sun Dec 12 22:42:09 2004 From: halostatue at gmail.com (Austin Ziegler) Date: Sun Dec 12 22:40:55 2004 Subject: [Rubygems-developers] 0.8.3 Documentation In-Reply-To: <3EEE0180-4C80-11D9-8C6F-000A95848050@hexane.org> References: <200412110101.55488.jim@weirichhouse.org> <3EEE0180-4C80-11D9-8C6F-000A95848050@hexane.org> Message-ID: <9e7db91104121219425a898a0e@mail.gmail.com> On Sun, 12 Dec 2004 15:56:07 -0500, Patrick May wrote: > On Saturday, December 11, 2004, at 01:01 AM, Jim Weirich wrote: >> We have been hit *bad* with wiki spam. In some causes actual page >> content was lost because we did not uncover the spam in time. In >> particular, the Environment variable page was trashed and the >> original contest was no longer in the history. I've rewritten >> that page from scratch (and if someone would like to review, I >> would appreciate it). > It might be good to allow project managers to lock their wikis > down. Maybe the default should only allow project developers to > edit the wiki. Much of the documentation is broadcast from the > project developer. Ruby-forge provides functionality that is good > for spammers, but isn't really used by many developers yet. This is possible with Ruwiki, but it does require a bit of additional work that I'll make sure is in the Ruwiki 0.9.1 that I will be releasing shortly to fix a bug in Ruwiki. The folks at RubyForge and I will then discuss making Ruwiki available for users. -austin -- Austin Ziegler * halostatue@gmail.com * Alternate: austin@halostatue.ca From jim at weirichhouse.org Mon Dec 13 09:34:24 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Mon Dec 13 09:33:09 2004 Subject: [Rubygems-developers] I've replaced LoadPathManager; good performance results In-Reply-To: <921411649614.20041212031355@soyabean.com.au> References: <921411649614.20041212031355@soyabean.com.au> Message-ID: <29541.192.223.163.6.1102948464.squirrel@weirichhouse.org> Gavin Sinclair said: > Folks, > > I've made a risky change to the CVS: reimplemented loadpath_manager.rb > to speed up the custom "require". The new implementation is in > "custom_require.rb". I just want to point out that the build is currently broken. It looks like it is failing in custom_require.rb at line 18 with an infinite recursion problem. See http://onestepback.org/cgi-bin/rubygems_build.cgi -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From gsinclair at soyabean.com.au Mon Dec 13 09:53:41 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Mon Dec 13 09:53:55 2004 Subject: [Rubygems-developers] I've replaced LoadPathManager; good performance results In-Reply-To: <29541.192.223.163.6.1102948464.squirrel@weirichhouse.org> References: <921411649614.20041212031355@soyabean.com.au> <29541.192.223.163.6.1102948464.squirrel@weirichhouse.org> Message-ID: <1821579635045.20041214015341@soyabean.com.au> On Tuesday, December 14, 2004, 1:34:24 AM, Jim wrote: > Gavin Sinclair said: >> Folks, >> >> I've made a risky change to the CVS: reimplemented loadpath_manager.rb >> to speed up the custom "require". The new implementation is in >> "custom_require.rb". > I just want to point out that the build is currently broken. It looks > like it is failing in custom_require.rb at line 18 with an infinite > recursion problem. Thanks Jim. I invalidated test_loadmanager.rb to avoid this; it's incompatible with custom_require.rb. There's now 3 failures in the test suite, all in test_gemloadpath.rb. I'll look at these in the morning. Gavin From gsinclair at soyabean.com.au Wed Dec 15 08:36:04 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Wed Dec 15 08:34:17 2004 Subject: [Rubygems-developers] ruby setup.rb all not working? Message-ID: <141747634305.20041216003604@soyabean.com.au> Guys, I don't know about you, but I can't seem to install RubyGems using setup.rb. I've removed my previous installation, to ensure I was testing against my project files, not installed files. Now I can't reinstall. A quick look through setup.rb doesn't enlighten me. Here is the transcript: $ RUBYOPT= ruby setup.rb all Processing the package `sources' ... ---> lib <--- lib Processing the package `sources' ... ---> lib <--- lib Processing the package `sources' ... rm -f InstalledFiles ---> lib mkdir -p /usr/local/lib/ruby/site_ruby/1.8/ install sources.rb /usr/local/lib/ruby/site_ruby/1.8/ <--- lib As of RubyGems 0.8.0, library stubs are no longer needed. Searching $LOAD_PATH for stubs to optionally delete (may take a while)... ...done. No library stubs found. Successfully built RubyGem Name: sources Version: 0.0.1 File: sources-0.0.1.gem Any suggestions? Any other experiences? Gavin From chad at chadfowler.com Wed Dec 15 08:39:16 2004 From: chad at chadfowler.com (Chad Fowler) Date: Wed Dec 15 08:38:01 2004 Subject: [Rubygems-developers] ruby setup.rb all not working? In-Reply-To: <141747634305.20041216003604@soyabean.com.au> References: <141747634305.20041216003604@soyabean.com.au> Message-ID: On 15-Dec-04, at 8:36 AM, Gavin Sinclair wrote: > Guys, > > I don't know about you, but I can't seem to install RubyGems using > setup.rb. I've removed my previous installation, to ensure I was > testing against my project files, not installed files. Now I can't > reinstall. > > A quick look through setup.rb doesn't enlighten me. > > Here is the transcript: > > $ RUBYOPT= ruby setup.rb all > Processing the package `sources' ... > ---> lib > <--- lib > Processing the package `sources' ... > ---> lib > <--- lib > Processing the package `sources' ... > rm -f InstalledFiles > ---> lib > mkdir -p /usr/local/lib/ruby/site_ruby/1.8/ > install sources.rb /usr/local/lib/ruby/site_ruby/1.8/ > <--- lib > > As of RubyGems 0.8.0, library stubs are no longer needed. > Searching $LOAD_PATH for stubs to optionally delete (may take a > while)... > ...done. > No library stubs found. > > Successfully built RubyGem > Name: sources > Version: 0.0.1 > File: sources-0.0.1.gem > > > Any suggestions? Any other experiences? > See if you have a directory called "packages". If so, rm -rf it. Chad From gsinclair at soyabean.com.au Wed Dec 15 08:48:04 2004 From: gsinclair at soyabean.com.au (Gavin Sinclair) Date: Wed Dec 15 08:46:13 2004 Subject: [Rubygems-developers] ruby setup.rb all not working? In-Reply-To: References: <141747634305.20041216003604@soyabean.com.au> Message-ID: <601748351927.20041216004804@soyabean.com.au> On Thursday, December 16, 2004, 12:39:16 AM, Chad wrote: >> Any suggestions? Any other experiences? >> > See if you have a directory called "packages". If so, rm -rf it. That's better. What's behind that? A legacy CVS thing that gets omitted from release? Gavin From chad at chadfowler.com Wed Dec 15 08:55:33 2004 From: chad at chadfowler.com (Chad Fowler) Date: Wed Dec 15 08:54:15 2004 Subject: [Rubygems-developers] ruby setup.rb all not working? In-Reply-To: <601748351927.20041216004804@soyabean.com.au> References: <141747634305.20041216003604@soyabean.com.au> <601748351927.20041216004804@soyabean.com.au> Message-ID: On 15-Dec-04, at 8:48 AM, Gavin Sinclair wrote: > On Thursday, December 16, 2004, 12:39:16 AM, Chad wrote: > >>> Any suggestions? Any other experiences? >>> > >> See if you have a directory called "packages". If so, rm -rf it. > > That's better. What's behind that? A legacy CVS thing that gets > omitted from release? > Yes. And a somewhat annoying quality of setup.rb. Chad From assaph at gmail.com Mon Dec 20 07:39:57 2004 From: assaph at gmail.com (Assaph Mehr) Date: Mon Dec 20 07:38:26 2004 Subject: [Rubygems-developers] RubyGem Issues Message-ID: <3ff47be904122004394b4d32db@mail.gmail.com> Hi Guys, Two issues: 1. Something I recieved re installing Pimki as a gem. As far as I can tell it's not a Pimki issue as I can install it fine. It maybe a gem under linux issue. This is the complete info I recieved from the user: ----------------------------------------- gem install pimki Attempting local installation of 'pimki' Local gem file not found: pimki*.gem Attempting remote installation of 'pimki' Updating Gem source index for: http://gems.rubyforge.org /usr/local/lib/ruby/1.8/yaml.rb:39: [BUG] rb_gc_mark(): unknown data type 0x10(0xb7f79840) non object ruby 1.8.1 (2003-12-25) [i686-linux] Aborted ----------------------------------------- 2. 'require_gem' in 0.8.3 seem to have lost the capacity to receive a single parameter. In 0.8.1 I was able to write both: require_gem 'redcloth', '< 3.0.0' require_gem 'bluecloth' In 0.8.3, the second line bombs. Was the behaviour changed intentionally? (ruby 1.8.2-pr3, gems 0.8.3, WinXP). Cheers, Assaph From chad at chadfowler.com Mon Dec 20 21:55:39 2004 From: chad at chadfowler.com (Chad Fowler) Date: Mon Dec 20 21:54:14 2004 Subject: [Rubygems-developers] RubyGem Issues In-Reply-To: <3ff47be904122004394b4d32db@mail.gmail.com> References: <3ff47be904122004394b4d32db@mail.gmail.com> Message-ID: Hi Assaph! On 20-Dec-04, at 7:39 AM, Assaph Mehr wrote: > Hi Guys, > > Two issues: > > 1. Something I recieved re installing Pimki as a gem. As far as I can > tell it's not a Pimki issue as I can install it fine. It maybe a gem > under linux issue. This is the complete info I recieved from the user: > > ----------------------------------------- > gem install pimki > Attempting local installation of 'pimki' > Local gem file not found: pimki*.gem > Attempting remote installation of 'pimki' > Updating Gem source index for: http://gems.rubyforge.org > /usr/local/lib/ruby/1.8/yaml.rb:39: [BUG] rb_gc_mark(): unknown data > type 0x10(0xb7f79840) non object > ruby 1.8.1 (2003-12-25) [i686-linux] > > Aborted > ----------------------------------------- > I believe this is a known issue with the release of Ruby the user is using, though I've heard of similar (though possibly not fully related) symptoms in more recent releases. It seems to be syck-related. > 2. 'require_gem' in 0.8.3 seem to have lost the capacity to receive a > single parameter. In 0.8.1 I was able to write both: > require_gem 'redcloth', '< 3.0.0' > require_gem 'bluecloth' > > In 0.8.3, the second line bombs. Was the behaviour changed > intentionally? > > (ruby 1.8.2-pr3, gems 0.8.3, WinXP). > Really? What is the error message? Could it be a problem with case sensitivity? The BlueCloth gem is called "BlueCloth"--not "bluecloth". Might that be the problem? Chad From jim at weirichhouse.org Wed Dec 22 08:08:53 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Wed Dec 22 08:06:41 2004 Subject: [Rubygems-developers] RubyGems build is broken Message-ID: <200412220808.53162.jim@weirichhouse.org> The current build is broken: http://onestepback.org/cgi-bin/rubygems_build.cgi -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From jim at weirichhouse.org Wed Dec 29 23:09:45 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Wed Dec 29 23:06:33 2004 Subject: [Rubygems-developers] Wiki Spam -- GRRRRRR! Message-ID: <200412292309.45629.jim@weirichhouse.org> Twenty minutes ago I discovered 30 pages of Wiki spam on the RubyGems wiki pages. I just got done cleaning this up. This is the third time this week. Shoot, they even spammed us on Christmas day. Grrrrrrrrr We have lost content on the wiki from excessive spam. It is costing us developer time. And it is doing serious damage to my mental health. Something has got to be done. We are rather limited in what we can do with the RubyForge wikis. It is an old version of UseMod without RSS, and no access to the Ban lists. And the RubyForge conversion to RuWiki is still a ways off (heavens, Ruwiki doesn't even support viewing history yet... that would make spam 10 times more annoying than it is now). Here are some ideas. (1) Move the wiki off of RubyForge and setup a tarpit like the one on RubyGarden. (2) Make the RubyForge a shadow wiki. The real wiki exists elsewhere in a secret location and we automatically update the public shadow wiki from the real, but hidden, one. (3) Abandon the wiki for documentation purposes. Convert the existing docs RDOC, or Textile or Markdown and generate static web pages. If we need user feedback, put it one small wiki somewhere else. (4) Give up and start coding in Python. (OK, I'm not serious about the last one). If someone else has some more ideas, I'd love to hear them. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) From curt at hibbs.com Thu Dec 30 10:54:07 2004 From: curt at hibbs.com (Curt Hibbs) Date: Thu Dec 30 10:52:17 2004 Subject: [Rubygems-developers] Wiki Spam -- GRRRRRR! In-Reply-To: <200412292309.45629.jim@weirichhouse.org> Message-ID: I thought that your usemod patch had been installed on the RubyGems wiki (the one that requires HTTP instead of http for external links). Am I wrong about this? Since I had Tom Copeland install this one my 4 RubyForge project wikis, I have had zero problems with spam (I guess I should go check to make sure this is still true). Of course, this is just a temporary fix. But very-soon-now, RuWiki should be ready for use on RubyForge, and we should all move our wiki's to that since it will require a RubyForge login to modify pages. Curt Jim Weirich wrote: > > Twenty minutes ago I discovered 30 pages of Wiki spam on the > RubyGems wiki > pages. I just got done cleaning this up. This is the third time > this week. > Shoot, they even spammed us on Christmas day. Grrrrrrrrr > > We have lost content on the wiki from excessive spam. It is costing us > developer time. And it is doing serious damage to my mental health. > > Something has got to be done. We are rather limited in what we > can do with > the RubyForge wikis. It is an old version of UseMod without RSS, and no > access to the Ban lists. And the RubyForge conversion to RuWiki > is still a > ways off (heavens, Ruwiki doesn't even support viewing history > yet... that > would make spam 10 times more annoying than it is now). > > Here are some ideas. > > (1) Move the wiki off of RubyForge and setup a tarpit like the one on > RubyGarden. > > (2) Make the RubyForge a shadow wiki. The real wiki exists > elsewhere in a > secret location and we automatically update the public shadow > wiki from the > real, but hidden, one. > > (3) Abandon the wiki for documentation purposes. Convert the > existing docs > RDOC, or Textile or Markdown and generate static web pages. If > we need user > feedback, put it one small wiki somewhere else. > > (4) Give up and start coding in Python. > > (OK, I'm not serious about the last one). > > If someone else has some more ideas, I'd love to hear them. > > -- > -- Jim Weirich jim@weirichhouse.org http://onestepback.org > ----------------------------------------------------------------- > "Beware of bugs in the above code; I have only proved it correct, > not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas) > _______________________________________________ > Rubygems-developers mailing list > Rubygems-developers@rubyforge.org > http://rubyforge.org/mailman/listinfo/rubygems-developers > > > -- > Internal Virus Database is out-of-date. > Checked by AVG Anti-Virus. > Version: 7.0.298 / Virus Database: 265.6.4 - Release Date: 12/22/2004 > > > -- > No virus found in this incoming message. > Checked by AVG Anti-Virus. > Version: 7.0.296 / Virus Database: 265.6.6 - Release Date: 12/28/2004 > From tom at infoether.com Thu Dec 30 11:08:19 2004 From: tom at infoether.com (Tom Copeland) Date: Thu Dec 30 11:06:31 2004 Subject: [Rubygems-developers] Wiki Spam -- GRRRRRR! In-Reply-To: References: Message-ID: <1104422899.8234.18.camel@hal> On Thu, 2004-12-30 at 10:54, Curt Hibbs wrote: > I thought that your usemod patch had been installed on the RubyGems wiki > (the one that requires HTTP instead of http for external links). Am I wrong > about this? It wasn't, but now it is, and I've converted the "http"s on the front page to be "HTTP"s. > Of course, this is just a temporary fix. But very-soon-now, RuWiki should be > ready for use on RubyForge, and we should all move our wiki's to that since > it will require a RubyForge login to modify pages. Right on! Yours, Tom From jim at weirichhouse.org Thu Dec 30 11:34:45 2004 From: jim at weirichhouse.org (Jim Weirich) Date: Thu Dec 30 11:32:53 2004 Subject: [Rubygems-developers] Wiki Spam -- GRRRRRR! In-Reply-To: <1104422899.8234.18.camel@hal> References: <1104422899.8234.18.camel@hal> Message-ID: <48424.192.223.163.6.1104424485.squirrel@weirichhouse.org> Tom Copeland said: > On Thu, 2004-12-30 at 10:54, Curt Hibbs wrote: >> I thought that your usemod patch had been installed on the RubyGems wiki >> (the one that requires HTTP instead of http for external links). Am I >> wrong >> about this? > > It wasn't, but now it is, and I've converted the "http"s on the front > page to be "HTTP"s. Thanks. I was actually despamming pages (yes, again, the same 30 pages even!) while you did the install. For about a 30 seconds the pages were all coming back empty! Wow, did I panic. Then the pages came back and I started getting external link errors and I suspected that you were busy behind the scenes. Thanks! I hope this will hold us until the RuWiki authorization is in place. -- -- Jim Weirich jim@weirichhouse.org http://onestepback.org ----------------------------------------------------------------- "Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)