From dennis at lauschmusik.de Wed Jan 4 07:53:58 2006 From: dennis at lauschmusik.de (Dennis Oelkers) Date: Wed, 04 Jan 2006 13:53:58 +0100 Subject: [Support-mirrors] new mirror In-Reply-To: <1135891433.4658.12.camel@hal> References: <43B426D2.9080109@iasi.roedu.net> <43B43B8E.7010904@gmail.com> <1135891433.4658.12.camel@hal> Message-ID: <43BBC566.8090307@lauschmusik.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hey Tom, hey list, Tom Copeland wrote: > Hm, Dennis, do you have all the info you need to sync to James' mirror > server? Sorry for the late response, but I was out of town for quite a while, and then came christmas and new year's eve and everything ... (Happy new year to everybody btw!) I have a problem using the password James sent me to sync his server, but we're working things out off-list. I will contact Manuel and send him all the info he needs to set things up and get running. Kind regards, Dennis Oelkers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDu8VmcT9mJF158yMRApVpAKC5QH2VBfx90hU7MIb3XPBHaMHVFgCfUPD8 BmJm4VPd7lzGEAf0DiDpD5M= =LuzY -----END PGP SIGNATURE----- From tom at infoether.com Wed Jan 4 09:07:48 2006 From: tom at infoether.com (Tom Copeland) Date: Wed, 04 Jan 2006 09:07:48 -0500 Subject: [Support-mirrors] new mirror In-Reply-To: <43BBC566.8090307@lauschmusik.de> References: <43B426D2.9080109@iasi.roedu.net> <43B43B8E.7010904@gmail.com> <1135891433.4658.12.camel@hal> <43BBC566.8090307@lauschmusik.de> Message-ID: <1136383668.12967.46.camel@hal> On Wed, 2006-01-04 at 13:53 +0100, Dennis Oelkers wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hey Tom, hey list, > > Tom Copeland wrote: > > Hm, Dennis, do you have all the info you need to sync to James' mirror > > server? > > Sorry for the late response, but I was out of town for quite a while, > and then came christmas and new year's eve and everything ... > (Happy new year to everybody btw!) No problemo, and same to you! > I have a problem using the password James sent me to sync his server, > but we're working things out off-list. Super! Just let me know when it's sorted out and I'll add his server to the gem and mirror rotations. Right now you and Austin are shouldering the entire load... > I will contact Manuel and send him all the info he needs to set things > up and get running. Great, thanks! Yours, Tom From gregory.t.brown at gmail.com Thu Jan 5 12:50:44 2006 From: gregory.t.brown at gmail.com (Gregory Brown) Date: Thu, 5 Jan 2006 12:50:44 -0500 Subject: [Support-mirrors] Interested in running a mirror on dreamhost Message-ID: Hello all, I've got a lot of bandwidth and space to share, so if you'd like, i'll be happy to host a mirror. There are a couple of minor caveats... dreamhost shared hosting plans are not rooted, so they allow rsync only through ssh. It's no trouble to offer up an account for whoever is administering the mirror, and we'll probably need to put the actual content in a fake root, as I don't have write access to /var However, we can easily drop a mirrored file system into a folder on a shell account, and if this doesn't pose a problem, then, let me know what i need to do and I'll get started! -Greg From tom at infoether.com Thu Jan 5 13:49:29 2006 From: tom at infoether.com (Tom Copeland) Date: Thu, 5 Jan 2006 13:49:29 -0500 Subject: [Support-mirrors] Interested in running a mirror on dreamhost In-Reply-To: Message-ID: <012301c61228$c2686090$6601a8c0@tomhplaptop> > Hello all, I've got a lot of bandwidth and space to share, so > if you'd like, i'll be happy to host a mirror. > > There are a couple of minor caveats... dreamhost shared > hosting plans are not rooted, so they allow rsync only through ssh. > > It's no trouble to offer up an account for whoever is > administering the mirror, and we'll probably need to put the > actual content in a fake root, as I don't have write access to /var > > However, we can easily drop a mirrored file system into a > folder on a shell account, and if this doesn't pose a > problem, then, let me know what i need to do and I'll get started! Hi Greg - Sounds great! Dennis, pls let me know if you need anything from my end... Yours, Tom From dennis at lauschmusik.de Mon Jan 16 08:00:52 2006 From: dennis at lauschmusik.de (Dennis Oelkers) Date: Mon, 16 Jan 2006 13:00:52 +0000 Subject: [Support-mirrors] Interested in running a mirror on dreamhost In-Reply-To: References: Message-ID: <43CB9904.9080007@lauschmusik.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Gregory, hi list, Gregory Brown wrote: > Hello all, I've got a lot of bandwidth and space to share, so if you'd > like, i'll be happy to host a mirror. > > There are a couple of minor caveats... dreamhost shared hosting plans > are not rooted, so they allow rsync only through ssh. Should not be a problem. > > It's no trouble to offer up an account for whoever is administering > the mirror, and we'll probably need to put the actual content in a > fake root, as I don't have write access to /var > > However, we can easily drop a mirrored file system into a folder on a > shell account, and if this doesn't pose a problem, then, let me know > what i need to do and I'll get started! How would the URLs look like? I guess Tom should be able to judge if your mirror is usable depending on the capabilities of his rewrite-script doing the mirror rotation. If it's all fine, I will send you a public ssh key and set things up. Kind regards, Dennis Oelkers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDy5kEcT9mJF158yMRArsaAKC5cXG9qrsxTT1WiTD59u+/v9kAeQCgyUTO CclBMpZokwmT0Z9sH+nsOoE= =zJkD -----END PGP SIGNATURE----- From gregory.t.brown at gmail.com Mon Jan 16 13:14:38 2006 From: gregory.t.brown at gmail.com (Gregory Brown) Date: Mon, 16 Jan 2006 13:14:38 -0500 Subject: [Support-mirrors] Interested in running a mirror on dreamhost In-Reply-To: <43CB9904.9080007@lauschmusik.de> References: <43CB9904.9080007@lauschmusik.de> Message-ID: On 1/16/06, Dennis Oelkers wrote: > How would the URLs look like? I guess Tom should be able to judge if > your mirror is usable depending on the capabilities of his > rewrite-script doing the mirror rotation. > > If it's all fine, I will send you a public ssh key and set things up. I can remap subdirs, so i don't think it should be a problem. Feel free to send the public key and some instructions. From tom at infoether.com Mon Jan 16 13:38:16 2006 From: tom at infoether.com (Tom Copeland) Date: Mon, 16 Jan 2006 13:38:16 -0500 Subject: [Support-mirrors] Interested in running a mirror on dreamhost In-Reply-To: References: <43CB9904.9080007@lauschmusik.de> Message-ID: <1137436696.26184.40.camel@hal> On Mon, 2006-01-16 at 13:14 -0500, Gregory Brown wrote: > On 1/16/06, Dennis Oelkers wrote: > > > How would the URLs look like? I guess Tom should be able to judge if > > your mirror is usable depending on the capabilities of his > > rewrite-script doing the mirror rotation. > > > > If it's all fine, I will send you a public ssh key and set things up. > > I can remap subdirs, so i don't think it should be a problem. Feel > free to send the public key and some instructions. I've updated the mirror doc here: http://rubyforge.org/docman/view.php/5/231/mirror_setup.html to reflect the URL patterns that RubyForge wants to see. It's pretty much just this for files: http://mirror.host.name/projectname/filename and this for RubyGems: http://mirror.host.name/gems/somegemname-1.0.gem Dennis, pls let me know when Gregory is all synced up and I'll add him to the rotation. Also, Matthew Bloch of bytemark has offered up another mirror via a hosted virtual machine - so Dennis, as soon as I get the account set up there I'll coordinate getting everything sync'd there. That'll get us up to 4 mirrors, good times! Yours, Tom From vince at litrium.com Tue Jan 17 14:32:47 2006 From: vince at litrium.com (Vince Hoang) Date: Tue, 17 Jan 2006 09:32:47 -1000 Subject: [Support-mirrors] mirror setup questions Message-ID: <20060117193247.GG28430@litrium.com> I am interested in helping mirroring RubyForge. I like the idea of using a mirror master to offloa the bandwidth requirements to a faster host, but would greatly prefer pulling the mirror than having it pushed. With the current URL schema of http://mirror/projectname/filename, would I need to setup a separate virtual host for it, rather than use something like http://mirror/rubyforge/projectname/filename? Lastly, I believe the documented rsync setup leaves the mirror vulnerable to malicious behavior. The 'hosts allow' option should only list IP addresses. If I know the contents of the rsync.secret file, I can resolve my IP address to rubyforge.lauschmusik.de and overwrite the contents of any mirror that uses the listed configuration: [file-mirror] path = /var/www/rubyforge/htdocs auth users = rubyforge secrets file = /var/www/rubyforge/etc/rsync.secrets hosts allow = rubyforge.lauschmusik.de hosts deny = * -Vince From dennis at lauschmusik.de Tue Jan 17 17:14:48 2006 From: dennis at lauschmusik.de (Dennis Oelkers) Date: Tue, 17 Jan 2006 23:14:48 +0100 Subject: [Support-mirrors] mirror setup questions In-Reply-To: <20060117193247.GG28430@litrium.com> References: <20060117193247.GG28430@litrium.com> Message-ID: <43CD6C58.9000005@lauschmusik.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Vince, Vince Hoang wrote: > I am interested in helping mirroring RubyForge. I like the idea of > using a mirror master to offloa the bandwidth requirements to a > faster host, but would greatly prefer pulling the mirror than > having it pushed. > > With the current URL schema of http://mirror/projectname/filename, > would I need to setup a separate virtual host for it, rather than > use something like http://mirror/rubyforge/projectname/filename? I guess Tom is able to answer those questions. > > Lastly, I believe the documented rsync setup leaves the mirror > vulnerable to malicious behavior. The 'hosts allow' option should > only list IP addresses. If I know the contents of the rsync.secret > file, I can resolve my IP address to rubyforge.lauschmusik.de and > overwrite the contents of any mirror that uses the listed > configuration: > This setup is used only for the master server, where the rsync-modules are read-only anyway. The limitation is there to save bandwidth, not to protect the content, but thanks for the hint, although it's already known :) Kind regards, Dennis Oelkers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDzWxXcT9mJF158yMRAn+9AJ49GlxKYBlnpsK/nkAKlY++LlFuIQCgnSZ+ VrTQXIZZqk8oap25rDbDQlg= =AC10 -----END PGP SIGNATURE----- From drbrain at segment7.net Tue Jan 17 17:42:20 2006 From: drbrain at segment7.net (Eric Hodel) Date: Tue, 17 Jan 2006 14:42:20 -0800 Subject: [Support-mirrors] Broken Rubygems mirror Message-ID: <42308357-6FE0-45C4-985A-75D68F456C55@segment7.net> I uploaded memcache-client-1.0.3.gem and cached_model-1.0.0.gem earlier today, but they have not shown up on the mirrors yet, so I can't install them. $ sudo gem install memcache-client Attempting local installation of 'memcache-client' Local gem file not found: memcache-client*.gem Attempting remote installation of 'memcache-client' ERROR: While executing gem ... (OpenURI::HTTPError) 404 Not Found ... so the gem is in the Rubygems source index, but not on the mirror: $ fetch -v http://gems.rubyforge.org/gems/memcache-client-1.0.3.gem looking up gems.rubyforge.org connecting to gems.rubyforge.org:80 requesting http://gems.rubyforge.org/gems/memcache-client-1.0.3.gem 302 redirect to http://rubyforge.lauschmusik.de/gems/memcache- client-1.0.3.gem looking up rubyforge.lauschmusik.de connecting to rubyforge.lauschmusik.de:80 requesting http://rubyforge.lauschmusik.de/gems/memcache- client-1.0.3.gem 404 - Not Found 404 - Not Found fetch: http://gems.rubyforge.org/gems/memcache-client-1.0.3.gem: Not Found But I can see the gem on rubyforge when using scp: $ scp rubyforge.org:/var/www/gems/gems/memcache-client-1.0.3.gem . memcache-client-1.0.3.gem 100% 8192 8.0KB/s 00:01 I'm seeing the same behavior with cached_model-1.0.0.gem. -- Eric Hodel - drbrain at segment7.net - http://segment7.net This implementation is HODEL-HASH-9600 compliant http://trackmap.robotcoop.com From dennis at lauschmusik.de Tue Jan 17 17:52:46 2006 From: dennis at lauschmusik.de (Dennis Oelkers) Date: Tue, 17 Jan 2006 23:52:46 +0100 Subject: [Support-mirrors] Broken Rubygems mirror In-Reply-To: <42308357-6FE0-45C4-985A-75D68F456C55@segment7.net> References: <42308357-6FE0-45C4-985A-75D68F456C55@segment7.net> Message-ID: <43CD753E.6060100@lauschmusik.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Eric, Eric Hodel schrieb: > I uploaded memcache-client-1.0.3.gem and cached_model-1.0.0.gem > earlier today, but they have not shown up on the mirrors yet, so I > can't install them. [...] We are currently working on the sync-scripts to be able to do a triggered sync instead of a timed one, so I deactivated the sync for a short while. I'll manually sync it now though, to let the recent changes through to the mirrors. Kind regards, Dennis Oelkers -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDzXU+cT9mJF158yMRAovTAKCPatngm3DKOY7Cj+hX9AXTG/ZNjwCeL7hZ wQlc/dWNZTqp195jHvAtD/s= =3EPj -----END PGP SIGNATURE----- From tom at infoether.com Tue Jan 17 22:25:24 2006 From: tom at infoether.com (Tom Copeland) Date: Tue, 17 Jan 2006 22:25:24 -0500 Subject: [Support-mirrors] mirror setup questions In-Reply-To: <20060117193247.GG28430@litrium.com> References: <20060117193247.GG28430@litrium.com> Message-ID: <1137554724.1306.30.camel@hal> On Tue, 2006-01-17 at 09:32 -1000, Vince Hoang wrote: > I am interested in helping mirroring RubyForge. I like the idea > of using a mirror master to offloa the bandwidth requirements to > a faster host, but would greatly prefer pulling the mirror than > having it pushed. > > With the current URL schema of > http://mirror/projectname/filename, would I need to setup a > separate virtual host for it, rather than use something like > http://mirror/rubyforge/projectname/filename? It would make things easier on me; my mod_rewrite skills are not so good. But if creating a new vhost is a problem, I can try to enhance the current rewrite setup... > Lastly, I believe the documented rsync setup leaves the > mirror vulnerable to malicious behavior. The 'hosts allow' > option should only list IP addresses. Hm, that's probably a good idea, I've updated the docs, thanks! Yours, Tom From tom at infoether.com Tue Jan 17 22:27:37 2006 From: tom at infoether.com (Tom Copeland) Date: Tue, 17 Jan 2006 22:27:37 -0500 Subject: [Support-mirrors] mirror setup questions In-Reply-To: <20060117193247.GG28430@litrium.com> References: <20060117193247.GG28430@litrium.com> Message-ID: <1137554857.1306.33.camel@hal> On Tue, 2006-01-17 at 09:32 -1000, Vince Hoang wrote: > I am interested in helping mirroring RubyForge. I like the idea > of using a mirror master to offloa the bandwidth requirements to > a faster host, but would greatly prefer pulling the mirror than > having it pushed. Sorry, forgot to address this the first time around... I think the advantage of the "mirror master push" schema is the speed at which files can get sync'd out with very little load on the machines involved. By pushing, it could take less than a minute for a new file to get sync'd out to all the mirror, and that will only happen when a large file needs to get transferred - not every 10-15 minutes. I'd kind of like to stick with this new "push" thing as long as possible... Yours, Tom From vince at litrium.com Wed Jan 18 03:21:03 2006 From: vince at litrium.com (Vince Hoang) Date: Tue, 17 Jan 2006 22:21:03 -1000 Subject: [Support-mirrors] mirror setup questions In-Reply-To: <1137554724.1306.30.camel@hal> References: <20060117193247.GG28430@litrium.com> <1137554724.1306.30.camel@hal> Message-ID: <20060118082103.GO28430@litrium.com> On Tue, Jan 17, 2006 at 10:25:24PM -0500, Tom Copeland wrote: > On Tue, 2006-01-17 at 09:32 -1000, Vince Hoang wrote: > > With the current URL schema of > > http://mirror/projectname/filename, would I need to setup a > > separate virtual host for it, rather than use something like > > http://mirror/rubyforge/projectname/filename? > > It would make things easier on me; my mod_rewrite skills are not so > good. But if creating a new vhost is a problem, I can try to enhance > the current rewrite setup... Having /rubyforge prefix the path would make it easier for larger mirror sites to support it. Think CPAN. I can help with rewrite rules if needed. > > Lastly, I believe the documented rsync setup leaves the > > mirror vulnerable to malicious behavior. The 'hosts allow' > > option should only list IP addresses. > > Hm, that's probably a good idea, I've updated the docs, thanks! Excellent. -Vince From vince at litrium.com Wed Jan 18 04:22:47 2006 From: vince at litrium.com (Vince Hoang) Date: Tue, 17 Jan 2006 23:22:47 -1000 Subject: [Support-mirrors] mirror setup questions In-Reply-To: <1137554857.1306.33.camel@hal> References: <20060117193247.GG28430@litrium.com> <1137554857.1306.33.camel@hal> Message-ID: <20060118092247.GP28430@litrium.com> On Tue, Jan 17, 2006 at 10:27:37PM -0500, Tom Copeland wrote: > Sorry, forgot to address this the first time around... I think > the advantage of the "mirror master push" schema is the speed > at which files can get sync'd out with very little load on the > machines involved. By pushing, it could take less than a minute > for a new file to get sync'd out to all the mirror, and that > will only happen when a large file needs to get transferred - > not every 10-15 minutes. I'd kind of like to stick with this > new "push" thing as long as possible... Fair enough. I am a little squeemish about having a writable rsync module, even with the chroot, but I enabled it for mirrors.hosef.org. I just need the secrets to grant access to my rsync server. -Vince From dennis at lauschmusik.de Wed Jan 18 05:25:52 2006 From: dennis at lauschmusik.de (Dennis Oelkers) Date: Wed, 18 Jan 2006 11:25:52 +0100 Subject: [Support-mirrors] mirror setup questions In-Reply-To: <20060118092247.GP28430@litrium.com> References: <20060117193247.GG28430@litrium.com> <1137554857.1306.33.camel@hal> <20060118092247.GP28430@litrium.com> Message-ID: <43CE17B0.3020909@lauschmusik.de> Vince Hoang wrote: >On Tue, Jan 17, 2006 at 10:27:37PM -0500, Tom Copeland wrote: > > >>Sorry, forgot to address this the first time around... I think >>the advantage of the "mirror master push" schema is the speed >>at which files can get sync'd out with very little load on the >>machines involved. By pushing, it could take less than a minute >>for a new file to get sync'd out to all the mirror, and that >>will only happen when a large file needs to get transferred - >>not every 10-15 minutes. I'd kind of like to stick with this >>new "push" thing as long as possible... >> >> > >Fair enough. > >I am a little squeemish about having a writable rsync module, >even with the chroot, but I enabled it for mirrors.hosef.org. >I just need the secrets to grant access to my rsync server. > > > Just generate a random one, and send all the data pgp-encrypted to me. Kind regards, Dennis Oelkers -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: 0x5D79F323.asc Url: http://rubyforge.org/pipermail/support-mirrors/attachments/20060118/b771bf63/0x5D79F323.bat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://rubyforge.org/pipermail/support-mirrors/attachments/20060118/b771bf63/signature.bin From halostatue at gmail.com Thu Jan 26 16:31:17 2006 From: halostatue at gmail.com (Austin Ziegler) Date: Thu, 26 Jan 2006 16:31:17 -0500 Subject: [Support-mirrors] What's up? Message-ID: <9e7db9110601261331u725663b9x@mail.gmail.com> Tom, What's up with the mirror situation? -austin -- Austin Ziegler * halostatue at gmail.com * Alternate: austin at halostatue.ca From tom at infoether.com Thu Jan 26 16:34:28 2006 From: tom at infoether.com (Tom Copeland) Date: Thu, 26 Jan 2006 16:34:28 -0500 Subject: [Support-mirrors] Mirror master down Message-ID: <011e01c622c0$49e2e6b0$1301a8c0@tomhplaptop> Hi all - You may have seen this thread on ruby-talk: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/177135 I daresay Dennis is working on things on his end; seems like the entirely lauschmusik.com domain has gone offline. Anyhow, I am currently rsync'ing the files and gems over to an InfoEther-provided mirror server and hope to get things back online in perhaps an hour or two. Yours, Tom From tom at infoether.com Thu Jan 26 18:45:06 2006 From: tom at infoether.com (Tom Copeland) Date: Thu, 26 Jan 2006 18:45:06 -0500 Subject: [Support-mirrors] Mirror master down In-Reply-To: <011e01c622c0$49e2e6b0$1301a8c0@tomhplaptop> References: <011e01c622c0$49e2e6b0$1301a8c0@tomhplaptop> Message-ID: <1138319106.29006.7.camel@hal> On Thu, 2006-01-26 at 16:34 -0500, Tom Copeland wrote: > Hi all - > > You may have seen this thread on ruby-talk: > > http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/177135 > > I daresay Dennis is working on things on his end; seems like the > entirely lauschmusik.com domain has gone offline. Just got a note from Dennis; service has been restored and all's well, it was an unscheduled provider outage. So RubyForge is back to full capacity. A thought on this - once we get this new mirror architecture firmly in place and a few mirrors are getting files pushed to them, the mirror master going down for a few hours won't be too big of a deal since we'll just drop the master server out of the rotation for a few hours. As things are now, rubyforge.lauschmusik.de is handling the load almost entirely. Dennis and I will keep working on getting the new "push to mirrors" system working and good times will result. Yours, Tom From wilsonb at gmail.com Thu Jan 26 20:42:17 2006 From: wilsonb at gmail.com (Wilson Bilkovich) Date: Thu, 26 Jan 2006 20:42:17 -0500 Subject: [Support-mirrors] Another Dreamhost gem mirror Message-ID: I'm willing to stand up another RubyGems mirror at my Dreamhost account. I have a jillion gigs of bandwidth and disk quota now; much more than I am likely to use. Is there need for another? --Wilson. From tom at infoether.com Fri Jan 27 10:54:16 2006 From: tom at infoether.com (Tom Copeland) Date: Fri, 27 Jan 2006 10:54:16 -0500 Subject: [Support-mirrors] Another Dreamhost gem mirror In-Reply-To: References: Message-ID: <1138377256.29006.73.camel@hal> On Thu, 2006-01-26 at 20:42 -0500, Wilson Bilkovich wrote: > I'm willing to stand up another RubyGems mirror at my Dreamhost account. > I have a jillion gigs of bandwidth and disk quota now; much more than > I am likely to use. > Is there need for another? Hi Wilson - Thanks! Let's see... right now we're right in the midst of setting up a new mirror architecture; if you can stay on the list for a week or two we should have finished that up and it should be easier to then add new mirror providers. Thanks for the offer! Yours, Tom