From citizen428 at gentoo.org Tue Nov 1 14:27:17 2005 From: citizen428 at gentoo.org (Michael Kohl) Date: Tue, 1 Nov 2005 20:27:17 +0100 Subject: Bookmarklet for ProxyLike Message-ID: <20051101202717.1c5682f4@localhost> Hi all, I tried to add this little nothing to the Wiki, put the thing insists on complaining about "illegal external urls" or something, and I seem to be too stupid to figure out how to avoid that... Anyway, if you are currently using the ProxyLike user script, the following bookmarklet may be for you: javascript:location.href='http://localhost:37004/'+location.href I'm one lazy bastard, and clicking this is much less work than manually prefixing the URL. Maybe someone finds this useful... Cheers, Michael -- web at citizen428.net citizen428 at gentoo.org http://citizen428.net/ http://dev.gentoo.org/~citizen428/ GnuPG key: 0x90CA09E3/4D21 916E DBCE 72B8 CDC5 BD87 DE2D 91A2 90CA 09E3 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051101/d272ce3a/attachment.bin From daniels at pronto.com.au Thu Nov 3 21:46:56 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Fri, 4 Nov 2005 13:46:56 +1100 Subject: Caching that hoodwink'd mousehole Message-ID: Because my connection is often slowed to a crawl (stupid australian broadband limits), I've put in a cache for hoodwink'd settings, so you make a hell of a lot less calls to the hoodwink'd server. Bit messy, but this way you don't have to go messing with anything else in your hoodwink.user.rb file. def json_read( url, meth ) if meth == :setup obj = @db["cache:#{url}"] if obj last_update = @db["last-update:#{url}"] if Time.now - last_update > 24*60*60 obj = nil end end unless obj p "Going to #{url}" obj = open( url ) do |f| JSON::Lexer.new( f.read ).nextvalue end @db["cache:#{url}"] = obj @db["last-update:#{url}"] = Time.now end method( meth ).call( obj ) if obj else open( url ) do |f| obj = JSON::Lexer.new( f.read ).nextvalue method( meth ).call( obj ) if obj end end end I'd probably suggest that the json_read method return the obj instead of passing in a symbol.... unless there's a reason why the connection isn't closed until after the method is called? I also threw in a couple of "raise MouseHole::CancelRewrite" lines elsewhere to speed it up/stop messing with non-wink'd pages. ##################################################################################### This email has been scanned by MailMarshal, an email content filter. ##################################################################################### -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051104/998d0bed/attachment-0001.htm From why at hobix.com Tue Nov 8 12:01:27 2005 From: why at hobix.com (why the lucky stiff) Date: Tue, 08 Nov 2005 10:01:27 -0700 Subject: Comparison between Monkeygrease and MouseHole Message-ID: <4370D9E7.30005@hobix.com> A new Greasemonkey-style proxy was released yesterday. It's written in Java. Here's an overview I wrote up for the Greasemonkey list: - Monkeygrease injects HTML elements into four possible positions on a page (top and bottom of or top and bottom of ). MouseHole hands the user an XHTML object which can be freely transformed. - Monkeygrease user scripts are written in XML. MouseHole scripts can be written in Ruby or Javascript (GM-style). - Monkeygrease requires Java and a configured web application server. MouseHole requires Ruby and leverages its own personal web server written in Ruby. However, MouseHole's Windows installer has no prerequisites. - Monkeygrease scripts must be added to an XML file by hand and requires a reboot. MouseHole presents the user with an installation window when scripts are encountered on the web. - Monkeygrease URL matching is done by hand in the XML config. MouseHole URL matching is done in the browser using an interface similar to GM. - Monkeygrease offers no database API like GM does (GM_setValue, GM_getValue). MouseHole offers each script a tiny, transparent object database. (i.e. myScript.db['lastVisited'] = Time.now.) - I'm unsure about Monkeygrease's ability to do cross-domain XMLHttpRequest. I can't see a way to do this. I know MH and GM can both do this (i.e. BookBurro, you're on amazon.com and the user script is given permission to hit bn.com, powells.com, etc.) - MouseHole also lets user scripts mount their own tiny web applications. This is a huge advantage. For example, there is a MouseWiki, an Instiki-clone which mounts at http://mouse.hole/wiki. A better example is MouseCommand, which is a wiki for storing small Ruby scripts which can be execute via URL. Think YubNub. _why From mental at rydia.net Tue Nov 8 14:58:05 2005 From: mental at rydia.net (mental@rydia.net) Date: Tue, 08 Nov 2005 14:58:05 -0500 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <4370D9E7.30005@hobix.com> References: <4370D9E7.30005@hobix.com> Message-ID: <1131479885.4371034d5bd60@www.rydia.net> Quoting why the lucky stiff : > A new Greasemonkey-style proxy was released yesterday. It's > written in Java. > - Monkeygrease scripts must be added to an XML file by hand and > requires a reboot. MouseHole presents the user with an > installation window when scripts are encountered on the web. Naturally, the only thing to do is to add support for Monkeygrease scripts to mouseHole. I mean, how else are Monkeygrease script writers to test their scripts? Any doco on Monkeygrease XML scripts floating about? -mental From why at hobix.com Tue Nov 8 15:25:43 2005 From: why at hobix.com (why the lucky stiff) Date: Tue, 08 Nov 2005 13:25:43 -0700 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <1131479885.4371034d5bd60@www.rydia.net> References: <4370D9E7.30005@hobix.com> <1131479885.4371034d5bd60@www.rydia.net> Message-ID: <437109C7.4060208@hobix.com> mental at rydia.net wrote: >Any doco on Monkeygrease XML scripts floating about? > > Check this: http://monkeygrease.org/documentation _why From mental at rydia.net Tue Nov 8 15:37:59 2005 From: mental at rydia.net (mental@rydia.net) Date: Tue, 08 Nov 2005 15:37:59 -0500 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <437109C7.4060208@hobix.com> References: <4370D9E7.30005@hobix.com> <1131479885.4371034d5bd60@www.rydia.net> <437109C7.4060208@hobix.com> Message-ID: <1131482279.43710ca760453@www.rydia.net> Quoting why the lucky stiff : > mental at rydia.net wrote: > > >Any doco on Monkeygrease XML scripts floating about? > > > Check this: http://monkeygrease.org/documentation Hot noodles. Looks simple. Should our monkey-greasing engine look for a monkeygrease.xml in the mouseHole scripts directory, or let it break into separate *.mg files, or what? Also ... hmm. How do we confine monkeygrease rules to specific domains? My first thought is to add an XML namespace for mousehole, and annotate each monkeygrease rule with a mh:domain attribute. Got better ones? -mental From why at hobix.com Tue Nov 8 16:01:49 2005 From: why at hobix.com (why the lucky stiff) Date: Tue, 08 Nov 2005 14:01:49 -0700 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <1131482279.43710ca760453@www.rydia.net> References: <4370D9E7.30005@hobix.com> <1131479885.4371034d5bd60@www.rydia.net> <437109C7.4060208@hobix.com> <1131482279.43710ca760453@www.rydia.net> Message-ID: <4371123D.4080600@hobix.com> mental at rydia.net wrote: >Also ... hmm. How do we confine monkeygrease rules to specific >domains? My first thought is to add an XML namespace for >mousehole, and annotate each monkeygrease rule with a mh:domain >attribute. Got better ones? > > Well domain rules are going to be in the web-based config anyway. So I guess we could just start out with empty rules and leave it up to the user to add those. It might even make sense to have the MG files mount. Since they're supposed to act like hosted web apps anyway. _why From why at hobix.com Tue Nov 8 16:01:49 2005 From: why at hobix.com (why the lucky stiff) Date: Tue, 08 Nov 2005 14:01:49 -0700 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <1131482279.43710ca760453@www.rydia.net> References: <4370D9E7.30005@hobix.com> <1131479885.4371034d5bd60@www.rydia.net> <437109C7.4060208@hobix.com> <1131482279.43710ca760453@www.rydia.net> Message-ID: <4371123D.4080600@hobix.com> mental at rydia.net wrote: >Also ... hmm. How do we confine monkeygrease rules to specific >domains? My first thought is to add an XML namespace for >mousehole, and annotate each monkeygrease rule with a mh:domain >attribute. Got better ones? > > Well domain rules are going to be in the web-based config anyway. So I guess we could just start out with empty rules and leave it up to the user to add those. It might even make sense to have the MG files mount. Since they're supposed to act like hosted web apps anyway. _why From why at hobix.com Tue Nov 8 18:02:45 2005 From: why at hobix.com (why the lucky stiff) Date: Tue, 08 Nov 2005 16:02:45 -0700 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <1131482279.43710ca760453@www.rydia.net> References: <4370D9E7.30005@hobix.com> <1131479885.4371034d5bd60@www.rydia.net> <437109C7.4060208@hobix.com> <1131482279.43710ca760453@www.rydia.net> Message-ID: <43712E95.8000403@hobix.com> mental at rydia.net wrote: >Also ... hmm. How do we confine monkeygrease rules to specific >domains? My first thought is to add an XML namespace for >mousehole, and annotate each monkeygrease rule with a mh:domain >attribute. Got better ones? > Maybe these rule files should correspond to a mount. Since the whole idea behind Monkeygrease is to use the ruleset as an independent web application. So we offer the 'domain' and 'mount' settings in the configuration. _why From mental at rydia.net Tue Nov 8 18:12:22 2005 From: mental at rydia.net (mental@rydia.net) Date: Tue, 08 Nov 2005 18:12:22 -0500 Subject: Comparison between Monkeygrease and MouseHole In-Reply-To: <43712E95.8000403@hobix.com> References: <4370D9E7.30005@hobix.com> <1131479885.4371034d5bd60@www.rydia.net> <437109C7.4060208@hobix.com> <1131482279.43710ca760453@www.rydia.net> <43712E95.8000403@hobix.com> Message-ID: <1131491542.437130d677f59@www.rydia.net> Quoting why the lucky stiff : > Maybe these rule files should correspond to a mount. Since the > whole idea behind Monkeygrease is to use the ruleset as an > independent web application. So we offer the 'domain' and > 'mount' settings in the configuration. Hmm, yes. Yes, this sounds good. -mental From mental at rydia.net Tue Nov 8 23:51:42 2005 From: mental at rydia.net (MenTaLguY) Date: Tue, 08 Nov 2005 23:51:42 -0500 Subject: HOSTS in CVS Message-ID: <1131511902.6830.30.camel@localhost.localdomain> Just a quick note that I've updated the IPs for hoodwink.d and that underground, underscored, place in CVS's mouseHole.rb. But are they used now? I'd tend to think they would have gotten changed sooner if they were... or is everyone still Greasemonkeying their way through life? We have streaming, people. (Granted, I'm not one to talk; I haven't touched mouseHole for a while, until just now.) -mental -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051108/6dd1e976/attachment.bin From daniels at pronto.com.au Wed Nov 9 17:39:11 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Thu, 10 Nov 2005 09:39:11 +1100 Subject: HOSTS in CVS Message-ID: I've been mouseholing it up constantly... but can't get to the CVS from behind a firewall, so just assumed that the CVS is up to date and patched my own. I actually had to take out the host entries, as I was having some sort of trouble with my upstream proxy and hoodwink.d - but I can't remember what it was now - I should probably reproduce it. I've been mainly using it for hoodwinking - I patched the hoodwink.user.rb to cache the calls to the server, and also for some RSS rewriting goodness. I'm planning on, in the near future: - adding a mountpoint to the hoodwink.user.rb and/or a new script to add some personal winking for: - personal comments - playing around with xpaths+regex before submitting it to hoodwink.d. - make a user script to add information about winks to my RSS feeds. - this will require changes to mousehole to provide an ordering of scripts, as I'll need to guarentee this runs AFTER AtomicMouse. - profit That near future however, was also the near future about a month ago... so horses must be held. > -----Original Message----- > From: mousehole-scripters-bounces at rubyforge.org > [mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf > Of MenTaLguY > Sent: Wednesday, 9 November 2005 3:52 PM > To: mousehole > Subject: HOSTS in CVS > > Just a quick note that I've updated the IPs for hoodwink.d > and that underground, underscored, place in CVS's mouseHole.rb. > > But are they used now? I'd tend to think they would have > gotten changed sooner if they were... or is everyone still > Greasemonkeying their way through life? We have streaming, people. > > (Granted, I'm not one to talk; I haven't touched mouseHole > for a while, until just now.) > > -mental > > ##################################################################################### This email has been scanned by MailMarshal, an email content filter. ##################################################################################### From why at hobix.com Thu Nov 10 00:23:20 2005 From: why at hobix.com (why the lucky stiff) Date: Wed, 09 Nov 2005 22:23:20 -0700 Subject: Keep-alive Message-ID: <4372D948.2030104@hobix.com> If you've been running CVS, you'll probably want to update. I've turned Keep-Alive on between the proxy and the browser. And killed keep-alives from the proxy to the world. (This should solve a lot of the churning-churning-churning without progress.) I also shortened the timeout. Connections will timeout after like twenty seconds without activity. Keep-alive on outgoing connections could buy us some speed as well. I'm gonna take a punch at this under lighttpd. _why From mental at rydia.net Sat Nov 12 15:20:32 2005 From: mental at rydia.net (MenTaLguY) Date: Sat, 12 Nov 2005 15:20:32 -0500 Subject: monkeygrease thoughts Message-ID: <1131826832.6830.97.camel@localhost.localdomain> Okay, I've been poking at this for a bit now, and it's looking like monkeygrease should be handled a bit differently from greasemonkey scripts (a la starmonkey). Basically, monkeygrease has a totally different use model from greasemonkey. So we've got to do things different for monkeygrease users. I think the right way forward is going to be just having a monkeygrease.user.rb which keeps sets of monkeygrease rules for different hosts/domains in its conf database and can import monkeygrease.xml. Do we even need to integrate it with mouseHole, really? Tho it would be nice to support monkeygrease rules out of the box, just becauses. -mental -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051112/c3be0056/attachment-0001.bin From why at hobix.com Wed Nov 16 13:57:04 2005 From: why at hobix.com (why the lucky stiff) Date: Wed, 16 Nov 2005 11:57:04 -0700 Subject: MouseHole repository moved Message-ID: <437B8100.1090309@hobix.com> I've been getting increasing resistance from helpers who prefer Subversion over CVS. I feel that it's slowing down development. So, done. Mailing lists, releases, news will still be at Rubyforge. Bug tracking will likely move to the new repository soon as well. The new repository is at: http://code.whytheluckystiff.net/ Checkout MouseHole from http://code.whytheluckystiff.net/svn/mouseHole/trunk I'm sorry but no one has commit rights at the moment. I'm working on a means of using your Rubyforge account through the new repo. I'll let you know. Thanks, people. _why From mental at rydia.net Wed Nov 16 15:11:52 2005 From: mental at rydia.net (mental@rydia.net) Date: Wed, 16 Nov 2005 15:11:52 -0500 Subject: repository moved to SVN In-Reply-To: <437B806E.3080208@hobix.com> References: <437B806E.3080208@hobix.com> Message-ID: <1132171912.437b92885b8b0@www.rydia.net> Quoting why the lucky stiff : > I've been getting increasing resistance from helpers who prefer > Subversion over CVS. I feel that it's slowing down development. > So, done. It's worth noting that if anyone was hoping for a more bitkeeper/monotone/git-style tool, that is available in the form of SVK (http://svk.elixus.org). Disconnected operation, local repositories, all that. But it is a normal Subversion client as far as the server and other SVN users are concerned. So, everybody wins. -mental From kevin at sb.org Fri Nov 18 00:09:04 2005 From: kevin at sb.org (Kevin Ballard) Date: Thu, 17 Nov 2005 21:09:04 -0800 Subject: MouseHole broken under lighttpd Message-ID: I seem to recall having this same issue about a month ago, but I've been offline for 3 weeks so I haven't touched MouseHole in a while. In any case, whenever I run ./mouseHole -s lighttpd, it treats every single domain as if it was localhost:37004, i.e. going to http:// www.google.com gives me the MouseHole start page, and http:// redhanded.hobix.com/inspect/some-blog-post gives me an error about no handler for `inspect'. Anybody know what's going on? This is a fresh checkout of MouseHole from subversion, at r54 (current HEAD). I have a self-built version of ruby 1.8.3 and a DarwinPorts-built version of lighttpd v1.4.4 installed. -- Kevin Ballard kevin at sb.org http://www.tildesoft.com http://kevin.sb.org -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2378 bytes Desc: not available Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051117/44f70035/smime.bin From why at hobix.com Fri Nov 18 12:39:58 2005 From: why at hobix.com (why the lucky stiff) Date: Fri, 18 Nov 2005 10:39:58 -0700 Subject: MouseHole repository moved In-Reply-To: <437B8100.1090309@hobix.com> References: <437B8100.1090309@hobix.com> Message-ID: <437E11EE.1030409@hobix.com> why the lucky stiff wrote: >The new repository is at: http://code.whytheluckystiff.net/ > > There is now a feed for MouseHole changes! http://code.whytheluckystiff.net/mouseHole.xml Overall discussion of the repository changes here: http://redhanded.hobix.com/inspect/myRepositoryTransplant.html _why From daniels at pronto.com.au Sun Nov 20 19:33:15 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Mon, 21 Nov 2005 11:33:15 +1100 Subject: hoodwinkd.user.rb Message-ID: Now that I've started to take more of an interesting in hoodwinking, I've noticed a bit of a problem with the way that the mousehole script works. When you visit a page that's been wink'd, if the underlying page has not changed, odds are that the server is going to be returning a 304 response - since the page is only rewritten on a 200 response, this means that new winks aren't injected into the page unless you do the old shift-refresh in the browser. Also, pages will often set an expires header, in which case the proxy never hears about requests until that date is passed. Should the hoodwink script mess with the expires header? Should it be able to intercept 304 responses and head off to hoodwink'd to make sure that the comments haven't changed either (in which case a second request needs to be made without the if-modified header so the whole page can be rewritten? Alternatively, should mousehole start behaving as a caching webserver, obeying the 304 responses and the expires header keeping the content of modified pages around so that they can be rewritten constantly - the expires header should be set to expire immediately for the browser, so that it will always call the proxy for the content. But whether this is desirable for more static scripts (things like the mailto: link script) - those pages should only change when their underlying content changes. But I'm sure if there's a dying need to keep generated pages cached in the browser, it would be easy enough to chuck in a flag to say that certain scripts shouldn't be constantly re-calculated. This would also be useful for doing things where you need to grab content off the web - the AtomicMouse script could take advantage of the cache to perform real caching of the linked web pages so that it doesn't need to store the content in the script's settings (with the problem that changes to the web page will not be picked up). It would be great for any script that wants to pull further information off the net. ##################################################################################### This email has been scanned by MailMarshal, an email content filter. ##################################################################################### -------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051121/c0537c84/attachment.htm From why at hobix.com Mon Nov 21 01:46:54 2005 From: why at hobix.com (why the lucky stiff) Date: Sun, 20 Nov 2005 23:46:54 -0700 Subject: hoodwinkd.user.rb In-Reply-To: References: Message-ID: <43816D5E.5090708@hobix.com> Daniel Sheppard wrote: > Also, pages will often set an expires header, in which case the proxy > never hears about requests until that date is passed. > MouseHole should be stripping the Expires, is it not? MouseHole does cache when an ETag is seen, but it has no support for Last Modified or Expires. (Soon.) > Should the hoodwink script mess with the expires header? > MouseHole can. It can fool the browser as it needs. I don't know about "should." I never know the answer to shoulds! > Alternatively, should mousehole start behaving as a caching webserver, > obeying the 304 responses and the expires header keeping the content > of modified pages around so that they can be rewritten constantly - > the expires header should be set to expire immediately for the > browser, so that it will always call the proxy for the content. > The check_cache and no_cache methods in MouseHole::ProxyServer are designed to do exactly that. > But whether this is desirable for more static scripts (things like the > mailto: link script) - those pages should only change when their > underlying content changes. But I'm sure if there's a dying need to > keep generated pages cached in the browser, it would be easy enough to > chuck in a flag to say that certain scripts shouldn't be constantly > re-calculated. > Sure. Some kind of heuristic that says, "If the resources haven't changed and the database hasn't changed, then go braindead." You're doing good, Shep. I'm working on RedCloth and Hobix right now, I'll be back to MouseHole in the coming days to make these changes. _why From daniels at pronto.com.au Mon Nov 21 23:40:29 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Tue, 22 Nov 2005 15:40:29 +1100 Subject: hoodwinkd.user.rb Message-ID: > > Alternatively, should mousehole start behaving as a caching > webserver, > > obeying the 304 responses and the expires header keeping > the content > > of modified pages around so that they can be rewritten constantly - > > the expires header should be set to expire immediately for the > > browser, so that it will always call the proxy for the content. > > > The check_cache and no_cache methods in > MouseHole::ProxyServer are designed to do exactly that. Hmmm... looking at the code, I see how it stops some caching, but mainly it's just concerned with the mousehole content. It doesn't, however modify the 'Expires' header on responses to rewritten requests. This means the browser won't make a call back to mousehole if, for example, you make a hoodwink post on a page with an Expires date in the future. The code in the prewink method appears to be ensuring 304's only if the page HAS been rewritten by mousehole?? Or am I reading the code wrong? What I'm suggesting is that mousehole constantly tell the browser that every page (with a mimetype that mousehole understands, or maybe just everything that is rewritten... not sure yet) is not cachable, and is changed each and every time it is asked for. It then performs it's own caching of those pages internally, sending if-modified-since to the server and obeying 304s and Expires headers. Possibly even hacking this down to the Net::HTTP level so that all requests get passed through the cache, even those as a result of a call from within a script. Note that I'm looking a snapshot that's over a month old - I was having trouble getting through the firewall for CVS and haven't gotten my hands on a new copy since the repository change-over. Hope to get some mousehole hacking on later in the week.... or later in the month/year/decade.... ##################################################################################### This email has been scanned by MailMarshal, an email content filter. ##################################################################################### From mental at rydia.net Tue Nov 22 10:01:24 2005 From: mental at rydia.net (MenTaLguY) Date: Tue, 22 Nov 2005 10:01:24 -0500 Subject: hoodwinkd.user.rb In-Reply-To: References: Message-ID: <1132671684.6830.201.camel@localhost.localdomain> On Tue, 2005-11-22 at 15:40 +1100, Daniel Sheppard wrote: > What I'm suggesting is that mousehole constantly tell the browser that > every page (with a mimetype that mousehole understands, or maybe just > everything that is rewritten... not sure yet) is not cachable, and is > changed each and every time it is asked for. For the pages that mousehole _doesn't_ rewrite, the HTTP specification says what to do -- just do that (it's what any "real" proxy does) and it should work for those cases. And anyway, if you don't do that you're violating the spec. I'm not sure what the best option for the rewritten stuff would be. Ideally, that would depend on the particular userscript. Some scripts may rewrite in a fashion which won't change unless the underlying page(s) change; others may change whenever. -mental -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20051122/01de1c1c/attachment.bin From daniels at pronto.com.au Tue Nov 22 17:31:12 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Wed, 23 Nov 2005 09:31:12 +1100 Subject: hoodwinkd.user.rb Message-ID: > On Tue, 2005-11-22 at 15:40 +1100, Daniel Sheppard wrote: > > What I'm suggesting is that mousehole constantly tell the > browser that > > every page (with a mimetype that mousehole understands, or > maybe just > > everything that is rewritten... not sure yet) is not > cachable, and is > > changed each and every time it is asked for. > > For the pages that mousehole _doesn't_ rewrite, the HTTP > specification says what to do -- just do that (it's what any > "real" proxy does) and it should work for those cases. And > anyway, if you don't do that you're violating the spec. Well, I think one of us doesn't understand the spec (or I don't understand what you're saying) AFIAK, it's perfectly legitimate to have a proxy which performs no caching at all, and it's perfectly legitimate to have a proxy that caches everything that is not specifically marked as not cachable. I'm pretty sure I know how to configure apache to do both of those things. ##################################################################################### This email has been scanned by MailMarshal, an email content filter. ##################################################################################### From mental at rydia.net Tue Nov 22 17:50:07 2005 From: mental at rydia.net (mental@rydia.net) Date: Tue, 22 Nov 2005 17:50:07 -0500 Subject: hoodwinkd.user.rb In-Reply-To: References: Message-ID: <1132699807.4383a09f5f325@www.rydia.net> Quoting Daniel Sheppard : > Well, I think one of us doesn't understand the spec (or I don't > understand what you're saying) Maybe both. I'd just woken up when I wrote that, sorry. -mental From why at hobix.com Tue Nov 22 18:34:41 2005 From: why at hobix.com (why the lucky stiff) Date: Tue, 22 Nov 2005 16:34:41 -0700 Subject: Caching in MH (was Re: hoodwinkd.user.rb) In-Reply-To: References: Message-ID: <4383AB11.9010304@hobix.com> Daniel Sheppard wrote: > AFIAK, it's perfectly legitimate to have a proxy which performs no > caching at all, and it's perfectly legitimate to have a proxy that > caches everything that is not specifically marked as not cachable. I'm > pretty sure I know how to configure apache to do both of those things. > I agree with mental that we shouldn't infere with caching of non-rewritten pages. And Daniel's point is that we can cache the rest of these pages in many situations. The current caching code should work like this (from an email to this list, dated 9/14/05): MouseHole basically adds a "MH-" prefix to the Etags. So if proxylike.user.js normally has an Etag of "7a7b949d311cc51:119e", MouseHole's version of it is called "MH-7a7b949d311cc51:119e". This means that as you switch between the proxy and the browser, you should be able to separately pull cached pages for each. This only happens for pages which list 'text/html' in the 'Accept' header. I think we're okay on this part. MouseHole manages its own cache of pages just like the browser would if it were there supervising. Sounds like I just need to crack open LiveHTTPHeaders again and see what's going on with the status codes. The biggest problem I foresee is caching on a shared proxy. If Louise is logged in and pulls http://private-page.com/, which gets cached and rewritten, we can't have Peter Stockingtail popping in on her session just because they're sharing a MouseHole installation. _why From daniels at pronto.com.au Tue Nov 22 19:40:44 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Wed, 23 Nov 2005 11:40:44 +1100 Subject: Caching in MH (was Re: hoodwinkd.user.rb) Message-ID: > The biggest problem I foresee is caching on a shared proxy. > If Louise is logged in and pulls http://private-page.com/, > which gets cached and rewritten, we can't have Peter > Stockingtail popping in on her session just because they're > sharing a MouseHole installation. If we start caching things, we need to start obeying the Cache-Control response header. It is the responsibility of the web application to send a Cache-Control header of either no-cache or private when the pages are private, unless the request included Authorization headers, in which case it is only cachable with an explicit public Cache-Control header - in all those cases Mousehole should perform no caching. Some applications may send Expires or Pragma: No-Cache headers instead of Cache-Control headers, because Cache-Control was only included in the HTTP 1.1 specification, so potentially a proxy server may not recognise it. This is probably all much bigger than MouseHole - if I stop spending all my time writing about things and actually start doing some things any time soon, I'll throw together a quick library that wraps and/or modifies Net:HTTP to turn it into a caching client. I don't imagine it being too difficult to get something up and running.... (famous last words). ##################################################################################### This email has been scanned by MailMarshal, an email content filter. ##################################################################################### From daniels at pronto.com.au Tue Nov 22 20:56:48 2005 From: daniels at pronto.com.au (Daniel Sheppard) Date: Wed, 23 Nov 2005 12:56:48 +1100 Subject: Caching in MH (was Re: hoodwinkd.user.rb) Message-ID: Here's a skeleton that just through the hoops of turning caching on/off. Now I just need to actually make it do something. It's a giant mess - I can't think of a decent interface for this considering that we need to replace the HTTP#request method, and also the HTTPIO#request method and they all need to use the same cache. I was considering making the cache a module and doing some aliasing in an append_features method, having the caching work on singleton methods and then including the module in both HTTP and HTTPIO - it'd work for mousehole, but I don't think it would be very flexible for use in the big wide world. So it's messy messy messy. If you can think of better suggestions, feel free to pipe-up. In the meantime, I'll work on getting get_cached_response and save_in_cache doing something meaningful. require 'net/http' module Net class HTTPCache def get_cached_response(req, body = nil, &block) p "getting cached response" end def save_in_cache(req, res) p "saving in cache" end def request(req, body = nil, &block) res = get_cached_response(req, body, &block) unless(res) res = non_cached_request(req, body, &block) save_in_cache(req, res) end res end def use_always(clazz) return if clazz.method_defined? :stop_caching cache = self clazz.class_eval { alias :non_cached_request :request define_method(:cache) { cache } def request(req, body = nil, &block) res = cache.get_cached_response(req, body, &block) unless(res) res = non_cached_request(req, body, &block) cache.save_in_cache(req, res) end res end def self.stop_caching class_eval {alias :request :non_cached_request} remove_method :non_cached_request remove_method :cache class << self remove_method :stop_caching end end def stop_caching class << self; stop_caching; end end } end def use(http) cache = self cache.use_always(class << http; self; end) end end end if $0 == __FILE__ cache = Net::HTTPCache.new Net::HTTP.start('localhost', 80) {|http| p "trying a cached response" cache.use(http) http.get('/index.html') p "trying a non cached response" http.stop_caching http.get('/index.html') } p "trying a non cached response" Net::HTTP.start('localhost', 80) {|http| http.get('/index.html') } p "trying a cached response" cache.use_always(Net::HTTP) Net::HTTP.start('localhost', 80) {|http| http.get('/index.html') } p "trying a non cached response" Net::HTTP.stop_caching Net::HTTP.start('localhost', 80) {|http| http.get('/index.html') } end ##################################################################################### This email has been scanned by MailMarshal, an email content filter. #####################################################################################