From why at hobix.com  Fri Sep  2 14:57:32 2005
From: why at hobix.com (why the lucky stiff)
Date: Fri Sep  2 14:50:44 2005
Subject: ! MouseHole 1.1 released
Message-ID: <4318A09C.1040906@hobix.com>

A walkthrough: 
http://redhanded.hobix.com/inspect/mousehole11InPlainView.html

Links from the bottom:

    * Download MouseHole 1.1:
          o Windows standalone.
            <http://rubyforge.org/frs/download.php/5867/mouseHole-1.1-win32-standalone.zip>
            (No Ruby required.)
          o or Source zip.
            <http://rubyforge.org/frs/download.php/5868/mouseHole-1.1.zip>
            (Ruby, yaml, dbm, rexml libraries required.)
          o or Source tarball.
            <http://rubyforge.org/frs/download.php/5869/mouseHole-1.1.tar.gz>
            (Ruby, yaml, dbm, rexml libraries required.)
    * My own MouseHole scripts. <http://whytheluckystiff.net/mouseHole/>
    * The MouseHole wiki. <http://mousehole.rubyforge.org>
    * Mailing list is mousehole-scripters
      <http://rubyforge.org/mailman/listinfo/mousehole-scripters>

_why
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050902/14752696/attachment.htm
From jake at bitmechanic.com  Sat Sep  3 23:11:09 2005
From: jake at bitmechanic.com (Jake Donham)
Date: Sat Sep  3 22:59:47 2005
Subject: wikish
Message-ID: <200509040311.j843BAmv006373@sparkplug.bitmechanic.com>

Hi,

A couple hours' hack: a wiki (based on why's) with a "run" action which
runs the page as Ruby code:

  http://jaked.org/wikish.user.rb

(You shouldn't put this on a MouseHole that is publically accessible,
since it has no protection of any kind at the moment.)

The idea is that it is a web-based shell like YubNub, but you can
execute arbitrary code. There is no input form at the moment--I'm using
it by setting keyword.URL to http://127.0.0.1:37004/wikish/run/ from
about:config in Firefox so I can type commands at the URL bar.

This doesn't have anything particularly to do with proxying (neither
does why's wiki) but if you are already running a personal server, that
seems like the right place to put it.

So far I haven't done anything except redirections. Here are a few:

del: (change username to taste)

  response['location'] = "http://del.icio.us/jaked/#{args.join('+')}"
  raise WEBrick::HTTPStatus::Found

amaz:

  response['location'] = "http://www.amazon.com/exec/obidos/external-search?mode=blended&field-keywords=#{args.join('+')}"
  raise WEBrick::HTTPStatus::Found

edit: (so you can say e.g. "edit amaz")

  response['location'] = "http://127.0.0.1:37004/wikish/edit/#{args[0]}"
  raise WEBrick::HTTPStatus::Found

Some ideas for the future:

  security
  more shell syntax--variables, pipes? or more ruby-ish? see scsh
  call code on a wiki page from another page's code--wiki page defines a subroutine
  OS auth so you can look at your files etc
  wiki-ish editing of OS files
  other web interfaces to things on the OS
  commands to control MouseHole--enable/disable scripts etc.
  forward commands to a different wikish or to an OS command shell
  convenience methods for things like redirections

Jake
From why at hobix.com  Mon Sep  5 02:09:29 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 02:03:13 2005
Subject: ! MouseHole 1.1 Windows Installer
Message-ID: <431BE119.1030502@hobix.com>

Okay, I've got an installer up for Windows now.  I'm also working on a 
generic gem, a win32 gem and a mac osx dmg installer.  Once I've got 
Rake building all of these, we'll have a good smorgasbord of possible 
installers to give everyone their options. 
<http://rubyforge.org/frs/download.php/5901/mouseHole-1.1-win32.exe>

_why

From why at hobix.com  Mon Sep  5 02:19:00 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 02:12:37 2005
Subject: wikish
In-Reply-To: <200509040311.j843BAmv006373@sparkplug.bitmechanic.com>
References: <200509040311.j843BAmv006373@sparkplug.bitmechanic.com>
Message-ID: <431BE354.1020503@hobix.com>

Jake Donham wrote:

>The idea is that it is a web-based shell like YubNub, but you can
>execute arbitrary code. There is no input form at the moment--I'm using
>it by setting keyword.URL to http://127.0.0.1:37004/wikish/run/ from
>about:config in Firefox so I can type commands at the URL bar.
>
Incredibly cool idea!  People are going to love this.  The cool thing 
about having a wiki behind it: all your commands are versioned and 
attributed.

This makes me wonder if the wiki's internals should be distilled and 
built into MouseHole's API.  You don't need RedCloth support I guess.

_why

From why at hobix.com  Mon Sep  5 02:31:33 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 02:25:09 2005
Subject: Security concerns
Message-ID: <431BE645.10408@hobix.com>

I'm really concerned about MouseHole's security.  To a degree, user 
scripts are inherently granted permissions and open the user to attack 
(just as a downloaded executable can do any damage it pleases.)

Anyway, here are the security concerns that need to be addressed:

* Installed user scripts: are they allowed to do anything at all once 
the user has authorized installation?  Or should they be forced to state 
their intentions? "Hi, I require at least $SAFE = 2!"
* Uninstalled user scripts which are loaded for metadata scanning: are 
allowed to do nothing at all!
* Javascript access to MouseHole: A MouseHole script injects Javascript 
into a page and that Javascript needs to store data back inside 
MouseHole.  Uhh, this exposes MouseHole to possible attack from other 
Javascripts.  What APIs do we allow Javascript to access.  (You don't 
want a rogue Javascript deactivating all your scripts or wiping out your 
configurations, right?)
* Javascript cross-site access: A MouseHole script injects Javascript 
into a page and that Javascript needs to pass data to another domain 
(normally a JS failure.)
** So, if we're on boingboing and we want to access metafilter.. 
http://boingboing.net/http://metafilter.com/api/post.  That opens a 
security hole (the same one Greasemonkey just bumped up on.)
** Instead, we could have the MouseHole script register URLs before 
hand: register_url("http://metafilter.com/api/post")
** Then, the script is given a hash to access that domain.  (Or is 
allowed access to it through 
http://boingboing.net/http://metafilter.com/api/post.)
** Or, just a whitelist with regexps.

I'm sure there are other security issues.  We'll start with these, though.

_why

From kevin at sb.org  Sun Sep  4 05:11:54 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sun Sep  4 05:05:08 2005
Subject: New filtering in MouseHole
Message-ID: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>

Ok, I've hacked on MouseHole a bit to make it possible for JavaScript  
to have access to MouseHole. I've done it in a fairly restrictive  
manner. Basically, I've made it possible for a script to handle any  
URL whose path starts with /.mouseHole/, matching based on domain and  
path. Because of naming issues, I decided to rename rewrite to filter  
and call this new feature rewrite (after all, the existing rewrite is  
really just filtering). But to preserve compatibility I've made  
rewrite with no args act like the new filter.

How you use the new feature is you simply add a call to rewrite(). It  
takes 2 args. The first is a regex to match against host:port, so you  
might use /^(www\.)?google\.com:80/ to match against google. The  
second is a regex to match against the path after /.mouseHole/, so to  
match http://www.google.com/.mouseHole/ping you would use %r{^/ping 
$}. On a request to the proxy, if the request method is GET/POST/PUT/ 
HEAD and the path starts with /.mouseHole/, it will match against all  
existing rewriting, and the first one it finds that works it calls.  
The rewrite block should set both req['Content-Type'] and req.body to  
appropriate values. req['Content-Length'] will be filled in for you  
based on req.body. The scripts are matched in whatever order  
each_fresh_script uses and the individual rewrites will be matched in  
order of definition. I would recommend that the path match be  
something fairly unique, so multiple generic scripts (i.e. ones not  
limited to a specific domain) don't try to use the same path. I  
thought about sticking a script identifier into the matched path, but  
I wasn't sure what to use.

The reason I restricted the rewriting to /.mouseHole/ is because it  
makes writing scripts a bit easier (so you don't end up trying to  
rewrite way too many pages when you meant to rewrite a specific page)  
and faster (because it doesn't have to match all scripts for every  
request). Also because this feature is meant primarily for JavaScript  
XMLHttpRequest calls, although you could use this to toss in a  
<script> or <img> tag or the like.

The patch can be found at <http://kevin.sb.org/files/ 
mousehole-1.patch>. It was created against CVS TOT right now (check  
the email time if you care).

An example script is the following. It adds a new button to google's  
index page that simply asks MouseHole for a random number. If you  
don't care, do keep reading. I talk more after the script.

---------------
MouseHole.script do
   # declaration
   name "Rewriting Test"
   namespace "kevin@sb.org"
   description "Tests the new rewriting stuff."
   include_match %r{^http://(www\.)?google\.com(/(index\.html)?)?$}
   version "0.1"

   filter do |req, res|
     document.elements['//head'].add_element 'script', 'type' =>  
'text/javascript', 'src' => '/.mouseHole/test.js'
     lucky = document.elements['//input[@name="btnI"]']
     lucky.parent.add_element 'input', 'type' => 'submit', 'value' =>  
'Random', 'onclick' => 'pingMouseHole(); return false;'
   end

   rewrite(%r{^(www\.)?google\.com:80$}, %r{^/test\.js$}) do |uri,  
req, res|
     res['Content-Type'] = 'text/javascript'
     res.body = <<-EOF
function createRequestObject() {
   var ro;
   var browser = navigator.appName;
   if(browser == "Microsoft Internet Explorer"){
     ro = new ActiveXObject("Microsoft.XMLHTTP");
   }else{
     ro = new XMLHttpRequest();
   }
   return ro;
}

var http = createRequestObject();

function sndReq(action, handler) {
   http.open('get', action);
   http.onreadystatechange = function() {
     if(http.readyState == 4){
       handler(http.responseText);
     }
   };
   http.send(null);
}

function pingMouseHole() {
   sndReq('/.mouseHole/ping', function(txt) {
     alert(txt);
   })
}
     EOF
   end

   rewrite(%r{^(www\.)google\.com:80$}, %r{^/ping$}) do |uri, req, res|
     res['Content-Type'] = 'text/plain'
     res.body = uri.to_s + ": " + rand(100).to_s
   end
end
-------------------

On Sep 5, 2005, at 2:31 AM, why the lucky stiff wrote:

> * Javascript access to MouseHole: A MouseHole script injects  
> Javascript into a page and that Javascript needs to store data back  
> inside MouseHole.  Uhh, this exposes MouseHole to possible attack  
> from other Javascripts.  What APIs do we allow Javascript to  
> access.  (You don't want a rogue Javascript deactivating all your  
> scripts or wiping out your configurations, right?)

Well, this provides the requested functionality. The only APIs  
accessible from javascript are the ones that your user scripts  
provide, so as long as they behave appropriately, there should be no  
problem. Feel free to add some $SAFE magick'ry if you wish around  
this, I don't have any experience twiddling $SAFE so I just ignored it.

> * Javascript cross-site access: A MouseHole script injects  
> Javascript into a page and that Javascript needs to pass data to  
> another domain (normally a JS failure.)
> ** So, if we're on boingboing and we want to access metafilter..  
> http://boingboing.net/http://metafilter.com/api/post.  That opens a  
> security hole (the same one Greasemonkey just bumped up on.)
> ** Instead, we could have the MouseHole script register URLs before  
> hand: register_url("http://metafilter.com/api/post")
> ** Then, the script is given a hash to access that domain.  (Or is  
> allowed access to it through http://boingboing.net/http:// 
> metafilter.com/api/post.)
> ** Or, just a whitelist with regexps.

With this new filtering, you just ask MouseHole to do the cross- 
domain request. Only vulnerabilities are again what the script itself  
provides.

Thoughts or comments?

Oh, and I don't suppose MouseHole could be hosted in an SVN repo  
instead of a CVS repo? I'm using SVK to wrap it (because I hate using  
CVS), but it's slow to sync changes in the repo.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050904/e8086156/smime.bin
From kevin at sb.org  Sun Sep  4 05:14:02 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sun Sep  4 05:07:13 2005
Subject: IRC channel
Message-ID: <E90EC779-919D-410D-A7B7-F57080E1183F@sb.org>

I registered #mousehole on freenode (irc://irc.freenode.net/ 
#mousehole) if anybody wants to chat in real-time.

Hopefully MouseHole will be come the Next Big Thing? and the channel  
will become popular ;) If not, I guess I'll just discard it.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050904/7b07e546/smime-0001.bin
From why at hobix.com  Sun Sep  4 14:59:08 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 14:52:45 2005
Subject: New filtering in MouseHole
In-Reply-To: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>
References: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>
Message-ID: <431B43FC.6030001@hobix.com>

Kevin Ballard wrote:

> How you use the new feature is you simply add a call to rewrite(). It  
> takes 2 args. The first is a regex to match against host:port, so you  
> might use /^(www\.)?google\.com:80/ to match against google. The  
> second is a regex to match against the path after /.mouseHole/, so to  
> match http://www.google.com/.mouseHole/ping you would use %r{^/ping $}.

Great, thankyou for the patch.  This is lovely.  Do you mind if I mix 
this with the register_url idea?

For a foreign URL:

  register_url( "http://google.com/test.js" ) do |req, res| 
    # no `uri' arg needed, we'll do magic to make the req.request_uri 
work fine.
  end

For accessing MouseHole directly:

  register_url( "/wiki" ) do |req, res| 
  end

We could normalize the ports and subdomains from inside MouseHole.

> Oh, and I don't suppose MouseHole could be hosted in an SVN repo  
> instead of a CVS repo? I'm using SVK to wrap it (because I hate using  
> CVS), but it's slow to sync changes in the repo.

Well, I'd like to stick with Rubyforge.  I'm pretty impartial on 
repository software.  They'll switch to SVN soon I'm sure.

_why

From kevin at sb.org  Sun Sep  4 15:41:32 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sun Sep  4 15:34:41 2005
Subject: New filtering in MouseHole
In-Reply-To: <431B43FC.6030001@hobix.com>
References: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>
	<431B43FC.6030001@hobix.com>
Message-ID: <DB3B907B-6826-481E-B163-DFCC6A387862@sb.org>

That won't work if you have a fairly generic script. What if I mutate  
all sites to throw a link in the upper-right that, say, pops open a  
box with input fields to let me post a note to my Backpack pages from  
any site? That would require handling // for domain and %r{/ 
backpack_ping/} for path. That can't be done as a single string  
register, and doing it as a regex register_url is no "better" than my  
approach. Although I would like to make req.request_uri be proper and  
drop the uri arg to the block - I didn't in my patch because  
request_uri is read-only and I didn't really feel like extending the  
class to get access to it, but it's probably appropriate here.

On Sep 4, 2005, at 2:59 PM, why the lucky stiff wrote:

> Great, thankyou for the patch.  This is lovely.  Do you mind if I  
> mix this with the register_url idea?
>
> For a foreign URL:
>
>  register_url( "http://google.com/test.js" ) do |req, res|    # no  
> `uri' arg needed, we'll do magic to make the req.request_uri work  
> fine.
>  end
>
> For accessing MouseHole directly:
>
>  register_url( "/wiki" ) do |req, res|  end
>
> We could normalize the ports and subdomains from inside MouseHole.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050904/c640e045/smime.bin
From kevin at sb.org  Sun Sep  4 15:53:45 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sun Sep  4 15:46:58 2005
Subject: Script configuration
Message-ID: <48219A97-6081-4832-8C08-A3685A77073C@sb.org>

When poking around in MouseHole I saw that scripts can define a block  
that gets inserted into the page for the script in MouseHole, for use  
in configuring the script. However, there's no good way to actually  
pass the user's chosen configuration back to the script, except for  
mounting a path and using JavaScript XMLHttpRequest to ping that  
path. And that doesn't seem like the best way IMHO.

I was thinking of adding a generic server_config method that took a  
script identifier (which requires giving all scripts identifiers,  
although that might be a good idea to prevent conflicts in my new URL  
rewriting as well) and name=value pairs, and it would pass them back  
to the script (as a hash) to do with however it pleased. And we'd  
also provide a JavaScript method that the configure block can use to  
easily send the config data back to MouseHole via XMLHttpRequest.

Any thoughts?

Oh, and on this subject, the main config for MouseHole has a problem.  
Since it uses a single XMLHttpRequest object for all communication,  
if I click a checkbox twice rapidly, the first toggle never manages  
to get sent to MouseHole, and the current visual state of the  
checkbox is now out of sync with the actual config value (as  
reloading the page will show). I think we need to make a new  
XMLHttpRequest object per message, but we don't want to store them  
forever. I would have done it but I don't know if it's safe to get  
rid of our reference to the request object after handling the result  
text (perhaps it has to clean up the connection?). Yes, JS uses a  
garbage collector, so the object wouldn't disappear immediately, but  
it would as soon as the GC is run. Oh, and by safe I really just mean  
nice and clean - dropping the object wouldn't damage anything, but I  
don't like to keep it from cleaning up the connection. Anybody know  
more about this?

Hrm, I just had a thought. In terms of config, we could actually  
handle sending the config info back to the script ourselves so the  
script just has to output a block of HTML. What we can do is slap a  
Submit button on the end and wrap it all in a form tag and throw an  
onclick on the submit that sends the form data back with  
XMLHttpRequest. Perhaps I should go do that now - we only need a  
single XMLHttpRequest object for this, because it's only triggered  
when clicking the submit button, so the above issue doesn't apply.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050904/f5e4a006/smime.bin
From why at hobix.com  Sun Sep  4 17:16:46 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 17:10:20 2005
Subject: Speeding up MouseHole
Message-ID: <431B643E.1010408@hobix.com>

Okay, I've had some good success speeding things up.  I've attached a 
benchmark script I'm playing with.  You can take a look at that later.

For now, here's my benchmarks.  50 iterations cleaning Boing Boing's 
home page, parsing it and building the HTML string again.  (I use Boing 
Boing since it's riddled with heinous Javascript and unclosed tags and 
stuff.)  This is on my Pentium-M 1.6GHz laptop on Gentoo Linux.

                        user     system      total        real
tidy+rexml:        19.010000   0.030000  19.040000 ( 20.467225) * 1.2 *
cached tidy+rexml: 19.580000   0.020000  19.600000 ( 20.823490)
htree+rexml:       48.630000   0.030000  48.660000 ( 51.742457) * 1.1 *
htree only:        14.490000   0.010000  14.500000 ( 15.717621)
tidy+smart:         3.150000   0.010000   3.160000 (  3.217260) * 2.0 *

The quickest combination is Tidy <http://tidy.rubyforge.org> for 
cleaning the HTML and XML-Smart 
<http://raa.ruby-lang.org/project/ruby-xml-smart/> for parsing the XML, 
altering it, displaying.  Tidy is a Ruby/DL wrapper for a C extension.  
XML-Smart is a C extension.

I'd like to move to Tidy and away from HTree.  Replacing HTree with Tidy 
will roughly improve speed by a factor of 3.  There are binaries for 
most operating systems: <http://tidy.sourceforge.net/>

I'd like to move to XML-Smart in time as well.  It's faster than REXML 
by a factor of 5 to 6 generally.  However, we'd have to fill in some 
gaps in the API, it's still beta software, and there's no binaries 
readily available.  If we can solve these problems, then I think we're 
looking at MouseHole 2.0.

So, for now, I'm checking support for Tidy, but holding off on other XML 
libs.  I've looked at XML-Simple, libxml bindings, etc.  Nothing else 
has both the speed and the XPath support, ease of scripting.

_why
From why at hobix.com  Sun Sep  4 17:26:59 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 17:20:32 2005
Subject: New filtering in MouseHole
In-Reply-To: <DB3B907B-6826-481E-B163-DFCC6A387862@sb.org>
References: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>	<431B43FC.6030001@hobix.com>
	<DB3B907B-6826-481E-B163-DFCC6A387862@sb.org>
Message-ID: <431B66A3.1080704@hobix.com>

Kevin Ballard wrote:

> That can't be done as a single string  register, and doing it as a 
> regex register_url is no "better" than my  approach.

Well, it's consistent with include_match and exclude_match.  I want all 
URL matching schemes to be identical.

So, let's say down the road I entertain an idea for per-part matching:

  include_match :scheme => "file"
  include_match :scheme => /https?/
  include_match :domain => /google\.com$/, :path => "user.js"
  exclude_match :scheme => "http", :domain => /boingboing/

If I add that, I want it to be consistent for exclude_match and 
register_url as well.  Consistency and ease of remembering MouseHole's 
API is paramount.

_why
From why at hobix.com  Sun Sep  4 17:28:53 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep  4 17:22:27 2005
Subject: Speeding up MouseHole (ATTACHMENT)
In-Reply-To: <431B643E.1010408@hobix.com>
References: <431B643E.1010408@hobix.com>
Message-ID: <431B6715.8000309@hobix.com>

why the lucky stiff wrote:

> Okay, I've had some good success speeding things up.  I've attached a 
> benchmark script I'm playing with.  You can take a look at that later.

And here is the script.  Happy.

_why
-------------- next part --------------
require './lib/mouseHole'
require 'htree'
require 'xml/smart'
require 'benchmark'

puts Benchmark::CAPTION

Tidy.path = '/usr/lib/libtidy.so'

xml = ""
test_html = "bb.html"
n = 50
Benchmark.bm do |x|
    x.report "tidy+rexml:" do
        html = File.read( test_html )
        n.times do
            Tidy.open(:output_xhtml=>true) do |tidy|
                xml = REXML::Document.new(tidy.clean html)
            end
            xml = xml.to_s
        end
    end

    File.open( "tidy+rexml-#{ test_html }", 'w' ) do |f|
        f << xml
    end

    x.report "cached tidy+rexml:" do
        html = File.read( "tidy+rexml-#{ test_html }" )
        n.times do
            xml = REXML::Document.new html
            xml = xml.to_s
        end
    end

    x.report "htree+rexml:" do
        html = File.read test_html
        n.times do
            xml = nil
            HTree.parse( html ).each_child do |child|
                if child.respond_to? :qualified_name
                    if child.qualified_name == 'html'
                        xml = HTree::Doc.new( child ).to_rexml
                        break
                    end
                end
            end
            xml = xml.to_s
        end
    end

    File.open( "htree+rexml-#{ test_html }", 'w' ) do |f|
        f << xml
    end

    x.report "htree only:" do
        html = File.read test_html
        n.times do
            xml = HTree.parse( html )
        end
        xml = xml.to_s
    end

    File.open( "htree-#{ test_html }", 'w' ) do |f|
        f << xml
    end

    x.report "tidy+smart:" do
        html = File.read test_html
        n.times do
            Tidy.open(:output_xhtml=>true) do |tidy|
                xml = XML::Smart.string(tidy.clean html)
            end
        end
    end

    File.open( "tidy+smart-#{ test_html }", 'w' ) do |f|
        f << xml
    end
end

From kevin at sb.org  Sun Sep  4 18:10:17 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sun Sep  4 18:03:32 2005
Subject: New filtering in MouseHole
In-Reply-To: <431B66A3.1080704@hobix.com>
References: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>	<431B43FC.6030001@hobix.com>
	<DB3B907B-6826-481E-B163-DFCC6A387862@sb.org>
	<431B66A3.1080704@hobix.com>
Message-ID: <57DC1C94-AACB-4DC5-A38B-B6C3FF238E41@sb.org>

That's actually a really good idea. It could take a single argument - 
a regex matches against the whole URL, a string gets compared to the  
URL (as opposed to translating it to a regex), and a hash gets  
compared to the URL components.

I'll work on that now, and post a revised patch that makes those  
changes (as well as reverts filter back to rewrite and turns my new  
rewrite to register_url).

On Sep 4, 2005, at 5:26 PM, why the lucky stiff wrote:

> Well, it's consistent with include_match and exclude_match.  I want  
> all URL matching schemes to be identical.
>
> So, let's say down the road I entertain an idea for per-part matching:
>
>  include_match :scheme => "file"
>  include_match :scheme => /https?/
>  include_match :domain => /google\.com$/, :path => "user.js"
>  exclude_match :scheme => "http", :domain => /boingboing/
>
> If I add that, I want it to be consistent for exclude_match and  
> register_url as well.  Consistency and ease of remembering  
> MouseHole's API is paramount.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050904/5cb5d9f9/smime-0001.bin
From tomato at WPI.EDU  Mon Sep  5 11:50:10 2005
From: tomato at WPI.EDU (Christopher T. Osborn)
Date: Mon Sep  5 13:09:14 2005
Subject: configure documentation
Message-ID: <op.swmzlwbkxfx8e4@rikku.res.wpi.net>

Is there a chance that someone who knows can write some more documentation for the configure method?  I can make it (based on one of _why's examples) show checkboxes on my script's configuration page, but how to use a mountpoint to save information completely escapes me.

-- 
Christopher T. Osborn  <tomato@wpi.edu>
From why at hobix.com  Mon Sep  5 16:36:34 2005
From: why at hobix.com (why the lucky stiff)
Date: Mon Sep  5 16:30:06 2005
Subject: MouseHole RSS feed
Message-ID: <431CAC52.8050905@hobix.com>

I've checked in some code this morning for an RSS feed available from 
the proxy.

  <http://127.0.0.1:37004/mouseHole/rss>

The feed list the configuration page and, if available, the mount point 
for each plugin.  In Firefox, you can visit <http://127.0.0.1:37004/> 
and the browser will give you a little radar symbol in the lower 
right-hand of the status bar.  Click on it, a menu will appear, select 
"Subscribe to RSS..."  I put the bookmark in my toolbar.  This gives an 
actively updated list of your user scripts from the toolbar.

_why
From why at hobix.com  Tue Sep  6 18:06:42 2005
From: why at hobix.com (why the lucky stiff)
Date: Tue Sep  6 17:59:47 2005
Subject: ProxyLike 0.2
Message-ID: <431E12F2.2090702@hobix.com>

ProxyLike has been updated to rewrite links on a filtered page to use 
ProxyLike as well.  
<http://whytheluckystiff.net/mouseHole/proxylike.user.rb>

For now, only links which are inside a filtered page's domain are rewritten.

_why
From bg-ruby at infofiend.com  Tue Sep  6 18:39:52 2005
From: bg-ruby at infofiend.com (Ben Giddings)
Date: Tue Sep  6 18:33:03 2005
Subject: What am I looking at here?
Message-ID: <200509061839.52344.bg-ruby@infofiend.com>

Ok, I managed a nice little mousehole script, but doing it brought up all 
kinds of questions.

Here's the script:

MouseHole.script do
  # declaration
  name "Google Ad Scrubber"
  namespace "bg-ruby@infofiend.com"
  description "Removes ads from Google search results"

  # Match google search results
  include_match %r{^http://(www\.)?google.com/search}
  version "0.1"

  rewrite do |request, response|
    right_sponsored_links = document.elements['//table[@width="25%" and 
@align="right"]']

    if right_sponsored_links
      right_sponsored_links.parent.delete(right_sponsored_links)
    end

    loop do
      top_sponsored_links = document.elements['//div[@class="ch"]']
      break if top_sponsored_links.nil?
      top_sponsored_links.parent.delete(top_sponsored_links)
    end
  end
end

And here are the questions:

* Does it have to be named foo.user.rb to be recognized when I put it in 
the userScripts directory?
* What if I want it to match every URL, is there a good way to do that?
* What is "document"?  The wiki docs say it's an HTree, but when I click on 
that link I see nothing named "elements".  Primitive debugging seems to 
suggest it's a REXML::Document...?
* XPath seems to say that '//div[@class="ch"]' should give me all the div 
nodes with a class "ch", but document.elements[XPATH] seems to only give 
me one, so I have to loop

If you can help me understand this stuff, I'll try to update the wiki dox.

Ben

P.S. Right now the proxy job in my web browser is being handled by Privoxy, 
but I think with a little work I can get Mousehole to do all that and 
more...
From grant at antiflux.org  Tue Sep  6 19:03:49 2005
From: grant at antiflux.org (Grant Hollingworth)
Date: Tue Sep  6 18:56:52 2005
Subject: ProxyLike 0.2
In-Reply-To: <431E12F2.2090702@hobix.com>
References: <431E12F2.2090702@hobix.com>
Message-ID: <20050906230349.GA27605@okcomputer.antiflux.org>

* why the lucky stiff <why@hobix.com> [2005-09-06 16:06]:
> ProxyLike has been updated to rewrite links on a filtered page to use 
> ProxyLike as well.  

Excellent... that's what I was thinking of when I wrote Hoodwink'd
Rewrite (now obsolete).

ProxyLike says:
  ERROR NameError: uninitialized constant MouseHole::MOUSEHOST
Is defining MOUSEHOST and MOUSEPORT enough, or are there other gotchas
for using this in 1.1?
From grant at antiflux.org  Tue Sep  6 19:06:14 2005
From: grant at antiflux.org (Grant Hollingworth)
Date: Tue Sep  6 18:59:16 2005
Subject: ProxyLike 0.2
In-Reply-To: <20050906230349.GA27605@okcomputer.antiflux.org>
References: <431E12F2.2090702@hobix.com>
	<20050906230349.GA27605@okcomputer.antiflux.org>
Message-ID: <20050906230614.GB27605@okcomputer.antiflux.org>

* Grant Hollingworth <grant@antiflux.org> [2005-09-06 17:03]:
> Excellent... that's what I was thinking of when I wrote Hoodwink'd
> Rewrite (now obsolete).
           ^^^^^^^^^^^^
Actually, no, I was getting ahead of myself.  Not until (if) ProxyLike
rewrites offsite links.
From kevin at sb.org  Tue Sep  6 20:20:29 2005
From: kevin at sb.org (Kevin Ballard)
Date: Tue Sep  6 20:13:36 2005
Subject: What am I looking at here?
In-Reply-To: <200509061839.52344.bg-ruby@infofiend.com>
References: <200509061839.52344.bg-ruby@infofiend.com>
Message-ID: <FEF08CA7-E761-4F8E-972D-AD4CE0DAA53E@sb.org>

On Sep 6, 2005, at 6:39 PM, Ben Giddings wrote:

> * Does it have to be named foo.user.rb to be recognized when I put  
> it in
> the userScripts directory?

Yes.

> * What if I want it to match every URL, is there a good way to do  
> that?

include_match %r{} should work.

> * What is "document"?  The wiki docs say it's an HTree, but when I  
> click on
> that link I see nothing named "elements".  Primitive debugging  
> seems to
> suggest it's a REXML::Document...?

I believe HTree is some sort of wrapper around REXML::Document, or at  
the very least forwards its messages on to that.

> * XPath seems to say that '//div[@class="ch"]' should give me all  
> the div
> nodes with a class "ch", but document.elements[XPATH] seems to only  
> give
> me one, so I have to loop

document.elements[XPATH] fetches a single element. You want  
document.get_elements[XPATH] to fetch all, or (and this should be a  
little faster) document.each_element(XPATH) { |e| } or  
document.elements.each(XPATH) { |e| }.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050906/b85bfc26/smime.bin
From kevin at sb.org  Wed Sep  7 01:41:46 2005
From: kevin at sb.org (Kevin Ballard)
Date: Wed Sep  7 01:34:53 2005
Subject: Make Tidy work on OS X
Message-ID: <AF18443E-EF5A-4367-984A-D01AD710ED1B@sb.org>

Here's a small patch that makes the Tidy detection work under OS X  
(where I have a libtidy.dylib rather than libtidy.so):

=== lib/mouseHole.rb
==================================================================
--- lib/mouseHole.rb    (revision 24)
+++ lib/mouseHole.rb    (revision 26)
@@ -841,8 +841,11 @@
          libdirs = ['/usr/lib', '/usr/local/lib'] + $:
          libdirs << File.dirname( RUBYSCRIPT2EXE_APPEXE ) if  
defined? RUBYSCRIPT2EXE_APPEXE
          libdirs.each do |libdir|
-            libtidy = File.join( libdir, "libtidy.#{ Config::CONFIG 
['arch'] =~ /win32/ ? 'dll' : 'so' }" )
-            if File.exists? libtidy
+            libtidies = ['so']
+            libtidies.unshift 'dll' if Config::CONFIG['arch'] =~ / 
win32/
+            libtidies.unshift 'dylib' if Config::CONFIG['arch'] =~ / 
darwin/
+            libtidies.collect! { |lib| File.join( libdir, "libtidy.# 
{lib}") }
+            if libtidy = libtidies.find { |lib| File.exists? lib }
                  puts "Found Tidy! #{ libtidy }"
                  require 'tidy'
                  require 'htree/htmlinfo'


-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050907/971afee2/smime.bin
From why at hobix.com  Wed Sep  7 02:12:19 2005
From: why at hobix.com (why the lucky stiff)
Date: Wed Sep  7 02:05:48 2005
Subject: Make Tidy work on OS X
In-Reply-To: <AF18443E-EF5A-4367-984A-D01AD710ED1B@sb.org>
References: <AF18443E-EF5A-4367-984A-D01AD710ED1B@sb.org>
Message-ID: <431E84C3.20404@hobix.com>

Kevin Ballard wrote:

> Here's a small patch that makes the Tidy detection work under OS X  
> (where I have a libtidy.dylib rather than libtidy.so):

Merged into CVS.  Thanks, Kevin!

I've also just included mfp's patch that was causing winks to never ever 
appear sometimes.

_why
From why at hobix.com  Wed Sep  7 14:37:09 2005
From: why at hobix.com (why the lucky stiff)
Date: Wed Sep  7 14:30:12 2005
Subject: configure documentation
In-Reply-To: <op.swmzlwbkxfx8e4@rikku.res.wpi.net>
References: <op.swmzlwbkxfx8e4@rikku.res.wpi.net>
Message-ID: <431F3355.8050401@hobix.com>

Christopher T. Osborn wrote:

> Is there a chance that someone who knows can write some more 
> documentation for the configure method?  I can make it (based on one 
> of _why's examples) show checkboxes on my script's configuration page, 
> but how to use a mountpoint to save information completely escapes me.

Well, there's a couple ways to do this.

1) Ajax + mount.  MouseHole has the simple sendReq function you can use 
to communicate setting alterations.  If your script mounts a directory, 
pass information to the mount.  (You'll probably want to clearly mark 
that these are updated without submission.)

2) The HTTPRequest#query object.  If you have form fields in your 
configure block, the submitted contents will be stored in the 
WEBrick::HTTPRequest#query object.  (The second parameter coming into 
the object.)

  configure do |req, res|
    if req.request_method == "POST"
      self['username'] = req.query['username']
    end
    %{<input type="text" name="username" value="#{ self['username'] }" />}
  end

Your script has its own object database (using YAML::DBM).  You can 
stuff anything you like in the database by using self[]=.  Read it back 
out with self[].

_why
From kevin at sb.org  Wed Sep  7 20:02:42 2005
From: kevin at sb.org (Kevin Ballard)
Date: Wed Sep  7 19:55:48 2005
Subject: configure documentation
In-Reply-To: <431F3355.8050401@hobix.com>
References: <op.swmzlwbkxfx8e4@rikku.res.wpi.net> <431F3355.8050401@hobix.com>
Message-ID: <68E22B39-4850-4E08-AE10-220D003C2118@sb.org>

I'd just like to mention that on my to-do list (something I hope to  
clear up within the next few days) is a way to make doing script  
configuration much easier. That said, nothing is certain, so feel  
free to implement it with the current functionality.

On Sep 7, 2005, at 2:37 PM, why the lucky stiff wrote:

> 1) Ajax + mount.  MouseHole has the simple sendReq function you can  
> use to communicate setting alterations.  If your script mounts a  
> directory, pass information to the mount.  (You'll probably want to  
> clearly mark that these are updated without submission.)
>
> 2) The HTTPRequest#query object.  If you have form fields in your  
> configure block, the submitted contents will be stored in the  
> WEBrick::HTTPRequest#query object.  (The second parameter coming  
> into the object.)
>
>  configure do |req, res|
>    if req.request_method == "POST"
>      self['username'] = req.query['username']
>    end
>    %{<input type="text" name="username" value="#{ self 
> ['username'] }" />}
>  end
>
> Your script has its own object database (using YAML::DBM).  You can  
> stuff anything you like in the database by using self[]=.  Read it  
> back out with self[].

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050907/cc26887a/smime.bin
From tomato at WPI.EDU  Wed Sep  7 23:38:12 2005
From: tomato at WPI.EDU (Christopher T. Osborn)
Date: Wed Sep  7 23:31:12 2005
Subject: configure documentation
In-Reply-To: <431F3355.8050401@hobix.com>
References: <op.swmzlwbkxfx8e4@rikku.res.wpi.net> <431F3355.8050401@hobix.com>
Message-ID: <op.swrlpycjxfx8e4@rikku.res.wpi.net>

Well, would it be possible for someone to tell me what I'm doing wrong with configure in this code?

http://users.wpi.edu/~tomato/linkd.user.rb

I'm afraid I have little HTML experience, so that may be why I can't wrap my head around it.


-- 
Christopher T. Osborn  <tomato@wpi.edu>
From kevin at sb.org  Thu Sep  8 13:28:40 2005
From: kevin at sb.org (Kevin Ballard)
Date: Thu Sep  8 13:21:45 2005
Subject: configure documentation
In-Reply-To: <op.swrlpycjxfx8e4@rikku.res.wpi.net>
References: <op.swmzlwbkxfx8e4@rikku.res.wpi.net> <431F3355.8050401@hobix.com>
	<op.swrlpycjxfx8e4@rikku.res.wpi.net>
Message-ID: <EE1DF439-30F0-4AB8-A2FF-62B8FBA53028@sb.org>

Well, you have no form or XMLHttpRequest, so the configuration data  
is not being passed back. Plus, you're checking for any given  
configuration data in your configure method, which is not where you  
will see it. You need to register for a mount and check for  
configuration there.

I'd give you more information as to how to pass the data to the  
mount, but I really have to be going. Sorry.

On Sep 7, 2005, at 11:38 PM, Christopher T. Osborn wrote:

> Well, would it be possible for someone to tell me what I'm doing  
> wrong with configure in this code?
>
> http://users.wpi.edu/~tomato/linkd.user.rb
>
> I'm afraid I have little HTML experience, so that may be why I  
> can't wrap my head around it.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050908/5425ceda/smime.bin
From kevin at sb.org  Thu Sep  8 22:23:52 2005
From: kevin at sb.org (Kevin Ballard)
Date: Thu Sep  8 22:16:55 2005
Subject: The Underground Blog and URI hacking
Message-ID: <82C50F7B-901B-4A3E-8AA6-15CEEF0461AC@sb.org>

The Underground Blog cannot be accessed via MouseHole, because  
URI.parse() doesn't accept ___._ as a valid hostname.

Here's a patch that hacks URI to allow non-standard hosts. Applying  
this allows me to access ___._ via MouseHole.

Note: I added an entry for ___._ to ::HOSTS just for kicks. After  
looking at it, it seems that the entry for hoodwink.d isn't doing  
anything (AFAICT ::HOSTS is only used for mounts), so adding an entry  
for ___._ is useless, but I did it anyway. Or am I missing something  
about ::HOSTS?

=== mouseHole
==================================================================
--- mouseHole    (revision 28)
+++ mouseHole    (revision 29)
@@ -1,10 +1,12 @@
#!/usr/bin/env ruby
require './lib/mouseHole'
+require './lib/uri-hack'
MOUSEHOST = ARGV[0] || "127.0.0.1"
MOUSEPORT = ARGV[1] || 37004
::HOSTS = Hash[ *%W[
      hoodwink.d  65.125.236.166
+    ___._       65.125.236.166
      mouse.hole  #{ MOUSEHOST }:#{ MOUSEPORT }
      mh          #{ MOUSEHOST }:#{ MOUSEPORT }
] ]
=== lib/uri-hack.rb
==================================================================
--- lib/uri-hack.rb    (revision 28)
+++ lib/uri-hack.rb    (revision 29)
@@ -0,0 +1,18 @@
+# Hacks URI to allow a host of ___._
+# Actually, it basically just hacks URI to allow any host for HTTP URLs
+# but it does it in somewhat of a roundabout way
+
+URI::HTTP.class_eval {
+  # Allow http URIs to accept a registry field
+  def self.use_registry
+    true
+  end
+
+  # If we get a nil host, set the host to the registry.
+  # This allows us to accept non-standard hosts, like ___._
+  def initialize(*args)
+    super(*args)
+    self.set_host(self.registry) if self.host.nil?
+    self.set_registry(nil)
+  end
+}

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050908/6fe06e22/smime.bin
From why at hobix.com  Thu Sep  8 22:29:51 2005
From: why at hobix.com (why the lucky stiff)
Date: Thu Sep  8 22:23:16 2005
Subject: The Underground Blog and URI hacking
In-Reply-To: <82C50F7B-901B-4A3E-8AA6-15CEEF0461AC@sb.org>
References: <82C50F7B-901B-4A3E-8AA6-15CEEF0461AC@sb.org>
Message-ID: <4320F39F.50907@hobix.com>

Kevin Ballard wrote:

> Note: I added an entry for ___._ to ::HOSTS just for kicks. After  
> looking at it, it seems that the entry for hoodwink.d isn't doing  
> anything (AFAICT ::HOSTS is only used for mounts), so adding an entry  
> for ___._ is useless, but I did it anyway. Or am I missing something  
> about ::HOSTS?

Yeah.  See dnshack.rb.

Excellent patch.  Now that's going in!

_why

From kevin at sb.org  Thu Sep  8 23:39:23 2005
From: kevin at sb.org (Kevin Ballard)
Date: Thu Sep  8 23:32:25 2005
Subject: The Underground Blog and URI hacking
In-Reply-To: <4320F39F.50907@hobix.com>
References: <82C50F7B-901B-4A3E-8AA6-15CEEF0461AC@sb.org>
	<4320F39F.50907@hobix.com>
Message-ID: <5FABCAAA-9AAB-4F7C-81D6-48686736A9F2@sb.org>

On Sep 8, 2005, at 10:29 PM, why the lucky stiff wrote:

> Yeah.  See dnshack.rb.

Ah hah.

> Excellent patch.  Now that's going in!

Glad to hear it :)

Now that I think about it some more, this hack isn't going to work if  
you pass a port number in (i.e. http://___._:8080/). Unfortunately I  
can't think of a fix, because this is a limitation in the regular  
expressions (it simply won't match, because _ isn't valid in a host  
and : isn't valid in a registry name). The only thing that would work  
is to completely re-define the regular expressions, which would be  
pretty ugly and fragile. However, right now that's not an issue, so  
as long as you don't try to run any services on a non-standard port  
from ___._, it should be ok.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050908/e9f87d22/smime.bin
From James at grayproductions.net  Fri Sep  9 16:34:52 2005
From: James at grayproductions.net (James Edward Gray II)
Date: Fri Sep  9 16:27:51 2005
Subject: Hoodwink'd Rewrite Question
Message-ID: <DD483AC4-E0A8-4C72-B28E-9E4EA2E7DD47@grayproductions.net>

Finally getting into playing around with all this Top Secret  
technology.  I have mouseHole installed and am using it with the  
ProxyLike script (I view Redhanded at http://127.0.0.1:37004/http:// 
redhanded.hobix.com/, for example.)

I can now see the hidden winking, but I cannot add my own.  Pushing  
the submit button on the form gives me a non-proxied link, which  
fails.  I do have Hoodwink'd-rewrite installed.  Any tips?

Thanks.

James Edward Gray II
From mental at rydia.net  Fri Sep  9 17:16:08 2005
From: mental at rydia.net (mental@rydia.net)
Date: Fri Sep  9 17:09:06 2005
Subject: Hoodwink'd Rewrite Question
In-Reply-To: <DD483AC4-E0A8-4C72-B28E-9E4EA2E7DD47@grayproductions.net>
References: <DD483AC4-E0A8-4C72-B28E-9E4EA2E7DD47@grayproductions.net>
Message-ID: <1126300568.4321fb98a84d9@www.rydia.net>

Quoting James Edward Gray II <James@grayproductions.net>:

> I can now see the hidden winking, but I cannot add my own.
> Pushing the submit button on the form gives me a non-proxied link,
> which fails.  I do have Hoodwink'd-rewrite installed.  Any tips?

It's likely a case of ProxyLike not rewriting the form submission
URL...

-mental
From mental at rydia.net  Fri Sep  9 17:24:21 2005
From: mental at rydia.net (mental@rydia.net)
Date: Fri Sep  9 17:17:22 2005
Subject: Hoodwink'd Rewrite Question
In-Reply-To: <1126300568.4321fb98a84d9@www.rydia.net>
References: <DD483AC4-E0A8-4C72-B28E-9E4EA2E7DD47@grayproductions.net>
	<1126300568.4321fb98a84d9@www.rydia.net>
Message-ID: <1126301061.4321fd8528d40@www.rydia.net>

I've not tested it, but try changing the first ProxyLike regexp,
from:

/(href\s*=\s*["']?)(#{ uri }|\/+)/

to:

/((?:href|action|src)\s*=\s*["']?)(#{ uri }|\/+)/

That ought to fill in the various gaps.

-mental
From james at grayproductions.net  Fri Sep  9 22:05:28 2005
From: james at grayproductions.net (James Edward Gray II)
Date: Fri Sep  9 21:58:27 2005
Subject: Skipping Problem Pages
Message-ID: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>

I went to ProxyLike, because mouseHole was breaking some sites for  
me.  Any way to get it not to process a certain domain?  I tried  
excluding the pages from the user scripts, but that didn't seem to do  
it.

James Edward Gray II

From kevin at sb.org  Fri Sep  9 22:14:57 2005
From: kevin at sb.org (Kevin Ballard)
Date: Fri Sep  9 22:07:57 2005
Subject: New filtering in MouseHole
In-Reply-To: <57DC1C94-AACB-4DC5-A38B-B6C3FF238E41@sb.org>
References: <1A8FD71D-047A-47DF-AA8E-436FA515E1BD@sb.org>	<431B43FC.6030001@hobix.com>
	<DB3B907B-6826-481E-B163-DFCC6A387862@sb.org>
	<431B66A3.1080704@hobix.com>
	<57DC1C94-AACB-4DC5-A38B-B6C3FF238E41@sb.org>
Message-ID: <48D83F1F-7F03-4B8B-847C-14B8C85BB6EA@sb.org>

Ok, I have a new patch that implements this. include/exclude now take  
the same arguments as the new register_url (and the rewrite=>filter  
change has been dropped). The arguments can be a String, which is  
matched against both the entire URL and just the path, a Regexp,  
which is matched against the entire URL (I recommend ending a regex  
with (?:$|\?|#) if you want to anchor it to the end - this way query  
and fragment are irrelevant). It can be a URI, which is matched  
against the entire URL, with caveats: if the matching URI has no  
query, the query is stripped from the target URL (same for fragment),  
so that way query and fragment won't affect URLs unless you want them  
to. And lastly, if the argument is a hash, the values of the hash are  
compared to the named components of the target URL, either as a  
String or a Regexp.

register_url still passes a uri parameter to the associated block -  
you can "fix" that if you want, but I personally see no problem with  
doing that.

The patch can be found at <http://kevin.sb.org/files/mousehole-2.patch>.

On Sep 4, 2005, at 6:10 PM, Kevin Ballard wrote:

> That's actually a really good idea. It could take a single argument  
> -a regex matches against the whole URL, a string gets compared to  
> the URL (as opposed to translating it to a regex), and a hash gets  
> compared to the URL components.
>
> I'll work on that now, and post a revised patch that makes those  
> changes (as well as reverts filter back to rewrite and turns my new  
> rewrite to register_url).
>
> On Sep 4, 2005, at 5:26 PM, why the lucky stiff wrote:
>
>
>> Well, it's consistent with include_match and exclude_match.  I  
>> want all URL matching schemes to be identical.
>>
>> So, let's say down the road I entertain an idea for per-part  
>> matching:
>>
>>  include_match :scheme => "file"
>>  include_match :scheme => /https?/
>>  include_match :domain => /google\.com$/, :path => "user.js"
>>  exclude_match :scheme => "http", :domain => /boingboing/
>>
>> If I add that, I want it to be consistent for exclude_match and  
>> register_url as well.  Consistency and ease of remembering  
>> MouseHole's API is paramount.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050909/b899f73a/smime.bin
From kevin at sb.org  Fri Sep  9 22:15:25 2005
From: kevin at sb.org (Kevin Ballard)
Date: Fri Sep  9 22:08:24 2005
Subject: Skipping Problem Pages
In-Reply-To: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>
References: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>
Message-ID: <A6CFD30F-154D-4055-BA5E-8F5D890F0F1E@sb.org>

What do you mean by breaking?

On Sep 9, 2005, at 10:05 PM, James Edward Gray II wrote:

> I went to ProxyLike, because mouseHole was breaking some sites for  
> me.  Any way to get it not to process a certain domain?  I tried  
> excluding the pages from the user scripts, but that didn't seem to  
> do it.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050909/a65ed1ce/smime.bin
From james at grayproductions.net  Fri Sep  9 23:54:56 2005
From: james at grayproductions.net (James Edward Gray II)
Date: Fri Sep  9 23:47:56 2005
Subject: Skipping Problem Pages
In-Reply-To: <A6CFD30F-154D-4055-BA5E-8F5D890F0F1E@sb.org>
References: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>
	<A6CFD30F-154D-4055-BA5E-8F5D890F0F1E@sb.org>
Message-ID: <70731AAE-6328-4FEE-9E5A-D6D8211500AD@grayproductions.net>

On Sep 9, 2005, at 9:15 PM, Kevin Ballard wrote:

> What do you mean by breaking?

I have a my.yahoo.com page setup the way I like it.  If I activate  
mouseHole, is all gets collapsed into one long, column, instead of  
the typical two column layout.

James Edward Gray II
From kevin at sb.org  Sat Sep 10 00:23:28 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sat Sep 10 00:16:27 2005
Subject: Skipping Problem Pages
In-Reply-To: <70731AAE-6328-4FEE-9E5A-D6D8211500AD@grayproductions.net>
References: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>
	<A6CFD30F-154D-4055-BA5E-8F5D890F0F1E@sb.org>
	<70731AAE-6328-4FEE-9E5A-D6D8211500AD@grayproductions.net>
Message-ID: <B9CEA70A-8CF3-4EDB-B656-A9DA884D807A@sb.org>

On Sep 9, 2005, at 11:54 PM, James Edward Gray II wrote:

> On Sep 9, 2005, at 9:15 PM, Kevin Ballard wrote:
>
>> What do you mean by breaking?
>
> I have a my.yahoo.com page setup the way I like it.  If I activate  
> mouseHole, is all gets collapsed into one long, column, instead of  
> the typical two column layout.

Hrm. Perhaps it's because MouseHole is using Tidy (assuming it can  
find it on your system) and the my.yahoo.com page is so fragile that  
fixing the HTML breaks the layout? Although that seems kinda odd...

Have you tried looking at the source to see what's different?

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050910/962d7425/smime.bin
From swschulz at astrum.com  Sat Sep 10 22:54:54 2005
From: swschulz at astrum.com (S. William Schulz)
Date: Sat Sep 10 22:46:07 2005
Subject: Introduction
Message-ID: <20050911025454.GA27218@draco.astrum.net>

Hello all,

Just wanted to send a quick not as I have recently joined the list after
finding the software based on a referral from the Greasemonkey list.  I
was trying to get GM to filter out applets, and found it would only do
it after the fact.  I quickly threw together a little code for MH to get
it to do the work, and now I'll be playing with other features.

After that, though, I have several questions and a pointer.

First, the pointer, and this follows up on the Skipping Pages thread.
When I was trying to join this list, I noticed I was unable to do so as
Clicking the submit button had no effect.  I turned off the proxy,
reloaded the page, and tried again.  This time successfully.  In the
process, I noticed a slight formatting change, and having read that
previous thread, I noticed that the Tidy function does indeed break the
form for joining this list.  Basically, it adds a closing form tag
before the closing TD tag, thereby breaking the rest of the subscribe
form.  I can provide a diff of the differences if anyone is interested.

Second, the question.  Should clicking the hoodwink.d link lead
somewhere?  Clicking on it either generates a DNS error, or the
following:

Internal Server Error
undefined method `message' for nil:NilClass
WEBrick/1.3.1 (Ruby/1.8.2/2004-12-25) at hoodwink.d:80 

This is the latest release on Windows for now.

Thanks,

SWS

From why at hobix.com  Sun Sep 11 15:51:29 2005
From: why at hobix.com (why the lucky stiff)
Date: Sun Sep 11 15:44:50 2005
Subject: Introduction
In-Reply-To: <20050911025454.GA27218@draco.astrum.net>
References: <20050911025454.GA27218@draco.astrum.net>
Message-ID: <43248AC1.50207@hobix.com>

S. William Schulz wrote:

>Just wanted to send a quick not as I have recently joined the list after
>finding the software based on a referral from the Greasemonkey list.  I
>was trying to get GM to filter out applets, and found it would only do
>it after the fact.  I quickly threw together a little code for MH to get
>it to do the work, and now I'll be playing with other features.
>  
>
Great, glad to have you.  Greasemonkey's obviously a big inspiration for 
MouseHole. I've wanted to discuss the idea on the GM list, but I don't 
really want to interrupt the flow of development they have going there.

>First, the pointer, and this follows up on the Skipping Pages thread.
>When I was trying to join this list, I noticed I was unable to do so as
>Clicking the submit button had no effect.  I turned off the proxy,
>reloaded the page, and tried again.  This time successfully.  In the
>process, I noticed a slight formatting change, and having read that
>previous thread, I noticed that the Tidy function does indeed break the
>form for joining this list.  Basically, it adds a closing form tag
>before the closing TD tag, thereby breaking the rest of the subscribe
>form.  I can provide a diff of the differences if anyone is interested.
>  
>
Sure enough.  I guess we need to move form tags around before processing 
the HTML.

As a general question to the list: are there other HTML tags you've seen 
behave this way?  Tags which should enclose large sections of a page, 
but which are closed early by the tidy-clean-lint stage?

>Second, the question.  Should clicking the hoodwink.d link lead
>somewhere?
>  
>
If you are going through the proxy directly: I believe it should.

_why
From splaestro at gmail.com  Mon Sep 12 02:25:43 2005
From: splaestro at gmail.com (Brian Young)
Date: Mon Sep 12 02:18:35 2005
Subject: Introduction
Message-ID: <9658d5a205091123255b171a89@mail.gmail.com>

swschulz at astrum.com asked:

>>Second, the question.  Should clicking the hoodwink.d link lead
>>somewhere?
>>  

_why answered:

>If you are going through the proxy directly: I believe it should.

>_why

I am encountering the same problem William is having with trying to
install the hoodwink.d script; here is the full stack trace:

[2005-09-12 01:11:09] ERROR NoMethodError: undefined method `message'
for nil:NilClass
        C:/Temp/eee.mouseHole.exe.4/lib/mouseHole.rb:409:in `installer_pane'
        C:/Temp/eee.mouseHole.exe.4/lib/mouseHole.rb:141:in `upwink'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/httpproxy.rb:156:in `call'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/httpproxy.rb:156:in
`proxy_service'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/httpproxy.rb:29:in `service'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/httpserver.rb:65:in `run'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:155:in `start_thread'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:144:in `start'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:144:in `start_thread'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:94:in `start'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:89:in `each'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:89:in `start'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:79:in `start'
        C:/Temp/eee.mouseHole.exe.4/lib/webrick/server.rb:79:in `start'
        C:\Temp\eee.mouseHole.exe.4\app\mouseHole.rb:27
        C:\Temp\eee.mouseHole.exe.4\bootstrap.rb:31:in `load'
        C:\Temp\eee.mouseHole.exe.4\bootstrap.rb:31
localhost - - [12/Sep/2005:01:11:09 US Eastern Standard Time] "GET
http://hoodwink.d/rbx/hoodwinkd.user.rb HTTP/1.1" 500 322

Like William, I am running MH v1.1 under WIndows XP Home (with Firefox).

Also, when I try http://whytheluckystiff.net/mouseHole/wiki.user.rb,
the proxy doesn't bring it up in the installer-- instead, I get the
script as if it were any other document.

Thanks.  Brian

From james at grayproductions.net  Sat Sep 10 13:05:10 2005
From: james at grayproductions.net (James Edward Gray II)
Date: Mon Sep 12 11:39:39 2005
Subject: Skipping Problem Pages
In-Reply-To: <B9CEA70A-8CF3-4EDB-B656-A9DA884D807A@sb.org>
References: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>
	<A6CFD30F-154D-4055-BA5E-8F5D890F0F1E@sb.org>
	<70731AAE-6328-4FEE-9E5A-D6D8211500AD@grayproductions.net>
	<B9CEA70A-8CF3-4EDB-B656-A9DA884D807A@sb.org>
Message-ID: <3B648891-CDA8-42A2-865F-C446EB8759C0@grayproductions.net>

On Sep 9, 2005, at 11:23 PM, Kevin Ballard wrote:


> On Sep 9, 2005, at 11:54 PM, James Edward Gray II wrote:
>
>
>
>> On Sep 9, 2005, at 9:15 PM, Kevin Ballard wrote:
>>
>>
>>
>>> What do you mean by breaking?
>>>
>>>
>>
>> I have a my.yahoo.com page setup the way I like it.  If I activate  
>> mouseHole, is all gets collapsed into one long, column, instead of  
>> the typical two column layout.
>>
>>
>
> Hrm. Perhaps it's because MouseHole is using Tidy (assuming it can  
> find it on your system) and the my.yahoo.com page is so fragile  
> that fixing the HTML breaks the layout? Although that seems kinda  
> odd...
>
> Have you tried looking at the source to see what's different?
>

Nothing is jumping out at me, but it's a huge source.  Here's the  
before and after, for the curious.

James Edward Gray II

-------------- next part --------------
A non-text attachment was scrubbed...
Name: pages.zip
Type: application/zip
Size: 36167 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050910/7a441b3a/pages-0001.zip
From kevin at sb.org  Mon Sep 12 13:22:37 2005
From: kevin at sb.org (Kevin Ballard)
Date: Mon Sep 12 13:15:30 2005
Subject: Skipping Problem Pages
In-Reply-To: <3B648891-CDA8-42A2-865F-C446EB8759C0@grayproductions.net>
References: <E4898F53-F63B-4810-B29B-4639732B62CD@grayproductions.net>
	<A6CFD30F-154D-4055-BA5E-8F5D890F0F1E@sb.org>
	<70731AAE-6328-4FEE-9E5A-D6D8211500AD@grayproductions.net>
	<B9CEA70A-8CF3-4EDB-B656-A9DA884D807A@sb.org>
	<3B648891-CDA8-42A2-865F-C446EB8759C0@grayproductions.net>
Message-ID: <9E02DD76-8A6C-4715-802A-A6E6B135133B@sb.org>

Pop it open in FileMerge and you can see differences. Unfortunately,  
about halfway through it seems to have gotten lost and is now  
comparing sections that appear to not be related.

One interesting bit is a tidbit of CSS that was inside a <noscript>  
block had the <noscript> block removed, so now it affects the page  
all the time.

On Sep 10, 2005, at 1:05 PM, James Edward Gray II wrote:

> Nothing is jumping out at me, but it's a huge source.  Here's the  
> before and after, for the curious.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050912/e6f8db32/smime.bin
From why at hobix.com  Mon Sep 12 13:40:40 2005
From: why at hobix.com (why the lucky stiff)
Date: Mon Sep 12 13:34:01 2005
Subject: Preliminary GM support in CVS
Message-ID: <4325BD98.3020501@hobix.com>

Hi, I've started merging the GM support from Hoodlum (pre-MouseHole) 
into MouseHole.

Basically, if you hit a user.js file, you'll get the same options you 
get with user.rb files.  When a page qualifies for script inject, I'm 
just injecting into a <script> element at the bottom of the page.

None of the GM_API is supported yet.  We shouldn't have any trouble 
supporting all of it.

It's biggest weakness is that it leverages the window.onload handler.  
I'm not sure if leeching the onload off another element or something is 
the way to go yet.  Whatever the case, I want to work on making it 
browser agnostic.

_why
From why at hobix.com  Mon Sep 12 17:11:05 2005
From: why at hobix.com (why the lucky stiff)
Date: Mon Sep 12 17:04:24 2005
Subject: Oh and dissected URL matching is in...
Message-ID: <4325EEE9.4080600@hobix.com>

Meaning, you can do this:

  include_match :scheme => "https"
  include_match :domain => "boingboing.net"
  include_match :domain => /google\.com$/, :path => "search"

_why
From kevin at sb.org  Mon Sep 12 19:51:17 2005
From: kevin at sb.org (Kevin Ballard)
Date: Mon Sep 12 19:45:51 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <4325EEE9.4080600@hobix.com>
References: <4325EEE9.4080600@hobix.com>
Message-ID: <0C516774-2497-4957-84B4-76609042BCFD@sb.org>

I noticed you added that when I updated my checkout and found my  
spiffy new register_url stuff (that I posted a patch for to this  
list) conflicted heavily. I haven't had time to really go through  
this, but at first glance the matching function appears a lot less  
powerful than the one I wrote in my patch. Should I bother updating  
my patch to work properly with the new register_url stuff in here, or  
just forget about it?

On Sep 12, 2005, at 5:11 PM, why the lucky stiff wrote:

> Meaning, you can do this:
>
>  include_match :scheme => "https"
>  include_match :domain => "boingboing.net"
>  include_match :domain => /google\.com$/, :path => "search"

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050912/e8651997/smime.bin
From why at hobix.com  Mon Sep 12 20:29:22 2005
From: why at hobix.com (why the lucky stiff)
Date: Mon Sep 12 20:22:40 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <0C516774-2497-4957-84B4-76609042BCFD@sb.org>
References: <4325EEE9.4080600@hobix.com>
	<0C516774-2497-4957-84B4-76609042BCFD@sb.org>
Message-ID: <43261D62.8090700@hobix.com>

Kevin Ballard wrote:

> I noticed you added that when I updated my checkout and found my  
> spiffy new register_url stuff (that I posted a patch for to this  
> list) conflicted heavily. I haven't had time to really go through  
> this, but at first glance the matching function appears a lot less  
> powerful than the one I wrote in my patch. Should I bother updating  
> my patch to work properly with the new register_url stuff in here, or  
> just forget about it?

I can't get your patch to come up in my browser, so I committed what I 
had.  Can you send it as an attachment to the list?  Or just to me if 
you like?

_why
From kevin at sb.org  Mon Sep 12 20:45:47 2005
From: kevin at sb.org (Kevin Ballard)
Date: Mon Sep 12 20:38:41 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <43261D62.8090700@hobix.com>
References: <4325EEE9.4080600@hobix.com>
	<0C516774-2497-4957-84B4-76609042BCFD@sb.org>
	<43261D62.8090700@hobix.com>
Message-ID: <07443DCA-D44E-450A-9F49-E1FD5F6FC6E4@sb.org>

Probably because Dreamhost (my webhost) is in LA and they had massive  
power outages today, so DH is slowly bringing themselves back online  
and my blog isn't up yet. But it was up when I posted the patch :P

I can't reconstruct the patch as it was, because I tossed the  
original and my trunk checkout was updated (so I can't re-diff). I  
could give you a new patch that turns current trunk into what I had  
(which means some irrelevant changes would be deleted), or,  
preferably, you could try again in a couple hours (or perhaps  
tomorrow) when my website will be back up.

On Sep 12, 2005, at 8:29 PM, why the lucky stiff wrote:

> I can't get your patch to come up in my browser, so I committed  
> what I had.  Can you send it as an attachment to the list?  Or just  
> to me if you like?

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050912/70b5d700/smime.bin
From why at hobix.com  Mon Sep 12 21:38:46 2005
From: why at hobix.com (why the lucky stiff)
Date: Mon Sep 12 21:32:07 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <07443DCA-D44E-450A-9F49-E1FD5F6FC6E4@sb.org>
References: <4325EEE9.4080600@hobix.com>	<0C516774-2497-4957-84B4-76609042BCFD@sb.org>	<43261D62.8090700@hobix.com>
	<07443DCA-D44E-450A-9F49-E1FD5F6FC6E4@sb.org>
Message-ID: <43262DA6.20605@hobix.com>

Kevin Ballard wrote:

> I  could give you a new patch that turns current trunk into what I 
> had  (which means some irrelevant changes would be deleted)

If you have time, that would be great.

_why
From kevin at sb.org  Mon Sep 12 23:16:47 2005
From: kevin at sb.org (Kevin Ballard)
Date: Mon Sep 12 23:09:50 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <43262DA6.20605@hobix.com>
References: <4325EEE9.4080600@hobix.com>	<0C516774-2497-4957-84B4-76609042BCFD@sb.org>	<43261D62.8090700@hobix.com>
	<07443DCA-D44E-450A-9F49-E1FD5F6FC6E4@sb.org>
	<43262DA6.20605@hobix.com>
Message-ID: <D52AE513-9F5E-4BBB-AD0B-C8C6810801A4@sb.org>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050912/ac1ea2d9/smime-0001.bin
From why at hobix.com  Tue Sep 13 01:04:47 2005
From: why at hobix.com (why the lucky stiff)
Date: Tue Sep 13 00:58:05 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <D52AE513-9F5E-4BBB-AD0B-C8C6810801A4@sb.org>
References: <4325EEE9.4080600@hobix.com>	<0C516774-2497-4957-84B4-76609042BCFD@sb.org>	<43261D62.8090700@hobix.com>	<07443DCA-D44E-450A-9F49-E1FD5F6FC6E4@sb.org>	<43262DA6.20605@hobix.com>
	<D52AE513-9F5E-4BBB-AD0B-C8C6810801A4@sb.org>
Message-ID: <43265DEF.4090202@hobix.com>

Kevin Ballard wrote:

> Ok, the attached patch should work on current trunk and revert all  
> the recent lib/mouseHole.rb changes (back to 9/09) and apply my  
> patch. The hack for ___._ will be left untouched, but the Tidy  
> reversion, the GM support, and the uri hash matching stuff that's  
> been committed to lib/mouseHole.rb will be gone and left will be my  
> register_uri stuff.

I've merged the stuff.  And I'm going to the crappiest thing I can do 
right now.  I'm going to totally change it all.  It's rotten and I hate 
myself right now.  I hope you will all join with me in hating me.

The thing is: most people are going to want to use the register_url 
stuff to do cross-domain Ajax.  Adding resources to the site is 
important as well, yes.

I just think that if a guygal is writing a user script for Google that 
adds Yahoo search results, it's confusing to have to register 
"http://google.com/.mouseHole/yahoo.com".

In addition, it seems like a registered URL should be available wherever 
the script runs.

And, in the Javascript, the guygal should be given a bit of convenient 
JS so that calls to "http://yahoo.com" will be translated to the 
registered URL.

I'm not sure how this will all go together.  But it will.  Now you see.  
There is much cause for hate.

_why
From kevin at sb.org  Tue Sep 13 02:02:48 2005
From: kevin at sb.org (Kevin Ballard)
Date: Tue Sep 13 01:55:44 2005
Subject: Oh and dissected URL matching is in...
In-Reply-To: <43265DEF.4090202@hobix.com>
References: <4325EEE9.4080600@hobix.com>	<0C516774-2497-4957-84B4-76609042BCFD@sb.org>	<43261D62.8090700@hobix.com>	<07443DCA-D44E-450A-9F49-E1FD5F6FC6E4@sb.org>	<43262DA6.20605@hobix.com>
	<D52AE513-9F5E-4BBB-AD0B-C8C6810801A4@sb.org>
	<43265DEF.4090202@hobix.com>
Message-ID: <A7D62631-BB83-4B16-978C-04BFA6836438@sb.org>

On Sep 13, 2005, at 1:04 AM, why the lucky stiff wrote:

> I've merged the stuff.  And I'm going to the crappiest thing I can  
> do right now.  I'm going to totally change it all.  It's rotten and  
> I hate myself right now.  I hope you will all join with me in  
> hating me.

Go ahead and change it, as long as you make it better :)

> The thing is: most people are going to want to use the register_url  
> stuff to do cross-domain Ajax.  Adding resources to the site is  
> important as well, yes.

I agree that cross-domain AJAX is the primary purpose.

> I just think that if a guygal is writing a user script for Google  
> that adds Yahoo search results, it's confusing to have to register  
> "http://google.com/.mouseHole/yahoo.com".

What else would you register? It has to be accessible from the source  
domain, which means it needs to be a google.com URL.

> In addition, it seems like a registered URL should be available  
> wherever the script runs.
>
> And, in the Javascript, the guygal should be given a bit of  
> convenient JS so that calls to "http://yahoo.com" will be  
> translated to the registered URL.

Easy enough to do.

register_url(:path => 'yahoo.com/google_aggregation') { |uri,req,res|  
do_stuff() }

The only problem with making it available everywhere is then you have  
the potential for name conflicts. An appropriate solution might be to  
give all scripts namespaces (and I don't mean the email address  
namespace we seem to be using right now - I mean a unique name for  
each script. Could be taken from the filename, but I think it should  
be declared in the script), and have register_url instead simply take  
a resource name, and you would then access that through http:// 
any.domain.com/.mouseHole/identifier/resource_name. And then we could  
do the JavaScript voodoo that hides the details about the URL from  
the user.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050913/91e17217/smime.bin
From mental at rydia.net  Tue Sep 13 21:38:29 2005
From: mental at rydia.net (MenTaLguY)
Date: Tue Sep 13 21:30:30 2005
Subject: Preliminary GM support in CVS
In-Reply-To: <4325BD98.3020501@hobix.com>
References: <4325BD98.3020501@hobix.com>
Message-ID: <1126661909.26436.130.camel@localhost.localdomain>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050913/9418c425/attachment.bin
From why at hobix.com  Tue Sep 13 23:06:02 2005
From: why at hobix.com (why the lucky stiff)
Date: Tue Sep 13 22:59:17 2005
Subject: Preliminary GM support in CVS
In-Reply-To: <1126661909.26436.130.camel@localhost.localdomain>
References: <4325BD98.3020501@hobix.com>
	<1126661909.26436.130.camel@localhost.localdomain>
Message-ID: <4327939A.6030003@hobix.com>

MenTaLguY wrote:
> onload basically isn't reliable. I've found the most reliable thing is 
> to attach a <script src=""> to the end of the document. Provided we 
> use src= rather than putting anything in the body of the script tag, 
> we shouldn't have any of the problems that Greasemonkey did from doing
> that.
>
> Since I probably won't have any mouseHole time for a while...
>
> I've attached the work I had been doing on the hoodlum-ng ("starmonkey")
> stuff... it's a diff against 1.1.  Unfortunately I just haven't had time
> to finish and it's in a pretty icky state, but maybe you would want to
> cherry-pick some of it at least.
>
> -mental
>   
Thanks, mental.  I'm workin on this now.  Things are truly coming 
together in a way that I haven't experienced since elementary school 
when we all danced around the edge of a parachute and synchronized of 
course.

I'll let everyone know when these things are all nestled.  In white 
chocolate.

_why
From why at hobix.com  Wed Sep 14 00:58:20 2005
From: why at hobix.com (why the lucky stiff)
Date: Wed Sep 14 00:51:37 2005
Subject: + Using register_uri
Message-ID: <4327ADEC.2080209@hobix.com>

I've just merged in the last of the major alterations for 1.2.  Two 
things: the foundation for cross-site Ajax and basic Greasemonkey support.

== register_uri ==

First, a minor change: some things were named 'home_url' and 
'register_url' and other things were named 'match_uri' and 
'request_uri'.  I'm no longer using 'url' in the code, everything uses 
'uri'.

The new `register_uri' basically lets you mount a proc on arbitrary 
domains.  So you can mount 'test.js' on every domain your script 
rewrites.  However, a script-specific token is used in the URI.

Let's look at a MouseHole script fragment:

  include_match :domain => "*google.com"
  register_uri "test.js" do |path|
     response['content-type'] = 'text/javascript'
     response.body = "var js_loaded = 1;"
  end

In the above example, we're registering a simple path.  The MouseHole 
script will then be available from 
http://google.com/SCRIPTTOKEN/test.js.  Or 
http://images.google.com/SCRIPTTOKEN/test.js.

You don't need to fiddle with the script token, though.  You can call 
the `reg' method to use these URLs in your code.

  rewrite do
    document.elements['//head'].add_element 'script',
      'type' => 'text/javascript', 'src' => reg('test.js')
  end

For a great example of how this works, check out samples/google-rand.js 
in CVS.  It's a modified version of the MouseHole pinging script Kevin 
posted.

== mount, configure, rewrite, register_uri conventions ==

You can now rely on:
1. The `request' and `response' instance variables are available for use 
in each of the above.
2. Most people won't care, but... Each of the above has a corresponding 
`do_' method, which is a wrapper for its proc.  So, do_rewrite, 
do_configure, etc...  Useful for plugins that want to extend 
UserScript.  (Like StarmonkeyUserScript does.)

_why
From kevin at sb.org  Wed Sep 14 03:18:23 2005
From: kevin at sb.org (Kevin Ballard)
Date: Wed Sep 14 03:11:15 2005
Subject: + Using register_uri
In-Reply-To: <4327ADEC.2080209@hobix.com>
References: <4327ADEC.2080209@hobix.com>
Message-ID: <E7B82BF2-73F1-4B47-9ABA-A14DD89F0F26@sb.org>

Hehehe, I just think it's really funny that my pointless little  
example script is now saved for posterity :D

On Sep 14, 2005, at 12:58 AM, why the lucky stiff wrote:

> For a great example of how this works, check out samples/google- 
> rand.js in CVS.  It's a modified version of the MouseHole pinging  
> script Kevin posted.

-- 
Kevin Ballard
kevin@sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050914/9ba39655/smime-0001.bin
From why at hobix.com  Wed Sep 14 15:31:03 2005
From: why at hobix.com (why the lucky stiff)
Date: Wed, 14 Sep 2005 13:31:03 -0600
Subject: + browser caching and mousehole
Message-ID: <43287A77.1070306@hobix.com>

Well, I might as well be straightforward on this.

Just checked in: MouseHole does allow browser-caching of pages.  
However, it assists the browser in distinguishing between a plain page 
and a proxied page.

The most common use cache is user scripts.  If you request 
proxylike.user.js, you'll get an Etag and a modification time and the 
browser will hold on to that unless MouseHole intervenes in the request 
headers.

MouseHole basically adds a "MH-" prefix to the Etags.  So if 
proxylike.user.js normally has an Etag of "7a7b949d311cc51:119e", 
MouseHole's version of it is called "MH-7a7b949d311cc51:119e".  This 
means that as you switch between the proxy and the browser, you should 
be able to separately pull cached pages for each.  This only happens for 
pages which list 'text/html' in the 'Accept' header.

MouseHole will dump its Etag list when you restart it.  I will probably 
allow this to be toggled off/on per-script.  Some scripts will rewrite 
pages several times.

_why


From why at hobix.com  Fri Sep 16 03:02:00 2005
From: why at hobix.com (why the lucky stiff)
Date: Fri, 16 Sep 2005 01:02:00 -0600
Subject: doorway improvements: uninstall, uri matching, character encodings
Message-ID: <432A6DE8.2060101@hobix.com>

Okay, all my changes are in for 1.2.  If there are no complaints, I will 
hand-wrap it tomorrow.

The biggest change concerns uri matching.  I scrapped the 
include/exclude boxes.  It's all one box now.  And matching happens in 
the order you dictate.  I've also added uninstall and factory defaults 
buttons.

I did a bit of work on character encodings for Tidy.  It'll try to 
detect utf8 and give it faithful processing through Tidy.  Other 
encodings don't appear to need such attention.

1.3 Todo:
* The blocking on timed-out threads is seriously annoying.
* User scripting hooks for rewriting of outgoing headers.
* Javascript library to ease common tasks.
* GM_API.

_why

From kevin at sb.org  Fri Sep 16 04:57:56 2005
From: kevin at sb.org (Kevin Ballard)
Date: Fri, 16 Sep 2005 04:57:56 -0400
Subject: doorway improvements: uninstall, uri matching, character encodings
In-Reply-To: <432A6DE8.2060101@hobix.com>
References: <432A6DE8.2060101@hobix.com>
Message-ID: <3D0FBFC7-8557-420F-B13D-1DE76A2EB49F@sb.org>

The Factory Defaults button remove the Installed From data.

On Sep 16, 2005, at 3:02 AM, why the lucky stiff wrote:

> The biggest change concerns uri matching.  I scrapped the
> include/exclude boxes.  It's all one box now.  And matching happens in
> the order you dictate.  I've also added uninstall and factory defaults
> buttons.

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050916/c9054105/smime.bin

From kevin at sb.org  Fri Sep 16 04:58:13 2005
From: kevin at sb.org (Kevin Ballard)
Date: Fri, 16 Sep 2005 04:58:13 -0400
Subject: README patch
Message-ID: <D6DE688E-85E8-4D6D-8DCC-8B7798DE5DE0@sb.org>

The README appears to be pretty much a direct dump of the wiki page,  
which means it's not wrapped for an 80-character display. I've  
attached a patch which re-wraps it for such a display (except for the  
one URL that was longer than 80 characters - I didn't know what to do  
with that, but having the display character-wrap that is fine). It  
also removes any extraneous spaces that were at the end of lines.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: readme.patch
Type: application/octet-stream
Size: 9696 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050916/568f5ce5/readme.obj
-------------- next part --------------

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org





-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050916/568f5ce5/smime.bin

From kevin at sb.org  Fri Sep 16 05:03:40 2005
From: kevin at sb.org (Kevin Ballard)
Date: Fri, 16 Sep 2005 05:03:40 -0400
Subject: Parse errors prevent proper usage
Message-ID: <73C00FC0-A12F-4F79-9669-E0ECCEB59457@sb.org>

I just went to http://127.0.0.1:37004/http://___._/2005/09/13/ 
theHoodwinkdBirthdayCake.html and saw no hoodwinks, so I checked the  
output from MouseHole and found the following:

Is there any way to make it gracefully handle parse errors like this?

[2005-09-16 04:58:50] ERROR Exception: Illegal character '&' in raw  
string "this just keeps getting creepier&"
         /usr/lib/ruby/1.8/rexml/text.rb:90:in `initialize'
         /usr/lib/ruby/1.8/rexml/parsers/treeparser.rb:38:in `new'
         /usr/lib/ruby/1.8/rexml/parsers/treeparser.rb:38:in `parse'
         /usr/lib/ruby/1.8/rexml/document.rb:176:in `build'
         /usr/lib/ruby/1.8/rexml/document.rb:45:in `initialize'
         (eval):339:in `new'
         (eval):339:in `archive_html'
         (eval):350:in `call'
         (eval):350:in `json_read'
         (eval):348:in `open_uri'
         /usr/lib/ruby/1.8/open-uri.rb:424:in `open'
         /usr/lib/ruby/1.8/open-uri.rb:85:in `open'
         (eval):348:in `json_read'
         (eval):269:in `setup'
         (eval):350:in `call'
         (eval):350:in `json_read'
         (eval):348:in `open_uri'
         /usr/lib/ruby/1.8/open-uri.rb:424:in `open'
         /usr/lib/ruby/1.8/open-uri.rb:85:in `open'
         (eval):348:in `json_read'
         (eval):258:in `load_user_script'
         (eval):255:in `[]'
         ./lib/mouseHole.rb:851:in `do_rewrite'
         ./lib/mouseHole.rb:277:in `upwink'
         ./lib/mouseHole.rb:265:in `each_fresh_script'
         ./lib/mouseHole.rb:117:in `each'
         ./lib/mouseHole.rb:117:in `each_fresh_script'
         ./lib/mouseHole.rb:265:in `upwink'
         /usr/lib/ruby/1.8/webrick/httpproxy.rb:156:in `call'
         /usr/lib/ruby/1.8/webrick/httpproxy.rb:156:in `proxy_service'
         /usr/lib/ruby/1.8/webrick/httpproxy.rb:29:in `service'
         ./lib/mouseHole.rb:100:in `service'
         /usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
         /usr/lib/ruby/1.8/webrick/server.rb:155:in `start_thread'
         /usr/lib/ruby/1.8/webrick/server.rb:144:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:144:in `start_thread'
         /usr/lib/ruby/1.8/webrick/server.rb:94:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:89:in `each'
         /usr/lib/ruby/1.8/webrick/server.rb:89:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:79:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:79:in `start'
         ./bin/mouseHole:64

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050916/68b1814e/smime-0001.bin

From why at hobix.com  Fri Sep 16 11:14:17 2005
From: why at hobix.com (why the lucky stiff)
Date: Fri, 16 Sep 2005 09:14:17 -0600
Subject: Parse errors prevent proper usage
In-Reply-To: <73C00FC0-A12F-4F79-9669-E0ECCEB59457@sb.org>
References: <73C00FC0-A12F-4F79-9669-E0ECCEB59457@sb.org>
Message-ID: <432AE149.3080308@hobix.com>

Kevin Ballard wrote:

> I just went to http://127.0.0.1:37004/http://___._/2005/09/13/ 
> theHoodwinkdBirthdayCake.html and saw no hoodwinks, so I checked the  
> output from MouseHole and found the following:
>
> Is there any way to make it gracefully handle parse errors like this?

It's an incompatibility between the hoodwink.d script and the Tidy 
character encodings.  I'll be updating the hoodwink.d script shortly.

I guess we could try a fallback 'ascii' encoding should the 'raw' 
encoding not work right.

_why

From why at hobix.com  Fri Sep 16 17:44:25 2005
From: why at hobix.com (why the lucky stiff)
Date: Fri, 16 Sep 2005 15:44:25 -0600
Subject: doorway improvements: uninstall, uri matching, character encodings
In-Reply-To: <3D0FBFC7-8557-420F-B13D-1DE76A2EB49F@sb.org>
References: <432A6DE8.2060101@hobix.com>
	<3D0FBFC7-8557-420F-B13D-1DE76A2EB49F@sb.org>
Message-ID: <432B3CB9.3040708@hobix.com>

Kevin Ballard wrote:

> The Factory Defaults button remove the Installed From data.

Okay, this is fixed.  Your README patch is in as well.  Well done, amigo.

_why

From why at hobix.com  Sun Sep 18 01:54:07 2005
From: why at hobix.com (why the lucky stiff)
Date: Sat, 17 Sep 2005 23:54:07 -0600
Subject: mouseHole is great on ruby 1.8.3
Message-ID: <432D00FF.9060809@hobix.com>

I've been hacking YAML stuff for Ruby 1.8.3 all weekend and I've noticed 
there's WEBrick updates in this release.

After a few alterations, mouseHole is running really well under the new 
version.  I haven't read the diffs, so I don't the reasons, but I'm not 
getting as much spinner-crunching in mouseHole when I'm loading several 
sites at once.

This is a hot tip.

_why

From why at hobix.com  Thu Sep 22 15:13:28 2005
From: why at hobix.com (why the lucky stiff)
Date: Thu, 22 Sep 2005 13:13:28 -0600
Subject: MouseCommand, a branch of wikish
Message-ID: <43330258.2000907@hobix.com>

I've been playing around with Jake's wikish script and altering the 
look, adding some helper methods.
  
  Overview: <http://redhanded.hobix.com/inspect/wikiShellOnMousehole.html>
  Script: <http://whytheluckystiff.net/mouseHole/cmd.user.rb>

I really only have one feature I'd like to add.  I love to be able to 
run a service like Yubnub where you can store your commands and copy 
others commands.

_why

From jake at bitmechanic.com  Thu Sep 22 20:55:47 2005
From: jake at bitmechanic.com (Jake Donham)
Date: 22 Sep 2005 20:55:47 -0400
Subject: MouseCommand, a branch of wikish
In-Reply-To: <43330258.2000907@hobix.com>
References: <43330258.2000907@hobix.com>
Message-ID: <m33bnwiovg.fsf@sparkplug.bitmechanic.com>

why the lucky stiff <why at hobix.com> writes:

> I've been playing around with Jake's wikish scrip

Thanks for running with it--I would like to contribute more myself but
I just started a comp sci PhD program and it's not leaving me any
recreational hack time at the moment.

Here is a thought: I work from school and from home, and I would like
to use this from both places (this is part of why I love del.icio.us),
but I don't know about opening up a MouseHole instance to the wide,
weird net. One alternative would be to provide a way to sync the wiki
from one MouseHole to another--you want your MouseHole local anyway.

Another thought, in the personal server department: a local
del.icio.us interface (syncable between MouseHoles as above), which
syncs to del.icio.us via its API. I use del.icio.us enough that I
sometimes find it annoyingly slow, and I rarely use the social
features. The trick would be to avoid duplicating the interface (as
much as possible). Maybe adds/edits still go direct to del.icio.us,
but lookups are local? Or have a less-fancy interface where you can
add/edit locally and have changes synced to del.icio.us later.

(The central-server design kind of bugs me in general; when we are
living in a truly Web N.0 for some high N world where everything is
making web service calls to everything else, it's going to be a
reliability nightmare. Things that can be handled locally should be
handled locally. This is why I like the personal server idea.)

Jake

From why at hobix.com  Fri Sep 23 02:39:38 2005
From: why at hobix.com (why the lucky stiff)
Date: Fri, 23 Sep 2005 00:39:38 -0600
Subject: MouseCommand, a branch of wikish
In-Reply-To: <m33bnwiovg.fsf@sparkplug.bitmechanic.com>
References: <43330258.2000907@hobix.com>
	<m33bnwiovg.fsf@sparkplug.bitmechanic.com>
Message-ID: <4333A32A.4060505@hobix.com>

Jake Donham wrote:

> One alternative would be to provide a way to sync the wiki from one 
> MouseHole to another--you want your MouseHole local anyway.

All your settings are stored in ~/.mouseHole.  Rsync that directory.  Or 
mount over sshfs or store it in Subversion or something like that.

_why

From why at hobix.com  Sun Sep 25 00:39:10 2005
From: why at hobix.com (why the lucky stiff)
Date: Sat, 24 Sep 2005 22:39:10 -0600
Subject: Centralizing the MouseHole DB (was Re: MouseCommand, a branch of
	wikish)
In-Reply-To: <m33bnwiovg.fsf@sparkplug.bitmechanic.com>
References: <43330258.2000907@hobix.com>
	<m33bnwiovg.fsf@sparkplug.bitmechanic.com>
Message-ID: <433629EE.6080204@hobix.com>

Jake Donham wrote:

> Here is a thought: I work from school and from home, and I would like 
> to use this from both places (this is part of why I love del.icio.us), 
> but I don't know about opening up a MouseHole instance to the wide, 
> weird net. One alternative would be to provide a way to sync the wiki 
> from one MouseHole to another--you want your MouseHole local anyway.

Over the past few days, this idea has really haunted me.  Moving 
forward, this is necessary.  And my suggestions so far (sshfs or rsync) 
are just too klunky for most people to put into action.

But syncing is a pain.  (Blech, syncing PDAs??)   And a central DB would 
require an internet connection, which I don't really want to require.  
So what else is there?  A peer-to-peer database?  Ugghh.

For now, the best solution may be to start simple.  I'm thinking of 
adding a backend API to MouseHole 1.3.  It'll mimick the DBM interface.  
That way at least we could support a central DB, which would be pretty 
cool to have anyway.

_why

From kevin at sb.org  Sun Sep 25 02:12:33 2005
From: kevin at sb.org (Kevin Ballard)
Date: Sun, 25 Sep 2005 02:12:33 -0400
Subject: Centralizing the MouseHole DB (was Re: MouseCommand,
	a branch of wikish)
In-Reply-To: <433629EE.6080204@hobix.com>
References: <43330258.2000907@hobix.com>
	<m33bnwiovg.fsf@sparkplug.bitmechanic.com>
	<433629EE.6080204@hobix.com>
Message-ID: <C91DDF25-586B-4533-9EC1-4ECE69C11B48@sb.org>

Here's an idea - write a plugin that provides a mountpoint as well as  
a plugin configuration. The mountpoint provides a way for external MH  
installations to download the wiki database (probably with some sort  
of auth), and the plugin configuration lets you enter a host to fetch  
the wiki database from. Or maybe that would be part of the mountpoint  
as well. Anyway, this way if you want to keep a home wiki and then  
sync from it when you're on the go, you could do that.

I'd start working on it, except I don't need the feature so I'm not  
sure how you'd want it to behave. I don't even use the wiki.

On Sep 25, 2005, at 12:39 AM, why the lucky stiff wrote:

> Over the past few days, this idea has really haunted me.  Moving
> forward, this is necessary.  And my suggestions so far (sshfs or  
> rsync)
> are just too klunky for most people to put into action.
>
> But syncing is a pain.  (Blech, syncing PDAs??)   And a central DB  
> would
> require an internet connection, which I don't really want to require.
> So what else is there?  A peer-to-peer database?  Ugghh.
>
> For now, the best solution may be to start simple.  I'm thinking of
> adding a backend API to MouseHole 1.3.  It'll mimick the DBM  
> interface.
> That way at least we could support a central DB, which would be pretty
> cool to have anyway.

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050925/633ae6c9/smime.bin

From grant at antiflux.org  Sun Sep 25 12:41:04 2005
From: grant at antiflux.org (Grant Hollingworth)
Date: Sun, 25 Sep 2005 10:41:04 -0600
Subject: bug: add_match with index
Message-ID: <20050925164104.GA32056@okcomputer.antiflux.org>

I don't think it's used anywhere, but UserScript.add_match (and so
include_match and exclude_match) can add a match at a specific index in
the match list.

  def add_match r, m, i = nil
      r.strip! if r.respond_to? :strip!
      if i; self.matches[i+1, 0] = [r, m]
      else; self.matches << [r, m] end
  end

However:
  irb(main):001:0> a = [[1,1],[2,2],[3,3]]
  => [[1, 1], [2, 2], [3, 3]]
  irb(main):002:0> a << [4,4]
  => [[1, 1], [2, 2], [3, 3], [4, 4]]
  irb(main):003:0> a[1,0] = [5,5]
  => [5, 5]
  irb(main):004:0> a
  => [[1, 1], 5, 5, [2, 2], [3, 3], [4, 4]]

So it should be
  if i; self.matches[i+1, 0] = [[r, m]]

I'm not sure about the '+1', either.

From kevin at sb.org  Mon Sep 26 01:32:22 2005
From: kevin at sb.org (Kevin Ballard)
Date: Mon, 26 Sep 2005 01:32:22 -0400
Subject: bug: add_match with index
In-Reply-To: <20050925164104.GA32056@okcomputer.antiflux.org>
References: <20050925164104.GA32056@okcomputer.antiflux.org>
Message-ID: <A4C6F83D-5620-4A1C-A53C-362078BB2FE2@sb.org>

Why not just use Array#insert instead of Array#[]=?

On Sep 25, 2005, at 12:41 PM, Grant Hollingworth wrote:

> I don't think it's used anywhere, but UserScript.add_match (and so
> include_match and exclude_match) can add a match at a specific  
> index in
> the match list.
>
>   def add_match r, m, i = nil
>       r.strip! if r.respond_to? :strip!
>       if i; self.matches[i+1, 0] = [r, m]
>       else; self.matches << [r, m] end
>   end
>
> However:
>   irb(main):001:0> a = [[1,1],[2,2],[3,3]]
>   => [[1, 1], [2, 2], [3, 3]]
>   irb(main):002:0> a << [4,4]
>   => [[1, 1], [2, 2], [3, 3], [4, 4]]
>   irb(main):003:0> a[1,0] = [5,5]
>   => [5, 5]
>   irb(main):004:0> a
>   => [[1, 1], 5, 5, [2, 2], [3, 3], [4, 4]]
>
> So it should be
>   if i; self.matches[i+1, 0] = [[r, m]]
>
> I'm not sure about the '+1', either.

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050926/8e9a81d9/smime-0001.bin

From grant at antiflux.org  Mon Sep 26 02:06:55 2005
From: grant at antiflux.org (Grant Hollingworth)
Date: Mon, 26 Sep 2005 00:06:55 -0600
Subject: bug: add_match with index
In-Reply-To: <A4C6F83D-5620-4A1C-A53C-362078BB2FE2@sb.org>
References: <20050925164104.GA32056@okcomputer.antiflux.org>
	<A4C6F83D-5620-4A1C-A53C-362078BB2FE2@sb.org>
Message-ID: <20050926060655.GA29382@okcomputer.antiflux.org>

* Kevin Ballard <kevin at sb.org> [2005-09-25 23:32]:
> Why not just use Array#insert instead of Array#[]=?

Well, that's much nicer.  why, is there a reason for the +1, or can it
be self.matches.insert(i, [r, m])?

From daniels at pronto.com.au  Mon Sep 26 03:48:31 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Mon, 26 Sep 2005 17:48:31 +1000
Subject: MouseCommand security/feature thoughts.
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E39B5@mlbmail.pronto.com.au>

I was having a play with mousecommand to have a go at some way of
downloading the commands from the web (tossing up between having
user.cmd.rb or using a microformat), and realised that I could do this:

http://localhost:37004/cmd/save/monkey/?content=p+nil&author=YourMomma&d
oc=%3Cscript%3Ealert(%22code+injection%22)%3C%2Fscript%3E

Put a form in a remote site that posts to that address and have some
javascript submit the form, add in some javascript in the doc section to
redirect the user somewhere else and bam, you've replaced one of their
commands.

Might add a check to the save command to make sure that the referrer is
local - can the referrer string be modified in javascript? Also, should
probably modify the doc section to escape html.

Also, once I figure out what I'm doing with downloading commands, I'm
also considering modifying mousecommand to either:
1) make a list of commands that will be redirected through yubnub for
processing (probably wont do this, as there's not much use if I do the
other two below), or have an option to just do this by default with
unknown commands.

2) import commands from yubnub (either convert them to ruby, or allow
the writing of yubnub("yubnub commands") into mousehole (which will
either go run it through yubnub, or do it locally - not sure yet).
   - additionally, change the new command to check for, grab and convert
a yubnub command and enter that in as the default values for a new
command.

3) duplicating yubnub's method of chaining commands (with the curly
brackets).



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050926/e75c77d4/attachment.htm

From kevin at sb.org  Mon Sep 26 04:03:15 2005
From: kevin at sb.org (Kevin Ballard)
Date: Mon, 26 Sep 2005 04:03:15 -0400
Subject: MouseCommand security/feature thoughts.
In-Reply-To: <EE0A2C5DE8D7CA4792891604D4FED52E034E39B5@mlbmail.pronto.com.au>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E39B5@mlbmail.pronto.com.au>
Message-ID: <82AD3BB0-4BFE-4D2F-A2DD-72F8FFF9613C@sb.org>

On Sep 26, 2005, at 3:48 AM, Daniel Sheppard wrote:

> I was having a play with mousecommand to have a go at some way of
> downloading the commands from the web (tossing up between having
> user.cmd.rb or using a microformat), and realised that I could do  
> this:
>
> http://localhost:37004/cmd/save/monkey/?content=p 
> +nil&author=YourMomma&d
> oc=%3Cscript%3Ealert(%22code+injection%22)%3C%2Fscript%3E
>
> Put a form in a remote site that posts to that address and have some
> javascript submit the form, add in some javascript in the doc  
> section to
> redirect the user somewhere else and bam, you've replaced one of their
> commands.
>
> Might add a check to the save command to make sure that the  
> referrer is
> local - can the referrer string be modified in javascript? Also,  
> should
> probably modify the doc section to escape html.

It would probably be more secure to simply add an opaque UUID to  
mousecommand (that it auto-generates, say, each time you run  
MouseHole) and you can only edit commands if the opaque UUID is  
passed along with the other arguments.

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050926/2733f0c2/smime.bin

From why at hobix.com  Mon Sep 26 12:09:44 2005
From: why at hobix.com (why the lucky stiff)
Date: Mon, 26 Sep 2005 10:09:44 -0600
Subject: MouseCommand security/feature thoughts.
In-Reply-To: <82AD3BB0-4BFE-4D2F-A2DD-72F8FFF9613C@sb.org>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E39B5@mlbmail.pronto.com.au>
	<82AD3BB0-4BFE-4D2F-A2DD-72F8FFF9613C@sb.org>
Message-ID: <43381D48.3090802@hobix.com>

Kevin Ballard wrote:

> It would probably be more secure to simply add an opaque UUID to  
> mousecommand (that it auto-generates, say, each time you run  
> MouseHole) and you can only edit commands if the opaque UUID is  
> passed along with the other arguments.

Each script has @token, a 32-character random string.  It's used by 
register_uri but certainly can be used in other cases such as this.

_why

From daniels at pronto.com.au  Tue Sep 27 01:45:04 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Tue, 27 Sep 2005 15:45:04 +1000
Subject: rexml/httpwrite, why must you mock me?
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3B95@mlbmail.pronto.com.au>

I got struck by a little inspiration last night, and thought "Hey,
mousehole, how would you like to have some fun manipulating rss feeds?".

MouseHole didn't seem too impressed by the idea.

Firstly, after littering my code with debug messages, I found that
MouseHole only wanted to deal with the text/html and
application/xhtml+xml mime types. Fair enough - if want to parse
everything as HTML, it should probably obey that restriction. So I
fiddled with the checking:

                    unless script.match req.request_uri
			    logger.info "Skipping #{script.name} - no
match"
			    next
		    end
                    unless
script.document_converter.handles_type?(res.content_type)
			    logger.info "Skipping #{script.name} - wrong
content type"
			    next
		    end
                    logger.info "Executing #{script.name}"
                    script.execute( req, res )

And I added in this bit of code (which I think is a pretty messy way of
going about this, but it works in a pinch):

    class HtmlDocumentConverter
        def parse_string(body)
	    parse_xhtml(HTree.parse(body))
	end
        def output_string(document, stream)
            document.write(stream)
        end
        def parse_xhtml( htree )
            htree.each_child do |child|
            if child.respond_to? :qualified_name
                if child.qualified_name == 'html'
                return HTree::Doc.new( child ).to_rexml
                break
                end
            end
            end
        end	  
        def handles_type?(type)
            [
                /^text\/html/, 
                /^application\/xhtml+xml/
            ].any? {|x| x === type }
        end        
    end
    
    class XmlDocumentConverter
        def parse_string(body)
            REXML::Document.new(body)
        end
        def output_string(document, stream)
            document.write(stream)
        end
        def handles_type?(type)
	    p type
            /^text\/xml/ === type
        end
    end

    class UserScript

        attr_accessor :document, :matches, :db, :request, :response,
:mtime, :active, :install_url, :document_converter
        def document_converter s = nil; s ? @document_converter = s :
(@document_converter || HtmlDocumentConverter.new) end
        def name s = nil; s ? @name = s : @name; end
        ......

Which allowed me to write this rule for rewriting the slashdot RSS feed:

MouseHole.script do
	name "Slashdot Fullfeed RSS"
	namespace "http://members.iinet.net.au/~soxbox/"
	description "Converts the slashdot RSS feed to a full-content
feed"
	include_match "http://rss.slashdot.org/Slashdot/slashdot"
	document_converter XmlDocumentConverter.new
	version "0.1"

	rewrite do |req,res|
		p "rewriting"
		document.each_element('//item/') do |e|
			e.each_element('description') {|x| x.remove}
			doc = read_xhtml_from(e.attributes['rdf:about']
+ "&mode=nocomment")
			desc = REXML::Element.new('description')
			doc.each_element('//div[@class="intro"]') do |x|
				s = ""
				x.write(s)
				s.gsub!("&mdash;","&#8212;")
				desc.text = s
			end
			e << desc
		end
	end
end

Then I noticed something strange - my rewritten feed had no content in
the <link/> elements. Apparently, the reason for this is the
rexml/httpwrite.rb code - which seems to be designed to ensure that HTML
elements that aren't meant to have content don't end up having any
content... Why does this code exist? Shouldn't it by the task of the
tree builder to put the right things in the right nodes? Otherwise,
wouldn't it be better to have something walking the tree and trimming
the bad nodes before it gets output?

#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050927/d309c7da/attachment-0001.htm

From daniels at pronto.com.au  Tue Sep 27 01:47:57 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Tue, 27 Sep 2005 15:47:57 +1000
Subject: rexml/httpwrite, why must you mock me?
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3B96@mlbmail.pronto.com.au>

Oh, and I forgot this change in UserScript as well:
 
        def execute( req, res )
            return unless rewrite_proc
            @document = document_converter.parse_string( res.body )
            return unless @document
            rewrite_proc[ req, res ]
            document_converter.output_string(document, res.body = "" )
        end
 
        def read_xhtml_from( uri )
            HtmlDocumentConverter.new.parse_xhtml( open( uri ) { |f|
HTree.parse f } )
        end

#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################


From why at hobix.com  Tue Sep 27 02:27:05 2005
From: why at hobix.com (why the lucky stiff)
Date: Tue, 27 Sep 2005 00:27:05 -0600
Subject: rexml/httpwrite, why must you mock me?
In-Reply-To: <EE0A2C5DE8D7CA4792891604D4FED52E034E3B95@mlbmail.pronto.com.au>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E3B95@mlbmail.pronto.com.au>
Message-ID: <4338E639.6060306@hobix.com>

Daniel Sheppard wrote:

> Then I noticed something strange - my rewritten feed had no content in 
> the <link/> elements. Apparently, the reason for this is the 
> rexml/httpwrite.rb code - which seems to be designed to ensure that 
> HTML elements that aren't meant to have content don't end up having 
> any content? Why does this code exist? Shouldn't it by the task of the 
> tree builder to put the right things in the right nodes? Otherwise, 
> wouldn't it be better to have something walking the tree and trimming 
> the bad nodes before it gets output?
>
Sounds like HtmlDocumentConverter just needs REXML::HtmlElements to go 
along with it, leaving regular REXML::Elements uneffected.

Regarding the rewrite of non-HTML goods, maybe a `mime_type' directive 
or `allow_non_html' directive?

_why


From daniels at pronto.com.au  Tue Sep 27 02:39:27 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Tue, 27 Sep 2005 16:39:27 +1000
Subject: rexml/httpwrite, why must you mock me?
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3BD8@mlbmail.pronto.com.au>

Okay - think I've got everything resolved.

If you include the attached fix_xml immediately after including
mousehole (ie. line 8 of the mousehole.rb wrapper script), it will give
you the ability to write XML transforming user scripts by including:

	document_converter XmlDocumentConverter.new

in your script definition.

I was planning on creating an AtomicMouse user script to manipulate
RSS/Atom feeds by picking out the links in a summary feed, grabbing the
linked pages and using an Xpath to specify which part of the page should
be grabbed as the feed content.

However, I don't know my way around the ruby RSS libraries enough to do
this yet - After parsing a feed with SimpleRss or the stdlib Rss, how do
you turn that data back into an RSS feed?

#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_xml.rb
Type: application/octet-stream
Size: 4425 bytes
Desc: fix_xml.rb
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050927/14da7119/fix_xml.obj

From grant at antiflux.org  Tue Sep 27 08:50:02 2005
From: grant at antiflux.org (Grant Hollingworth)
Date: Tue, 27 Sep 2005 06:50:02 -0600
Subject: rexml/httpwrite, why must you mock me?
In-Reply-To: <EE0A2C5DE8D7CA4792891604D4FED52E034E3BD8@mlbmail.pronto.com.au>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E3BD8@mlbmail.pronto.com.au>
Message-ID: <20050927125002.GA28344@okcomputer.antiflux.org>

* Daniel Sheppard <daniels at pronto.com.au> [2005-09-27 00:40]:
> However, I don't know my way around the ruby RSS libraries enough to do
> this yet - After parsing a feed with SimpleRss or the stdlib Rss, how do
> you turn that data back into an RSS feed?

I've been thinking about adding feed support for the last week.  I'm
glad you actually did it.

I was going to use FeedTools.  It has a lot more dependencies than the
two you tried, but it should allow for easy rebuilding of a feed.

Once you have that abstraction in place, you could have a script that
changes links to full pages for any feed, not just Slashdot.  Users
could add the feeds they want using mouseHole's web interface.

From grant at antiflux.org  Tue Sep 27 08:55:26 2005
From: grant at antiflux.org (Grant Hollingworth)
Date: Tue, 27 Sep 2005 06:55:26 -0600
Subject: rexml/httpwrite, why must you mock me?
In-Reply-To: <4338E639.6060306@hobix.com>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E3B95@mlbmail.pronto.com.au>
	<4338E639.6060306@hobix.com>
Message-ID: <20050927125526.GB28344@okcomputer.antiflux.org>

* why the lucky stiff <why at hobix.com> [2005-09-27 00:36]:
> Regarding the rewrite of non-HTML goods, maybe a `mime_type' directive 
> or `allow_non_html' directive?

Two MIME type directives might be needed: one to state the type the URI
will return (default text/html), and one for the type the feed should be
treated as.  The second would be mostly for RSS feeds, which are
notorious (well, in some circles) for using text/xml or even text/plain.

FeedTools (or similar) could handle application/atom+xml
and application/rss+xml (and whatever other feed types there are) while
REXML would take application/xml and text/xml (and whatever other plain
XML types).

From nicolas.delsaux at gmail.com  Tue Sep 27 10:13:49 2005
From: nicolas.delsaux at gmail.com (Nicolas Delsaux)
Date: Tue, 27 Sep 2005 16:13:49 +0200
Subject: using MouseHole behind corporate proxy
Message-ID: <418361960509270713796c0a9e@mail.gmail.com>

Hi,
I would like to use MouseHole at my company. Unfortunatly, I can't
connect to any web site, since my company use a corporate proxy.
I've tried diving into source code but, since I'm only a Ruby padawan,
I didn't encounter success.
So, is there any way to ask gently MouseHole (or even the WEBrick
underneath) to access the web through that proxy ?
Even involving code, i'm ready to produce some Ruby gem (well, in my
case, it won't be really a cool gem, but at least a first try).
Notice I already have a Ruby fragment for doing such proxied
connection, used for a bookmark synchronization script.
Thanks.
--
Nicolas Delsaux
Depuis GMail qui vous espionne, mais qui est bien
Et pour ceux que ?a int?resse, j'ai 50 invitations


From daniels at pronto.com.au  Tue Sep 27 19:46:32 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Wed, 28 Sep 2005 09:46:32 +1000
Subject: using MouseHole behind corporate proxy
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3C79@mlbmail.pronto.com.au>

You can set the http_proxy environment variable.

However, I had some trouble with that, as there's quite a few sites running internally that I don't want sent through the proxy, and also some other sites that need to go through a separate proxy - so I added in a little fragment (attached) which allows me to have a separate configuration file that defines which proxy to use for a given host.

Require this directly after requiring the mousehole.rb script, and you can put a proxy_config.rb file in your mousehole directory that will be eval'd constantly to determine what proxy to use. It gets the host passed in, so you can write a file something like this:

case host
	when "localhost",
		"127.0.0.1",
		"192.168.0.2",
		"168.168.8.88",
		/^172\.17\./
		nil
	when "www.example.com"
		"http://proxy1:5466"
	else
		"http://proxy2:8080"
end

You should install the zipped version of mousehole, not the installer version so that you can play around with the mousehole source and add in the new require statement.

Also, if your firewall happens to be running NTLM authentication (ie. it's the microsoft proxy), you can use http://apserver.sourceforge.net/ to provide a local proxy that runs the NTLM part of the puzzle. Does anybody know if this has been ported to ruby?


-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org [mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Nicolas Delsaux
Sent: Wednesday, 28 September 2005 12:14 AM
To: mousehole-scripters at rubyforge.org
Subject: using MouseHole behind corporate proxy

Hi,
I would like to use MouseHole at my company. Unfortunatly, I can't connect to any web site, since my company use a corporate proxy.
I've tried diving into source code but, since I'm only a Ruby padawan, I didn't encounter success.
So, is there any way to ask gently MouseHole (or even the WEBrick
underneath) to access the web through that proxy ?
Even involving code, i'm ready to produce some Ruby gem (well, in my case, it won't be really a cool gem, but at least a first try).
Notice I already have a Ruby fragment for doing such proxied connection, used for a bookmark synchronization script.
Thanks.
--
Nicolas Delsaux
Depuis GMail qui vous espionne, mais qui est bien Et pour ceux que ?a int?resse, j'ai 50 invitations

_______________________________________________
Mousehole-scripters mailing list
Mousehole-scripters at rubyforge.org
http://rubyforge.org/mailman/listinfo/mousehole-scripters



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_proxy.rb
Type: application/octet-stream
Size: 736 bytes
Desc: fix_proxy.rb
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050928/dd78d07e/fix_proxy.obj

From daniels at pronto.com.au  Tue Sep 27 20:48:57 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Wed, 28 Sep 2005 10:48:57 +1000
Subject: rexml/httpwrite, why must you mock me?
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3CC8@mlbmail.pronto.com.au>

My initial thought was to put in an allowed_content_type directive, but
then how should MouseHole perform the document translation?

If it's served as text/xml, in some cases you'll want it processed by an
RSS Handler to turn it into an RSS document, in others you'll want it
handled by the XML handler as a xml document, in others you might just
want a Plain Text Handler to manipulate it directly as a string.

The attachment in my previous email undoes the work of rexml/httpwrite
and does a scan of the document in the write_output method of
HtmlDocumentConverter to make sure the write elements are left empty - I
take it the only place that's actually causing this problem is the
<script/> element?

-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of why the
lucky stiff
Sent: Tuesday, 27 September 2005 4:27 PM
To: mousehole-scripters at rubyforge.org
Subject: Re: rexml/httpwrite, why must you mock me?

Daniel Sheppard wrote:

> Then I noticed something strange - my rewritten feed had no content in

> the <link/> elements. Apparently, the reason for this is the 
> rexml/httpwrite.rb code - which seems to be designed to ensure that 
> HTML elements that aren't meant to have content don't end up having 
> any content... Why does this code exist? Shouldn't it by the task of
the 
> tree builder to put the right things in the right nodes? Otherwise, 
> wouldn't it be better to have something walking the tree and trimming 
> the bad nodes before it gets output?
>
Sounds like HtmlDocumentConverter just needs REXML::HtmlElements to go
along with it, leaving regular REXML::Elements uneffected.

Regarding the rewrite of non-HTML goods, maybe a `mime_type' directive
or `allow_non_html' directive?

_why

_______________________________________________
Mousehole-scripters mailing list
Mousehole-scripters at rubyforge.org
http://rubyforge.org/mailman/listinfo/mousehole-scripters


#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################


From daniels at pronto.com.au  Tue Sep 27 22:38:20 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Wed, 28 Sep 2005 12:38:20 +1000
Subject: FeedTools
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3D3C@mlbmail.pronto.com.au>

Hmmm... poking about with feedtools, there doesn't seem to be a way to
make it parse a string rather than going to a URL.

No problems - MouseHole needs to be intercepting requests and subverting
them without necessarily visiting the requested URL.

Basically, I'm looking at replacing a section of the proxy_service call,
and creating an "intercept" method. The problem is the interface for
this method - what parameters should it get? what should it need to be
returning? Looking at the source-code for proxy_service in Webrick, I'm
not quiet sure at what point I should be handing over to the
interceptor.

Options are:
	- rely on the interceptor to handle everything that
proxy_service handles - this gives complete control, but makes things
hard.
	- pass in the request, the path and the header - expect a
HTTPResponse back - probably the minimum you can do and still leave a
lot of control, but it requires a fair bit of work to get it back to the
existing level .
	- ???

for option 2, in order to intercept the request, and say - send it to a
slightly different path (ie. add some more query params), you need to
do:

intercept do |request, path, header|
        path << path.include? '?' : '&' : '?'
        path << "nostyle=true";
        http = Net::HTTP.new(request.uri.host, request.uri.port)
        http.start{
          case req.request_method
          when "GET"  then response = http.get(path, header)
          when "POST" then response = http.post(path, req.body || "",
header)
          when "HEAD" then response = http.head(path, header)
          else
            raise HTTPStatus::MethodNotAllowed,
              "unsupported method `#{req.request_method}'."
          end
        }	
end

A helper method might be useful to encapsulate that core functionality,
and let the script writer change the request,path & header before
passing them through (if that's what they want to do).

intercept do |request, path, header|
        path << path.include? '?' : '&' : '?'
        path << "nostyle=true";
        need_a_name_for_this_thing(request, path, header)
end


Any thoughts on the matter?



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050928/fd35b828/attachment.htm

From daniels at pronto.com.au  Wed Sep 28 03:47:56 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Wed, 28 Sep 2005 17:47:56 +1000
Subject: rexml/httpwrite, why must you mock me?
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E3E6A@mlbmail.pronto.com.au>

Changed it to use feed-tools and spit out an atom feed.

The attached fix_xml also contains my changes to allow the interception
of requests -- not that I need that any longer as Bob Aman showed me how
to use feed-tools to process a string rather than a URL.

Should be able to now go convertifying anything I want once I set up a
mount. The only bit that should need to change between feeds is the
xpath for finding the full content. I've changed the interface for the
DocumentConverters so that they have access to the request and response
objects (so they can discover further information and change the
contenttype on the way out). I'm thinking that the parsing of the
document and the outputting of the document should probably be handed
off to separate objects (in which case, I'd just have a
MouseHole::Converter module with a bunch of constants that are procs to
handle conversion).

But before that, I think the first step will be to add in caching of the
downloaded full-content pages. I don't think it's very nice to grab
everybody pages constantly. 

This is getting fun.

------

MouseHole.script do
	name "Slashdot Fullfeed RSS"
	namespace "http://members.iinet.net.au/~soxbox/"
	description "Converts the slashdot RSS feed to a full-content
feed"
	include_match "*"
	version "0.1"
	document_converter FeedToolsDocumentConverter.new
	rewrite do |req,res|
		document.items.each do |feed_item|
			p feed_item.link
			doc = read_xhtml_from(feed_item.link +
"&mode=nocomment")
			s = ""
			doc.each_element('//div[@class="intro"]') do |x|
				x.write(s)
			end
			feed_item.description = s
		end
	end
end 

-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Grant
Hollingworth
Sent: Tuesday, 27 September 2005 10:50 PM
To: mousehole-scripters at rubyforge.org
Subject: Re: rexml/httpwrite, why must you mock me?

* Daniel Sheppard <daniels at pronto.com.au> [2005-09-27 00:40]:
> However, I don't know my way around the ruby RSS libraries enough to 
> do this yet - After parsing a feed with SimpleRss or the stdlib Rss, 
> how do you turn that data back into an RSS feed?

I've been thinking about adding feed support for the last week.  I'm
glad you actually did it.

I was going to use FeedTools.  It has a lot more dependencies than the
two you tried, but it should allow for easy rebuilding of a feed.

Once you have that abstraction in place, you could have a script that
changes links to full pages for any feed, not just Slashdot.  Users
could add the feeds they want using mouseHole's web interface.
_______________________________________________
Mousehole-scripters mailing list
Mousehole-scripters at rubyforge.org
http://rubyforge.org/mailman/listinfo/mousehole-scripters



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_xml.rb
Type: application/octet-stream
Size: 8404 bytes
Desc: fix_xml.rb
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050928/b012a150/fix_xml.obj

From nicolas.delsaux at gmail.com  Wed Sep 28 05:58:14 2005
From: nicolas.delsaux at gmail.com (Nicolas Delsaux)
Date: Wed, 28 Sep 2005 11:58:14 +0200
Subject: scripts location ?
Message-ID: <418361960509280258716d33b1@mail.gmail.com>

Just a small question from a beginner. Where are the scripts located
once installed ? I would like to create one, but don't know where to
put it.
--
Nicolas Delsaux
Depuis GMail qui vous espionne, mais qui est bien
Et pour ceux que ?a int?resse, j'ai 50 invitations


From splaestro at gmail.com  Wed Sep 28 10:26:59 2005
From: splaestro at gmail.com (Brian C Young)
Date: Wed, 28 Sep 2005 09:26:59 -0500
Subject: scripts location ?
In-Reply-To: <418361960509280258716d33b1@mail.gmail.com>
References: <418361960509280258716d33b1@mail.gmail.com>
Message-ID: <433AA833.6020208@gmail.com>

Nicolas Delsaux wrote:

>Just a small question from a beginner. Where are the scripts located
>once installed ? I would like to create one, but don't know where to
>put it.
>--
>  
>
On  Windows, looks like it's %APPDATA%\MouseHole\userScripts.  Linux/Mac 
equivalent is probably in an analogous directory structure under 
/home/$user somewhere...

Brian

From dougkearns at gmail.com  Wed Sep 28 11:24:13 2005
From: dougkearns at gmail.com (Doug Kearns)
Date: Thu, 29 Sep 2005 01:24:13 +1000
Subject: scripts location ?
In-Reply-To: <433AA833.6020208@gmail.com>
References: <418361960509280258716d33b1@mail.gmail.com>
	<433AA833.6020208@gmail.com>
Message-ID: <20050928152413.GC861@localhost.localdomain>

On Wed, Sep 28, 2005 at 09:26:59AM -0500, Brian C Young wrote:
> Nicolas Delsaux wrote:
> 
> >Just a small question from a beginner. Where are the scripts located
> >once installed ? I would like to create one, but don't know where to
> >put it.
> >--
> >  
> >
> On  Windows, looks like it's %APPDATA%\MouseHole\userScripts.  Linux/Mac 
> equivalent is probably in an analogous directory structure under 
> /home/$user somewhere...

~/.mouseHole/userScripts/

Regards,
Doug

From nicolas.delsaux at gmail.com  Wed Sep 28 11:40:42 2005
From: nicolas.delsaux at gmail.com (Nicolas Delsaux)
Date: Wed, 28 Sep 2005 17:40:42 +0200
Subject: My first script
Message-ID: <41836196050928084050e0829d@mail.gmail.com>

Hi, in a long tradition, I've revamped one or two old scripts to
produce my very own link marker.
It simply highlights links to other servers, links using Javascript
and links opening in new frames.
Are you interested in ?
Strangely enough, none of the scripts I'vbe tested so far seems to
work with bloglines. Have you got any explanation ?
--
Nicolas Delsaux
Depuis GMail qui vous espionne, mais qui est bien
Et pour ceux que ?a int?resse, j'ai 50 invitations


From me at benschumacher.com  Wed Sep 28 14:07:46 2005
From: me at benschumacher.com (Ben Schumacher)
Date: Wed, 28 Sep 2005 12:07:46 -0600
Subject: My first script
In-Reply-To: <41836196050928084050e0829d@mail.gmail.com>
References: <41836196050928084050e0829d@mail.gmail.com>
Message-ID: <9859143f050928110729dc3358@mail.gmail.com>

I'm relatively certain that Bloglines does much of their rendering of
"my feeds" (http://www.bloglines.com/myblogs) with Javascript. I
wonder if that could be the reason?

On 9/28/05, Nicolas Delsaux <nicolas.delsaux at gmail.com> wrote:
> Hi, in a long tradition, I've revamped one or two old scripts to
> produce my very own link marker.
> It simply highlights links to other servers, links using Javascript
> and links opening in new frames.
> Are you interested in ?
> Strangely enough, none of the scripts I'vbe tested so far seems to
> work with bloglines. Have you got any explanation ?
> --
> Nicolas Delsaux
> Depuis GMail qui vous espionne, mais qui est bien
> Et pour ceux que ?a int?resse, j'ai 50 invitations
>
> _______________________________________________
> Mousehole-scripters mailing list
> Mousehole-scripters at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mousehole-scripters
>


From why at hobix.com  Wed Sep 28 15:23:32 2005
From: why at hobix.com (why the lucky stiff)
Date: Wed, 28 Sep 2005 13:23:32 -0600
Subject: ! MouseHole 1.2 is out
Message-ID: <433AEDB4.7040307@hobix.com>

Hey.  MouseHole 1.2 is done.

  * Windows installer
    <http://rubyforge.org/frs/download.php/6262/mouseHole-1.2-win32.exe>

  * Source zip
    <http://rubyforge.org/frs/download.php/6260/mouseHole-1.2.zip>
   (Ruby, yaml, dbm, rexml libraries required.)

  * Source tarball
    <http://rubyforge.org/frs/download.php/6259/mouseHole-1.2.tar.bz2>
    (Ruby, yaml, dbm, rexml libraries required.)

I'm attaching the Changlog for convenience.

_why
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CHANGELOG
Url: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050928/21151507/CHANGELOG.bat

From me at benschumacher.com  Wed Sep 28 16:31:07 2005
From: me at benschumacher.com (Ben Schumacher)
Date: Wed, 28 Sep 2005 14:31:07 -0600
Subject: ! MouseHole 1.2 is out
In-Reply-To: <433AEDB4.7040307@hobix.com>
References: <433AEDB4.7040307@hobix.com>
Message-ID: <9859143f0509281331703c5328@mail.gmail.com>

Q: (cause i'm too lazy to figure this out myself) Does this Windows
installer version default to Tidy or HTree?

bs. (aka rynok)


From why at hobix.com  Wed Sep 28 17:18:48 2005
From: why at hobix.com (why the lucky stiff)
Date: Wed, 28 Sep 2005 15:18:48 -0600
Subject: ! MouseHole 1.2 is out
In-Reply-To: <9859143f0509281331703c5328@mail.gmail.com>
References: <433AEDB4.7040307@hobix.com>
	<9859143f0509281331703c5328@mail.gmail.com>
Message-ID: <433B08B8.6030108@hobix.com>

Ben Schumacher wrote:

>Q: (cause i'm too lazy to figure this out myself) Does this Windows
>installer version default to Tidy or HTree?
>
>bs. (aka rynok)
>  
>
Tidy.  Sorry, I should put that in the --help.

_why

From daniels at pronto.com.au  Thu Sep 29 01:59:23 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Thu, 29 Sep 2005 15:59:23 +1000
Subject: ! MouseHole 1.2 is out
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E40A4@mlbmail.pronto.com.au>

There's a bug with the paths. In the scripted_mounts method:

     path_parts = request.path_info.split( '/' ).reject { |x|
x.to_s.strip.size == 0 }
     mount = path_parts.shift.to_s.strip
     each_fresh_script do |path, script|
         if mount =~ /^\/*#{ script.mount }$/
             script.do_mount( path_parts.join( '/' ), request, response
)
             no_cache response
             return
         end
     end 

Why is the reject { |x| x.to_s.strip.size == 0 } there? It has the
consequence that any double forward slash is stripped from the path (so
that urls cannot be included in the url for a mount). I take it that
it's there to filter out localhost:37004//cmd/ ? Would the following
suffice:

     path_parts = request.path_info.split( '/' )
     begin
         mount = path_parts.shift 
     end until mount.to_s.strip.size > 0 || path_parts.empty?


-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of why the
lucky stiff
Sent: Thursday, 29 September 2005 7:19 AM
To: Ben Schumacher; mousehole-scripters at rubyforge.org
Subject: Re: ! MouseHole 1.2 is out

Ben Schumacher wrote:

>Q: (cause i'm too lazy to figure this out myself) Does this Windows 
>installer version default to Tidy or HTree?
>
>bs. (aka rynok)
>  
>
Tidy.  Sorry, I should put that in the --help.

_why
_______________________________________________
Mousehole-scripters mailing list
Mousehole-scripters at rubyforge.org
http://rubyforge.org/mailman/listinfo/mousehole-scripters


#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################


From kevin at sb.org  Thu Sep 29 02:02:11 2005
From: kevin at sb.org (Kevin Ballard)
Date: Thu, 29 Sep 2005 02:02:11 -0400
Subject: ! MouseHole 1.2 is out
In-Reply-To: <EE0A2C5DE8D7CA4792891604D4FED52E034E40A4@mlbmail.pronto.com.au>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E40A4@mlbmail.pronto.com.au>
Message-ID: <E8C381DA-C4B8-4886-BD2B-426EAD04B016@sb.org>

I believe it's because /foo//bar and /foo/bar are equivalent URLs,  
and so stripping double-slashes everywhere makes the comparison work  
better.

If you want to include a URL in the mount, look at what ProxyLike  
does. It registers http: and then just takes the path and pretends it  
starts with //.

On Sep 29, 2005, at 1:59 AM, Daniel Sheppard wrote:

> There's a bug with the paths. In the scripted_mounts method:
>
>      path_parts = request.path_info.split( '/' ).reject { |x|
> x.to_s.strip.size == 0 }
>      mount = path_parts.shift.to_s.strip
>      each_fresh_script do |path, script|
>          if mount =~ /^\/*#{ script.mount }$/
>              script.do_mount( path_parts.join( '/' ), request,  
> response
> )
>              no_cache response
>              return
>          end
>      end
>
> Why is the reject { |x| x.to_s.strip.size == 0 } there? It has the
> consequence that any double forward slash is stripped from the path  
> (so
> that urls cannot be included in the url for a mount). I take it that
> it's there to filter out localhost:37004//cmd/ ? Would the following
> suffice:
>
>      path_parts = request.path_info.split( '/' )
>      begin
>          mount = path_parts.shift
>      end until mount.to_s.strip.size > 0 || path_parts.empty?

-- 
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2378 bytes
Desc: not available
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050929/33b97f33/smime.bin

From daniels at pronto.com.au  Thu Sep 29 02:08:56 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Thu, 29 Sep 2005 16:08:56 +1000
Subject: ! MouseHole 1.2 is out
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E40B5@mlbmail.pronto.com.au>

/foo//bar may be equivalent to /foo/bar (I think that's just a
convention though, not the http spec), but there should be a definite
difference between /foo%2F%2Fbar and /foo%2Fbar 

But it appears that the bug is in Net::HTTP, not mousehole - the extra
backslash has already be stripped before mousehole gets its paws on
things.

I'll take the ProxyLike route.

-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Kevin
Ballard
Sent: Thursday, 29 September 2005 4:02 PM
To: mousehole-scripters at rubyforge.org
Subject: Re: ! MouseHole 1.2 is out

I believe it's because /foo//bar and /foo/bar are equivalent URLs, and
so stripping double-slashes everywhere makes the comparison work better.

If you want to include a URL in the mount, look at what ProxyLike does.
It registers http: and then just takes the path and pretends it starts
with //.

On Sep 29, 2005, at 1:59 AM, Daniel Sheppard wrote:

> There's a bug with the paths. In the scripted_mounts method:
>
>      path_parts = request.path_info.split( '/' ).reject { |x| 
> x.to_s.strip.size == 0 }
>      mount = path_parts.shift.to_s.strip
>      each_fresh_script do |path, script|
>          if mount =~ /^\/*#{ script.mount }$/
>              script.do_mount( path_parts.join( '/' ), request, 
> response
> )
>              no_cache response
>              return
>          end
>      end
>
> Why is the reject { |x| x.to_s.strip.size == 0 } there? It has the 
> consequence that any double forward slash is stripped from the path 
> (so that urls cannot be included in the url for a mount). I take it 
> that it's there to filter out localhost:37004//cmd/ ? Would the 
> following
> suffice:
>
>      path_parts = request.path_info.split( '/' )
>      begin
>          mount = path_parts.shift
>      end until mount.to_s.strip.size > 0 || path_parts.empty?

--
Kevin Ballard
kevin at sb.org
http://www.tildesoft.com
http://kevin.sb.org



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################


From daniels at pronto.com.au  Thu Sep 29 02:56:24 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Thu, 29 Sep 2005 16:56:24 +1000
Subject: RSS feed manipulation is alive
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E40E9@mlbmail.pronto.com.au>

 <<fix_xml.rb>> Requires some fiddling. You need to require the fix_xml
in your mousehole launcher, this provides:

- the ability to manipulate RSS/ATOM feeds
	- only for application/xml+atom and text/xml mime types
	- all output will be as an ATOM 1.0 feed

- the ability to cancel the manipulation of a page by raising
MouseHole::CancelRewrite
	- this allows you to decide whether or not your script should
execute from within the script.
	- helps to avoid both the overhead and error potential of page
reconstruction.

Then you can use the attached "Atomic Mouse" user script, wherein you
enter in an RSS/ATOM path and an Xpath for extracting from the
full-content page to create a full-content ATOM feed. The script will
cache page content so that you don't incur the wrath of an angry
webmaster. The cache is cleared upon modification of the xpath rule -
I'll probably change it to also keep an eye on the last updated time on
the feed entry as well.



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
A non-text attachment was scrubbed...
Name: atomic.user.rb
Type: application/octet-stream
Size: 15538 bytes
Desc: atomic.user.rb
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050929/5136f9c3/atomic.user-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_xml.rb
Type: application/octet-stream
Size: 5164 bytes
Desc: fix_xml.rb
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050929/5136f9c3/fix_xml-0001.obj

From daniels at pronto.com.au  Thu Sep 29 03:05:19 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Thu, 29 Sep 2005 17:05:19 +1000
Subject: RSS feed manipulation is alive
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E40F8@mlbmail.pronto.com.au>

Bah, it seems every time I make a post anywhere this week, it's followed
by a correction.

It's currently only picking up the application/xml and text/xml mime
types - if you want application/atom+xml, you'll have to edit line 29. 

-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Daniel
Sheppard
Sent: Thursday, 29 September 2005 4:56 PM
To: mousehole-scripters at rubyforge.org
Subject: RSS feed manipulation is alive

 <<fix_xml.rb>> Requires some fiddling. You need to require the fix_xml
in your mousehole launcher, this provides:

- the ability to manipulate RSS/ATOM feeds
	- only for application/xml+atom and text/xml mime types
	- all output will be as an ATOM 1.0 feed

- the ability to cancel the manipulation of a page by raising
MouseHole::CancelRewrite
	- this allows you to decide whether or not your script should
execute from within the script.
	- helps to avoid both the overhead and error potential of page
reconstruction.

Then you can use the attached "Atomic Mouse" user script, wherein you
enter in an RSS/ATOM path and an Xpath for extracting from the
full-content page to create a full-content ATOM feed. The script will
cache page content so that you don't incur the wrath of an angry
webmaster. The cache is cleared upon modification of the xpath rule -
I'll probably change it to also keep an eye on the last updated time on
the feed entry as well.



########################################################################
#############
This email has been scanned by MailMarshal, an email content filter.
########################################################################
#############


#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################


From why at hobix.com  Thu Sep 29 03:06:51 2005
From: why at hobix.com (why the lucky stiff)
Date: Thu, 29 Sep 2005 01:06:51 -0600
Subject: RSS feed manipulation is alive
In-Reply-To: <EE0A2C5DE8D7CA4792891604D4FED52E034E40E9@mlbmail.pronto.com.au>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E40E9@mlbmail.pronto.com.au>
Message-ID: <433B928B.1080201@hobix.com>

Daniel Sheppard wrote:

> <<fix_xml.rb>> Requires some fiddling. You need to require the fix_xml
>in your mousehole launcher, this provides:
>  
>
Daniel, you're going to leave a tear in the universe at this rate.  Very 
speedy, very magnificent.  I'll look at this in the morning and get it 
checked in.  I'm aiming for MouseHole 1.3 in two weeks.

_why

From daniels at pronto.com.au  Thu Sep 29 03:07:48 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Thu, 29 Sep 2005 17:07:48 +1000
Subject: At least I learn from my mistakes
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E40FC@mlbmail.pronto.com.au>

Because I just did the same thing, I noticed that the mime-type matching
is wrong:

                elsif @conf[:rewrites_on]
                    case res.content_type
                    when /^text\/html/, /^application\/xhtml+xml/

you need to escape the + in the regex.

#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050929/5ac6c6ac/attachment.htm

From daniels at pronto.com.au  Thu Sep 29 03:20:15 2005
From: daniels at pronto.com.au (Daniel Sheppard)
Date: Thu, 29 Sep 2005 17:20:15 +1000
Subject: RSS feed manipulation is alive
Message-ID: <EE0A2C5DE8D7CA4792891604D4FED52E034E4104@mlbmail.pronto.com.au>

Bah, bugs abound.

This fixes the mime-types and also no longer barfs when it tries to
parse a page and it can't. 

Mousehole laughs in the face of partial-content feeds.

-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Daniel
Sheppard
Sent: Thursday, 29 September 2005 5:05 PM
To: mousehole-scripters at rubyforge.org
Subject: RE: RSS feed manipulation is alive

Bah, it seems every time I make a post anywhere this week, it's followed
by a correction.

It's currently only picking up the application/xml and text/xml mime
types - if you want application/atom+xml, you'll have to edit line 29. 

-----Original Message-----
From: mousehole-scripters-bounces at rubyforge.org
[mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Daniel
Sheppard
Sent: Thursday, 29 September 2005 4:56 PM
To: mousehole-scripters at rubyforge.org
Subject: RSS feed manipulation is alive

 <<fix_xml.rb>> Requires some fiddling. You need to require the fix_xml
in your mousehole launcher, this provides:

- the ability to manipulate RSS/ATOM feeds
	- only for application/xml+atom and text/xml mime types
	- all output will be as an ATOM 1.0 feed

- the ability to cancel the manipulation of a page by raising
MouseHole::CancelRewrite
	- this allows you to decide whether or not your script should
execute from within the script.
	- helps to avoid both the overhead and error potential of page
reconstruction.

Then you can use the attached "Atomic Mouse" user script, wherein you
enter in an RSS/ATOM path and an Xpath for extracting from the
full-content page to create a full-content ATOM feed. The script will
cache page content so that you don't incur the wrath of an angry
webmaster. The cache is cleared upon modification of the xpath rule -
I'll probably change it to also keep an eye on the last updated time on
the feed entry as well.



########################################################################
#############
This email has been scanned by MailMarshal, an email content filter.
########################################################################
#############


########################################################################
#############
This email has been scanned by MailMarshal, an email content filter.
########################################################################
#############

_______________________________________________
Mousehole-scripters mailing list
Mousehole-scripters at rubyforge.org
http://rubyforge.org/mailman/listinfo/mousehole-scripters



#####################################################################################
This email has been scanned by MailMarshal, an email content filter.
#####################################################################################
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_xml.rb
Type: application/octet-stream
Size: 5320 bytes
Desc: fix_xml.rb
Url : http://rubyforge.org/pipermail/mousehole-scripters/attachments/20050929/c27b2a39/fix_xml.obj

From nicolas.delsaux at gmail.com  Thu Sep 29 04:51:29 2005
From: nicolas.delsaux at gmail.com (Nicolas Delsaux)
Date: Thu, 29 Sep 2005 10:51:29 +0200
Subject: My first script
In-Reply-To: <41836196050928084050e0829d@mail.gmail.com>
References: <41836196050928084050e0829d@mail.gmail.com>
Message-ID: <41836196050929015138b78732@mail.gmail.com>

On 9/28/05, Nicolas Delsaux <nicolas.delsaux at gmail.com> wrote:
> It simply highlights links to other servers, links using Javascript
> and links opening in new frames.

I've added it to the wiki. Could some Ruby experts take a glance at it ?
--
Nicolas Delsaux
Depuis GMail qui vous espionne, mais qui est bien
Et pour ceux que ?a int?resse, j'ai 50 invitations


From me at benschumacher.com  Thu Sep 29 12:04:45 2005
From: me at benschumacher.com (Ben Schumacher)
Date: Thu, 29 Sep 2005 10:04:45 -0600
Subject: ! MouseHole 1.2 is out
In-Reply-To: <EE0A2C5DE8D7CA4792891604D4FED52E034E40B5@mlbmail.pronto.com.au>
References: <EE0A2C5DE8D7CA4792891604D4FED52E034E40B5@mlbmail.pronto.com.au>
Message-ID: <9859143f0509290904501c1a89@mail.gmail.com>

I think it's lower level than just by convention at the HTTP layer. It
probably has more to do with filesystem behavior. I would venture a
guess that nearly all *nix operating systems treat "//" as a single
"/" and I have just confirmed on the Windows box I'm on that it
behaves the same way (and I am certain that FreeBSD does as well). I
think the idea is to be forgiving of users.

That being said, I think it would be nice to pass it through for the
benifit of things like ProxyLike. Maybe we can add the ability to get
the non-normalized URL...?

Just a thought.

bs.

On 9/29/05, Daniel Sheppard <daniels at pronto.com.au> wrote:
> /foo//bar may be equivalent to /foo/bar (I think that's just a
> convention though, not the http spec), but there should be a definite
> difference between /foo%2F%2Fbar and /foo%2Fbar
>
> But it appears that the bug is in Net::HTTP, not mousehole - the extra
> backslash has already be stripped before mousehole gets its paws on
> things.
>
> I'll take the ProxyLike route.
>
> -----Original Message-----
> From: mousehole-scripters-bounces at rubyforge.org
> [mailto:mousehole-scripters-bounces at rubyforge.org] On Behalf Of Kevin
> Ballard
> Sent: Thursday, 29 September 2005 4:02 PM
> To: mousehole-scripters at rubyforge.org
> Subject: Re: ! MouseHole 1.2 is out
>
> I believe it's because /foo//bar and /foo/bar are equivalent URLs, and
> so stripping double-slashes everywhere makes the comparison work better.
>
> If you want to include a URL in the mount, look at what ProxyLike does.
> It registers http: and then just takes the path and pretends it starts
> with //.
>
> On Sep 29, 2005, at 1:59 AM, Daniel Sheppard wrote:
>
> > There's a bug with the paths. In the scripted_mounts method:
> >
> >      path_parts = request.path_info.split( '/' ).reject { |x|
> > x.to_s.strip.size == 0 }
> >      mount = path_parts.shift.to_s.strip
> >      each_fresh_script do |path, script|
> >          if mount =~ /^\/*#{ script.mount }$/
> >              script.do_mount( path_parts.join( '/' ), request,
> > response
> > )
> >              no_cache response
> >              return
> >          end
> >      end
> >
> > Why is the reject { |x| x.to_s.strip.size == 0 } there? It has the
> > consequence that any double forward slash is stripped from the path
> > (so that urls cannot be included in the url for a mount). I take it
> > that it's there to filter out localhost:37004//cmd/ ? Would the
> > following
> > suffice:
> >
> >      path_parts = request.path_info.split( '/' )
> >      begin
> >          mount = path_parts.shift
> >      end until mount.to_s.strip.size > 0 || path_parts.empty?
>
> --
> Kevin Ballard
> kevin at sb.org
> http://www.tildesoft.com
> http://kevin.sb.org
>
>
>
> #####################################################################################
> This email has been scanned by MailMarshal, an email content filter.
> #####################################################################################
>
> _______________________________________________
> Mousehole-scripters mailing list
> Mousehole-scripters at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mousehole-scripters
>