From drbrain at segment7.net  Sat Apr  9 21:31:51 2011
From: drbrain at segment7.net (Eric Hodel)
Date: Sat, 9 Apr 2011 18:31:51 -0700
Subject: [Mechanize-users] [ANN] mechanize 2.0.pre.1
Message-ID: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>

I have released Mechanize 2.0.pre.1 to rubygems.org.

  gem install --pre mechanize

My goal for mechanize is to move the HTTP layer into its own gem and hopefully into ruby itself.  The main changes to support this are the removal of Mechanize::Chain and the removal of the options hash that #fetch_page supported.

Passing an options hash to the mechanize methods #get, #head, #post, #put etc. will be deprecated in 2.0 (this has not been done yet).

Changes so far:

Mechanize is now under the MIT license

* API changes
  * WWW::Mechanize has been removed.
  * Pre connect hooks are now called with the agent and the request.  See
    Mechanize#pre_connect_hooks.
  * Post connect hooks are now called with the agent and the response.  See
    Mechanize#post_connect_hooks.
  * Mechanize::Chain is gone, as an internal API this should cause no problems.
  * Mechanize#fetch_page no longer accepts an options Hash.
  * Mechanize#put now accepts headers instead of an options Hash as the last
    argument
  * Mechanize#delete now accepts headers instead of an options Hash as the
    last argument
  * Mechanize#request_with_entity now accepts headers instead of an options
    Hash as the last argument
  * Mechanize no longer raises RuntimeError directly, Mechanize::Error or
    ArgumentError are raised instead.
  * The User-Agent header has changed.  It no longer includes the WWW- prefix
    and now includes the ruby version.  The URL has been updated as well.
  * Mechanize now requires ruby 1.8.7 or newer.

* New Features

  * Add header reference methods to Mechanize::File so that a reponse
    object gets compatible with Net::HTTPResponse.
  * Mechanize#click accepts a regexp or string to click a button/link in the
    current page. It works as expected when not passed a string or regexp.
  * Provide a way to only follow permanent redirects (301)
    automatically: <tt>agent.redirect_ok = :permanent</tt>  GH #73
  * Documented various Mechanize accessors.  GH #66
  * Mechanize now uses net-http-digest_auth.  GH #31
  * Mechanize now implements session cookies.  GH #78
  * Mechanize now implements deflate decoding.  GH #40
  * Mechanize now allows a certificate and key to be passed directly.  GH #71
  * Mechanize::Form::MultiSelectList now implements #option_with and
    #options_with.  GH #42
  * Add Mechanize::Page::Link#rel and #rel?(kind) to read and test the rel
    attribute.
  * Add Mechanize::Page#canonical_uri to read a </tt><link
    rel="canonical"></tt> tag.
  * Add support for Robots Exclusion Protocol (i.e. robots.txt) and
    nofollow/noindex in meta tags and the rel attribute.  Automatic
    exclusion can be turned on by setting:
      agent.robots = true
  * Manual robots.txt test can be performed with
    Mechanize#robots_allowed? and #robots_disallowed?.

* Bug Fixes:

  * Fixed a bug where Referer is not sent when accessing a relative
    URI starting with "http".
  * Fix handling of Meta Refresh with relative paths.  GH #39
  * Mechanize::CookieJar now supports RFC 2109 correctly.  GH #85
  * Fixed typo in EXAMPLES.rdoc.  GH #74
  * The base element is now handled correctly for images.  GH #72
  * Image buttons with no name attribute are now included in the form's button
    list.  GH#56
  * Improved handling of non ASCII-7bit compatible characters in links (only
    an issue on ruby 1.8).  GH #36, GH #75
  * Loading cookies.txt is faster.  GH #38
  * Mechanize no longer sends cookies for a.b.example to axb.example.  GH #41
  * Mechanize no longer sends the button name as a form field for image
    buttons.  GH #45
  * Blank cookie values are now skipped.  GH #80
  * Mechanize now adds a '.' to cookie domains if no '.' was sent.  This is
    not allowed by RFC 2109 but does appear in RFC 2965.  GH #86
  * file URIs are now read in binary mode.  GH #83
  * Content-Encoding: x-gzip is now treated like gzip per RFC 2616.
  * Mechanize now unescapes URIs for meta refresh.  GH #68
  * Mechanize now has more robust HTML charset detection.  GH #43


From ermaker at gmail.com  Mon Apr 11 04:06:43 2011
From: ermaker at gmail.com (Minwoo Lee)
Date: Mon, 11 Apr 2011 17:06:43 +0900
Subject: [Mechanize-users] html parsing problem with encoding
Message-ID: <BANLkTikUOVhxMQ0yB75USfZo4oUKJbktSw@mail.gmail.com>

Hi.
I have a question about parsing and encoding.

I tried this out.

>> agent = Mechanize.new
>> p = agent.get('http://log.kaist.ac.kr/enc.html')
>> p.forms
=> []

However, it has a form.
I think the reason is encoding of 'enc.html'

'enc.html' is with encoding 'euc-kr' and it conflicts with the parser.

I tried out
>> p.encoding = 'utf-8'
or
>> p.encoding = 'euc-kr'
but it doesn't work.


Thank you for reading this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20110411/3c3605f4/attachment.html>

From drbrain at segment7.net  Mon Apr 11 13:38:40 2011
From: drbrain at segment7.net (Eric Hodel)
Date: Mon, 11 Apr 2011 10:38:40 -0700
Subject: [Mechanize-users] html parsing problem with encoding
In-Reply-To: <BANLkTikUOVhxMQ0yB75USfZo4oUKJbktSw@mail.gmail.com>
References: <BANLkTikUOVhxMQ0yB75USfZo4oUKJbktSw@mail.gmail.com>
Message-ID: <7806C174-EC52-41C7-A029-5504EEBAF59C@segment7.net>

On Apr 11, 2011, at 1:06 AM, Minwoo Lee wrote:

> Hi.
> I have a question about parsing and encoding.
> 
> I tried this out.
> 
> >> agent = Mechanize.new
> >> p = agent.get('http://log.kaist.ac.kr/enc.html')
> >> p.forms
> => []
> 
> However, it has a form.
> I think the reason is encoding of 'enc.html'
> 
> 'enc.html' is with encoding 'euc-kr' and it conflicts with the parser.
> 
> I tried out
> >> p.encoding = 'utf-8'
> or
> >> p.encoding = 'euc-kr'
> but it doesn't work.

This appears to work with the mechanize beta.  Try `gem install mechanize --pre` or install from source.

From barjunk at attglobal.net  Mon Apr 11 22:07:30 2011
From: barjunk at attglobal.net (barsalou)
Date: Mon, 11 Apr 2011 18:07:30 -0800
Subject: [Mechanize-users] [ANN] mechanize 2.0.pre.1
In-Reply-To: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
References: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
Message-ID: <20110411180730.az7dp1on400ss80s@192.168.0.101>

Quoting Eric Hodel <drbrain at segment7.net>:

> I have released Mechanize 2.0.pre.1 to rubygems.org.
>
>   gem install --pre mechanize
<snip>
> Changes so far:
>
> Mechanize is now under the MIT license
>
> * API changes
>   * WWW::Mechanize has been removed.
>   * Pre connect hooks are now called with the agent and the request.  See
>     Mechanize#pre_connect_hooks.
>   * Post connect hooks are now called with the agent and the response.  See
>     Mechanize#post_connect_hooks.
>   * Mechanize::Chain is gone, as an internal API this should cause   
> no problems.
>   * Mechanize#fetch_page no longer accepts an options Hash.
>   * Mechanize#put now accepts headers instead of an options Hash as the last
>     argument
>   * Mechanize#delete now accepts headers instead of an options Hash as the
>     last argument
>   * Mechanize#request_with_entity now accepts headers instead of an options
>     Hash as the last argument
>   * Mechanize no longer raises RuntimeError directly, Mechanize::Error or
>     ArgumentError are raised instead.
>   * The User-Agent header has changed.  It no longer includes the WWW- prefix
>     and now includes the ruby version.  The URL has been updated as well.
>   * Mechanize now requires ruby 1.8.7 or newer.
>
> * New Features
>
>   * Add header reference methods to Mechanize::File so that a reponse
>     object gets compatible with Net::HTTPResponse.
>   * Mechanize#click accepts a regexp or string to click a button/link in the
>     current page. It works as expected when not passed a string or regexp.
>   * Provide a way to only follow permanent redirects (301)
>     automatically: <tt>agent.redirect_ok = :permanent</tt>  GH #73
>   * Documented various Mechanize accessors.  GH #66
>   * Mechanize now uses net-http-digest_auth.  GH #31
>   * Mechanize now implements session cookies.  GH #78
>   * Mechanize now implements deflate decoding.  GH #40
>   * Mechanize now allows a certificate and key to be passed directly.  GH #71
>   * Mechanize::Form::MultiSelectList now implements #option_with and
>     #options_with.  GH #42
>   * Add Mechanize::Page::Link#rel and #rel?(kind) to read and test the rel
>     attribute.
>   * Add Mechanize::Page#canonical_uri to read a </tt><link
>     rel="canonical"></tt> tag.
>   * Add support for Robots Exclusion Protocol (i.e. robots.txt) and
>     nofollow/noindex in meta tags and the rel attribute.  Automatic
>     exclusion can be turned on by setting:
>       agent.robots = true
>   * Manual robots.txt test can be performed with
>     Mechanize#robots_allowed? and #robots_disallowed?.
>
> * Bug Fixes:
>
>   * Fixed a bug where Referer is not sent when accessing a relative
>     URI starting with "http".
>   * Fix handling of Meta Refresh with relative paths.  GH #39
>   * Mechanize::CookieJar now supports RFC 2109 correctly.  GH #85
>   * Fixed typo in EXAMPLES.rdoc.  GH #74
>   * The base element is now handled correctly for images.  GH #72
>   * Image buttons with no name attribute are now included in the   
> form's button
>     list.  GH#56
>   * Improved handling of non ASCII-7bit compatible characters in links (only
>     an issue on ruby 1.8).  GH #36, GH #75
>   * Loading cookies.txt is faster.  GH #38
>   * Mechanize no longer sends cookies for a.b.example to axb.example.  GH #41
>   * Mechanize no longer sends the button name as a form field for image
>     buttons.  GH #45
>   * Blank cookie values are now skipped.  GH #80
>   * Mechanize now adds a '.' to cookie domains if no '.' was sent.  This is
>     not allowed by RFC 2109 but does appear in RFC 2965.  GH #86
>   * file URIs are now read in binary mode.  GH #83
>   * Content-Encoding: x-gzip is now treated like gzip per RFC 2616.
>   * Mechanize now unescapes URIs for meta refresh.  GH #68
>   * Mechanize now has more robust HTML charset detection.  GH #43
>

Thanks for all the work!

What drove the change to the MIT license?

Mike B.

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.


From drbrain at segment7.net  Tue Apr 12 16:31:10 2011
From: drbrain at segment7.net (Eric Hodel)
Date: Tue, 12 Apr 2011 13:31:10 -0700
Subject: [Mechanize-users] [ANN] mechanize 2.0.pre.1
In-Reply-To: <20110411180730.az7dp1on400ss80s@192.168.0.101>
References: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
	<20110411180730.az7dp1on400ss80s@192.168.0.101>
Message-ID: <AC7CEDC8-5551-43C6-ABF6-7E3A7AD48371@segment7.net>

On Apr 11, 2011, at 7:07 PM, barsalou wrote:

> Quoting Eric Hodel <drbrain at segment7.net>:
> 
>> I have released Mechanize 2.0.pre.1 to rubygems.org.
>> 
>>  gem install --pre mechanize
> 
> Thanks for all the work!
> 
> What drove the change to the MIT license?

I would like to pull the HTTP parts of mechanize away from the browser parts and contribute that back to ruby.  Since ruby has a dual BSD/ruby license now I needed a compatible license before I could begin the work.

From apoc at sixserv.org  Tue Apr 12 18:22:06 2011
From: apoc at sixserv.org (Matthias -apoc- Hecker)
Date: Wed, 13 Apr 2011 00:22:06 +0200
Subject: [Mechanize-users] [ANN] mechanize 2.0.pre.1
In-Reply-To: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
References: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
Message-ID: <4DA4D08E.7090008@sixserv.org>

Wow 2.0! :) I would also like to thank you for your amazing work with
this outstanding library!

On 04/10/2011 03:31 AM, Eric Hodel wrote:
> I have released Mechanize 2.0.pre.1 to rubygems.org.
> 
>   gem install --pre mechanize
> 
> My goal for mechanize is to move the HTTP layer into its own gem and hopefully into ruby itself.  The main changes to support this are the removal of Mechanize::Chain and the removal of the options hash that #fetch_page supported.
> 
> Passing an options hash to the mechanize methods #get, #head, #post, #put etc. will be deprecated in 2.0 (this has not been done yet).
> 
> Changes so far:
> 
> Mechanize is now under the MIT license
> 
> * API changes
>   * WWW::Mechanize has been removed.
>   * Pre connect hooks are now called with the agent and the request.  See
>     Mechanize#pre_connect_hooks.
>   * Post connect hooks are now called with the agent and the response.  See
>     Mechanize#post_connect_hooks.
>   * Mechanize::Chain is gone, as an internal API this should cause no problems.
>   * Mechanize#fetch_page no longer accepts an options Hash.
>   * Mechanize#put now accepts headers instead of an options Hash as the last
>     argument
>   * Mechanize#delete now accepts headers instead of an options Hash as the
>     last argument
>   * Mechanize#request_with_entity now accepts headers instead of an options
>     Hash as the last argument
>   * Mechanize no longer raises RuntimeError directly, Mechanize::Error or
>     ArgumentError are raised instead.
>   * The User-Agent header has changed.  It no longer includes the WWW- prefix
>     and now includes the ruby version.  The URL has been updated as well.
>   * Mechanize now requires ruby 1.8.7 or newer.
> 
> * New Features
> 
>   * Add header reference methods to Mechanize::File so that a reponse
>     object gets compatible with Net::HTTPResponse.
>   * Mechanize#click accepts a regexp or string to click a button/link in the
>     current page. It works as expected when not passed a string or regexp.
>   * Provide a way to only follow permanent redirects (301)
>     automatically: <tt>agent.redirect_ok = :permanent</tt>  GH #73
>   * Documented various Mechanize accessors.  GH #66
>   * Mechanize now uses net-http-digest_auth.  GH #31
>   * Mechanize now implements session cookies.  GH #78
>   * Mechanize now implements deflate decoding.  GH #40
>   * Mechanize now allows a certificate and key to be passed directly.  GH #71
>   * Mechanize::Form::MultiSelectList now implements #option_with and
>     #options_with.  GH #42
>   * Add Mechanize::Page::Link#rel and #rel?(kind) to read and test the rel
>     attribute.
>   * Add Mechanize::Page#canonical_uri to read a </tt><link
>     rel="canonical"></tt> tag.
>   * Add support for Robots Exclusion Protocol (i.e. robots.txt) and
>     nofollow/noindex in meta tags and the rel attribute.  Automatic
>     exclusion can be turned on by setting:
>       agent.robots = true
>   * Manual robots.txt test can be performed with
>     Mechanize#robots_allowed? and #robots_disallowed?.
> 
> * Bug Fixes:
> 
>   * Fixed a bug where Referer is not sent when accessing a relative
>     URI starting with "http".
>   * Fix handling of Meta Refresh with relative paths.  GH #39
>   * Mechanize::CookieJar now supports RFC 2109 correctly.  GH #85
>   * Fixed typo in EXAMPLES.rdoc.  GH #74
>   * The base element is now handled correctly for images.  GH #72
>   * Image buttons with no name attribute are now included in the form's button
>     list.  GH#56
>   * Improved handling of non ASCII-7bit compatible characters in links (only
>     an issue on ruby 1.8).  GH #36, GH #75
>   * Loading cookies.txt is faster.  GH #38
>   * Mechanize no longer sends cookies for a.b.example to axb.example.  GH #41
>   * Mechanize no longer sends the button name as a form field for image
>     buttons.  GH #45
>   * Blank cookie values are now skipped.  GH #80
>   * Mechanize now adds a '.' to cookie domains if no '.' was sent.  This is
>     not allowed by RFC 2109 but does appear in RFC 2965.  GH #86
>   * file URIs are now read in binary mode.  GH #83
>   * Content-Encoding: x-gzip is now treated like gzip per RFC 2616.
>   * Mechanize now unescapes URIs for meta refresh.  GH #68
>   * Mechanize now has more robust HTML charset detection.  GH #43
> 
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users

From drbrain at segment7.net  Tue Apr 12 23:22:49 2011
From: drbrain at segment7.net (Eric Hodel)
Date: Tue, 12 Apr 2011 20:22:49 -0700
Subject: [Mechanize-users] [ANN] mechanize 2.0.pre.1
In-Reply-To: <4DA4D08E.7090008@sixserv.org>
References: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
	<4DA4D08E.7090008@sixserv.org>
Message-ID: <57BB689C-F018-4B8F-B2BF-F76D0A54E305@segment7.net>

On Apr 12, 2011, at 3:22 PM, Matthias -apoc- Hecker wrote:
> Wow 2.0! :) I would also like to thank you for your amazing work with
> this outstanding library!

I didn't do that much, just moved some stuff around enough to break the api which means the major version must be bumped.

Oh, I applied some patches and fixed some bugs, too.

From apoc at sixserv.org  Wed Apr 13 06:54:26 2011
From: apoc at sixserv.org (Matthias -apoc- Hecker)
Date: Wed, 13 Apr 2011 12:54:26 +0200
Subject: [Mechanize-users] [ANN] mechanize 2.0.pre.1
In-Reply-To: <57BB689C-F018-4B8F-B2BF-F76D0A54E305@segment7.net>
References: <7BE8EA71-69FD-4429-B224-A6F40D0D4913@segment7.net>
	<4DA4D08E.7090008@sixserv.org>
	<57BB689C-F018-4B8F-B2BF-F76D0A54E305@segment7.net>
Message-ID: <4DA580E2.9050001@sixserv.org>

On 04/13/2011 05:22 AM, Eric Hodel wrote:
> On Apr 12, 2011, at 3:22 PM, Matthias -apoc- Hecker wrote:
>> Wow 2.0! :) I would also like to thank you for your amazing work with
>> this outstanding library!
> 
> I didn't do that much, just moved some stuff around enough to break the api which means the major version must be bumped.
> 
> Oh, I applied some patches and fixed some bugs, too.

well still appreciated ;)

From ermaker at gmail.com  Thu Apr 14 07:39:10 2011
From: ermaker at gmail.com (Minwoo Lee)
Date: Thu, 14 Apr 2011 20:39:10 +0900
Subject: [Mechanize-users] html parsing problem with encoding
In-Reply-To: <7806C174-EC52-41C7-A029-5504EEBAF59C@segment7.net>
References: <BANLkTikUOVhxMQ0yB75USfZo4oUKJbktSw@mail.gmail.com>
	<7806C174-EC52-41C7-A029-5504EEBAF59C@segment7.net>
Message-ID: <BANLkTim21C+ORb+UTOg2JTH3g2=YC7__oA@mail.gmail.com>

Thank you!
Your answer is perfect. I solved the problem.

2011/4/12 Eric Hodel <drbrain at segment7.net>

> On Apr 11, 2011, at 1:06 AM, Minwoo Lee wrote:
>
> > Hi.
> > I have a question about parsing and encoding.
> >
> > I tried this out.
> >
> > >> agent = Mechanize.new
> > >> p = agent.get('http://log.kaist.ac.kr/enc.html')
> > >> p.forms
> > => []
> >
> > However, it has a form.
> > I think the reason is encoding of 'enc.html'
> >
> > 'enc.html' is with encoding 'euc-kr' and it conflicts with the parser.
> >
> > I tried out
> > >> p.encoding = 'utf-8'
> > or
> > >> p.encoding = 'euc-kr'
> > but it doesn't work.
>
> This appears to work with the mechanize beta.  Try `gem install mechanize
> --pre` or install from source.
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20110414/39b66dea/attachment.html>

From drbrain at segment7.net  Mon Apr 18 01:24:18 2011
From: drbrain at segment7.net (Eric Hodel)
Date: Sun, 17 Apr 2011 22:24:18 -0700
Subject: [Mechanize-users] [ANN] Mechanize 2.0.pre.2
Message-ID: <FD172BFD-EDA2-4A84-A852-D0A1470D80F9@segment7.net>

mechanize version 2.0.pre.2 has been released!  Please test this release and let me know if I have broken anything important.

* <http://mechanize.rubyforge.org>
* <http://github.com/tenderlove/mechanize/tree/master>

The Mechanize library is used for automating interaction with websites.
Mechanize automatically stores and sends cookies, follows redirects,
can follow links, and submit forms.  Form fields can be populated and
submitted.  Mechanize also keeps track of the sites that you have visited as
a history.

Changes:

### 2.0.pre.2 / 2011-04-17

Mechanize is now under the MIT license

* API changes
  * WWW::Mechanize has been removed.  Use Mechanize.
  * Pre connect hooks are now called with the agent and the request.  See
    Mechanize#pre_connect_hooks.
  * Post connect hooks are now called with the agent and the response.  See
    Mechanize#post_connect_hooks.
  * Mechanize::Chain is gone, as an internal API this should cause no problems.
  * Mechanize#fetch_page no longer accepts an options Hash.
  * Mechanize#put now accepts headers instead of an options Hash as the last
    argument
  * Mechanize#delete now accepts headers instead of an options Hash as the
    last argument
  * Mechanize#request_with_entity now accepts headers instead of an options
    Hash as the last argument
  * Mechanize no longer raises RuntimeError directly, Mechanize::Error or
    ArgumentError are raised instead.
  * The User-Agent header has changed.  It no longer includes the WWW- prefix
    and now includes the ruby version.  The URL has been updated as well.
  * Mechanize now requires ruby 1.8.7 or newer.
  * Hpricot support has been removed as webrobots requires nokogiri.
  * Mechanize#get no longer accepts the referer as the second argument.
  * Mechanize#get no longer allows the HTTP method to be changed (:verb
    option).

* Deprecations
  * Mechanize#get with an options hash is deprecated and will be removed after
    October, 2011.
  * Mechanize::Util::to_native_charset is deprecated as it is no longer used
    by Mechanize.

* New Features

  * Add header reference methods to Mechanize::File so that a reponse
    object gets compatible with Net::HTTPResponse.
  * Mechanize#click accepts a regexp or string to click a button/link in the
    current page. It works as expected when not passed a string or regexp.
  * Provide a way to only follow permanent redirects (301)
    automatically: <tt>agent.redirect_ok = :permanent</tt>  GH #73
  * Documented various Mechanize accessors.  GH #66
  * Mechanize now uses net-http-digest_auth.  GH #31
  * Mechanize now implements session cookies.  GH #78
  * Mechanize now implements deflate decoding.  GH #40
  * Mechanize now allows a certificate and key to be passed directly.  GH #71
  * Mechanize::Form::MultiSelectList now implements #option_with and
    #options_with.  GH #42
  * Add Mechanize::Page::Link#rel and #rel?(kind) to read and test the rel
    attribute.
  * Add Mechanize::Page#canonical_uri to read a </tt><link
    rel="canonical"></tt> tag.
  * Add support for Robots Exclusion Protocol (i.e. robots.txt) and
    nofollow/noindex in meta tags and the rel attribute.  Automatic
    exclusion can be turned on by setting:
      agent.robots = true
  * Manual robots.txt test can be performed with
    Mechanize#robots_allowed? and #robots_disallowed?.
  * Mechanize::Form now supports the accept-charset attribute.  GH #96

* Bug Fixes:

  * Fixed a bug where Referer is not sent when accessing a relative
    URI starting with "http".
  * Fix handling of Meta Refresh with relative paths.  GH #39
  * Mechanize::CookieJar now supports RFC 2109 correctly.  GH #85
  * Fixed typo in EXAMPLES.rdoc.  GH #74
  * The base element is now handled correctly for images.  GH #72
  * Image buttons with no name attribute are now included in the form's button
    list.  GH#56
  * Improved handling of non ASCII-7bit compatible characters in links (only
    an issue on ruby 1.8).  GH #36, GH #75
  * Loading cookies.txt is faster.  GH #38
  * Mechanize no longer sends cookies for a.b.example to axb.example.  GH #41
  * Mechanize no longer sends the button name as a form field for image
    buttons.  GH #45
  * Blank cookie values are now skipped.  GH #80
  * Mechanize now adds a '.' to cookie domains if no '.' was sent.  This is
    not allowed by RFC 2109 but does appear in RFC 2965.  GH #86
  * file URIs are now read in binary mode.  GH #83
  * Content-Encoding: x-gzip is now treated like gzip per RFC 2616.
  * Mechanize now unescapes URIs for meta refresh.  GH #68
  * Mechanize now has more robust HTML charset detection.  GH #43
  * Mechanize::Form::Textarea is now created from a textarea element.  GH #94


From sean at clipperadams.com  Sun Apr 24 07:42:13 2011
From: sean at clipperadams.com (DeNigris Sean)
Date: Sun, 24 Apr 2011 07:42:13 -0400
Subject: [Mechanize-users] Dynamic Captcha
References: <EA91E589-5F16-47ED-8AB4-D84AA94987E4@clipperadams.com>
Message-ID: <20E4DB06-F653-4394-BEE0-513D373670C5@clipperadams.com>

I never saw this post to the list (sorry if it's a duplicate)...

I'm trying to download my EZ-Pass statements from www.e-zpassny.com using Mechanize. I'm having trouble logging in because the source of the captcha image is https://www.e-zpassny.com/vector/jcaptcha.do, which generates a new image every time it's loaded. So when I attempt to look at it by opening it in a window, it's already changed.

How do I see the original image from my "login_page = agent.get 'https://www.e-zpassny.com/vector/account/home/accountLogin.do'"?

Thanks.
Sean



From benmanns at gmail.com  Sun Apr 24 12:00:06 2011
From: benmanns at gmail.com (Benjamin Manns)
Date: Sun, 24 Apr 2011 12:00:06 -0400
Subject: [Mechanize-users] Dynamic Captcha
In-Reply-To: <20E4DB06-F653-4394-BEE0-513D373670C5@clipperadams.com>
References: <EA91E589-5F16-47ED-8AB4-D84AA94987E4@clipperadams.com>
	<20E4DB06-F653-4394-BEE0-513D373670C5@clipperadams.com>
Message-ID: <BANLkTikMEnkBUox-Hc0hCqvSi+S7MgxSFg@mail.gmail.com>

Hey Sean,

I have had a similar problem in the past. The issue is that the jcaptcha.do
image is generating and validating captchas based on your session
information, so you will need to download the image to disk, then view the
image from your local copy. This way the session data used to get the
captcha image is the same that is used to submit the form.

Ben

On Sun, Apr 24, 2011 at 7:42 AM, DeNigris Sean <sean at clipperadams.com>wrote:

> I never saw this post to the list (sorry if it's a duplicate)...
>
> I'm trying to download my EZ-Pass statements from www.e-zpassny.com using
> Mechanize. I'm having trouble logging in because the source of the captcha
> image is https://www.e-zpassny.com/vector/jcaptcha.do, which generates a
> new image every time it's loaded. So when I attempt to look at it by opening
> it in a window, it's already changed.
>
> How do I see the original image from my "login_page = agent.get '
> https://www.e-zpassny.com/vector/account/home/accountLogin.do'"?
>
> Thanks.
> Sean
>
>
> _______________________________________________
> Mechanize-users mailing list
> Mechanize-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/mechanize-users
>



-- 
Benjamin Manns
benmanns at gmail.com
(434) 321-8324
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20110424/9fb18a8c/attachment-0001.html>

From sean at clipperadams.com  Sun Apr 24 13:25:36 2011
From: sean at clipperadams.com (DeNigris Sean)
Date: Sun, 24 Apr 2011 13:25:36 -0400
Subject: [Mechanize-users] Dynamic Captcha
In-Reply-To: <BANLkTikMEnkBUox-Hc0hCqvSi+S7MgxSFg@mail.gmail.com>
References: <EA91E589-5F16-47ED-8AB4-D84AA94987E4@clipperadams.com>
	<20E4DB06-F653-4394-BEE0-513D373670C5@clipperadams.com>
	<BANLkTikMEnkBUox-Hc0hCqvSi+S7MgxSFg@mail.gmail.com>
Message-ID: <267E4311-1A65-4220-BD8C-13E9EAB35B1D@clipperadams.com>

On Apr 24, 2011, at 12:00 PM, Benjamin Manns wrote:
> you will need to download the image to disk, then view the image from your local copy.

Thanks. I tried your suggestion to download the image with the agent, but it doesn't seem to work:
require 'rubygems'
require 'mechanize'

agent = Mechanize.new
login_page = agent.get 'https://www.e-zpassny.com/vector/account/home/accountLogin.do'
agent.get('https://www.e-zpassny.com:443/vector/jcaptcha.do').save_as('/Users/sean/Library/Scripts/save_ezpass_statements/captcha.jpg')
#agent.get('https://www.e-zpassny.com:443/vector/jcaptcha.do').save_as('/Users/sean/Library/Scripts/save_ezpass_statements/captcha2.jpg')

secret_code = gets.rstrip

my_page = login_page.form_with(:name => 'accountLoginForm') do |f|
	tagNumberRadioIndex = 1
	f.radiobuttons_with(:name => 'loginType')[tagNumberRadioIndex].check
	f.login = 'my username'
	f.password = 'my password'
	f.jcaptcha_response = secret_code
end.submit

The above gives me a new login page, after a failed login. I tried enabling the second download link, and it loads a new captcha image every time anyway.


From sean at clipperadams.com  Sun Apr 24 01:38:29 2011
From: sean at clipperadams.com (DeNigris Sean)
Date: Sun, 24 Apr 2011 01:38:29 -0400
Subject: [Mechanize-users] Dynamic Captcha
Message-ID: <EA91E589-5F16-47ED-8AB4-D84AA94987E4@clipperadams.com>

I'm trying to download my EZ-Pass statements from www.e-zpassny.com using Mechanize. I'm having trouble logging in because the source of the captcha image is https://www.e-zpassny.com/vector/jcaptcha.do, which generates a new image every time it's loaded. So when I attempt to look at it by opening it in a window, it's already changed.

How do I see the original image from my "login_page = agent.get 'https://www.e-zpassny.com/vector/account/home/accountLogin.do'"?

Thanks.
Sean