From rdpoor at gmail.com Wed Aug 1 04:14:52 2012 From: rdpoor at gmail.com (Robert Poor) Date: Tue, 31 Jul 2012 21:14:52 -0700 Subject: [Mechanize-users] preferred practice for comparing Mechanize::Page? Message-ID: I'm writing rspec tests for a specialized subclass of Mechanize. I'd like to test the values returned by generic.get() against my_subclass.get(). I know that the following doesn't work: it 'should produce the same result' do a = generic.get("http://www.google.com/") b = my_subclass.get("http://www.google.com/") a.should == b end ...because (for among a few other problems) two equal() urls may not compare as ==: Diff: @@ -1,5 +1,5 @@ #} + {url #} This may be too general a question, but what would constitute a sensible equality test for two Mechanize::Page objects? TIA. From drbrain at segment7.net Wed Aug 1 04:52:25 2012 From: drbrain at segment7.net (Eric Hodel) Date: Tue, 31 Jul 2012 21:52:25 -0700 Subject: [Mechanize-users] preferred practice for comparing Mechanize::Page? In-Reply-To: References: Message-ID: On Jul 31, 2012, at 21:14, Robert Poor wrote: > I'm writing rspec tests for a specialized subclass of Mechanize. I'd > like to test the values returned by generic.get() against > my_subclass.get(). I know that the following doesn't work: > > it 'should produce the same result' do > a = generic.get("http://www.google.com/") > b = my_subclass.get("http://www.google.com/") > a.should == b > end > > ...because (for among a few other problems) two equal() urls may not > compare as ==: > > Diff: > @@ -1,5 +1,5 @@ > # - {url #} > + {url #} Red herring: $ ruby -ruri -e 'p URI("http://google.com") == URI("http://google.com")' true (Apparently rspec is missing the super-useful object_id suppression that unit_diff has had since its first release. unit_diff, The Original Result Diffing Tool? unit_diff, Accept No Substitutes?, etc.) Neither Mechanize::Page nor Mechanize::File define #==, so two pages will never be equal in released versions. > This may be too general a question, but what would constitute a > sensible equality test for two Mechanize::Page objects? RFC 2616 section 13.3 appears to be where you want to look. At first glance this seems to include URI, Last-Modified header and Etag header. From trimolesi at gmail.com Wed Aug 1 18:54:39 2012 From: trimolesi at gmail.com (Thiago Imolesi) Date: Wed, 1 Aug 2012 15:54:39 -0300 Subject: [Mechanize-users] Button outside the form Message-ID: How I get a button outside the form to click him ? -- Att. Thiago Imolesi -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenny at kennymeyer.net Thu Aug 2 22:29:44 2012 From: kenny at kennymeyer.net (Kenny Meyer) Date: Thu, 2 Aug 2012 18:29:44 -0400 Subject: [Mechanize-users] Handling HTTP 500 errors Message-ID: <2C1E044C-7B27-49F8-BC07-5CDBED3F9914@kennymeyer.net> Hey guys, How do you handle HTTP 500 errors with Mechanize? After researching the web I thought something like this would work: begin Mechanize.new.get(url_which_returns_500) rescue Net::HTTPInternalServerError puts "Life is awesome. Let's move on" end Anyways, the code in the rescue block will never be executed. Can you guys help me out? Kenny From snowhitster at gmail.com Thu Aug 2 23:31:36 2012 From: snowhitster at gmail.com (T. Snowhite) Date: Thu, 2 Aug 2012 19:31:36 -0400 Subject: [Mechanize-users] Handling HTTP 500 errors In-Reply-To: <2C1E044C-7B27-49F8-BC07-5CDBED3F9914@kennymeyer.net> References: <2C1E044C-7B27-49F8-BC07-5CDBED3F9914@kennymeyer.net> Message-ID: Mechanize::ResponseCodeError will be raised on a 500. It has #code and #page attributes so you can inspect what went wrong, or switch off specific response codes. http://mechanize.rubyforge.org/Mechanize/ResponseCodeError.html On Aug 2, 2012, at 6:29 PM, Kenny Meyer wrote: Hey guys, How do you handle HTTP 500 errors with Mechanize? After researching the web I thought something like this would work: begin Mechanize.new.get(url_which_returns_500) rescue Net::HTTPInternalServerError puts "Life is awesome. Let's move on" end Anyways, the code in the rescue block will never be executed. Can you guys help me out? Kenny _______________________________________________ Mechanize-users mailing list Mechanize-users at rubyforge.org http://rubyforge.org/mailman/listinfo/mechanize-users From kenny at kennymeyer.net Fri Aug 3 00:28:15 2012 From: kenny at kennymeyer.net (Kenny Meyer) Date: Thu, 2 Aug 2012 20:28:15 -0400 Subject: [Mechanize-users] Handling HTTP 500 errors In-Reply-To: References: <2C1E044C-7B27-49F8-BC07-5CDBED3F9914@kennymeyer.net> Message-ID: Thank you, Tim. On Thu, Aug 2, 2012 at 7:31 PM, T. Snowhite wrote: > Mechanize::ResponseCodeError will be raised on a 500. > > It has #code and #page attributes so you can inspect what went wrong, or > switch off specific response codes. > > http://mechanize.rubyforge.org/Mechanize/ResponseCodeError.html > > > On Aug 2, 2012, at 6:29 PM, Kenny Meyer wrote: > > Hey guys, > > How do you handle HTTP 500 errors with Mechanize? > > After researching the web I thought something like this would work: > > begin > Mechanize.new.get(url_which_returns_500) > rescue Net::HTTPInternalServerError > puts "Life is awesome. Let's move on" > end > > Anyways, the code in the rescue block will never be executed. > > Can you guys help me out? > > Kenny > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Takahiro.Yamaguchi at jp.sony.com Wed Aug 8 07:39:48 2012 From: Takahiro.Yamaguchi at jp.sony.com (Takahiro Yamaguchi) Date: Wed, 08 Aug 2012 08:39:48 +0100 Subject: [Mechanize-users] [Question] my script with mechanize doesn't fill password field for login Message-ID: <12nfw7xomrf.wl%Takahiro.Yamaguchi@jp.sony.com> I'm Takahiro. My environment is as follows: yama at roswell:~/tmp$ uname -a Linux roswell 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:50:42 UTC 2011 i686 i686 i386 GNU/Linux yama at roswell:~/tmp$ ruby --version ruby 1.8.7 (2011-06-30 patchlevel 352) [i686-linux] mechanize v2.5.1-9 I'm trying to login with mechanize. I wrote the script for it. My script puts email field with value "tako" I specified,but doesn't put password I specified. The following is a snip of my script outputs. # this puts out as follows: #
"tako" is specified for email field in scipt is found. #
password vailue is "". Could you tell me what points are wrong? Here is my script. yama at roswell:~/tmp$ cat ndrive.rb #! /usr/bin/ruby # reference http://q.hatena.ne.jp/1213335523#a834923 require 'rubygems' require 'mechanize' agent = Mechanize.new # NAVER URL page = agent.get('https://ssl.naver.jp/login?fromUrl=http%3A%2F%2Fndrive.naver.jp%2F') # print form form = page.forms.first #form.fields.each { |f| puts f.name } # this puts out as follows: #rsaKeyName #fromUrl #encryptedLoginParam #enc #email #password # set for login authentication form['email'] = "tako" form['password'] = "xxxx" # submit form.submit #puts page.body puts agent.page.body -- Takahiro Yamaguchi From Takahiro.Yamaguchi at jp.sony.com Wed Aug 8 07:35:33 2012 From: Takahiro.Yamaguchi at jp.sony.com (Takahiro Yamaguchi) Date: Wed, 08 Aug 2012 08:35:33 +0100 Subject: [Mechanize-users] [Question] my script with mechanize doesn't fill password field for login Message-ID: <12nhasdomyi.wl%Takahiro.Yamaguchi@jp.sony.com> I'm Takahiro. My environment is as follows: yama at roswell:~/tmp$ uname -a Linux roswell 3.0.0-12-generic #20-Ubuntu SMP Fri Oct 7 14:50:42 UTC 2011 i686 i686 i386 GNU/Linux yama at roswell:~/tmp$ ruby --version ruby 1.8.7 (2011-06-30 patchlevel 352) [i686-linux] mechanize v2.5.1-9 I'm trying to login with mechanize. I wrote the script for it. My script puts email field with value "tako" I specified,but doesn't put password I specified. The following is a snip of my script outputs. # this puts out as follows: #
"tako" is specified for email field in scipt is found. #
password vailue is "". Could you tell me what points are wrong? Here is my script. yama at roswell:~/tmp$ cat ndrive.rb #! /usr/bin/ruby # reference http://q.hatena.ne.jp/1213335523#a834923 require 'rubygems' require 'mechanize' agent = Mechanize.new # NAVER URL page = agent.get('https://ssl.naver.jp/login?fromUrl=http%3A%2F%2Fndrive.naver.jp%2F') # print form form = page.forms.first #form.fields.each { |f| puts f.name } # this puts out as follows: #rsaKeyName #fromUrl #encryptedLoginParam #enc #email #password # set for login authentication form['email'] = "tako" form['password'] = "xxxx" # submit form.submit #puts page.body puts agent.page.body -- Takahiro Yamaguchi From joao.gsneto at gmail.com Wed Aug 8 15:18:07 2012 From: joao.gsneto at gmail.com (=?ISO-8859-1?Q?Jo=E3o_Neto?=) Date: Wed, 8 Aug 2012 12:18:07 -0300 Subject: [Mechanize-users] How to fetch a page with gzip enconded content Message-ID: Hello... I'm trying to fetch a page like this: agent = Mechanize.new agent.gzip_enabled = true require 'logger' logger = Logger.new $stdout agent.log = logger #agent.charset 'UTF-8' pp agent.get( 'http://www.peixeurbano.com.br/dailydeals.xml' ) But mechanize don't understand this page, the result is a Mechanize::FILE object, but this content is a wellformed xml document. How to fetch a compressed page with mechanize? I'm aready have gzip and mechanize 2.5 installed on my server. Thanks for help! Atenciosamente, *Jo?o Gomes Neto* Software Developer | ZCE Tel. (85) 9675-2423 Skype: joao.gsneto -------------- next part -------------- An HTML attachment was scrubbed... URL: From valdemirs at gmail.com Fri Aug 10 12:09:02 2012 From: valdemirs at gmail.com (Valdemir Santos) Date: Fri, 10 Aug 2012 09:09:02 -0300 Subject: [Mechanize-users] Clicking in fourth radio button Message-ID: Hi, I have a web page with 30 radio buttons, and I need click only in fourth radio button, named "Solaris". Looking at web page's source code I cannot see data, because it was made in javascript. Can Mechanize do this task for me ? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From sirbeep at gmail.com Fri Aug 10 13:35:22 2012 From: sirbeep at gmail.com (Brian Kennedy) Date: Fri, 10 Aug 2012 09:35:22 -0400 Subject: [Mechanize-users] Clicking in fourth radio button In-Reply-To: References: Message-ID: Yes, it can handle the job you have to do, but no it wont parse the javascript and build the form for you. You'll have to figure out what the javascript is doing to the html that's being posted and do it yourself with mechanize. One handy tool I use heavily for dealing with this type of overcomplicated scrape is WebScarab. I just proxy and capture my browser stepping through what the mechanize script will be stepping through and go through exactly what is being passed back and forth. No matter how complicated they do anything in javascript, or even worse the .asp.net crap pages I have to deal with, response to the service is still a simple html get or put, easily handled with Mechanize. On Fri, Aug 10, 2012 at 8:09 AM, Valdemir Santos wrote: > Hi, > I have a web page with 30 radio buttons, and I need click only in fourth > radio button, named "Solaris". > Looking at web page's source code I cannot see data, because it was made > in javascript. > Can Mechanize do this task for me ? > Thanks > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From trimolesi at gmail.com Fri Aug 10 13:40:46 2012 From: trimolesi at gmail.com (Thiago Imolesi) Date: Fri, 10 Aug 2012 10:40:46 -0300 Subject: [Mechanize-users] Clicking in fourth radio button In-Reply-To: References: Message-ID: I always submit the page via browser and so get the data submited, added the form with form.add_field! after only submit the form! On Fri, Aug 10, 2012 at 10:35 AM, Brian Kennedy wrote: > Yes, it can handle the job you have to do, but no it wont parse the > javascript and build the form for you. You'll have to figure out what the > javascript is doing to the html that's being posted and do it yourself with > mechanize. > > One handy tool I use heavily for dealing with this type of overcomplicated > scrape is WebScarab. I just proxy and capture my browser stepping through > what the mechanize script will be stepping through and go through exactly > what is being passed back and forth. > > No matter how complicated they do anything in javascript, or even worse > the .asp.net crap pages I have to deal with, response to the service is > still a simple html get or put, easily handled with Mechanize. > > On Fri, Aug 10, 2012 at 8:09 AM, Valdemir Santos wrote: > >> Hi, >> I have a web page with 30 radio buttons, and I need click only in fourth >> radio button, named "Solaris". >> Looking at web page's source code I cannot see data, because it was made >> in javascript. >> Can Mechanize do this task for me ? >> Thanks >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -- Att. Thiago Imolesi -------------- next part -------------- An HTML attachment was scrubbed... URL: From valdemirs at gmail.com Fri Aug 10 14:22:09 2012 From: valdemirs at gmail.com (Valdemir Santos) Date: Fri, 10 Aug 2012 11:22:09 -0300 Subject: [Mechanize-users] Clicking in fourth radio button In-Reply-To: References: Message-ID: Thanks for the answers. I need create an automated checklist filler - 150 radio buttons in web page, like below: for line in bkp_linux bkp_red_hat bkp_alpha bkp_solaris bkp_ibm do (go to the web page and click each radio button named like lines above). On Fri, Aug 10, 2012 at 10:40 AM, Thiago Imolesi wrote: > I always submit the page via browser and so get the data submited, added > the form with form.add_field! after only submit the form! > > > On Fri, Aug 10, 2012 at 10:35 AM, Brian Kennedy wrote: > >> Yes, it can handle the job you have to do, but no it wont parse the >> javascript and build the form for you. You'll have to figure out what the >> javascript is doing to the html that's being posted and do it yourself with >> mechanize. >> >> One handy tool I use heavily for dealing with this type of >> overcomplicated scrape is WebScarab. I just proxy and capture my browser >> stepping through what the mechanize script will be stepping through and go >> through exactly what is being passed back and forth. >> >> No matter how complicated they do anything in javascript, or even worse >> the .asp.net crap pages I have to deal with, response to the service is >> still a simple html get or put, easily handled with Mechanize. >> >> On Fri, Aug 10, 2012 at 8:09 AM, Valdemir Santos wrote: >> >>> Hi, >>> I have a web page with 30 radio buttons, and I need click only in fourth >>> radio button, named "Solaris". >>> Looking at web page's source code I cannot see data, because it was made >>> in javascript. >>> Can Mechanize do this task for me ? >>> Thanks >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > > -- > Att. > Thiago Imolesi > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sirbeep at gmail.com Fri Aug 10 14:30:16 2012 From: sirbeep at gmail.com (Brian Kennedy) Date: Fri, 10 Aug 2012 10:30:16 -0400 Subject: [Mechanize-users] Clicking in fourth radio button In-Reply-To: References: Message-ID: So you're not trying to navigate a page for the purposes of automating a task, you're trying to run a regression test on the site? Wrong tool, don't stick a screwdriver in that! If you're testing you'll want the javascript tested too and Mechanize can't do that for you. On Fri, Aug 10, 2012 at 10:22 AM, Valdemir Santos wrote: > Thanks for the answers. > I need create an automated checklist filler - 150 radio buttons in web > page, like below: > > for line in > bkp_linux > bkp_red_hat > bkp_alpha > bkp_solaris > bkp_ibm > do > (go to the web page and click each radio button named like lines above). > > > On Fri, Aug 10, 2012 at 10:40 AM, Thiago Imolesi wrote: > >> I always submit the page via browser and so get the data submited, added >> the form with form.add_field! after only submit the form! >> >> >> On Fri, Aug 10, 2012 at 10:35 AM, Brian Kennedy wrote: >> >>> Yes, it can handle the job you have to do, but no it wont parse the >>> javascript and build the form for you. You'll have to figure out what the >>> javascript is doing to the html that's being posted and do it yourself with >>> mechanize. >>> >>> One handy tool I use heavily for dealing with this type of >>> overcomplicated scrape is WebScarab. I just proxy and capture my browser >>> stepping through what the mechanize script will be stepping through and go >>> through exactly what is being passed back and forth. >>> >>> No matter how complicated they do anything in javascript, or even worse >>> the .asp.net crap pages I have to deal with, response to the service is >>> still a simple html get or put, easily handled with Mechanize. >>> >>> On Fri, Aug 10, 2012 at 8:09 AM, Valdemir Santos wrote: >>> >>>> Hi, >>>> I have a web page with 30 radio buttons, and I need click only in >>>> fourth radio button, named "Solaris". >>>> Looking at web page's source code I cannot see data, because it was >>>> made in javascript. >>>> Can Mechanize do this task for me ? >>>> Thanks >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> >> >> -- >> Att. >> Thiago Imolesi >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From valdemirs at gmail.com Fri Aug 10 15:04:56 2012 From: valdemirs at gmail.com (Valdemir Santos) Date: Fri, 10 Aug 2012 12:04:56 -0300 Subject: [Mechanize-users] Clicking in fourth radio button In-Reply-To: References: Message-ID: Hi, It's not a regression test. I need fill a checklist with 150 radio buttons. Tool needs searching for a string in the web page and click in that radio button. On Fri, Aug 10, 2012 at 11:30 AM, Brian Kennedy wrote: > So you're not trying to navigate a page for the purposes of automating a > task, you're trying to run a regression test on the site? Wrong tool, > don't stick a screwdriver in that! If you're testing you'll want the > javascript tested too and Mechanize can't do that for you. > > On Fri, Aug 10, 2012 at 10:22 AM, Valdemir Santos wrote: > >> Thanks for the answers. >> I need create an automated checklist filler - 150 radio buttons in web >> page, like below: >> >> for line in >> bkp_linux >> bkp_red_hat >> bkp_alpha >> bkp_solaris >> bkp_ibm >> do >> (go to the web page and click each radio button named like lines above). >> >> >> On Fri, Aug 10, 2012 at 10:40 AM, Thiago Imolesi wrote: >> >>> I always submit the page via browser and so get the data submited, added >>> the form with form.add_field! after only submit the form! >>> >>> >>> On Fri, Aug 10, 2012 at 10:35 AM, Brian Kennedy wrote: >>> >>>> Yes, it can handle the job you have to do, but no it wont parse the >>>> javascript and build the form for you. You'll have to figure out what the >>>> javascript is doing to the html that's being posted and do it yourself with >>>> mechanize. >>>> >>>> One handy tool I use heavily for dealing with this type of >>>> overcomplicated scrape is WebScarab. I just proxy and capture my browser >>>> stepping through what the mechanize script will be stepping through and go >>>> through exactly what is being passed back and forth. >>>> >>>> No matter how complicated they do anything in javascript, or even worse >>>> the .asp.net crap pages I have to deal with, response to the service >>>> is still a simple html get or put, easily handled with Mechanize. >>>> >>>> On Fri, Aug 10, 2012 at 8:09 AM, Valdemir Santos wrote: >>>> >>>>> Hi, >>>>> I have a web page with 30 radio buttons, and I need click only in >>>>> fourth radio button, named "Solaris". >>>>> Looking at web page's source code I cannot see data, because it was >>>>> made in javascript. >>>>> Can Mechanize do this task for me ? >>>>> Thanks >>>>> >>>>> _______________________________________________ >>>>> Mechanize-users mailing list >>>>> Mechanize-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>> >>> >>> >>> -- >>> Att. >>> Thiago Imolesi >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From valdemirs at gmail.com Tue Aug 14 19:43:54 2012 From: valdemirs at gmail.com (Valdemir Santos) Date: Tue, 14 Aug 2012 16:43:54 -0300 Subject: [Mechanize-users] Script error Message-ID: Hi, I?m trying run a ruby script, but I?m getting this output: :29:in `require': ld.so.1: ruby: fatal: libgcc_s.so.1: open failed: No such file or directory - /usr/local/lib/ruby/1.9.1/sparc-solaris2.10/etc.so (LoadError) from :29:in `require' from /usr/local/lib/ruby/1.9.1/rubygems.rb:17:in `' from :29:in `require' from :29:in `require' from ./goo2:2:in `
' -------------- next part -------------- An HTML attachment was scrubbed... URL: From drbrain at segment7.net Tue Aug 14 20:16:01 2012 From: drbrain at segment7.net (Eric Hodel) Date: Tue, 14 Aug 2012 13:16:01 -0700 Subject: [Mechanize-users] Script error In-Reply-To: References: Message-ID: On Aug 14, 2012, at 12:43, Valdemir Santos wrote: > I?m trying run a ruby script, but I?m getting this output: > > :29:in `require': ld.so.1: ruby: fatal: libgcc_s.so.1: open failed: No such file or directory - /usr/local/lib/ruby/1.9.1/sparc-solaris2.10/etc.so (LoadError) > from :29:in `require' > from /usr/local/lib/ruby/1.9.1/rubygems.rb:17:in `' > from :29:in `require' > from :29:in `require' > from ./goo2:2:in `
' You are missing etc.so. This is part of ruby, not mechanize. Please consult the ruby-talk mailing list (or forum) for help.