From whitethunder922 at yahoo.com  Sat Nov  1 03:46:15 2008
From: whitethunder922 at yahoo.com (Matt White)
Date: Sat, 1 Nov 2008 00:46:15 -0700 (PDT)
Subject: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question
Message-ID: <497107.760.qm@web53306.mail.re2.yahoo.com>

Cindy,

When you call split on this string, it splits it in to 3 pieces and puts them into an array. If you do:

array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

you can then access the first line with array[0] and the second with array[1]. If you want to just discard the last item in the array, just call pop on the array. This example should illustrate:

irb(main):001:0> string = "this is\na string\nto split"
=> "this is\na string\nto split"
irb(main):002:0> array = string.split("\n").map{|s| s.strip}
=> ["this is", "a string", "to split"]
irb(main):003:0> array.pop
=> "to split"
irb(main):004:0> array
=> ["this is", "a string"]

If this is confusing or unclear, you might try a basic Ruby tutorial like at tryruby.hobix.com. Good luck!

Matt White




________________________________
From: Cindy Schaller <cschaller at gmail.com>
To: Ruby Mechanize Users List <mechanize-users at rubyforge.org>
Sent: Friday, October 31, 2008 3:23:08 PM
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Thanks Aaron.

That worked great.

Now I know that this is going to show my lack of Ruby knowledge, but I'm
still learning.

How can I parse the same set of HTML, but only get the first 2 lines and not
the third line.

My current code is this:
listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }  


THANKS!!!


-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Aaron Patterson
Sent: Friday, October 31, 2008 11:16 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Hi Cindy,

On Fri, Oct 31, 2008 at 8:50 AM, Cindy Schaller <cschaller at gmail.com> wrote:
> Hello.
>
>
>
> I am new to Hpricot and Mechanize, but so far I am loving it.
>
>
>
> I am trying to parse out some text inside of an HTML page and hoped that I
> could get some advice from this group.
>
>
>
> I have the following code:
>
>
>
> <strong>
>
>      Wii Game for Sale<br />
>
>      American Idol<br />
>
>      Ad #: 12345
>
> </strong>

Assuming you have already been able to find the "strong" tag, I would
do something like this:

  strong_tag.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

Hope that helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/
_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081101/397fb831/attachment-0001.html>

From cschaller at gmail.com  Mon Nov  3 13:06:05 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Mon, 3 Nov 2008 12:06:05 -0600
Subject: [Mechanize-users] Multiple Submit Buttons
Message-ID: <108025FE000E46B4ABC2F823E60276BC@CSWeb1>

Hello.

 

I have run into a new form that contains multiple submit buttons.

 

Currently my code for working with forms looks just like this example.

 

require 'rubygems'
  require 'mechanize'
 
  a = WWW::Mechanize.new { |agent|
    agent.user_agent_alias = 'Mac Safari'
  }
 
  a.get('http://google.com/') do |page|
    search_result = page.form_with(:name => 'f') do |search|
      search.q = 'Hello world'
    end.submit
 
    search_result.links.each do |link|
      puts link.text
    end
  end

 

 

 

But now I have 2 <input type=submit.> values on my form.  How can I tell
Mechanize which submit button to use to get to the result set?

 

Thanks!

Cindy

 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081103/f3a1c807/attachment.html>

From cschaller at gmail.com  Mon Nov  3 14:03:10 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Mon, 3 Nov 2008 13:03:10 -0600
Subject: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question
In-Reply-To: <497107.760.qm@web53306.mail.re2.yahoo.com>
References: <497107.760.qm@web53306.mail.re2.yahoo.com>
Message-ID: <7065ACC47B7E40C58C70D552C9ECE3FE@CSWeb1>

Hi Matt -

 

I'm not sure what the problem is, but if when I run the script through my
code this is what I see.  The output is cleaned up for readability.

 

The value of array[0]  appears to be the entire array instead of just the
first value.  If I take the string 

 

Thanks!

 

 

[code]

result.search('strong').each do |listing|  

puts "listing here: #{listing}"

            puts "==="

            array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip
}           

            puts "array: #{array}"

            puts "==="

            puts "array 0: #{array[0]}"

            puts "==="

end

 

 

[output]

 

listing here: 

<strong>

Wii Game for Sale<br />

American Idol52<br />

Ad #: 12345

<img class="noLink" src="img_source" alt="image" />

</strong>

 

===

 

array: Wii Game for Sale

American Idol

Ad #: 12345

 

 

===

array 0: Wii Game for Sale

American Idol

Ad #:12345

 

 

===

 

 

Cindy

 

 

 

 

 

 

 

  _____  

From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Matt White
Sent: Saturday, November 01, 2008 2:46 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

 

Cindy,

When you call split on this string, it splits it in to 3 pieces and puts
them into an array. If you do:

array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

you can then access the first line with array[0] and the second with
array[1]. If you want to just discard the last item in the array, just call
pop on the array. This example should illustrate:

irb(main):001:0> string = "this is\na string\nto split"
=> "this is\na string\nto split"
irb(main):002:0> array = string.split("\n").map{|s| s.strip}
=> ["this is", "a string", "to split"]
irb(main):003:0> array.pop
=> "to split"
irb(main):004:0> array
=> ["this is", "a string"]

If this is confusing or unclear, you might try a basic Ruby tutorial like at
tryruby.hobix.com. Good luck!

Matt White

 

  _____  

From: Cindy Schaller <cschaller at gmail.com>
To: Ruby Mechanize Users List <mechanize-users at rubyforge.org>
Sent: Friday, October 31, 2008 3:23:08 PM
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Thanks Aaron.

That worked great.

Now I know that this is going to show my lack of Ruby knowledge, but I'm
still learning.

How can I parse the same set of HTML, but only get the first 2 lines and not
the third line.

My current code is this:
listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }  


THANKS!!!


-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Aaron Patterson
Sent: Friday, October 31, 2008 11:16 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Hi Cindy,

On Fri, Oct 31, 2008 at 8:50 AM, Cindy Schaller <cschaller at gmail.com> wrote:
> Hello.
>
>
>
> I am new to Hpricot and Mechanize, but so far I am loving it.
>
>
>
> I am trying to parse out some text inside of an HTML page and hoped that I
> could get some advice from this group.
>
>
>
> I have the following code:
>
>
>
> <strong>
>
>      Wii Game for Sale<br />
>
>      American Idol<br />
>
>      Ad #: 12345
>
> </strong>

Assuming you have already been able to find the "strong" tag, I would
do something like this:

  strong_tag.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

Hope that helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/
_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081103/3bde509b/attachment-0001.html>

From cschaller at gmail.com  Mon Nov  3 14:06:55 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Mon, 3 Nov 2008 13:06:55 -0600
Subject: [Mechanize-users] Multiple Submit Buttons
Message-ID: <A8C8A0876FFC440791BC9264687C3EB3@CSWeb1>

Hello.

 

Sorry if this is double posted, but I didn't see it come through the mailing
list yet.

 

 

I have run into a new form that contains multiple submit buttons.

 

Currently my code for working with forms looks just like this example.

 

require 'rubygems'
  require 'mechanize'
 
  a = WWW::Mechanize.new { |agent|
    agent.user_agent_alias = 'Mac Safari'
  }
 
  a.get('http://google.com/') do |page|
    search_result = page.form_with(:name => 'f') do |search|
      search.q = 'Hello world'
    end.submit
 
    search_result.links.each do |link|
      puts link.text
    end
  end

 

 

 

But now I have 2 <input type=submit.> values on my form.  How can I tell
Mechanize which submit button to use to get to the result set?

 

Thanks!

Cindy

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081103/0d4a66d5/attachment.html>

From whitethunder922 at yahoo.com  Mon Nov  3 15:38:46 2008
From: whitethunder922 at yahoo.com (Matt White)
Date: Mon, 3 Nov 2008 12:38:46 -0800 (PST)
Subject: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question
Message-ID: <846702.375.qm@web53306.mail.re2.yahoo.com>

Heh, I'm going to blame Aaron for this one because he wrote the code that I cut and paste, but it's my fault for not noticing either. Instead of listing.inner_text, try using listing.inner_html. That should do it.

Matt




________________________________
From: Cindy Schaller <cschaller at gmail.com>
To: Ruby Mechanize Users List <mechanize-users at rubyforge.org>
Sent: Monday, November 3, 2008 12:03:10 PM
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

 
Hi Matt ?
 
I?m not sure what the problem is,
but if when I run the script through my code this is what I see.  The
output is cleaned up for readability.
 
The value of array[0]  appears to be
the entire array instead of just the first value.  If I take the string 
 
Thanks!
 
 
[code]
result.search('strong').each do |listing|  
puts "listing here:
#{listing}"
          
 puts "==="
           
array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }           
           
puts "array: #{array}"
           
puts "==="
           
puts "array 0: #{array[0]}"
           
puts "==="
end
 
 
[output]
 
listing here: 
<strong>
Wii Game for Sale <br />
American Idol52<br />
Ad #: 12345
<img class="noLink"
src="img_source" alt="image" />
</strong>
 
===
 
array: Wii Game for Sale
American Idol
Ad #: 12345
 
 
===
array 0: Wii Game for Sale
American Idol
Ad #:12345
 
 
===
 
 
Cindy
 
 
 
 
 
 
 

________________________________
 
From:mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Matt White
Sent: Saturday, November 01, 2008
2:46 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users]
Mechanize/Hpricot -- Strings parsing question
 
Cindy,

When you call split on this string, it splits it in to 3 pieces and puts them
into an array. If you do:

array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

you can then access the first line with array[0] and the second with array[1].
If you want to just discard the last item in the array, just call pop on the
array. This example should illustrate:

irb(main):001:0> string = "this is\na string\nto split"
=> "this is\na string\nto split"
irb(main):002:0> array = string.split("\n").map{|s| s.strip}
=> ["this is", "a string", "to split"]
irb(main):003:0> array.pop
=> "to split"
irb(main):004:0> array
=> ["this is", "a string"]

If this is confusing or unclear, you might try a basic Ruby tutorial like at tryruby.hobix.com. Good luck!

Matt White
 

________________________________
 
From:Cindy Schaller
<cschaller at gmail.com>
To: Ruby Mechanize Users List
<mechanize-users at rubyforge.org>
Sent: Friday, October 31, 2008
3:23:08 PM
Subject: Re: [Mechanize-users]
Mechanize/Hpricot -- Strings parsing question

Thanks Aaron.

That worked great.

Now I know that this is going to show my lack of Ruby knowledge, but I'm
still learning.

How can I parse the same set of HTML, but only get the first 2 lines and not
the third line.

My current code is this:
listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }  


THANKS!!!


-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org]
On Behalf Of Aaron Patterson
Sent: Friday, October 31, 2008 11:16 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Hi Cindy,

On Fri, Oct 31, 2008 at 8:50 AM, Cindy Schaller <cschaller at gmail.com>
wrote:
> Hello.
>
>
>
> I am new to Hpricot and Mechanize, but so far I am loving it.
>
>
>
> I am trying to parse out some text inside of an HTML page and hoped that I
> could get some advice from this group.
>
>
>
> I have the following code:
>
>
>
> <strong>
>
>      Wii Game for Sale <br
/>
>
>      American Idol<br />
>
>      Ad #: 12345
>
> </strong>

Assuming you have already been able to find the "strong" tag, I would
do something like this:

  strong_tag.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

Hope that helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/
_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081103/cbac9a1c/attachment-0001.html>

From cschaller at gmail.com  Mon Nov  3 16:12:39 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Mon, 3 Nov 2008 15:12:39 -0600
Subject: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question
In-Reply-To: <846702.375.qm@web53306.mail.re2.yahoo.com>
References: <846702.375.qm@web53306.mail.re2.yahoo.com>
Message-ID: <6145FACC500D40AFAB98196277B13B09@CSWeb1>

Ahh, many thanks!  Works like a charm.

 

I see that my last email was pre-empted before I finished the sentence.  But
I had meant to type that in my console, the code to split the string had
worked great.

 

Lots to learn.  Thanks for being great teachers!

 

  _____  

From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Matt White
Sent: Monday, November 03, 2008 2:39 PM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

 

Heh, I'm going to blame Aaron for this one because he wrote the code that I
cut and paste, but it's my fault for not noticing either. Instead of
listing.inner_text, try using listing.inner_html. That should do it.

Matt

 

  _____  

From: Cindy Schaller <cschaller at gmail.com>
To: Ruby Mechanize Users List <mechanize-users at rubyforge.org>
Sent: Monday, November 3, 2008 12:03:10 PM
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Hi Matt -

 

I'm not sure what the problem is, but if when I run the script through my
code this is what I see.  The output is cleaned up for readability.

 

The value of array[0]  appears to be the entire array instead of just the
first value.  If I take the string 

 

Thanks!

 

 

[code]

result.search('strong').each do |listing|  

puts "listing here: #{listing}"

            puts "==="

            array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip
}           

            puts "array: #{array}"

            puts "==="

            puts "array 0: #{array[0]}"

            puts "==="

end

 

 

[output]

 

listing here: 

<strong>

Wii Game for Sale <br />

American Idol52<br />

Ad #: 12345

<img class="noLink" src="img_source" alt="image" />

</strong>

 

===

 

array: Wii Game for Sale

American Idol

Ad #: 12345

 

 

===

array 0: Wii Game for Sale

American Idol

Ad #:12345

 

 

===

 

 

Cindy

 

 

 

 

 

 

 

  _____  

From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Matt White
Sent: Saturday, November 01, 2008 2:46 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

 

Cindy,

When you call split on this string, it splits it in to 3 pieces and puts
them into an array. If you do:

array = listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

you can then access the first line with array[0] and the second with
array[1]. If you want to just discard the last item in the array, just call
pop on the array. This example should illustrate:

irb(main):001:0> string = "this is\na string\nto split"
=> "this is\na string\nto split"
irb(main):002:0> array = string.split("\n").map{|s| s.strip}
=> ["this is", "a string", "to split"]
irb(main):003:0> array.pop
=> "to split"
irb(main):004:0> array
=> ["this is", "a string"]

If this is confusing or unclear, you might try a basic Ruby tutorial like at
tryruby.hobix.com. Good luck!

Matt White

 

  _____  

From: Cindy Schaller <cschaller at gmail.com>
To: Ruby Mechanize Users List <mechanize-users at rubyforge.org>
Sent: Friday, October 31, 2008 3:23:08 PM
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Thanks Aaron.

That worked great.

Now I know that this is going to show my lack of Ruby knowledge, but I'm
still learning.

How can I parse the same set of HTML, but only get the first 2 lines and not
the third line.

My current code is this:
listing.inner_text.split(/<br[^>]*>/).map { |x| x.strip }  


THANKS!!!


-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Aaron Patterson
Sent: Friday, October 31, 2008 11:16 AM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question

Hi Cindy,

On Fri, Oct 31, 2008 at 8:50 AM, Cindy Schaller <cschaller at gmail.com> wrote:
> Hello.
>
>
>
> I am new to Hpricot and Mechanize, but so far I am loving it.
>
>
>
> I am trying to parse out some text inside of an HTML page and hoped that I
> could get some advice from this group.
>
>
>
> I have the following code:
>
>
>
> <strong>
>
>      Wii Game for Sale <br />
>
>      American Idol<br />
>
>      Ad #: 12345
>
> </strong>

Assuming you have already been able to find the "strong" tag, I would
do something like this:

  strong_tag.inner_text.split(/<br[^>]*>/).map { |x| x.strip }

Hope that helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/
_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081103/2980c39b/attachment.html>

From christopher.mcmahon at gmail.com  Mon Nov  3 16:20:29 2008
From: christopher.mcmahon at gmail.com (Chris McMahon)
Date: Mon, 3 Nov 2008 14:20:29 -0700
Subject: [Mechanize-users] Mechanize/Hpricot -- Strings parsing question
In-Reply-To: <6145FACC500D40AFAB98196277B13B09@CSWeb1>
References: <846702.375.qm@web53306.mail.re2.yahoo.com>
	<6145FACC500D40AFAB98196277B13B09@CSWeb1>
Message-ID: <72799cd70811031320s59053a9bwbde59f032295ffbb@mail.gmail.com>

> Lots to learn.  Thanks for being great teachers!

FWIW, open source projects don't need more developers;  they need more
users.  I've really enjoyed reading this thread, thanks.
-Chris

From aaron at tenderlovemaking.com  Mon Nov  3 16:04:30 2008
From: aaron at tenderlovemaking.com (Aaron Patterson)
Date: Mon, 3 Nov 2008 13:04:30 -0800
Subject: [Mechanize-users] Multiple Submit Buttons
In-Reply-To: <108025FE000E46B4ABC2F823E60276BC@CSWeb1>
References: <108025FE000E46B4ABC2F823E60276BC@CSWeb1>
Message-ID: <20081103210430.GA4104@mac-mini.local>

On Mon, Nov 03, 2008 at 12:06:05PM -0600, Cindy Schaller wrote:
> Hello.
> 
>  
> 
> I have run into a new form that contains multiple submit buttons.
> 
>  
> 
> Currently my code for working with forms looks just like this example.
> 
>  
> 
> require 'rubygems'
>   require 'mechanize'
>  
>   a = WWW::Mechanize.new { |agent|
>     agent.user_agent_alias = 'Mac Safari'
>   }
>  
>   a.get('http://google.com/') do |page|
>     search_result = page.form_with(:name => 'f') do |search|
>       search.q = 'Hello world'
>     end.submit
>  
>     search_result.links.each do |link|
>       puts link.text
>     end
>   end

Submit takes an argument which is the button to use.  You can probably
do something like this:

 a.get('http://google.com/') do |page|
   search_result = page.form_with(:name => 'f') do |search|
     search.q = 'Hello world'
   end.submit(page.form_with(:name => 'f').buttons[1])
 end

-- 
Aaron Patterson
http://tenderlovemaking.com/

From cschaller at gmail.com  Wed Nov 12 10:47:46 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Wed, 12 Nov 2008 09:47:46 -0600
Subject: [Mechanize-users] Following Pages Question
Message-ID: <8F7227B20B624D69A30AF01EB87B49C3@CSWeb1>

Hello.

 

I'm still working on learning Mechanize.

 

I've got a standard form that returns multiple pages of results.  What I
need to do is follow the links to the next pages until I've reached the end.

 

I have the code that brings back the results from the first page and I can
then see the link with the inner_html value of "Next".

 

How can I continue to move through the rest of the results set?

 

I have this 

 

 

 

 a = WWW::Mechanize.new { |agent|

            agent.user_agent_alias = 'Mac Safari'

            agent.set_proxy('proxy', '80', 'user', 'pwd')

          }      

 a.get('http://www.website.com) do |page|

          page = page.form_with(:name => 'search1') do |search|

            search.checkboxes.name('feature').check

          end.submit

 

     

next_links = page.at("a[text()*='Next']")

puts "next links: #{next_links}"

 

 

a.click page.at("a[text()*='Next']") do |n|

        inside = n.at("a[text()*='Next']")

        puts "inside #{inside}"

      end  

 

 

 

The next_links will give me the link to the next page so I know that the
initial search will give me the result that I need.  So how can I translate
that next_link into something that I can use to transverse through the rest
of the pages?

 

I tried to follow the "Finding Links" section in Guide.txt  and the transact
method in Examples.txt, but I couldn't put these 2 together to make it work
correctly.  I'm sure that I'm mixing it all together and making a mess.

 

Thanks so much for the great help!!!

Cindy

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081112/114599b8/attachment.html>

From cschaller at gmail.com  Thu Nov 13 00:23:47 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Wed, 12 Nov 2008 23:23:47 -0600
Subject: [Mechanize-users] MultiSelect Lists Help Needed
Message-ID: <30408F347A1647DEBC1E6E17E20CC897@CSWeb1>

Hello.

 

Sorry for the 2nd email of the day, but I've got a select list that is
giving me grief, along with the next links, but that was in my first email
of the day.  =)

 

I have a multi select list and I can't seem to select the values and submit
them.  Instead, it looks like they are being ignored.

 

Here's my code:

 

    page = page.form_with(:name => 'search1') do |search|

        search.fields.name('typeSelect').options[2].select 

        search.checkboxes.name('checkbox').check

      end.submit  

 

 

What am I doing wrong and how can I fix this?

 

THANKS!

Cindy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081112/ec5559c4/attachment.html>

From aaron at tenderlovemaking.com  Thu Nov 13 00:33:56 2008
From: aaron at tenderlovemaking.com (Aaron Patterson)
Date: Wed, 12 Nov 2008 21:33:56 -0800
Subject: [Mechanize-users] MultiSelect Lists Help Needed
In-Reply-To: <30408F347A1647DEBC1E6E17E20CC897@CSWeb1>
References: <30408F347A1647DEBC1E6E17E20CC897@CSWeb1>
Message-ID: <20081113053356.GA5037@mac-mini.local>

Hi Cindy!

On Wed, Nov 12, 2008 at 11:23:47PM -0600, Cindy Schaller wrote:
> Hello.
> 
>  
> 
> Sorry for the 2nd email of the day, but I've got a select list that is
> giving me grief, along with the next links, but that was in my first email
> of the day.  =)

No problem.  I try to respond as quickly as possible, but my paying job
sometimes gets in the way.  ;-)

> I have a multi select list and I can't seem to select the values and submit
> them.  Instead, it looks like they are being ignored.
> 
>  
> 
> Here's my code:
> 
>  
> 
>     page = page.form_with(:name => 'search1') do |search|
> 
>         search.fields.name('typeSelect').options[2].select 
> 
>         search.checkboxes.name('checkbox').check
> 
>       end.submit  
> 
>  
> 
>  
> 
> What am I doing wrong and how can I fix this?

Mechanize is probably submitting those fields to the server.
Unfortunately the server probably wants other parameters as well that
you aren't setting.

Typically what I recommend doing is installing LiveHTTPHeaders

  * http://livehttpheaders.mozdev.org/

Then submitting the form using Firefox with the livehttpheaders dialog
open.  That lets you see exactly what Firefox is sending to the server.

Step 2, set a logger on your mechanize agent:

  require 'logger'
  require 'mechanize'

  agent = WWW::Mechanize.new { |a| a.log = Logger.new('out.log') }

Then run your script.  Mechanize will log what it sends to the server.
Compare your Firefox headers to what mechanize sent, then adjust your
script until Mechanize sends the same thing that Firefox does.

I hope that helps.  Unfortunately since I have no context, namely a
short failing example, I can't help you much more than that.

-- 
Aaron Patterson
http://tenderlovemaking.com/

From cschaller at gmail.com  Thu Nov 13 10:40:52 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Thu, 13 Nov 2008 09:40:52 -0600
Subject: [Mechanize-users] MultiSelect Lists Help Needed
In-Reply-To: <20081113053356.GA5037@mac-mini.local>
References: <30408F347A1647DEBC1E6E17E20CC897@CSWeb1>
	<20081113053356.GA5037@mac-mini.local>
Message-ID: <BD01E668F3284F9F9BCC29B72E2B40D1@CSWeb1>

Hi Aaron -

Sorry, I didn't mean to say that the response time is slow, I just meant
that I was sending multiple emails due to just leaning mechanize.  I totally
appreciate all of your help.

My first email yesterday was about following "Next" links.  If anyone has
insight on that email, I'd appreciate it.


This email was the second issue of the day - select lists.


I have the LiveHTTPHeaders installed.  Works great.  When I added logger to
my Mechanize call, it broke the processing.  


I've ran variations of this, but currently I have the following:

agent = WWW::Mechanize.new { |a| a.log = Logger.new('out.log') }

page = page.form_with(:name => 'search1') do |search|
search.fields.name('selectlist').options[7].value 
search.checkboxes.name('checkbox').check
end.submit



The error message is this:

c:/ruby/lib/ruby/gems/1.8/gems/mechanize-0.8.0/lib/www/mechanize/chain/respo
nse_reader.rb:30:in `handle': undefined local variable or method `response'
for #<WWW::Mechanize::Chain::ResponseReader:0x3ef2024> (NameError)



Any help is appreciated,
Cindy




-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Aaron Patterson
Sent: Wednesday, November 12, 2008 11:34 PM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] MultiSelect Lists Help Needed

Hi Cindy!

On Wed, Nov 12, 2008 at 11:23:47PM -0600, Cindy Schaller wrote:
> Hello.
> 
>  
> 
> Sorry for the 2nd email of the day, but I've got a select list that is
> giving me grief, along with the next links, but that was in my first email
> of the day.  =)

No problem.  I try to respond as quickly as possible, but my paying job
sometimes gets in the way.  ;-)

> I have a multi select list and I can't seem to select the values and
submit
> them.  Instead, it looks like they are being ignored.
> 
>  
> 
> Here's my code:
> 
>  
> 
>     page = page.form_with(:name => 'search1') do |search|
> 
>         search.fields.name('typeSelect').options[2].select 
> 
>         search.checkboxes.name('checkbox').check
> 
>       end.submit  
> 
>  
> 
>  
> 
> What am I doing wrong and how can I fix this?

Mechanize is probably submitting those fields to the server.
Unfortunately the server probably wants other parameters as well that
you aren't setting.

Typically what I recommend doing is installing LiveHTTPHeaders

  * http://livehttpheaders.mozdev.org/

Then submitting the form using Firefox with the livehttpheaders dialog
open.  That lets you see exactly what Firefox is sending to the server.

Step 2, set a logger on your mechanize agent:

  require 'logger'
  require 'mechanize'

  agent = WWW::Mechanize.new { |a| a.log = Logger.new('out.log') }

Then run your script.  Mechanize will log what it sends to the server.
Compare your Firefox headers to what mechanize sent, then adjust your
script until Mechanize sends the same thing that Firefox does.

I hope that helps.  Unfortunately since I have no context, namely a
short failing example, I can't help you much more than that.

-- 
Aaron Patterson
http://tenderlovemaking.com/
_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users


From cschaller at gmail.com  Thu Nov 13 11:55:02 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Thu, 13 Nov 2008 10:55:02 -0600
Subject: [Mechanize-users] MultiSelect Lists Help Needed
In-Reply-To: <BD01E668F3284F9F9BCC29B72E2B40D1@CSWeb1>
References: <30408F347A1647DEBC1E6E17E20CC897@CSWeb1><20081113053356.GA5037@mac-mini.local>
	<BD01E668F3284F9F9BCC29B72E2B40D1@CSWeb1>
Message-ID: <2B2C5B3071BB42AEBD3DB9AC8201753A@CSWeb1>



From cschaller at gmail.com  Thu Nov 13 12:12:19 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Thu, 13 Nov 2008 11:12:19 -0600
Subject: [Mechanize-users] MultiSelect Lists Help Needed
In-Reply-To: <BD01E668F3284F9F9BCC29B72E2B40D1@CSWeb1>
References: <30408F347A1647DEBC1E6E17E20CC897@CSWeb1><20081113053356.GA5037@mac-mini.local>
	<BD01E668F3284F9F9BCC29B72E2B40D1@CSWeb1>
Message-ID: <6C21FEE078CC47A2BF75950E6127F2FC@CSWeb1>

Sorry, my last email didn't work.  Not sure what happened there.

I found the fix to the logger error and was able to generate the output.

Here's the output from LiveHTTPHeaders:

GET
/?classification=real+estate&temp_type=browse&finder=buy&ad_type=residential
-listing%2Cvacation-listing%2Cnew-homes-listing&tp=RE_nj&property=nj.com&fea
ture_exists_just_listed=on&feature_includes_property_type=single+family%2Cco
ndo HTTP/1.1



My mechanize form submits all of the variables on the form and because I
didn't select options, it gets all of the listings.

Net::HTTP::Get:
/?classification=real+estate&temp_type=browse&finder=buy&ad_type=residential
-listing%2Cvacation-listing%2Cnew-homes-listing&orderby=&advancedMtlSearch=&
tp=RE_nj&property=nj.com&neighborhood=&keywords=&feature_includes_condition=
&feature_includes_property_type=&feature_max_acres=&feature_exists_parking=&
feature_exists_garage_type=&ad_no=&zipcode=&map_region=&region=&adpricemin=&
adpricemax=&feature_min_beds=&feature_min_baths=&feature_min_acres=&feature_
min_number_sqfeet=&multiimage=&date_created=&feature_exists_just_listed=on


I am going to http://realestate.nj.com

The options that I want to submit are just listed, single family and condo.
I can't seem to submit the single family and condo options in the multi
select.

page = page.form_with(:name => 'search1') do |search|
 
search.fields.name('feature_includes_property_typeSelect').options[0].select
#single family
 
search.fields.name('feature_includes_property_typeSelect').options[1].select
#condo
search.checkboxes.name('feature_exists_just_listed').check

end.submit



Any ideas on how to submit this search properly?

Thanks,
Cindy





-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Cindy Schaller
Sent: Thursday, November 13, 2008 9:41 AM
To: 'Ruby Mechanize Users List'
Subject: Re: [Mechanize-users] MultiSelect Lists Help Needed

Hi Aaron -

Sorry, I didn't mean to say that the response time is slow, I just meant
that I was sending multiple emails due to just leaning mechanize.  I totally
appreciate all of your help.

My first email yesterday was about following "Next" links.  If anyone has
insight on that email, I'd appreciate it.


This email was the second issue of the day - select lists.


I have the LiveHTTPHeaders installed.  Works great.  When I added logger to
my Mechanize call, it broke the processing.  


I've ran variations of this, but currently I have the following:

agent = WWW::Mechanize.new { |a| a.log = Logger.new('out.log') }

page = page.form_with(:name => 'search1') do |search|
search.fields.name('selectlist').options[7].value 
search.checkboxes.name('checkbox').check
end.submit



The error message is this:

c:/ruby/lib/ruby/gems/1.8/gems/mechanize-0.8.0/lib/www/mechanize/chain/respo
nse_reader.rb:30:in `handle': undefined local variable or method `response'
for #<WWW::Mechanize::Chain::ResponseReader:0x3ef2024> (NameError)



Any help is appreciated,
Cindy




-----Original Message-----
From: mechanize-users-bounces at rubyforge.org
[mailto:mechanize-users-bounces at rubyforge.org] On Behalf Of Aaron Patterson
Sent: Wednesday, November 12, 2008 11:34 PM
To: Ruby Mechanize Users List
Subject: Re: [Mechanize-users] MultiSelect Lists Help Needed

Hi Cindy!

On Wed, Nov 12, 2008 at 11:23:47PM -0600, Cindy Schaller wrote:
> Hello.
> 
>  
> 
> Sorry for the 2nd email of the day, but I've got a select list that is
> giving me grief, along with the next links, but that was in my first email
> of the day.  =)

No problem.  I try to respond as quickly as possible, but my paying job
sometimes gets in the way.  ;-)

> I have a multi select list and I can't seem to select the values and
submit
> them.  Instead, it looks like they are being ignored.
> 
>  
> 
> Here's my code:
> 
>  
> 
>     page = page.form_with(:name => 'search1') do |search|
> 
>         search.fields.name('typeSelect').options[2].select 
> 
>         search.checkboxes.name('checkbox').check
> 
>       end.submit  
> 
>  
> 
>  
> 
> What am I doing wrong and how can I fix this?

Mechanize is probably submitting those fields to the server.
Unfortunately the server probably wants other parameters as well that
you aren't setting.

Typically what I recommend doing is installing LiveHTTPHeaders

  * http://livehttpheaders.mozdev.org/

Then submitting the form using Firefox with the livehttpheaders dialog
open.  That lets you see exactly what Firefox is sending to the server.

Step 2, set a logger on your mechanize agent:

  require 'logger'
  require 'mechanize'

  agent = WWW::Mechanize.new { |a| a.log = Logger.new('out.log') }

Then run your script.  Mechanize will log what it sends to the server.
Compare your Firefox headers to what mechanize sent, then adjust your
script until Mechanize sends the same thing that Firefox does.

I hope that helps.  Unfortunately since I have no context, namely a
short failing example, I can't help you much more than that.

-- 
Aaron Patterson
http://tenderlovemaking.com/
_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users

_______________________________________________
Mechanize-users mailing list
Mechanize-users at rubyforge.org
http://rubyforge.org/mailman/listinfo/mechanize-users


From cschaller at gmail.com  Fri Nov 14 17:33:43 2008
From: cschaller at gmail.com (Cindy Schaller)
Date: Fri, 14 Nov 2008 16:33:43 -0600
Subject: [Mechanize-users] Variation on my Mechanize MultiSelect Question
Message-ID: <1E3CC501EC654CD299F1FD62C891CFAE@CSWeb1>

Hello,

 

Has anyone had success submitting forms that have a multi select drop down
boxes?  

 

I have worked on this for 3 days and have not found the solution and today I
found a post that seemed to suggest that I need to do this in an alternative
way.  http://wiki.m001.net/technical/show/HowtoPostWithMultiselectList, but
I really see this as cumbersome and less than ideal.  I can already see
problems sending the parameters as opposed to submitting the form.

 

I sent an email to the group earlier this week so I won't repeat the code
that I was using unless someone would like to see it, but thought I'd check
to see if anyone has actually made this work submitting a form.

 

 

Thanks so much,

Cindy

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/mechanize-users/attachments/20081114/4671a831/attachment.html>

From michael.c.libby at gmail.com  Thu Nov 20 14:20:22 2008
From: michael.c.libby at gmail.com (Michael Libby)
Date: Thu, 20 Nov 2008 13:20:22 -0600
Subject: [Mechanize-users] Empty action
Message-ID: <db8524fb0811201120q493d35d8l69d11dc64f528608@mail.gmail.com>

Just getting started with Mechanize. Loving it. So thanks for the library.

Question about what is considered normal behavior when using a form's
#submit method.

I have a form that looks like

  #<WWW::Mechanize::Form
   {name "show_Next"}
   {method "POST"}
   {action ""}
   {fields ...

So action is empty.



When I submit this form with

  results = @agent.submit(show_next)

It doesn't appear to be submitting the form correctly.

From michael.c.libby at gmail.com  Thu Nov 20 14:28:13 2008
From: michael.c.libby at gmail.com (Michael Libby)
Date: Thu, 20 Nov 2008 13:28:13 -0600
Subject: [Mechanize-users] Empty action
Message-ID: <db8524fb0811201128l2e6609fdjc2039dc1a4dfe819@mail.gmail.com>

(sorry for duplicate email here, I fat-fingered the keyboard and sent
the previous email too soon)

Just getting started with Mechanize. Loving it. So thanks for the library.

Question about what is considered normal behavior when using a form's
#submit method.

I have a search form, assume it is page.forms[0], when I submit my
first search I get results back that contain the following form:

 #<WWW::Mechanize::Form
  {name "show_Next"}
  {method "POST"}
  {action ""}
  {fields ...

Action is empty. When I submit *this* form with

 results = agent.submit(show_next)

It doesn't appear to be submitting the form to the same URL the form
came from. My results doesn't contain anything like what I get if I'm
in a browser and I click the Next link. All I get back is the main
page for the web site in question. On the original search form that
I'm submitting, action is a URL with an embedded session ID.

However, if I do this:

   action = page.forms[0].action
   show_next.action = action
   results = agent.submit(show_next)

Then results contains the next results page for the query in question.

So obviously there is a way to resolve the underlying need of POSTing
the form to the right URL, but I'm wondering if this is an expected
behavior or if there is a better way to work around this.

Thanks,
Michael Libby.