Hi,
I just can’t get the next_page to work.
It only works for ebay site. It fails on google. Any reason why?
I am only trying a simple sample. It works fine without next_page.
require 'rubygems'
require 'scrubyt'
google_data = Scrubyt::Extractor.define do
fetch 'http://www.google.com/ncr'
fill_textfield 'q', 'ruby'
submit
desc "Ruby Programming Language"
next_page 'Next', :limit => 2
end
google_data.to_xml.write($stdout, 1)
Scrubyt::ResultDumper.print_statistics(google_data)
It gives the error:
[MODE] Learning
[ACTION] fetching document: http://www.google.com/ncr
[ACTION] typing ruby into the textfield named 'q'
[ACTION] submitting form...
[ACTION] fetched http://www.google.com/search?hl=en&ie=ISO-8859-1&q=ruby
d:/InstantRails/ruby/lib/ruby/gems/1.8/gems/scrubyt-0.2.3/lib/scrubyt/core/shared/evaluation_context
.rb:73:in `generate_next_page_link': private method `gsub' called for nil:NilClass (NoMethodError)
from d:/InstantRails/ruby/lib/ruby/gems/1.8/gems/scrubyt-0.2.3/lib/scrubyt/core/shared/evalu
ation_context.rb:31:in `crawl_to_new_page'
from d:/InstantRails/ruby/lib/ruby/gems/1.8/gems/scrubyt-0.2.3/lib/scrubyt/core/shared/extra
ctor.rb:103:in `evaluate_extractor'
from d:/InstantRails/ruby/lib/ruby/gems/1.8/gems/scrubyt-0.2.3/lib/scrubyt/core/shared/extra
ctor.rb:101:in `loop'
from d:/InstantRails/ruby/lib/ruby/gems/1.8/gems/scrubyt-0.2.3/lib/scrubyt/core/shared/extra
ctor.rb:101:in `evaluate_extractor'
from d:/InstantRails/ruby/lib/ruby/gems/1.8/gems/scrubyt-0.2.3/lib/scrubyt/core/shared/extra
ctor.rb:36:in `define'
from test.rb:4
|
Start New Thread
RE: next_page doesn't work [ reply ]