[Mechanize-users] mechanize: 400 Bad Request

[Friedrich, Axel]

Hello,

finally, I found time to create a test case:

With browser, the following page shows up correctly:

http://cafriedrich.netfirms.com/root/dir1/firstpage.htm



With the "standard mechanize code", I get the before mentioned error:
----------------------------------------------------------------------

require 'rubygems'
require 'mechanize'
require 'logger'
require 'fileutils'

# ========== CONFIG ===================================
@logFilePath = "G:/Ruby/FrankfurterFondsBank/downloaded/log.txt"
@downloadDir = 'G:/Ruby/FrankfurterFondsBank/downloaded/'
@firstPage = "http://cafriedrich.netfirms.com/root/dir1/firstpage.htm"
# ========== END CONFIG ===============================


FileUtils.mkpath( File.dirname(@logFilePath) )
FileUtils.mkpath( @downloadDir )

agent = WWW::Mechanize.new{|a| a.log = Logger.new(@logFilePath) }

page = agent.get(@firstPage)
txt = page.body
File.open("@downloadDir" + "firstpage.htm", 'w') {|f|  f.write(txt)  }

puts "\n\n================= frames:=================  "
page.frames.each { |f| puts sprintf("%-15s %-1s \n", f.text, f.href) }
puts "\n=================  /frames =================  \n\n"

begin
   mainframe = agent.click( page.frames.text('main') )
   txt = mainframe.body
   File.open(@downloadDir + "mainframe.htm", 'w') {|f|  f.write(txt)  }

   frameleft1 = agent.click( page.frames.text('left1') )
   txt = page.body
   File.open(@downloadDir + "left1.htm", 'w') {|f|  f.write(txt)  }
rescue WWW::Mechanize::ResponseCodeError => ex
   puts "====== ERROR =========="
   puts ex.page.body
   puts "====== /ERROR =========="
end

puts "-- Exiting now."



With the "workaround code" below, it works:
-------------------------------------------

require 'rubygems'
require 'mechanize'
require 'logger'
require 'fileutils'

# ========== CONFIG ===================================
@logFilePath = "G:/Ruby/FrankfurterFondsBank/downloaded/log.txt"
@downloadDir = 'G:/Ruby/FrankfurterFondsBank/downloaded/'
@firstPage = "http://cafriedrich.netfirms.com/root/dir1/firstpage.htm"
# ========== END CONFIG ===============================


FileUtils.mkpath( File.dirname(@logFilePath) )
FileUtils.mkpath( @downloadDir )

agent = WWW::Mechanize.new{|a| a.log = Logger.new(@logFilePath) }

page = agent.get(@firstPage)
txt = page.body
File.open("@downloadDir" + "firstpage.htm", 'w') {|f|  f.write(txt)  }

puts "\n\n================= frames:=================  "
page.frames.each { |f| puts sprintf("%-15s %-1s \n", f.text, f.href) }
puts "\n=================  /frames =================  \n\n"

begin
   mainframe = agent.click( page.frames.text('main') )
   txt = mainframe.body
   File.open(@downloadDir + "mainframe.htm", 'w') {|f|  f.write(txt)  }

   href = 'http://cafriedrich.netfirms.com/root/'        # <-  WORKAROUND
   href << page.frames.text('left1').href.sub('../', '') # <-  WORKAROUND
   frameleft1 = agent.get(href)                          # <-  WORKAROUND
   txt = page.body
   File.open(@downloadDir + "left1.htm", 'w') {|f|  f.write(txt)  }
rescue WWW::Mechanize::ResponseCodeError => ex
   puts "====== ERROR =========="
   puts ex.page.body
   puts "====== /ERROR =========="
end

puts "-- Exiting now."