From kevin at proofimaging.com Mon Feb 25 23:55:16 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Mon, 25 Feb 2013 18:55:16 -0500 Subject: [Mechanize-users] URI problems, need some love Message-ID: Got a form, hitting a button with Mechanize. Thus far have been going through login steps quite nicely so I know my Mechanize basics are up to snuff. Then I get: URI::InvalidURIError: bad URI(is not URI?): CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 from /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in `split' from /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in `parse' from /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in `parse' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in `rescue in resolve' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in `resolve' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in `fetch' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in `post_form' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in `submit' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in `submit' from (irb):69 from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in `start' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in `start' from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in `' from script/rails:6:in `require' from script/rails:6:in `
' >From the googling I've done, seems to be related to an invalid character in the re-direct, but really I have no idea... Any way of ignoring these weird characters and continuing? Anyway of fixing this? Best, Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From godfreykfc at gmail.com Tue Feb 26 02:06:26 2013 From: godfreykfc at gmail.com (Godfrey Chan) Date: Mon, 25 Feb 2013 18:06:26 -0800 (PST) Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: Message-ID: <1361844385378.a1b307de@Nodemailer> Hi Kevin, It looks like the "action" attribute of your form contains something along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". The semicolon in here is not strictly a valid character in an URL. Try replacing it with %3B. form = page.at( ...select the form with a selector... ) action = form.attr('action') action = action.gsub(';','%3B').gsub('=','%3D') form.attr('action', action) # then submit the form as usual (You can't do form.submit because this form object we have here is a nokogiri node. You need to go through mechanize to submit the form.) Godfrey? ? Sent from Mailbox for iPhone On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann wrote: > Got a form, hitting a button with Mechanize. Thus far have been going > through login steps quite nicely so I know my Mechanize basics are up to > snuff. > Then I get: > URI::InvalidURIError: bad URI(is not URI?): > CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 > from > /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in > `split' > from > /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in > `parse' > from > /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in > `parse' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in > `rescue in resolve' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in > `resolve' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in > `fetch' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in > `post_form' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in > `submit' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in > `submit' > from (irb):69 > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in > `start' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in > `start' > from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in > `' > from script/rails:6:in `require' > from script/rails:6:in `
' >>From the googling I've done, seems to be related to an invalid character in > the re-direct, but really I have no idea... > Any way of ignoring these weird characters and continuing? Anyway of fixing > this? > Best, > Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Tue Feb 26 03:29:28 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Mon, 25 Feb 2013 22:29:28 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: <1361844385378.a1b307de@Nodemailer> References: <1361844385378.a1b307de@Nodemailer> Message-ID: Hi Godfrey, Thanks for your help. Really working hard to figure this out. I ran into a road block that is my inexperience with nokogiri and mech. When I try to grab the action (action = form.attr('action')), I get undefined method... Here is what I'm doing exactly: agent = Mechanize.new agent.user_agent_alias = 'Mac Safari' page = agent.get('https://somesite.jsp') form = agent.page.forms.first action = form.attr('action') and undefined method... I tried seeing if I could grab the action various ways to no avail. My other question is, assuming I do this successfuly, I would normally use "form.submit" but you're saying this won't work? What should I try? Thanks again for your help!! Kevin On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: > Hi Kevin, > > It looks like the "action" attribute of your form contains something along > the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". The > semicolon in here is not strictly a valid character in an URL. Try > replacing it with %3B. > > form = page.at( ...select the form with a selector... ) > > action = form.attr('action') > > action = action.gsub(';','%3B').gsub('=','%3D') > > form.attr('action', action) > > # then submit the form as usual (You can't do form.submit because this > form object we have here is a nokogiri node. You need to go through > mechanize to submit the form.) > > Godfrey > ? > Sent from Mailbox for iPhone > > > On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann wrote: > >> Got a form, hitting a button with Mechanize. Thus far have been going >> through login steps quite nicely so I know my Mechanize basics are up to >> snuff. >> >> Then I get: >> >> URI::InvalidURIError: bad URI(is not URI?): >> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >> from >> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >> `split' >> from >> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >> `parse' >> from >> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >> `parse' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >> `rescue in resolve' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >> `resolve' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >> `fetch' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >> `post_form' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >> `submit' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >> `submit' >> from (irb):69 >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >> `start' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >> `start' >> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >> `' >> from script/rails:6:in `require' >> from script/rails:6:in `
' >> >> From the googling I've done, seems to be related to an invalid character >> in the re-direct, but really I have no idea... >> >> Any way of ignoring these weird characters and continuing? Anyway of >> fixing this? >> >> Best, >> Kevin >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Tue Feb 26 03:35:44 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Mon, 25 Feb 2013 22:35:44 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361844385378.a1b307de@Nodemailer> Message-ID: actually, this grabbed it: action = form.action and your gsub worked a charm. Just need to get this to submit... Because you're right form.submit didn't work... I tried what you gave me: action = form.attr('action') But undefined meth again on attr... On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann wrote: > Hi Godfrey, > > Thanks for your help. Really working hard to figure this out. I ran into a > road block that is my inexperience with nokogiri and mech. > > When I try to grab the action (action = form.attr('action')), I get > undefined method... Here is what I'm doing exactly: > > agent = Mechanize.new > agent.user_agent_alias = 'Mac Safari' > page = agent.get('https://somesite.jsp') > > form = agent.page.forms.first > action = form.attr('action') > > and undefined method... > I tried seeing if I could grab the action various ways to no avail. > > My other question is, assuming I do this successfuly, I would normally use > "form.submit" but you're saying this won't work? What should I try? > > Thanks again for your help!! > Kevin > > > On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: > >> Hi Kevin, >> >> It looks like the "action" attribute of your form contains something >> along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >> The semicolon in here is not strictly a valid character in an URL. Try >> replacing it with %3B. >> >> form = page.at( ...select the form with a selector... ) >> >> action = form.attr('action') >> >> action = action.gsub(';','%3B').gsub('=','%3D') >> >> form.attr('action', action) >> >> # then submit the form as usual (You can't do form.submit because this >> form object we have here is a nokogiri node. You need to go through >> mechanize to submit the form.) >> >> Godfrey >> ? >> Sent from Mailbox for iPhone >> >> >> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann wrote: >> >>> Got a form, hitting a button with Mechanize. Thus far have been going >>> through login steps quite nicely so I know my Mechanize basics are up to >>> snuff. >>> >>> Then I get: >>> >>> URI::InvalidURIError: bad URI(is not URI?): >>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>> from >>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>> `split' >>> from >>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>> `parse' >>> from >>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>> `parse' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>> `rescue in resolve' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>> `resolve' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>> `fetch' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>> `post_form' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>> `submit' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>> `submit' >>> from (irb):69 >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>> `start' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>> `start' >>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>> `' >>> from script/rails:6:in `require' >>> from script/rails:6:in `
' >>> >>> From the googling I've done, seems to be related to an invalid character >>> in the re-direct, but really I have no idea... >>> >>> Any way of ignoring these weird characters and continuing? Anyway of >>> fixing this? >>> >>> Best, >>> Kevin >>> >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Tue Feb 26 03:38:02 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Mon, 25 Feb 2013 22:38:02 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361844385378.a1b307de@Nodemailer> Message-ID: sorry meant I tried: form.attr('action', action) to no avail...? On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann wrote: > actually, this grabbed it: > > action = form.action > > and your gsub worked a charm. > > Just need to get this to submit... Because you're right form.submit didn't > work... I tried what you gave me: > > action = form.attr('action') > > But undefined meth again on attr... > > On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann wrote: > >> Hi Godfrey, >> >> Thanks for your help. Really working hard to figure this out. I ran into >> a road block that is my inexperience with nokogiri and mech. >> >> When I try to grab the action (action = form.attr('action')), I get >> undefined method... Here is what I'm doing exactly: >> >> agent = Mechanize.new >> agent.user_agent_alias = 'Mac Safari' >> page = agent.get('https://somesite.jsp') >> >> form = agent.page.forms.first >> action = form.attr('action') >> >> and undefined method... >> I tried seeing if I could grab the action various ways to no avail. >> >> My other question is, assuming I do this successfuly, I would normally >> use "form.submit" but you're saying this won't work? What should I try? >> >> Thanks again for your help!! >> Kevin >> >> >> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >> >>> Hi Kevin, >>> >>> It looks like the "action" attribute of your form contains something >>> along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>> The semicolon in here is not strictly a valid character in an URL. Try >>> replacing it with %3B. >>> >>> form = page.at( ...select the form with a selector... ) >>> >>> action = form.attr('action') >>> >>> action = action.gsub(';','%3B').gsub('=','%3D') >>> >>> form.attr('action', action) >>> >>> # then submit the form as usual (You can't do form.submit because this >>> form object we have here is a nokogiri node. You need to go through >>> mechanize to submit the form.) >>> >>> Godfrey >>> ? >>> Sent from Mailbox for iPhone >>> >>> >>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann >> > wrote: >>> >>>> Got a form, hitting a button with Mechanize. Thus far have been going >>>> through login steps quite nicely so I know my Mechanize basics are up to >>>> snuff. >>>> >>>> Then I get: >>>> >>>> URI::InvalidURIError: bad URI(is not URI?): >>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>> from >>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>> `split' >>>> from >>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>> `parse' >>>> from >>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>> `parse' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>> `rescue in resolve' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>> `resolve' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>> `fetch' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>> `post_form' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>> `submit' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>> `submit' >>>> from (irb):69 >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>> `start' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>> `start' >>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>> `' >>>> from script/rails:6:in `require' >>>> from script/rails:6:in `
' >>>> >>>> From the googling I've done, seems to be related to an invalid >>>> character in the re-direct, but really I have no idea... >>>> >>>> Any way of ignoring these weird characters and continuing? Anyway of >>>> fixing this? >>>> >>>> Best, >>>> Kevin >>>> >>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From godfreykfc at gmail.com Tue Feb 26 03:50:45 2013 From: godfreykfc at gmail.com (Godfrey Chan) Date: Mon, 25 Feb 2013 19:50:45 -0800 (PST) Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: Message-ID: <1361850644466.0c1912ce@Nodemailer> Hi Kevin, I?don't currently have my computer with me to check the code?- so sorry?I can't be more helpful. I might have made some of these methods up, so apply a good dose of common sense if things didn't work as expected :) I believe this should do the trick: form = agent.current_page.forms.first form.action = form.action.gsub(...).gsub(...) form.submit Forget what I said earlier about the nokogiri stuff and form.submit won't work etc. I wasn't aware that you can change the form action from mechanize. If form.submit somehow?didn't work, please post a trace. Godfrey ? Sent from Mailbox for iPhone On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann wrote: > sorry meant I tried: > form.attr('action', action) > to no avail...? > On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann wrote: >> actually, this grabbed it: >> >> action = form.action >> >> and your gsub worked a charm. >> >> Just need to get this to submit... Because you're right form.submit didn't >> work... I tried what you gave me: >> >> action = form.attr('action') >> >> But undefined meth again on attr... >> >> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann wrote: >> >>> Hi Godfrey, >>> >>> Thanks for your help. Really working hard to figure this out. I ran into >>> a road block that is my inexperience with nokogiri and mech. >>> >>> When I try to grab the action (action = form.attr('action')), I get >>> undefined method... Here is what I'm doing exactly: >>> >>> agent = Mechanize.new >>> agent.user_agent_alias = 'Mac Safari' >>> page = agent.get('https://somesite.jsp') >>> >>> form = agent.page.forms.first >>> action = form.attr('action') >>> >>> and undefined method... >>> I tried seeing if I could grab the action various ways to no avail. >>> >>> My other question is, assuming I do this successfuly, I would normally >>> use "form.submit" but you're saying this won't work? What should I try? >>> >>> Thanks again for your help!! >>> Kevin >>> >>> >>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >>> >>>> Hi Kevin, >>>> >>>> It looks like the "action" attribute of your form contains something >>>> along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>> The semicolon in here is not strictly a valid character in an URL. Try >>>> replacing it with %3B. >>>> >>>> form = page.at( ...select the form with a selector... ) >>>> >>>> action = form.attr('action') >>>> >>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>> >>>> form.attr('action', action) >>>> >>>> # then submit the form as usual (You can't do form.submit because this >>>> form object we have here is a nokogiri node. You need to go through >>>> mechanize to submit the form.) >>>> >>>> Godfrey >>>> ? >>>> Sent from Mailbox for iPhone >>>> >>>> >>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann >>> > wrote: >>>> >>>>> Got a form, hitting a button with Mechanize. Thus far have been going >>>>> through login steps quite nicely so I know my Mechanize basics are up to >>>>> snuff. >>>>> >>>>> Then I get: >>>>> >>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>> from >>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>> `split' >>>>> from >>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>> `parse' >>>>> from >>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>> `parse' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>> `rescue in resolve' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>> `resolve' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>> `fetch' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>> `post_form' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>> `submit' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>> `submit' >>>>> from (irb):69 >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>> `start' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>> `start' >>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>> `' >>>>> from script/rails:6:in `require' >>>>> from script/rails:6:in `
' >>>>> >>>>> From the googling I've done, seems to be related to an invalid >>>>> character in the re-direct, but really I have no idea... >>>>> >>>>> Any way of ignoring these weird characters and continuing? Anyway of >>>>> fixing this? >>>>> >>>>> Best, >>>>> Kevin >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Tue Feb 26 04:06:44 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Mon, 25 Feb 2013 23:06:44 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: <1361850644466.0c1912ce@Nodemailer> References: <1361850644466.0c1912ce@Nodemailer> Message-ID: Hi Godfrey, 1) Thanks again for your help, I really appreciate it. 2) yeah, so I can grab the action, gsub it nicely (I see the return). But when I 'form.submit' it errors out on the original URI, as if the new version wasn't changed at all... I'll keep trying and let you know how I do. ENough for tonight. thanks, Kevin On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: > Hi Kevin, > > I don't currently have my computer with me to check the code - so sorry I > can't be more helpful. I might have made some of these methods up, so apply > a good dose of common sense if things didn't work as expected :) > > I believe this should do the trick: > > form = agent.current_page.forms.first > > form.action = form.action.gsub(...).gsub(...) > > form.submit > > Forget what I said earlier about the nokogiri stuff and form.submit won't > work etc. I wasn't aware that you can change the form action from mechanize. > > If form.submit somehow didn't work, please post a trace. > > Godfrey > ? > Sent from Mailbox for iPhone > > > On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann wrote: > >> sorry meant I tried: >> >> form.attr('action', action) >> >> to no avail...? >> >> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann > > wrote: >> >>> actually, this grabbed it: >>> >>> action = form.action >>> >>> and your gsub worked a charm. >>> >>> Just need to get this to submit... Because you're right form.submit >>> didn't work... I tried what you gave me: >>> >>> action = form.attr('action') >>> >>> But undefined meth again on attr... >>> >>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann < >>> kevin at proofimaging.com> wrote: >>> >>>> Hi Godfrey, >>>> >>>> Thanks for your help. Really working hard to figure this out. I ran >>>> into a road block that is my inexperience with nokogiri and mech. >>>> >>>> When I try to grab the action (action = form.attr('action')), I get >>>> undefined method... Here is what I'm doing exactly: >>>> >>>> agent = Mechanize.new >>>> agent.user_agent_alias = 'Mac Safari' >>>> page = agent.get('https://somesite.jsp') >>>> >>>> form = agent.page.forms.first >>>> action = form.attr('action') >>>> >>>> and undefined method... >>>> I tried seeing if I could grab the action various ways to no avail. >>>> >>>> My other question is, assuming I do this successfuly, I would normally >>>> use "form.submit" but you're saying this won't work? What should I try? >>>> >>>> Thanks again for your help!! >>>> Kevin >>>> >>>> >>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >>>> >>>>> Hi Kevin, >>>>> >>>>> It looks like the "action" attribute of your form contains something >>>>> along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>>> The semicolon in here is not strictly a valid character in an URL. Try >>>>> replacing it with %3B. >>>>> >>>>> form = page.at( ...select the form with a selector... ) >>>>> >>>>> action = form.attr('action') >>>>> >>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>> >>>>> form.attr('action', action) >>>>> >>>>> # then submit the form as usual (You can't do form.submit because this >>>>> form object we have here is a nokogiri node. You need to go through >>>>> mechanize to submit the form.) >>>>> >>>>> Godfrey >>>>> ? >>>>> Sent from Mailbox for iPhone >>>>> >>>>> >>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann < >>>>> kevin at proofimaging.com> wrote: >>>>> >>>>>> Got a form, hitting a button with Mechanize. Thus far have been going >>>>>> through login steps quite nicely so I know my Mechanize basics are up to >>>>>> snuff. >>>>>> >>>>>> Then I get: >>>>>> >>>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>> from >>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>>> `split' >>>>>> from >>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>>> `parse' >>>>>> from >>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>>> `parse' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>>> `rescue in resolve' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>>> `resolve' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>>> `fetch' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>>> `post_form' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>>> `submit' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>>> `submit' >>>>>> from (irb):69 >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>>> `start' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>>> `start' >>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>>> `' >>>>>> from script/rails:6:in `require' >>>>>> from script/rails:6:in `
' >>>>>> >>>>>> From the googling I've done, seems to be related to an invalid >>>>>> character in the re-direct, but really I have no idea... >>>>>> >>>>>> Any way of ignoring these weird characters and continuing? Anyway of >>>>>> fixing this? >>>>>> >>>>>> Best, >>>>>> Kevin >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mechanize-users mailing list >>>>> Mechanize-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>> >>>> >>>> >>> >> > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From godfreykfc at gmail.com Tue Feb 26 06:58:10 2013 From: godfreykfc at gmail.com (Godfrey Chan) Date: Mon, 25 Feb 2013 22:58:10 -0800 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361850644466.0c1912ce@Nodemailer> Message-ID: Hi Kevin, I finally have access to my computer now, and it turns out the semi colon in the action is not the corrupt - the colon ("...:-1") was the issue. (Didn't notice that before, I thought it was a line number or some sort of error code. That's a *really* weird URI.) >> require 'mechanize' >> agent = Mechanize.new >> page = agent.get 'http://formtest.dev/form_test.html' >> form = page.forms.first >> form.submit URI::InvalidURIError: bad URI(is not URI?): CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >> form.action = CGI.escape(form.action) >> form.submit # this should work now If your server fail to URL decode that for some odd reason and refuse to process that, you can try: form.action = './' + form.action Hope it helps. Godfrey On Mon, Feb 25, 2013 at 8:06 PM, Kevin Kornemann wrote: > Hi Godfrey, > > 1) Thanks again for your help, I really appreciate it. > 2) yeah, so I can grab the action, gsub it nicely (I see the return). But > when I 'form.submit' it errors out on the original URI, as if the new > version wasn't changed at all... > > I'll keep trying and let you know how I do. ENough for tonight. > thanks, > Kevin > > > On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: > >> Hi Kevin, >> >> I don't currently have my computer with me to check the code - so sorry I >> can't be more helpful. I might have made some of these methods up, so apply >> a good dose of common sense if things didn't work as expected :) >> >> I believe this should do the trick: >> >> form = agent.current_page.forms.first >> >> form.action = form.action.gsub(...).gsub(...) >> >> form.submit >> >> Forget what I said earlier about the nokogiri stuff and form.submit won't >> work etc. I wasn't aware that you can change the form action from mechanize. >> >> If form.submit somehow didn't work, please post a trace. >> >> Godfrey >> ? >> Sent from Mailbox for iPhone >> >> >> On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann wrote: >> >>> sorry meant I tried: >>> >>> form.attr('action', action) >>> >>> to no avail...? >>> >>> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann < >>> kevin at proofimaging.com> wrote: >>> >>>> actually, this grabbed it: >>>> >>>> action = form.action >>>> >>>> and your gsub worked a charm. >>>> >>>> Just need to get this to submit... Because you're right form.submit >>>> didn't work... I tried what you gave me: >>>> >>>> action = form.attr('action') >>>> >>>> But undefined meth again on attr... >>>> >>>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann < >>>> kevin at proofimaging.com> wrote: >>>> >>>>> Hi Godfrey, >>>>> >>>>> Thanks for your help. Really working hard to figure this out. I ran >>>>> into a road block that is my inexperience with nokogiri and mech. >>>>> >>>>> When I try to grab the action (action = form.attr('action')), I get >>>>> undefined method... Here is what I'm doing exactly: >>>>> >>>>> agent = Mechanize.new >>>>> agent.user_agent_alias = 'Mac Safari' >>>>> page = agent.get('https://somesite.jsp') >>>>> >>>>> form = agent.page.forms.first >>>>> action = form.attr('action') >>>>> >>>>> and undefined method... >>>>> I tried seeing if I could grab the action various ways to no avail. >>>>> >>>>> My other question is, assuming I do this successfuly, I would normally >>>>> use "form.submit" but you're saying this won't work? What should I try? >>>>> >>>>> Thanks again for your help!! >>>>> Kevin >>>>> >>>>> >>>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >>>>> >>>>>> Hi Kevin, >>>>>> >>>>>> It looks like the "action" attribute of your form contains something >>>>>> along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>>>> The semicolon in here is not strictly a valid character in an URL. Try >>>>>> replacing it with %3B. >>>>>> >>>>>> form = page.at( ...select the form with a selector... ) >>>>>> >>>>>> action = form.attr('action') >>>>>> >>>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>>> >>>>>> form.attr('action', action) >>>>>> >>>>>> # then submit the form as usual (You can't do form.submit because >>>>>> this form object we have here is a nokogiri node. You need to go through >>>>>> mechanize to submit the form.) >>>>>> >>>>>> Godfrey >>>>>> ? >>>>>> Sent from Mailbox for iPhone >>>>>> >>>>>> >>>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann < >>>>>> kevin at proofimaging.com> wrote: >>>>>> >>>>>>> Got a form, hitting a button with Mechanize. Thus far have been >>>>>>> going through login steps quite nicely so I know my Mechanize basics are up >>>>>>> to snuff. >>>>>>> >>>>>>> Then I get: >>>>>>> >>>>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>>> from >>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>>>> `split' >>>>>>> from >>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>>>> `parse' >>>>>>> from >>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>>>> `parse' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>>>> `rescue in resolve' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>>>> `resolve' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>>>> `fetch' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>>>> `post_form' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>>>> `submit' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>>>> `submit' >>>>>>> from (irb):69 >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>>>> `start' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>>>> `start' >>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>>>> `' >>>>>>> from script/rails:6:in `require' >>>>>>> from script/rails:6:in `
' >>>>>>> >>>>>>> From the googling I've done, seems to be related to an invalid >>>>>>> character in the re-direct, but really I have no idea... >>>>>>> >>>>>>> Any way of ignoring these weird characters and continuing? Anyway of >>>>>>> fixing this? >>>>>>> >>>>>>> Best, >>>>>>> Kevin >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Mechanize-users mailing list >>>>>> Mechanize-users at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>>> >>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Tue Feb 26 14:02:09 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Tue, 26 Feb 2013 09:02:09 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361850644466.0c1912ce@Nodemailer> Message-ID: So, that got past the URI error, but the system just bounced me to a "session expired" page... Actually last night I did you gsub trick and submitted the new string with a manual agent.post but the same thing happened, it just bounced me to a session expired page. This is the last page (and form) of many pages/forms before it that all have similar action URI strings (with the same weird chars) that don't raise an mech exceptions. Which leads me to believe this is something about the agent not looking enough like a browser that this site doesn't like, and this last page is looking at headers or something and bouncing requests from non browsers... But I'm no expert this is just my hunch at this point. Is there a more robust method of setting up mech to look more like a browser??? besides the usual agent.user_agent_alias = 'Windows IE 6' Headers? Thanks again, Kevin On Tue, Feb 26, 2013 at 1:58 AM, Godfrey Chan wrote: > Hi Kevin, > > I finally have access to my computer now, and it turns out the semi colon > in the action is not the corrupt - the colon ("...:-1") was the issue. > (Didn't notice that before, I thought it was a line number or some sort of > error code. That's a *really* weird URI.) > > >> require 'mechanize' > >> agent = Mechanize.new > >> page = agent.get 'http://formtest.dev/form_test.html' > >> form = page.forms.first > >> form.submit > URI::InvalidURIError: bad URI(is not URI?): > CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 > >> form.action = CGI.escape(form.action) > >> form.submit # this should work now > > If your server fail to URL decode that for some odd reason and refuse to > process that, you can try: > > form.action = './' + form.action > > Hope it helps. > > Godfrey > > > > On Mon, Feb 25, 2013 at 8:06 PM, Kevin Kornemann wrote: > >> Hi Godfrey, >> >> 1) Thanks again for your help, I really appreciate it. >> 2) yeah, so I can grab the action, gsub it nicely (I see the return). But >> when I 'form.submit' it errors out on the original URI, as if the new >> version wasn't changed at all... >> >> I'll keep trying and let you know how I do. ENough for tonight. >> thanks, >> Kevin >> >> >> On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: >> >>> Hi Kevin, >>> >>> I don't currently have my computer with me to check the code - so >>> sorry I can't be more helpful. I might have made some of these methods up, >>> so apply a good dose of common sense if things didn't work as expected :) >>> >>> I believe this should do the trick: >>> >>> form = agent.current_page.forms.first >>> >>> form.action = form.action.gsub(...).gsub(...) >>> >>> form.submit >>> >>> Forget what I said earlier about the nokogiri stuff and form.submit >>> won't work etc. I wasn't aware that you can change the form action from >>> mechanize. >>> >>> If form.submit somehow didn't work, please post a trace. >>> >>> Godfrey >>> ? >>> Sent from Mailbox for iPhone >>> >>> >>> On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann >> > wrote: >>> >>>> sorry meant I tried: >>>> >>>> form.attr('action', action) >>>> >>>> to no avail...? >>>> >>>> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann < >>>> kevin at proofimaging.com> wrote: >>>> >>>>> actually, this grabbed it: >>>>> >>>>> action = form.action >>>>> >>>>> and your gsub worked a charm. >>>>> >>>>> Just need to get this to submit... Because you're right form.submit >>>>> didn't work... I tried what you gave me: >>>>> >>>>> action = form.attr('action') >>>>> >>>>> But undefined meth again on attr... >>>>> >>>>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann < >>>>> kevin at proofimaging.com> wrote: >>>>> >>>>>> Hi Godfrey, >>>>>> >>>>>> Thanks for your help. Really working hard to figure this out. I ran >>>>>> into a road block that is my inexperience with nokogiri and mech. >>>>>> >>>>>> When I try to grab the action (action = form.attr('action')), I get >>>>>> undefined method... Here is what I'm doing exactly: >>>>>> >>>>>> agent = Mechanize.new >>>>>> agent.user_agent_alias = 'Mac Safari' >>>>>> page = agent.get('https://somesite.jsp') >>>>>> >>>>>> form = agent.page.forms.first >>>>>> action = form.attr('action') >>>>>> >>>>>> and undefined method... >>>>>> I tried seeing if I could grab the action various ways to no avail. >>>>>> >>>>>> My other question is, assuming I do this successfuly, I would >>>>>> normally use "form.submit" but you're saying this won't work? What should I >>>>>> try? >>>>>> >>>>>> Thanks again for your help!! >>>>>> Kevin >>>>>> >>>>>> >>>>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >>>>>> >>>>>>> Hi Kevin, >>>>>>> >>>>>>> It looks like the "action" attribute of your form contains something >>>>>>> along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>>>>> The semicolon in here is not strictly a valid character in an URL. Try >>>>>>> replacing it with %3B. >>>>>>> >>>>>>> form = page.at( ...select the form with a selector... ) >>>>>>> >>>>>>> action = form.attr('action') >>>>>>> >>>>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>>>> >>>>>>> form.attr('action', action) >>>>>>> >>>>>>> # then submit the form as usual (You can't do form.submit because >>>>>>> this form object we have here is a nokogiri node. You need to go through >>>>>>> mechanize to submit the form.) >>>>>>> >>>>>>> Godfrey >>>>>>> ? >>>>>>> Sent from Mailbox for iPhone >>>>>>> >>>>>>> >>>>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann < >>>>>>> kevin at proofimaging.com> wrote: >>>>>>> >>>>>>>> Got a form, hitting a button with Mechanize. Thus far have been >>>>>>>> going through login steps quite nicely so I know my Mechanize basics are up >>>>>>>> to snuff. >>>>>>>> >>>>>>>> Then I get: >>>>>>>> >>>>>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>>>> from >>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>>>>> `split' >>>>>>>> from >>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>>>>> `parse' >>>>>>>> from >>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>>>>> `parse' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>>>>> `rescue in resolve' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>>>>> `resolve' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>>>>> `fetch' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>>>>> `post_form' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>>>>> `submit' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>>>>> `submit' >>>>>>>> from (irb):69 >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>>>>> `start' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>>>>> `start' >>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>>>>> `' >>>>>>>> from script/rails:6:in `require' >>>>>>>> from script/rails:6:in `
' >>>>>>>> >>>>>>>> From the googling I've done, seems to be related to an invalid >>>>>>>> character in the re-direct, but really I have no idea... >>>>>>>> >>>>>>>> Any way of ignoring these weird characters and continuing? Anyway >>>>>>>> of fixing this? >>>>>>>> >>>>>>>> Best, >>>>>>>> Kevin >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Mechanize-users mailing list >>>>>>> Mechanize-users at rubyforge.org >>>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattw922 at gmail.com Tue Feb 26 15:06:57 2013 From: mattw922 at gmail.com (Matt White) Date: Tue, 26 Feb 2013 07:06:57 -0800 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361850644466.0c1912ce@Nodemailer> Message-ID: <6B93798D-9994-4F2F-BF0C-33915BDD2F7C@gmail.com> You might try something other than IE6. See http://rubydoc.info/gems/mechanize/Mechanize, scroll down just a bit to see all the options. On Feb 26, 2013, at 6:02 AM, Kevin Kornemann wrote: > So, that got past the URI error, but the system just bounced me to a "session expired" page... Actually last night I did you gsub trick and submitted the new string with a manual agent.post but the same thing happened, it just bounced me to a session expired page. > > This is the last page (and form) of many pages/forms before it that all have similar action URI strings (with the same weird chars) that don't raise an mech exceptions. Which leads me to believe this is something about the agent not looking enough like a browser that this site doesn't like, and this last page is looking at headers or something and bouncing requests from non browsers... But I'm no expert this is just my hunch at this point. > > Is there a more robust method of setting up mech to look more like a browser??? besides the usual agent.user_agent_alias = 'Windows IE 6' > > Headers? > > Thanks again, > Kevin > > > On Tue, Feb 26, 2013 at 1:58 AM, Godfrey Chan wrote: >> Hi Kevin, >> >> I finally have access to my computer now, and it turns out the semi colon in the action is not the corrupt - the colon ("...:-1") was the issue. (Didn't notice that before, I thought it was a line number or some sort of error code. That's a *really* weird URI.) >> >> >> require 'mechanize' >> >> agent = Mechanize.new >> >> page = agent.get 'http://formtest.dev/form_test.html' >> >> form = page.forms.first >> >> form.submit >> URI::InvalidURIError: bad URI(is not URI?): CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >> >> form.action = CGI.escape(form.action) >> >> form.submit # this should work now >> >> If your server fail to URL decode that for some odd reason and refuse to process that, you can try: >> >> form.action = './' + form.action >> >> Hope it helps. >> >> Godfrey >> >> >> >> On Mon, Feb 25, 2013 at 8:06 PM, Kevin Kornemann wrote: >>> Hi Godfrey, >>> >>> 1) Thanks again for your help, I really appreciate it. >>> 2) yeah, so I can grab the action, gsub it nicely (I see the return). But when I 'form.submit' it errors out on the original URI, as if the new version wasn't changed at all... >>> >>> I'll keep trying and let you know how I do. ENough for tonight. >>> thanks, >>> Kevin >>> >>> >>> On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: >>>> Hi Kevin, >>>> >>>> I don't currently have my computer with me to check the code - so sorry I can't be more helpful. I might have made some of these methods up, so apply a good dose of common sense if things didn't work as expected :) >>>> >>>> I believe this should do the trick: >>>> >>>> form = agent.current_page.forms.first >>>> >>>> form.action = form.action.gsub(...).gsub(...) >>>> >>>> form.submit >>>> >>>> Forget what I said earlier about the nokogiri stuff and form.submit won't work etc. I wasn't aware that you can change the form action from mechanize. >>>> >>>> If form.submit somehow didn't work, please post a trace. >>>> >>>> Godfrey >>>> ? >>>> Sent from Mailbox for iPhone >>>> >>>> >>>> On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann wrote: >>>>> sorry meant I tried: >>>>> >>>>> >>>>> form.attr('action', action) >>>>> >>>>> to no avail...? >>>>> >>>>> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann wrote: >>>>>> actually, this grabbed it: >>>>>> >>>>>> action = form.action >>>>>> >>>>>> and your gsub worked a charm. >>>>>> >>>>>> Just need to get this to submit... Because you're right form.submit didn't work... I tried what you gave me: >>>>>> >>>>>> action = form.attr('action') >>>>>> >>>>>> But undefined meth again on attr... >>>>>> >>>>>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann wrote: >>>>>>> Hi Godfrey, >>>>>>> >>>>>>> Thanks for your help. Really working hard to figure this out. I ran into a road block that is my inexperience with nokogiri and mech. >>>>>>> >>>>>>> When I try to grab the action (action = form.attr('action')), I get undefined method... Here is what I'm doing exactly: >>>>>>> >>>>>>> agent = Mechanize.new >>>>>>> agent.user_agent_alias = 'Mac Safari' >>>>>>> page = agent.get('https://somesite.jsp') >>>>>>> >>>>>>> form = agent.page.forms.first >>>>>>> action = form.attr('action') >>>>>>> >>>>>>> and undefined method... >>>>>>> I tried seeing if I could grab the action various ways to no avail. >>>>>>> >>>>>>> My other question is, assuming I do this successfuly, I would normally use "form.submit" but you're saying this won't work? What should I try? >>>>>>> >>>>>>> Thanks again for your help!! >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >>>>>>>> Hi Kevin, >>>>>>>> >>>>>>>> >>>>>>>> It looks like the "action" attribute of your form contains something along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". The semicolon in here is not strictly a valid character in an URL. Try replacing it with %3B. >>>>>>>> >>>>>>>> form = page.at( ...select the form with a selector... ) >>>>>>>> >>>>>>>> action = form.attr('action') >>>>>>>> >>>>>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>>>>> >>>>>>>> form.attr('action', action) >>>>>>>> >>>>>>>> # then submit the form as usual (You can't do form.submit because this form object we have here is a nokogiri node. You need to go through mechanize to submit the form.) >>>>>>>> >>>>>>>> Godfrey >>>>>>>> ? >>>>>>>> Sent from Mailbox for iPhone >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann wrote: >>>>>>>>> Got a form, hitting a button with Mechanize. Thus far have been going through login steps quite nicely so I know my Mechanize basics are up to snuff. >>>>>>>>> >>>>>>>>> >>>>>>>>> Then I get: >>>>>>>>> >>>>>>>>> URI::InvalidURIError: bad URI(is not URI?): CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>>>>> from /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in `split' >>>>>>>>> from /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in `parse' >>>>>>>>> from /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in `parse' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in `rescue in resolve' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in `resolve' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in `fetch' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in `post_form' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in `submit' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in `submit' >>>>>>>>> from (irb):69 >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in `start' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in `start' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in `' >>>>>>>>> from script/rails:6:in `require' >>>>>>>>> from script/rails:6:in `
' >>>>>>>>> >>>>>>>>> From the googling I've done, seems to be related to an invalid character in the re-direct, but really I have no idea... >>>>>>>>> >>>>>>>>> Any way of ignoring these weird characters and continuing? Anyway of fixing this? >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Kevin >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Mechanize-users mailing list >>>>>>>> Mechanize-users at rubyforge.org >>>>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Tue Feb 26 15:49:07 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Tue, 26 Feb 2013 10:49:07 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: <6B93798D-9994-4F2F-BF0C-33915BDD2F7C@gmail.com> References: <1361850644466.0c1912ce@Nodemailer> <6B93798D-9994-4F2F-BF0C-33915BDD2F7C@gmail.com> Message-ID: Yeah I started with, "Mac Safari" It's an old site, so I thought try the oldest crappiest browser I could use. But I suppose trying them all will only be slightly futile. Now I'm wondering if it's a lost cookie thing. I understand Mech tracks these automatically but I see from googling people still also manually store them in a "cookie_jar" pain... On Tue, Feb 26, 2013 at 10:06 AM, Matt White wrote: > You might try something other than IE6. See > http://rubydoc.info/gems/mechanize/Mechanize, scroll down just a bit to > see all the options. > > > On Feb 26, 2013, at 6:02 AM, Kevin Kornemann > wrote: > > So, that got past the URI error, but the system just bounced me to a > "session expired" page... Actually last night I did you gsub trick and > submitted the new string with a manual agent.post but the same thing > happened, it just bounced me to a session expired page. > > This is the last page (and form) of many pages/forms before it that all > have similar action URI strings (with the same weird chars) that don't > raise an mech exceptions. Which leads me to believe this is something about > the agent not looking enough like a browser that this site doesn't like, > and this last page is looking at headers or something and bouncing requests > from non browsers... But I'm no expert this is just my hunch at this point. > > Is there a more robust method of setting up mech to look more like a > browser??? besides the usual agent.user_agent_alias = 'Windows IE 6' > > Headers? > > Thanks again, > Kevin > > > On Tue, Feb 26, 2013 at 1:58 AM, Godfrey Chan wrote: > >> Hi Kevin, >> >> I finally have access to my computer now, and it turns out the semi colon >> in the action is not the corrupt - the colon ("...:-1") was the issue. >> (Didn't notice that before, I thought it was a line number or some sort of >> error code. That's a *really* weird URI.) >> >> >> require 'mechanize' >> >> agent = Mechanize.new >> >> page = agent.get 'http://formtest.dev/form_test.html' >> >> form = page.forms.first >> >> form.submit >> URI::InvalidURIError: bad URI(is not URI?): >> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >> >> form.action = CGI.escape(form.action) >> >> form.submit # this should work now >> >> If your server fail to URL decode that for some odd reason and refuse to >> process that, you can try: >> >> form.action = './' + form.action >> >> Hope it helps. >> >> Godfrey >> >> >> >> On Mon, Feb 25, 2013 at 8:06 PM, Kevin Kornemann > > wrote: >> >>> Hi Godfrey, >>> >>> 1) Thanks again for your help, I really appreciate it. >>> 2) yeah, so I can grab the action, gsub it nicely (I see the return). >>> But when I 'form.submit' it errors out on the original URI, as if the new >>> version wasn't changed at all... >>> >>> I'll keep trying and let you know how I do. ENough for tonight. >>> thanks, >>> Kevin >>> >>> >>> On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: >>> >>>> Hi Kevin, >>>> >>>> I don't currently have my computer with me to check the code - so >>>> sorry I can't be more helpful. I might have made some of these methods up, >>>> so apply a good dose of common sense if things didn't work as expected :) >>>> >>>> I believe this should do the trick: >>>> >>>> form = agent.current_page.forms.first >>>> >>>> form.action = form.action.gsub(...).gsub(...) >>>> >>>> form.submit >>>> >>>> Forget what I said earlier about the nokogiri stuff and form.submit >>>> won't work etc. I wasn't aware that you can change the form action from >>>> mechanize. >>>> >>>> If form.submit somehow didn't work, please post a trace. >>>> >>>> Godfrey >>>> ? >>>> Sent from Mailbox for iPhone >>>> >>>> >>>> On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann < >>>> kevin at proofimaging.com> wrote: >>>> >>>>> sorry meant I tried: >>>>> >>>>> form.attr('action', action) >>>>> >>>>> to no avail...? >>>>> >>>>> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann < >>>>> kevin at proofimaging.com> wrote: >>>>> >>>>>> actually, this grabbed it: >>>>>> >>>>>> action = form.action >>>>>> >>>>>> and your gsub worked a charm. >>>>>> >>>>>> Just need to get this to submit... Because you're right form.submit >>>>>> didn't work... I tried what you gave me: >>>>>> >>>>>> action = form.attr('action') >>>>>> >>>>>> But undefined meth again on attr... >>>>>> >>>>>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann < >>>>>> kevin at proofimaging.com> wrote: >>>>>> >>>>>>> Hi Godfrey, >>>>>>> >>>>>>> Thanks for your help. Really working hard to figure this out. I ran >>>>>>> into a road block that is my inexperience with nokogiri and mech. >>>>>>> >>>>>>> When I try to grab the action (action = form.attr('action')), I get >>>>>>> undefined method... Here is what I'm doing exactly: >>>>>>> >>>>>>> agent = Mechanize.new >>>>>>> agent.user_agent_alias = 'Mac Safari' >>>>>>> page = agent.get('https://somesite.jsp') >>>>>>> >>>>>>> form = agent.page.forms.first >>>>>>> action = form.attr('action') >>>>>>> >>>>>>> and undefined method... >>>>>>> I tried seeing if I could grab the action various ways to no avail. >>>>>>> >>>>>>> My other question is, assuming I do this successfuly, I would >>>>>>> normally use "form.submit" but you're saying this won't work? What should I >>>>>>> try? >>>>>>> >>>>>>> Thanks again for your help!! >>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan wrote: >>>>>>> >>>>>>>> Hi Kevin, >>>>>>>> >>>>>>>> It looks like the "action" attribute of your form contains >>>>>>>> something along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>>>>>> The semicolon in here is not strictly a valid character in an URL. Try >>>>>>>> replacing it with %3B. >>>>>>>> >>>>>>>> form = page.at( ...select the form with a selector... ) >>>>>>>> >>>>>>>> action = form.attr('action') >>>>>>>> >>>>>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>>>>> >>>>>>>> form.attr('action', action) >>>>>>>> >>>>>>>> # then submit the form as usual (You can't do form.submit because >>>>>>>> this form object we have here is a nokogiri node. You need to go through >>>>>>>> mechanize to submit the form.) >>>>>>>> >>>>>>>> Godfrey >>>>>>>> ? >>>>>>>> Sent from Mailbox for iPhone >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann < >>>>>>>> kevin at proofimaging.com> wrote: >>>>>>>> >>>>>>>>> Got a form, hitting a button with Mechanize. Thus far have been >>>>>>>>> going through login steps quite nicely so I know my Mechanize basics are up >>>>>>>>> to snuff. >>>>>>>>> >>>>>>>>> Then I get: >>>>>>>>> >>>>>>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>>>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>>>>> from >>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>>>>>> `split' >>>>>>>>> from >>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>>>>>> `parse' >>>>>>>>> from >>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>>>>>> `parse' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>>>>>> `rescue in resolve' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>>>>>> `resolve' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>>>>>> `fetch' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>>>>>> `post_form' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>>>>>> `submit' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>>>>>> `submit' >>>>>>>>> from (irb):69 >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>>>>>> `start' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>>>>>> `start' >>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>>>>>> `' >>>>>>>>> from script/rails:6:in `require' >>>>>>>>> from script/rails:6:in `
' >>>>>>>>> >>>>>>>>> From the googling I've done, seems to be related to an invalid >>>>>>>>> character in the re-direct, but really I have no idea... >>>>>>>>> >>>>>>>>> Any way of ignoring these weird characters and continuing? Anyway >>>>>>>>> of fixing this? >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Kevin >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Mechanize-users mailing list >>>>>>>> Mechanize-users at rubyforge.org >>>>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From godfreykfc at gmail.com Wed Feb 27 02:43:05 2013 From: godfreykfc at gmail.com (Godfrey Chan) Date: Tue, 26 Feb 2013 18:43:05 -0800 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361850644466.0c1912ce@Nodemailer> <6B93798D-9994-4F2F-BF0C-33915BDD2F7C@gmail.com> Message-ID: I doubt that it's a UA string or cookie issue - I'd very carefully compare the URIs from the previous pages and the offending form target URI. Also, have you tried "form.action = './' + form.action" approach instead of the gsub/CGI.escape approach? It could be that your server doesn't decode the URL properly and hence missed the session ID. On Tue, Feb 26, 2013 at 7:49 AM, Kevin Kornemann wrote: > Yeah I started with, "Mac Safari" > > It's an old site, so I thought try the oldest crappiest browser I could > use. But I suppose trying them all will only be slightly futile. > > Now I'm wondering if it's a lost cookie thing. I understand Mech tracks > these automatically but I see from googling people still also manually > store them in a "cookie_jar" > > pain... > > > On Tue, Feb 26, 2013 at 10:06 AM, Matt White wrote: > >> You might try something other than IE6. See >> http://rubydoc.info/gems/mechanize/Mechanize, scroll down just a bit to >> see all the options. >> >> >> On Feb 26, 2013, at 6:02 AM, Kevin Kornemann >> wrote: >> >> So, that got past the URI error, but the system just bounced me to a >> "session expired" page... Actually last night I did you gsub trick and >> submitted the new string with a manual agent.post but the same thing >> happened, it just bounced me to a session expired page. >> >> This is the last page (and form) of many pages/forms before it that all >> have similar action URI strings (with the same weird chars) that don't >> raise an mech exceptions. Which leads me to believe this is something about >> the agent not looking enough like a browser that this site doesn't like, >> and this last page is looking at headers or something and bouncing requests >> from non browsers... But I'm no expert this is just my hunch at this point. >> >> Is there a more robust method of setting up mech to look more like a >> browser??? besides the usual agent.user_agent_alias = 'Windows IE 6' >> >> Headers? >> >> Thanks again, >> Kevin >> >> >> On Tue, Feb 26, 2013 at 1:58 AM, Godfrey Chan wrote: >> >>> Hi Kevin, >>> >>> I finally have access to my computer now, and it turns out the semi >>> colon in the action is not the corrupt - the colon ("...:-1") was the >>> issue. (Didn't notice that before, I thought it was a line number or some >>> sort of error code. That's a *really* weird URI.) >>> >>> >> require 'mechanize' >>> >> agent = Mechanize.new >>> >> page = agent.get 'http://formtest.dev/form_test.html' >>> >> form = page.forms.first >>> >> form.submit >>> URI::InvalidURIError: bad URI(is not URI?): >>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>> >> form.action = CGI.escape(form.action) >>> >> form.submit # this should work now >>> >>> If your server fail to URL decode that for some odd reason and refuse to >>> process that, you can try: >>> >>> form.action = './' + form.action >>> >>> Hope it helps. >>> >>> Godfrey >>> >>> >>> >>> On Mon, Feb 25, 2013 at 8:06 PM, Kevin Kornemann < >>> kevin at proofimaging.com> wrote: >>> >>>> Hi Godfrey, >>>> >>>> 1) Thanks again for your help, I really appreciate it. >>>> 2) yeah, so I can grab the action, gsub it nicely (I see the return). >>>> But when I 'form.submit' it errors out on the original URI, as if the new >>>> version wasn't changed at all... >>>> >>>> I'll keep trying and let you know how I do. ENough for tonight. >>>> thanks, >>>> Kevin >>>> >>>> >>>> On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: >>>> >>>>> Hi Kevin, >>>>> >>>>> I don't currently have my computer with me to check the code - so >>>>> sorry I can't be more helpful. I might have made some of these methods up, >>>>> so apply a good dose of common sense if things didn't work as expected :) >>>>> >>>>> I believe this should do the trick: >>>>> >>>>> form = agent.current_page.forms.first >>>>> >>>>> form.action = form.action.gsub(...).gsub(...) >>>>> >>>>> form.submit >>>>> >>>>> Forget what I said earlier about the nokogiri stuff and form.submit >>>>> won't work etc. I wasn't aware that you can change the form action from >>>>> mechanize. >>>>> >>>>> If form.submit somehow didn't work, please post a trace. >>>>> >>>>> Godfrey >>>>> ? >>>>> Sent from Mailbox for iPhone >>>>> >>>>> >>>>> On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann < >>>>> kevin at proofimaging.com> wrote: >>>>> >>>>>> sorry meant I tried: >>>>>> >>>>>> form.attr('action', action) >>>>>> >>>>>> to no avail...? >>>>>> >>>>>> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann < >>>>>> kevin at proofimaging.com> wrote: >>>>>> >>>>>>> actually, this grabbed it: >>>>>>> >>>>>>> action = form.action >>>>>>> >>>>>>> and your gsub worked a charm. >>>>>>> >>>>>>> Just need to get this to submit... Because you're right form.submit >>>>>>> didn't work... I tried what you gave me: >>>>>>> >>>>>>> action = form.attr('action') >>>>>>> >>>>>>> But undefined meth again on attr... >>>>>>> >>>>>>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann < >>>>>>> kevin at proofimaging.com> wrote: >>>>>>> >>>>>>>> Hi Godfrey, >>>>>>>> >>>>>>>> Thanks for your help. Really working hard to figure this out. I ran >>>>>>>> into a road block that is my inexperience with nokogiri and mech. >>>>>>>> >>>>>>>> When I try to grab the action (action = form.attr('action')), I >>>>>>>> get undefined method... Here is what I'm doing exactly: >>>>>>>> >>>>>>>> agent = Mechanize.new >>>>>>>> agent.user_agent_alias = 'Mac Safari' >>>>>>>> page = agent.get('https://somesite.jsp') >>>>>>>> >>>>>>>> form = agent.page.forms.first >>>>>>>> action = form.attr('action') >>>>>>>> >>>>>>>> and undefined method... >>>>>>>> I tried seeing if I could grab the action various ways to no avail. >>>>>>>> >>>>>>>> My other question is, assuming I do this successfuly, I would >>>>>>>> normally use "form.submit" but you're saying this won't work? What should I >>>>>>>> try? >>>>>>>> >>>>>>>> Thanks again for your help!! >>>>>>>> Kevin >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan >>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hi Kevin, >>>>>>>>> >>>>>>>>> It looks like the "action" attribute of your form contains >>>>>>>>> something along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>>>>>>> The semicolon in here is not strictly a valid character in an URL. Try >>>>>>>>> replacing it with %3B. >>>>>>>>> >>>>>>>>> form = page.at( ...select the form with a selector... ) >>>>>>>>> >>>>>>>>> action = form.attr('action') >>>>>>>>> >>>>>>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>>>>>> >>>>>>>>> form.attr('action', action) >>>>>>>>> >>>>>>>>> # then submit the form as usual (You can't do form.submit because >>>>>>>>> this form object we have here is a nokogiri node. You need to go through >>>>>>>>> mechanize to submit the form.) >>>>>>>>> >>>>>>>>> Godfrey >>>>>>>>> ? >>>>>>>>> Sent from Mailbox for iPhone >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann < >>>>>>>>> kevin at proofimaging.com> wrote: >>>>>>>>> >>>>>>>>>> Got a form, hitting a button with Mechanize. Thus far have been >>>>>>>>>> going through login steps quite nicely so I know my Mechanize basics are up >>>>>>>>>> to snuff. >>>>>>>>>> >>>>>>>>>> Then I get: >>>>>>>>>> >>>>>>>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>>>>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>>>>>> from >>>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>>>>>>> `split' >>>>>>>>>> from >>>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>>>>>>> `parse' >>>>>>>>>> from >>>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>>>>>>> `parse' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>>>>>>> `rescue in resolve' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>>>>>>> `resolve' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>>>>>>> `fetch' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>>>>>>> `post_form' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>>>>>>> `submit' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>>>>>>> `submit' >>>>>>>>>> from (irb):69 >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>>>>>>> `start' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>>>>>>> `start' >>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>>>>>>> `' >>>>>>>>>> from script/rails:6:in `require' >>>>>>>>>> from script/rails:6:in `
' >>>>>>>>>> >>>>>>>>>> From the googling I've done, seems to be related to an invalid >>>>>>>>>> character in the re-direct, but really I have no idea... >>>>>>>>>> >>>>>>>>>> Any way of ignoring these weird characters and continuing? Anyway >>>>>>>>>> of fixing this? >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Kevin >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Mechanize-users mailing list >>>>>>>>> Mechanize-users at rubyforge.org >>>>>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Mechanize-users mailing list >>>>> Mechanize-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin at proofimaging.com Wed Feb 27 04:06:20 2013 From: kevin at proofimaging.com (Kevin Kornemann) Date: Tue, 26 Feb 2013 23:06:20 -0500 Subject: [Mechanize-users] URI problems, need some love In-Reply-To: References: <1361850644466.0c1912ce@Nodemailer> <6B93798D-9994-4F2F-BF0C-33915BDD2F7C@gmail.com> Message-ID: that did it. The identifier strings (no idea if that is what they are called) were all preceded by a path of some sort, and this one wasn't. So simply adding the ./ before the id did the trick. I'm in! Just like Matthew Broderick in War Games... Can't express how my gratitude enough for your help. I googled and googled and googled, nothing suggested this, which now that I understand what was happening, well perhaps it's too simple a topic to deserve a stack overflow post. Anyway, hopefully this helps a fellow newb in the future. Thanks Godfrey. Kevin On Tue, Feb 26, 2013 at 9:43 PM, Godfrey Chan wrote: > I doubt that it's a UA string or cookie issue - I'd very carefully compare > the URIs from the previous pages and the offending form target URI. > > Also, have you tried "form.action = './' + form.action" approach instead > of the gsub/CGI.escape approach? It could be that your server doesn't > decode the URL properly and hence missed the session ID. > > > On Tue, Feb 26, 2013 at 7:49 AM, Kevin Kornemann wrote: > >> Yeah I started with, "Mac Safari" >> >> It's an old site, so I thought try the oldest crappiest browser I could >> use. But I suppose trying them all will only be slightly futile. >> >> Now I'm wondering if it's a lost cookie thing. I understand Mech tracks >> these automatically but I see from googling people still also manually >> store them in a "cookie_jar" >> >> pain... >> >> >> On Tue, Feb 26, 2013 at 10:06 AM, Matt White wrote: >> >>> You might try something other than IE6. See >>> http://rubydoc.info/gems/mechanize/Mechanize, scroll down just a bit to >>> see all the options. >>> >>> >>> On Feb 26, 2013, at 6:02 AM, Kevin Kornemann >>> wrote: >>> >>> So, that got past the URI error, but the system just bounced me to a >>> "session expired" page... Actually last night I did you gsub trick and >>> submitted the new string with a manual agent.post but the same thing >>> happened, it just bounced me to a session expired page. >>> >>> This is the last page (and form) of many pages/forms before it that all >>> have similar action URI strings (with the same weird chars) that don't >>> raise an mech exceptions. Which leads me to believe this is something about >>> the agent not looking enough like a browser that this site doesn't like, >>> and this last page is looking at headers or something and bouncing requests >>> from non browsers... But I'm no expert this is just my hunch at this point. >>> >>> Is there a more robust method of setting up mech to look more like a >>> browser??? besides the usual agent.user_agent_alias = 'Windows IE 6' >>> >>> Headers? >>> >>> Thanks again, >>> Kevin >>> >>> >>> On Tue, Feb 26, 2013 at 1:58 AM, Godfrey Chan wrote: >>> >>>> Hi Kevin, >>>> >>>> I finally have access to my computer now, and it turns out the semi >>>> colon in the action is not the corrupt - the colon ("...:-1") was the >>>> issue. (Didn't notice that before, I thought it was a line number or some >>>> sort of error code. That's a *really* weird URI.) >>>> >>>> >> require 'mechanize' >>>> >> agent = Mechanize.new >>>> >> page = agent.get 'http://formtest.dev/form_test.html' >>>> >> form = page.forms.first >>>> >> form.submit >>>> URI::InvalidURIError: bad URI(is not URI?): >>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>> >> form.action = CGI.escape(form.action) >>>> >> form.submit # this should work now >>>> >>>> If your server fail to URL decode that for some odd reason and refuse >>>> to process that, you can try: >>>> >>>> form.action = './' + form.action >>>> >>>> Hope it helps. >>>> >>>> Godfrey >>>> >>>> >>>> >>>> On Mon, Feb 25, 2013 at 8:06 PM, Kevin Kornemann < >>>> kevin at proofimaging.com> wrote: >>>> >>>>> Hi Godfrey, >>>>> >>>>> 1) Thanks again for your help, I really appreciate it. >>>>> 2) yeah, so I can grab the action, gsub it nicely (I see the return). >>>>> But when I 'form.submit' it errors out on the original URI, as if the new >>>>> version wasn't changed at all... >>>>> >>>>> I'll keep trying and let you know how I do. ENough for tonight. >>>>> thanks, >>>>> Kevin >>>>> >>>>> >>>>> On Mon, Feb 25, 2013 at 10:50 PM, Godfrey Chan wrote: >>>>> >>>>>> Hi Kevin, >>>>>> >>>>>> I don't currently have my computer with me to check the code - so >>>>>> sorry I can't be more helpful. I might have made some of these methods up, >>>>>> so apply a good dose of common sense if things didn't work as expected :) >>>>>> >>>>>> I believe this should do the trick: >>>>>> >>>>>> form = agent.current_page.forms.first >>>>>> >>>>>> form.action = form.action.gsub(...).gsub(...) >>>>>> >>>>>> form.submit >>>>>> >>>>>> Forget what I said earlier about the nokogiri stuff and form.submit >>>>>> won't work etc. I wasn't aware that you can change the form action from >>>>>> mechanize. >>>>>> >>>>>> If form.submit somehow didn't work, please post a trace. >>>>>> >>>>>> Godfrey >>>>>> ? >>>>>> Sent from Mailbox for iPhone >>>>>> >>>>>> >>>>>> On Mon, Feb 25, 2013 at 7:38 PM, Kevin Kornemann < >>>>>> kevin at proofimaging.com> wrote: >>>>>> >>>>>>> sorry meant I tried: >>>>>>> >>>>>>> form.attr('action', action) >>>>>>> >>>>>>> to no avail...? >>>>>>> >>>>>>> On Mon, Feb 25, 2013 at 10:35 PM, Kevin Kornemann < >>>>>>> kevin at proofimaging.com> wrote: >>>>>>> >>>>>>>> actually, this grabbed it: >>>>>>>> >>>>>>>> action = form.action >>>>>>>> >>>>>>>> and your gsub worked a charm. >>>>>>>> >>>>>>>> Just need to get this to submit... Because you're right form.submit >>>>>>>> didn't work... I tried what you gave me: >>>>>>>> >>>>>>>> action = form.attr('action') >>>>>>>> >>>>>>>> But undefined meth again on attr... >>>>>>>> >>>>>>>> On Mon, Feb 25, 2013 at 10:29 PM, Kevin Kornemann < >>>>>>>> kevin at proofimaging.com> wrote: >>>>>>>> >>>>>>>>> Hi Godfrey, >>>>>>>>> >>>>>>>>> Thanks for your help. Really working hard to figure this out. I >>>>>>>>> ran into a road block that is my inexperience with nokogiri and mech. >>>>>>>>> >>>>>>>>> When I try to grab the action (action = form.attr('action')), I >>>>>>>>> get undefined method... Here is what I'm doing exactly: >>>>>>>>> >>>>>>>>> agent = Mechanize.new >>>>>>>>> agent.user_agent_alias = 'Mac Safari' >>>>>>>>> page = agent.get('https://somesite.jsp') >>>>>>>>> >>>>>>>>> form = agent.page.forms.first >>>>>>>>> action = form.attr('action') >>>>>>>>> >>>>>>>>> and undefined method... >>>>>>>>> I tried seeing if I could grab the action various ways to no avail. >>>>>>>>> >>>>>>>>> My other question is, assuming I do this successfuly, I would >>>>>>>>> normally use "form.submit" but you're saying this won't work? What should I >>>>>>>>> try? >>>>>>>>> >>>>>>>>> Thanks again for your help!! >>>>>>>>> Kevin >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Feb 25, 2013 at 9:06 PM, Godfrey Chan < >>>>>>>>> godfreykfc at gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Kevin, >>>>>>>>>> >>>>>>>>>> It looks like the "action" attribute of your form contains >>>>>>>>>> something along the lines of "CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz". >>>>>>>>>> The semicolon in here is not strictly a valid character in an URL. Try >>>>>>>>>> replacing it with %3B. >>>>>>>>>> >>>>>>>>>> form = page.at( ...select the form with a selector... ) >>>>>>>>>> >>>>>>>>>> action = form.attr('action') >>>>>>>>>> >>>>>>>>>> action = action.gsub(';','%3B').gsub('=','%3D') >>>>>>>>>> >>>>>>>>>> form.attr('action', action) >>>>>>>>>> >>>>>>>>>> # then submit the form as usual (You can't do form.submit because >>>>>>>>>> this form object we have here is a nokogiri node. You need to go through >>>>>>>>>> mechanize to submit the form.) >>>>>>>>>> >>>>>>>>>> Godfrey >>>>>>>>>> ? >>>>>>>>>> Sent from Mailbox for iPhone >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Feb 25, 2013 at 4:01 PM, Kevin Kornemann < >>>>>>>>>> kevin at proofimaging.com> wrote: >>>>>>>>>> >>>>>>>>>>> Got a form, hitting a button with Mechanize. Thus far have been >>>>>>>>>>> going through login steps quite nicely so I know my Mechanize basics are up >>>>>>>>>>> to snuff. >>>>>>>>>>> >>>>>>>>>>> Then I get: >>>>>>>>>>> >>>>>>>>>>> URI::InvalidURIError: bad URI(is not URI?): >>>>>>>>>>> CBMenu.jsp;jsessionid=0000FWXUFvwFz8oQcES43L6CCz1:-1 >>>>>>>>>>> from >>>>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:176:in >>>>>>>>>>> `split' >>>>>>>>>>> from >>>>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:211:in >>>>>>>>>>> `parse' >>>>>>>>>>> from >>>>>>>>>>> /Users/kevin/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/uri/common.rb:747:in >>>>>>>>>>> `parse' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:621:in >>>>>>>>>>> `rescue in resolve' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:618:in >>>>>>>>>>> `resolve' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/http/agent.rb:214:in >>>>>>>>>>> `fetch' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:1229:in >>>>>>>>>>> `post_form' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize.rb:515:in >>>>>>>>>>> `submit' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/mechanize-2.5.1/lib/mechanize/form.rb:178:in >>>>>>>>>>> `submit' >>>>>>>>>>> from (irb):69 >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:47:in >>>>>>>>>>> `start' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands/console.rb:8:in >>>>>>>>>>> `start' >>>>>>>>>>> from /Users/kevin/.rvm/gems/ruby-1.9.3-p286 at texas/gems/railties-3.2.12/lib/rails/commands.rb:41:in >>>>>>>>>>> `' >>>>>>>>>>> from script/rails:6:in `require' >>>>>>>>>>> from script/rails:6:in `
' >>>>>>>>>>> >>>>>>>>>>> From the googling I've done, seems to be related to an invalid >>>>>>>>>>> character in the re-direct, but really I have no idea... >>>>>>>>>>> >>>>>>>>>>> Any way of ignoring these weird characters and continuing? >>>>>>>>>>> Anyway of fixing this? >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Kevin >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Mechanize-users mailing list >>>>>>>>>> Mechanize-users at rubyforge.org >>>>>>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Mechanize-users mailing list >>>>>> Mechanize-users at rubyforge.org >>>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mechanize-users mailing list >>>>> Mechanize-users at rubyforge.org >>>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Mechanize-users mailing list >>>> Mechanize-users at rubyforge.org >>>> http://rubyforge.org/mailman/listinfo/mechanize-users >>>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >>> >>> _______________________________________________ >>> Mechanize-users mailing list >>> Mechanize-users at rubyforge.org >>> http://rubyforge.org/mailman/listinfo/mechanize-users >>> >> >> >> _______________________________________________ >> Mechanize-users mailing list >> Mechanize-users at rubyforge.org >> http://rubyforge.org/mailman/listinfo/mechanize-users >> > > > _______________________________________________ > Mechanize-users mailing list > Mechanize-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/mechanize-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: