From dorikick at gmail.com  Thu Jun  3 20:56:02 2010
From: dorikick at gmail.com (doridori Jo)
Date: Thu, 3 Jun 2010 17:56:02 -0700
Subject: [Celerity-users] dealing with popups part 2
Message-ID: <AANLkTimL1rIXZcV0PVlKmLNf6pL1d5UYfsokcpS9nKaD@mail.gmail.com>

hi guys,

in previous email, someone mentioned getting directly to htmlunit.

however, i was wondering, if it's possible to somehow selectively "mute"
javascript popup ads without having to turn off javascript support ?

thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://rubyforge.org/pipermail/celerity-users/attachments/20100603/c9afb561/attachment.html>

From kris.leech at gmail.com  Fri Jun 11 03:59:23 2010
From: kris.leech at gmail.com (Kris Leech)
Date: Fri, 11 Jun 2010 08:59:23 +0100
Subject: [Celerity-users] Navigating list of links without href's (.NET page)
Message-ID: <4C11ECDB.5040500@gmail.com>

The preferred method for navigating a list of links is shown in the FAQ 
(http://wiki.github.com/jarib/celerity/faq). However the links I need to 
crawl do not have 'href' attributes, but inline onclick javascript  
events which result in the required page being loaded.

Can anyone suggest a better way than the one I am using, which according 
to the FAQ may result in an "invalid state".

browser.div(:id => 'ctl00_CPHL_divPeopleZone').links.each do | link |
   link.click
   puts 'HERE: ' + browser.url
   browser.back
end

Many thanks, Kris.

From jari.bakken at gmail.com  Fri Jun 11 04:27:44 2010
From: jari.bakken at gmail.com (Jari Bakken)
Date: Fri, 11 Jun 2010 10:27:44 +0200
Subject: [Celerity-users] Navigating list of links without href's (.NET
	page)
In-Reply-To: <4C11ECDB.5040500@gmail.com>
References: <4C11ECDB.5040500@gmail.com>
Message-ID: <AANLkTim0l7iSi9VU8fR_XFHteAm0O9BqRT-4S57xcZKT@mail.gmail.com>

Hi Kris,

On Fri, Jun 11, 2010 at 9:59 AM, Kris Leech <kris.leech at gmail.com> wrote:
> Can anyone suggest a better way than the one I am using, which according to
> the FAQ may result in an "invalid state".
>

You should be able to use :index to do the same, as along the list of
links doesn't change. Something like:

id = 'ctl00_CPHL_divPeopleZone'
count = browser.div(:id => id).links.size

1.upto(count).do |idx|
    browser.div(:id => id).link(:index => idx).click
    puts browser.url
    browser.back
end

From win at wincent.com  Mon Jun 21 19:02:12 2010
From: win at wincent.com (Wincent Colaiuta)
Date: Tue, 22 Jun 2010 01:02:12 +0200
Subject: [Celerity-users] URLs getting double-escaped
Message-ID: <97B61BFA-CBBD-4F00-9F2B-739CB75CA80D@wincent.com>

Hi,

I'm trying to get Celerity to visit a URL of the form:

  http://localhost:3000/wiki/has_%3Cstrange%3E_stuff

(That is, a wiki article with some strange chars in the title, "has <strange> stuff"):

  require 'rubygems'
  require 'celerity'
  b=Celerity::Browser.new
  b.goto('http://localhost:3000/wiki/has_%3Cstrange%3E_stuff')

Turns out though that Celerity gets a 404 error at this point, and looking at my server access log I see why; the URL is getting URL-double-escaped:

  Started GET "/wiki/has_%25253Cstrange%25253E_stuff" for 127.0.0.1 at Tue Jun 22 00:45:25 +0200 2010
    Processing by ArticlesController#show as */*
    Parameters: {"id"=>"has_%253Cstrange%253E_stuff"}

ie. "%3C" in the string passed to goto() is becoming "%25253C"

If I instead instruct it to visit '/wiki/has_<strange>_stuff', it's still getting escaped too much:

  Started GET "/wiki/has_%253Cstrange%253E_stuff" for 127.0.0.1 at Tue Jun 22 00:52:20 +0200 2010
    Processing by ArticlesController#show as */*
    Parameters: {"id"=>"has_%3Cstrange%3E_stuff"}

ie. now the "<" is becoming %253C

I'm wondering if this is a bug in Celerity, or perhaps in HtmlUnit...

Cheers,
Wincent


From jari.bakken at gmail.com  Tue Jun 22 05:57:32 2010
From: jari.bakken at gmail.com (Jari Bakken)
Date: Tue, 22 Jun 2010 11:57:32 +0200
Subject: [Celerity-users] URLs getting double-escaped
In-Reply-To: <97B61BFA-CBBD-4F00-9F2B-739CB75CA80D@wincent.com>
References: <97B61BFA-CBBD-4F00-9F2B-739CB75CA80D@wincent.com>
Message-ID: <AANLkTilHBk23v70WB3G9YTfoP9v2bIWHFKVa4gvceq3e@mail.gmail.com>

On Tue, Jun 22, 2010 at 1:02 AM, Wincent Colaiuta <win at wincent.com> wrote:
>
> ie. "%3C" in the string passed to goto() is becoming "%25253C"
>

We're not doing any escaping in Celerity, so probably an issue in
HtmlUnit. Does it work if you pass the URL unescaped?

  browser.goto "http://localhost:3000/wiki/has_<strange>_stuff"

From win at wincent.com  Tue Jun 22 06:18:20 2010
From: win at wincent.com (Wincent Colaiuta)
Date: Tue, 22 Jun 2010 12:18:20 +0200
Subject: [Celerity-users] URLs getting double-escaped
In-Reply-To: <AANLkTilHBk23v70WB3G9YTfoP9v2bIWHFKVa4gvceq3e@mail.gmail.com>
References: <97B61BFA-CBBD-4F00-9F2B-739CB75CA80D@wincent.com>
	<AANLkTilHBk23v70WB3G9YTfoP9v2bIWHFKVa4gvceq3e@mail.gmail.com>
Message-ID: <648F5542-AA3F-4C84-A517-160FCB6B8D98@wincent.com>

El 22/06/2010, a las 11:57, Jari Bakken escribi?:

> On Tue, Jun 22, 2010 at 1:02 AM, Wincent Colaiuta <win at wincent.com> wrote:
>> 
>> ie. "%3C" in the string passed to goto() is becoming "%25253C"
> 
> We're not doing any escaping in Celerity, so probably an issue in
> HtmlUnit. Does it work if you pass the URL unescaped?
> 
>  browser.goto "http://localhost:3000/wiki/has_<strange>_stuff"

No, if I go there, server logs show that the request URL ends up being:

  http://localhost:3000/wiki/has_%253Cstrange%253E_stuff

ie. "<" becomes "%3C" which is then double-escaped and becomes "%253C"

Cheers,
Wincent



From win at wincent.com  Tue Jun 22 07:10:39 2010
From: win at wincent.com (Wincent Colaiuta)
Date: Tue, 22 Jun 2010 13:10:39 +0200
Subject: [Celerity-users] URLs getting double-escaped
In-Reply-To: <AANLkTilHBk23v70WB3G9YTfoP9v2bIWHFKVa4gvceq3e@mail.gmail.com>
References: <97B61BFA-CBBD-4F00-9F2B-739CB75CA80D@wincent.com>
	<AANLkTilHBk23v70WB3G9YTfoP9v2bIWHFKVa4gvceq3e@mail.gmail.com>
Message-ID: <21F1E5B5-7705-4EB0-9242-DFA089529219@wincent.com>

El 22/06/2010, a las 11:57, Jari Bakken escribi?:

> On Tue, Jun 22, 2010 at 1:02 AM, Wincent Colaiuta <win at wincent.com> wrote:
>> 
>> ie. "%3C" in the string passed to goto() is becoming "%25253C"
>> 
> 
> We're not doing any escaping in Celerity, so probably an issue in
> HtmlUnit.

You're right. Done more investigation and have established that it's a bug which is already fixed upstream.

Have opened a ticket in the issue tracker here about updating the bundled HtmlUnit in order to pick up the fix:

http://github.com/jarib/celerity/issues/issue/19

Cheers,
Wincent




From jari.bakken at gmail.com  Tue Jun 22 08:09:43 2010
From: jari.bakken at gmail.com (Jari Bakken)
Date: Tue, 22 Jun 2010 14:09:43 +0200
Subject: [Celerity-users] URLs getting double-escaped
In-Reply-To: <21F1E5B5-7705-4EB0-9242-DFA089529219@wincent.com>
References: <97B61BFA-CBBD-4F00-9F2B-739CB75CA80D@wincent.com> 
	<AANLkTilHBk23v70WB3G9YTfoP9v2bIWHFKVa4gvceq3e@mail.gmail.com> 
	<21F1E5B5-7705-4EB0-9242-DFA089529219@wincent.com>
Message-ID: <AANLkTin_0OoGS54rTKprOE4oUueiVv-Oi-TeLNxMSQ8B@mail.gmail.com>

On Tue, Jun 22, 2010 at 1:10 PM, Wincent Colaiuta <win at wincent.com> wrote:
> Have opened a ticket in the issue tracker here about updating the bundled HtmlUnit in order to pick up the fix:
>
> http://github.com/jarib/celerity/issues/issue/19
>

Yes, a release with updated snapshots is long overdue. HEAD already
has pretty recent snapshots, but there are other issues (which you'll
see if you run the spec suite) which I haven't been able to
prioritize. Patches welcome, of course :)