[Wtr-general] Fwd: tutorial description: scripting web tests
bret at pettichord.com
Fri May 14 00:12:21 EDT 2004
Today was the deadline for the tutorial description that will appear in the
XP/AU proceedings. Our submission is below. It had to be written in the
past tense, so what it really describes is what happened this week when i
taught the class in Portland and Seattle. I will be teaching it again next
week in Orlando. After that, i'll be able to focus more fully on the next
revision of the class.
1. Using blocks to specify rules that identified forms and elements worked
rather well, certainly better than the dedidicated functions we'd used last
October. But learning about when to identify using actions vs. values vs.
names takes time -- time that we won't have in the half-day version in
Calgary. So we are going to have to insert unique names for all the forms
and elements. We'll also have to add named spans for text we want to verify.
2. The network is unnecessary. Having everyone test a single server is
pretty pointless because it starts crashing pretty fast. On monday, when
this started happening, i told them that they could run their own local
server, and five minutes later every one of them was doing this. I didn't
even set up the network on tuesday and won't be using it in orlando next
week. I'm not sure if the server is less stable that previously. It might
be, perhaps due to platform (windows vs. mac) or the current version of
3. A couple students were getting semi-reproducible seg faults in Ruby when
they were running harnessed tests. These were client side. I'm going to see
if i can track this down. In one case, the student was using 1.8.1-13.
4. As the description below implies, we pretty much got most of the
students to put together a suite of tests by the end of the day. It's
rather satisfying to get this far and will be a challenge to get as far in
only a half-day. The biggest area of streamlining has got to come from
using named elements. This also means we can use more Chris's original IEC
code and concept. A lot of students wasn't better logging. I know that Paul
has this and this really needs to get folded into the code we use for the
5. Even with named objects, the hardest part of the class is probably
getting students to associate the names of things with the actual objects.
After seeing Chris's Motunes demo, i'm thinking that it might be possible
to directly annotate objects by modifying the page via the DOM. This is
something i want to try out. It could be major cool thing that accellerates
the visibility of the testing hooks we use.
6. My policy has been that i need a TA for more than 12 students. On
Monday, i had 17 students, but the TA didn't work out. That was a bear for
me. I was constantly scrambling and didn't really have time to reflect on
what was happening. I had a TA (Dan) on Tuesday with about the same number
of students. Things went much much better.
7. In each class, i had one or two people who really ran with what we have.
We may have more of these at XP/AU, because of the kind of conference it
is. If so, we may want to have one of us run a fast-track in some corner.
8. The current docs are too disorganized and unspecific for a few people in
each class. They get discouraged and expect better doc. I plan to keep
working on this. I think we actually addressed this somewhat better in
early classes. In other words, Brian addressed it better. This is important
to ensure success with people who are not quick learners or experience
programmers. I think they can be reached.
9. I've given up on trying to make people pair. Everyone wants to use their
own machine. In one case, two students came with one computer between them
and that was ok (and we should continue to encourage this). XP/AU may give
us a better context to encourage this, but i really think that we should be
happy to let the students decide what to do on this point.
10. I need to drop all uses of variables from the materials for Monday.
(And later as well). The problem is that i had examples that stored
references to forms and elements in variables, but these references would
become stale when new pages were loaded. Yet, they would act and behave in
large part like real references. This lead to endless confusion that was
very difficult to explain.
11. The class was a big hit. In both classes, most of the students came up
to me and shook my hand and thanked me for the class. I suspect that half
of what they get comes not from my agenda, but simply from the fact they
are given an authentic and rich environment.
12. In several cases, students found bugs in the test tool. One was a
checkbox bug that Dan reported last week (it's fixed in CVS) in IEC. One
student in each class found this, and i showed each of them how to fix it.
They got a lot from this, from being shown the code and how things hooked
together, so i'm glad i didn't worry about updating the IEC version for the
class at the last minute. As the class proceeds, i try to explain to
students how they can change the tool to work they want it to and in a few
cases, we even do this. This is another big benefit of the class. I don't
see that it's necessary to have the perfect tool. Rather, students get more
from realizing that they can change it.
13. I have a stack of bug reports that students wrote during the class.
I'll have to sort through them and will send basically minor timeclock bugs
to Brian. Encouraging them to log bugs on index cards really seems to
diffuse potential dissatisfaction. And there may be some good stuff in
>Date: Thu, 13 May 2004 21:41:40 -0500
>To: xpau2004 at agilestl.com
>From: Bret Pettichord <bret at pettichord.com>
>Subject: tutorial description: scripting web tests
>Here is the description for the XP/AU proceedings. I just taught versions
>of this class on Monday and Tuesday, which allows me to be pretty sure
>that they claims i make here will in fact be true.
>Scripting Web Tests
>Bret Pettichord, Pettichord Consulting LLC, bret at pettichord.com
>Brian Marick, Testing Foundations, marick at testing.com
>Paul Rogers, Wireless Matrix, paul.rogers at shaw.ca
>Jonathan Kohl, Westjet Airlines, jkohl at telusplanet.net
>Students in this tutorial learned how to write automated customer
>tests for a web-based application. They used an open-source tool kit
>to create tests that drive a web browser. Most completed a suite of
>multiple tests by the conclusion of the tutorial. This hands-on
>tutorial used open-source software installed on student-provided
>laptops. Students learned to write these tests using the Ruby
>scripting language and the Web Testing with Ruby toolkit, available at
>http://rubyforge.org/projects/wtr/. A timeclock application was used
>as the target of the tests.
>The tool kit consists of a library to make it convenient to access the
>COM interface to Internet Explorer, including it's document object
>mode. Similar methods could be used by any language the provides
>access to COM (which is to say: most languages). Students also made
>extensive use of the interactive Ruby interface, which facilitates
>learning and exploration. (Similar interactive features are also
>part of the Python and Tcl languages.)
>The hands-on learning experience allowed students to focus on the
>issues of greatest personal interest. Many appreciated being able to
>continue to experiment with and review the class exercises after the
>completion of the tutorial. The tutorial instructors are contributors
>to the tool kit and have used it in testing commercial software
>developed by agile teams.
> Bret Pettichord, Software Tester
> Consultant - www.pettichord.com
> Author - www.testinglessons.com
> Blogger - www.io.com/~wazmo/blog
> Scripting for Testers Tutorial
> Portland, Seattle, Orlando, Austin, Calgary
Bret Pettichord, Software Tester
Consultant - www.pettichord.com
Author - www.testinglessons.com
Blogger - www.io.com/~wazmo/blog
Scripting for Testers Tutorial
Portland, Seattle, Orlando, Austin, Calgary
More information about the Wtr-general