[Wtr-general] Fwd: tutorial description: scripting web tests

[Bret Pettichord]

Today was the deadline for the tutorial description that will appear in the 
XP/AU proceedings. Our submission is below. It had to be written in the 
past tense, so what it really describes is what happened this week when i 
taught the class in Portland and Seattle. I will be teaching it again next 
week in Orlando. After that, i'll be able to focus more fully on the next 
revision of the class.

1. Using blocks to specify rules that identified forms and elements worked 
rather well, certainly better than the dedidicated functions we'd used last 
October. But learning about when to identify using actions vs. values vs. 
names takes time -- time that we won't have in the half-day version in 
Calgary. So we are going to have to insert unique names for all the forms 
and elements. We'll also have to add named spans for text we want to verify.

2. The network is unnecessary. Having everyone test a single server is 
pretty pointless because it starts crashing pretty fast. On monday, when 
this started happening, i told them that they could run their own local 
server, and five minutes later every one of them was doing this. I didn't 
even set up the network on tuesday and won't be using it in orlando next 
week. I'm not sure if the server is less stable that previously. It might 
be, perhaps due to platform (windows vs. mac) or the current version of 
Ruby (1.8.1-12).

3. A couple students were getting semi-reproducible seg faults in Ruby when 
they were running harnessed tests. These were client side. I'm going to see 
if i can track this down. In one case, the student was using 1.8.1-13.

4. As the description below implies, we pretty much got most of the 
students to put together a suite of tests by the end of the day. It's 
rather satisfying to get this far and will be a challenge to get as far in 
only a half-day. The biggest area of streamlining has got to come from 
using named elements. This also means we can use more Chris's original IEC 
code and concept. A lot of students wasn't better logging. I know that Paul 
has this and this really needs to get folded into the code we use for the 
class.

5. Even with named objects, the hardest part of the class is probably 
getting students to associate the names of things with the actual objects. 
After seeing Chris's Motunes demo, i'm thinking that it might be possible 
to directly annotate objects by modifying the page via the DOM. This is 
something i want to try out. It could be major cool thing that accellerates 
the visibility of the testing hooks we use.

6. My policy has been that i need a TA for more than 12 students. On 
Monday, i had 17 students, but the TA didn't work out. That was a bear for 
me. I was constantly scrambling and didn't really have time to reflect on 
what was happening. I had a TA (Dan) on Tuesday with about the same number 
of students. Things went much much better.

7. In each class, i had one or two people who really ran with what we have. 
We may have more of these at XP/AU, because of the kind of conference it 
is. If so, we may want to have one of us run a fast-track in some corner.

8. The current docs are too disorganized and unspecific for a few people in 
each class. They get discouraged and expect better doc. I plan to keep 
working on this. I think we actually addressed this somewhat better in 
early classes. In other words, Brian addressed it better. This is important 
to ensure success with people who are not quick learners or experience 
programmers. I think they can be reached.

9. I've given up on trying to make people pair. Everyone wants to use their 
own machine. In one case, two students came with one computer between them 
and that was ok (and we should continue to encourage this). XP/AU may give 
us a better context to encourage this, but i really think that we should be 
happy to let the students decide what to do on this point.

10. I need to drop all uses of variables from the materials for Monday. 
(And later as well). The problem is that i had examples that stored 
references to forms and elements in variables, but these references would 
become stale when new pages were loaded. Yet, they would act and behave in 
large part like real references. This lead to endless confusion that was 
very difficult to explain.

11. The class was a big hit. In both classes, most of the students came up 
to me and shook my hand and thanked me for the class. I suspect that half 
of what they get comes not from my agenda, but simply from the fact they 
are given an authentic and rich environment.

12. In several cases, students found bugs in the test tool. One was a 
checkbox bug that Dan reported last week (it's fixed in CVS) in IEC. One 
student in each class found this, and i showed each of them how to fix it. 
They got a lot from this, from being shown the code and how things hooked 
together, so i'm glad i didn't worry about updating the IEC version for the 
class at the last minute. As the class proceeds, i try to explain to 
students how they can change the tool to work they want it to and in a few 
cases, we even do this. This is another big benefit of the class. I don't 
see that it's necessary to have the perfect tool. Rather, students get more 
from realizing that they can change it.

13. I have a stack of bug reports that students wrote during the class. 
I'll have to sort through them and will send basically minor timeclock bugs 
to Brian. Encouraging them to log bugs on index cards really seems to 
diffuse potential dissatisfaction. And there may be some good stuff in 
their too.

Bret

>Date: Thu, 13 May 2004 21:41:40 -0500
>To: xpau2004 at agilestl.com
>From: Bret Pettichord <bret at pettichord.com>
>Subject: tutorial description: scripting web tests
>
>Brian,
>
>Here is the description for the XP/AU proceedings. I just taught versions 
>of this class on Monday and Tuesday, which allows me to be pretty sure 
>that they claims i make here will in fact be true.
>
>Bret
>
>
>
>Scripting Web Tests
>
>Bret Pettichord, Pettichord Consulting LLC, bret at pettichord.com
>Brian Marick, Testing Foundations, marick at testing.com
>Paul Rogers, Wireless Matrix, paul.rogers at shaw.ca
>Jonathan Kohl, Westjet Airlines, jkohl at telusplanet.net
>
>Students in this tutorial learned how to write automated customer
>tests for a web-based application. They used an open-source tool kit
>to create tests that drive a web browser. Most completed a suite of
>multiple tests by the conclusion of the tutorial. This hands-on
>tutorial used open-source software installed on student-provided
>laptops. Students learned to write these tests using the Ruby
>scripting language and the Web Testing with Ruby toolkit, available at
>http://rubyforge.org/projects/wtr/. A timeclock application was used
>as the target of the tests.
>
>The tool kit consists of a library to make it convenient to access the
>COM interface to Internet Explorer, including it's document object
>mode. Similar methods could be used by any language the provides
>access to COM (which is to say: most languages). Students also made
>extensive use of the interactive Ruby interface, which facilitates
>learning and exploration. (Similar interactive features are also
>part of the Python and Tcl languages.)
>
>The hands-on learning experience allowed students to focus on the
>issues of greatest personal interest. Many appreciated being able to
>continue to experiment with and review the class exercises after the
>completion of the tutorial. The tutorial instructors are contributors
>to the tool kit and have used it in testing commercial software
>developed by agile teams.
>
>______________________________________
>  Bret Pettichord, Software Tester
>    Consultant - www.pettichord.com
>    Author - www.testinglessons.com
>    Blogger - www.io.com/~wazmo/blog
>
>    Scripting for Testers Tutorial
>    Portland, Seattle, Orlando, Austin, Calgary
>    www.pettichord.com/training.html

______________________________________
  Bret Pettichord, Software Tester
    Consultant - www.pettichord.com
    Author - www.testinglessons.com
    Blogger - www.io.com/~wazmo/blog

    Scripting for Testers Tutorial
    Portland, Seattle, Orlando, Austin, Calgary
    www.pettichord.com/training.html