[rspec-devel] [ rspec-Feature Requests-13540 ] [Story Runner] dry run should show all of the steps, even after one that fails

noreply at rubyforge.org noreply at rubyforge.org
Mon Nov 19 00:23:16 EST 2007


Feature Requests item #13540, was opened at 2007-08-30 20:56
You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=3152&aid=13540&group_id=797

Category: None
Group: None
Status: Open
Priority: 3
Submitted By: David Chelimsky (dchelimsky)
Assigned to: Dan North (tastapod)
Summary: [Story Runner] dry run should show all of the steps, even after one that fails

Initial Comment:
I've got a Scenario which is failing at a step in the middle. When I run:

  ruby stories/user_changes_password.rb

the output does not include all of the steps, but does show the error that occurs in the last step that was executed.

When I run:

  ruby stories/user_changes_password.rb --dry-run

the output, again, does not include all of the steps, even though it doesn't give you the impression that any steps are actually getting executed.

I think you should see all of the steps when doing a dry run. In fact, I really think you should see all of the steps even when one fails. This is high level stuff - 'fail fast' doesn't have the same meaning here as it does when doing TDD at the object level.

----------------------------------------------------------------------

Comment By: Chad Humphries (spicycode)
Date: 2007-11-19 00:23

Message:
Moved to http://rspec.lighthouseapp.com/projects/5645-rspec/tickets/112-13540-story-runner-dry-run-should-show-all-of-the-steps-even-after-one-that-fails#ticket-112-2

----------------------------------------------------------------------

Comment By: Ian Dees (undees)
Date: 2007-08-31 14:52

Message:
FWIW, we ran into this all the time with our old, homegrown GUI tests (and will no doubt face it again when we start writing longer Stories).  Do we stop and give up, or keep going?

For us, the answer was, "it depends."  If a test failed -- a control caption didn't match or something -- we'd soldier bravely on.

But if a test threw an exception (other than an "expectation not met," of course) -- maybe a particular window completely failed to appear -- then we'd bail out of the entire script, and hope the next script had better luck.

----------------------------------------------------------------------

Comment By: David Chelimsky (dchelimsky)
Date: 2007-08-31 05:43

Message:
I think that, for a tool that operates at a high level, running subsequent makes perfect sense.

Using the map analogy, you wouldn't forget to turn left executing a Story - you would FAIL to turn left because there was some road block - but what if the subsequent two rights and a left all failed too? Now you're back on the same road! Then the subsequent steps make sense.

Also - not all steps in a Story are necessary at the code level, even though they may be useful at the story-telling level.

The story-level tool that I 'grew up on' was FitNesse, based on FIT. In FIT, the entire story gets played out on each page, regardless of where failures are. In my experience as a developer using those failures to guide my next steps at the code level, having the later steps pass or fail provides valuable information as to where I am in relation to DONE. And that's a very important point of this process, isn't it?

----------------------------------------------------------------------

Comment By: Dan North (tastapod)
Date: 2007-08-31 05:33

Message:
It's definitely a bug if it isn't printing all the steps in a dry run. Can you post an example?

A scenario should always "fail fast", but at the scenario level (i.e. any subsequent scenarios will always be run).

Running subsequent steps after a failure doesn't make any sense. Imagine following a list of map directions. Halfway down the list you forget to "turn left", but carrying on following the list anyway: "turn right at the warehouse" - what warehouse? "then straight ahead at the river" - eh?). By definition you are in an inconsistent or unrecoverable state after a failed step.

Although... that does suddenly introduce the possibility for a RecoverableStep, or maybe an OptionalStep - hmm. But not on this bug report :)

----------------------------------------------------------------------

You can respond by visiting: 
http://rubyforge.org/tracker/?func=detail&atid=3152&aid=13540&group_id=797


More information about the rspec-devel mailing list