[rspec-users] Thought on a quality metric for Cucumber test suites

Ben Mabey ben at benmabey.com
Thu Apr 2 18:43:36 EDT 2009

Matt Wynne wrote:
> Writing another post to Zach I had a thought about a quality metric 
> for Cucumber suites: Ratio of #steps executed / #step matchers
> For example, if you can run 100 steps with only 10 step matchers, you 
> have a nice low ratio of 100 / 10, meaning you are getting loads of 
> testing done with only a few step matchers to maintain.

I don't think a metric like that would be enough.  That doesn't take 
into account what type of steps these are.  If you are aiming for 
extremely high reuse you are probably going to end up with extremely 
imperative style scenarios.. Which *could* be a bad thing.  So I would 
be wary of using any such calculation to determine a "quality".  With 
something like Cucumber, the quality is really in the eye of the 
customer. :)  (I may be out of context however since I haven't read the 
other thread.. I will go do that now..)

If you are trying to get a metric then I would say you would need to 
also use the number of scenarios...  What if you have 100 steps, 10 step 
matchers, and *1* scenario?  Would you really want such a scenario in 
your app? :p

> Is there a way to pull out the stat of the number of step matchers 
> defined?
Use the usage formatter....


More information about the rspec-users mailing list