[rspec-users] A recently observed anti pattern: commented out tests

David Chelimsky dchelimsky at gmail.com
Wed Jul 25 13:12:49 UTC 2012

On Tue, Jul 24, 2012 at 1:55 PM, James Cox <james at imaj.es> wrote:
> Yeah, I love pending too. but it doesn't help me get a sense of the
> state of a suite before I start. now it's part of my practice to go in
> and find out how much is commented out.
> David,
> three concerns with pending as an option:
> a. it won't help the people who think it's ok to comment out whole
> tests. If you make that choice it's not a good thing (™). I don't
> think there's enough evangelism in the world to change them.

Those same people will work around whatever we make available to help them.

> b. how do you distinguish between a pending and a broken-but-fixing
> test? one means, "i've got no coverage here, and i haven't thought
> about it' whereas the other says, 'i used to have coverage, and i need
> to fix it'. I know it's semantics, but that's important here: i need
> to know where no effort for testing has been made vs where testing
> existed (which may imply some domain knowledge which was at one point
> true).

What Chris Flipse said.

> c. if you see # it 'should …' or similar, that's a commented test, not
> a test comment. This metric is always going to be loose… but it may
> give an indication, a sniff test. some kind of idea of what the state
> of a test is. It's the same as running rake stats - you and I know
> it's a bullshit metric (it can't possibly tell me how good the tests
> are), but it tells me at least if any effort to test has happened.
> Then, I run coverage and figure out how exercised the code is..
> somewhere along that line, it'd be good to know if there used to be
> tests but they are commented out.

"it" is an alias for example, specify, and pending (among others), so
we'd have to parse for all of those.

> an anecdote… i experienced this recently with a project, and a
> significant majority of the tests were just commented out. They used
> to work, and a lot of it modeled the domain reasonably well - but
> either due to a breaking gem upgrade or a refactor or something - the
> original dev just didn't move/fix the test. So, on the face of it, the
> test scenario looked horrible, but in the end, for the key components,
> fixing the tests wasn't that painful. Regardless, I got a pretty fast
> sense of how much water he was treading at the time (or, how much he
> was under it :/)
> so yes, pending is ok, but a second keyword "broken" might be nicer,
> which would act the same but output different info.

You can actually do that!

RSpec.configuration do |c|
  c.alias_example_to :broken, :pending => "Broken"

See http://rubydoc.info/gems/rspec-core/RSpec/Core/ExampleGroup.alias_example_to
and https://github.com/rspec/rspec-core/blob/master/lib/rspec/core/example_group.rb#L69-104


> -james
> On Mon, Jul 23, 2012 at 10:58 PM, Adam Sroka <adam.sroka at gmail.com> wrote:
>> I haven't posted in a while, but I want to say that as someone who spends a
>> significant portion of his time teaching (T/B)DD I am totally in love with
>> pending specs. There are analogous concepts in nearly every xUnit/xSpec, but
>> pending is by far the best. Kudos.
>> On Jul 23, 2012 9:57 PM, "David Chelimsky" <dchelimsky at gmail.com> wrote:
>>> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at imaj.es> wrote:
>>> > Hey,
>>> >
>>> > in a bunch of the rescues i've recently done, I see a pretty big anti
>>> > pattern: tests don't work, and so rather than making them work, the
>>> > dev team just comments them out till 'later'.
>>> >
>>> > Does anyone think it'd be useful/interesting to get a flag for rspec
>>> > which would compare lines vs lines-commented, and if the percentage
>>> > was higher than xx, it'd issue some kind of warning?
>>> The pending feature is designed to help with this problem by allowing
>>> you to disable an example while still keeping it visible.
>>> If we were to do what you propose, we'd need to offer an opt-out
>>> and/or the ability to configure the percentage. Consider a suite that
>>> uses a lot of comments to annotate the specs. The problem with making
>>> it configurable is that the folks who's priorities lead them to
>>> comment out examples instead of fixing them will likely just disable
>>> this feature.
>>> I'd say, let's encourage people to use 'pending' correctly. WDYT?
>>> Cheers,
>>> David
>>> _______________________________________________
>>> rspec-users mailing list
>>> rspec-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/rspec-users
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
> --
> James Cox,
> Consultant, Raconteur, Photographer, Entrepreneur
> t: +1 347 433 0567  e: james at imaj.es w: http://imaj.es/
> talk: http://twitter.com/imajes photos: http://500px.com/imajes
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

More information about the rspec-users mailing list