[rspec-users] A recently observed anti pattern: commented out tests

James Cox james at imaj.es
Tue Jul 24 17:55:57 UTC 2012

Yeah, I love pending too. but it doesn't help me get a sense of the
state of a suite before I start. now it's part of my practice to go in
and find out how much is commented out.


three concerns with pending as an option:

a. it won't help the people who think it's ok to comment out whole
tests. If you make that choice it's not a good thing (™). I don't
think there's enough evangelism in the world to change them.

b. how do you distinguish between a pending and a broken-but-fixing
test? one means, "i've got no coverage here, and i haven't thought
about it' whereas the other says, 'i used to have coverage, and i need
to fix it'. I know it's semantics, but that's important here: i need
to know where no effort for testing has been made vs where testing
existed (which may imply some domain knowledge which was at one point

c. if you see # it 'should …' or similar, that's a commented test, not
a test comment. This metric is always going to be loose… but it may
give an indication, a sniff test. some kind of idea of what the state
of a test is. It's the same as running rake stats - you and I know
it's a bullshit metric (it can't possibly tell me how good the tests
are), but it tells me at least if any effort to test has happened.
Then, I run coverage and figure out how exercised the code is..
somewhere along that line, it'd be good to know if there used to be
tests but they are commented out.

an anecdote… i experienced this recently with a project, and a
significant majority of the tests were just commented out. They used
to work, and a lot of it modeled the domain reasonably well - but
either due to a breaking gem upgrade or a refactor or something - the
original dev just didn't move/fix the test. So, on the face of it, the
test scenario looked horrible, but in the end, for the key components,
fixing the tests wasn't that painful. Regardless, I got a pretty fast
sense of how much water he was treading at the time (or, how much he
was under it :/)

so yes, pending is ok, but a second keyword "broken" might be nicer,
which would act the same but output different info.


On Mon, Jul 23, 2012 at 10:58 PM, Adam Sroka <adam.sroka at gmail.com> wrote:
> I haven't posted in a while, but I want to say that as someone who spends a
> significant portion of his time teaching (T/B)DD I am totally in love with
> pending specs. There are analogous concepts in nearly every xUnit/xSpec, but
> pending is by far the best. Kudos.
> On Jul 23, 2012 9:57 PM, "David Chelimsky" <dchelimsky at gmail.com> wrote:
>> On Mon, Jul 23, 2012 at 11:19 AM, James Cox <james at imaj.es> wrote:
>> > Hey,
>> >
>> > in a bunch of the rescues i've recently done, I see a pretty big anti
>> > pattern: tests don't work, and so rather than making them work, the
>> > dev team just comments them out till 'later'.
>> >
>> > Does anyone think it'd be useful/interesting to get a flag for rspec
>> > which would compare lines vs lines-commented, and if the percentage
>> > was higher than xx, it'd issue some kind of warning?
>> The pending feature is designed to help with this problem by allowing
>> you to disable an example while still keeping it visible.
>> If we were to do what you propose, we'd need to offer an opt-out
>> and/or the ability to configure the percentage. Consider a suite that
>> uses a lot of comments to annotate the specs. The problem with making
>> it configurable is that the folks who's priorities lead them to
>> comment out examples instead of fixing them will likely just disable
>> this feature.
>> I'd say, let's encourage people to use 'pending' correctly. WDYT?
>> Cheers,
>> David
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

James Cox,
Consultant, Raconteur, Photographer, Entrepreneur
t: +1 347 433 0567  e: james at imaj.es w: http://imaj.es/
talk: http://twitter.com/imajes photos: http://500px.com/imajes

More information about the rspec-users mailing list