Bugs: Browse | Submit New | Admin

[#17118] Expectations should take precedence over stubs

Date:
2008-01-15 16:52
Priority:
3
Submitted By:
Nobody
Assigned To:
Nobody (None)
Category:
None
State:
Open
Summary:
Expectations should take precedence over stubs

Detailed description
It's a very common practice to setup stubs in a testcase "setup" and set expectations on the same method at
the specific tests.

For example:
class FooTest < Test::Unit::TestCase
  def setup
    @foo = Foo.new
    @foo.stubs(:bar).return(true) # common case
  end

  def test_something_shouldnt_call_bar
    @foo.expects(:bar).never
    @foo.something
  end
end


I'd expect this test to fail if @foo.something calls :bar, but in the current version of Mocha (0.5.6) it will succeed.

A test that demonstrate the issue:
http://pastie.caboo.se/139119

Add A Comment: Notepad

Please login


Followup

Message
Date: 2009-01-02 16:44
Sender: James Mead

This bug report has moved to Lighthouse [1].

[1] http://floehopper.lighthouseapp.com/projects/22289/tickets/17
Date: 2008-08-06 21:05
Sender: James Mead

Yossef: I have responded to your comment on the same thread [1]
on the mailing list.

Regards, James.

[1] http://groups.google.com/group/mocha-developer/browse_thread/
thread/abd2c499db23a40
Date: 2008-08-06 18:54
Sender: James Mead

Rick: I hope you don't mind, but I've responded to your comment
on the mailing list [1] - I think that's a better forum for discussing
the issue.

Regards, James.

[1] http://groups.google.com/group/mocha-developer/browse_thread/
thread/abd2c499db23a40
Date: 2008-08-06 00:17
Sender: Yossef Mendelssohn

I'd like to add my vote to this issue. I'm glad that the state
machine feature makes what I want to test possible with a setup
method (or more precisely, a before block, since I like RSpec
-- and I think Mocha is better than the built-in RSpec mocking),
but it's kind of a pain and not entirely nice to work with or
look at.

I understand that you prefer it to work a specific way, that
you follow Jay Fields's examples of duplicating code in your
tests. What I'm not understanding is

 1. Why it's not possible to write Mocha so that it can be used
in a way others expect as well as your preferred style.
 2. Why Mocha needs to be brought into line with jMock v1, gotchas
and all.
 3. To take the words out of a colleague's mouth, why you would
build a mocking library in a dynamic language modeled after *anything*
written in Java?

Note that I have answers that may catch your initial responses,
or at least point out that I've done some research.

 1. I admit there's probably more to this than just this one
example. Constrained to just this, it seems like letting expectations
take precedence over stubs would work with the way I (and the
reporter, and some colleagues, and unknown others) want to write
the tests as well as the way you (and Jay Fields, and unknown
others) want to write them. It would also change other behavior
that some people (you, Jay Fields, unknown others?) might depend
on.
 2. Yes, I know that it's using "a syntax like that
of jMock", but that doesn't mean it has to be the Ruby version
of jMock. If it does mean that, at least you didn't call
it rMock.
 3. Dude, I don't know what to say. I'm not a fan of Java. At
all. Especially Java files that have .rb extensions.
Date: 2008-08-06 00:15
Sender: Rick Bradley

Argh.  I've been getting bitten by this too.  At least I understand
the source of the problem now.

Just my opinions, but:

(1) From what I'm reading here, jMock seems broken.

(2) It seems quite odd to emulate a Java mocking tool in a language
as dynamic as Ruby, I've never met a Java testing or mocking
tool that I've liked, though, to be fair, that's primarily because
of Java as a language.

(3) I've never bought Jay's argument that no setup is not only
a good thing but The Way It Should Be; until now I've not had
any reason to care that I was on the other side of that issue.

(4) The state-based mocking seems like it would promote more
bad development practices than prolific use of setup methods
-- most time I've seen the need for mocking states,
"scenarios", etc., it's masking the fact that the code
under test is poorly designed.  It SHOULD be painful to write
tests for bad code, state-machining relieves the pain of testing
poor code.

(5) Regardless, this was a surprising (and quiet) change to mocha
-- existing tests aren't going to break, and TDDing new code
exposes this (in a confusing, "wtf, mocha's busted for some
reason" way), but characterizing existing code gets a false
sense of certainty.

As far as I can tell, basically this change says "unless
you write tests without setup methods, prepare to be surprised
or be lulled into a false sense of confidence", without
explicitly saying so... except for here, which took me months
to find. :-/

Best,
Rick
Date: 2008-01-20 17:01
Sender: James Mead

You are correct that this behaviour did change between Mocha
v0.4.0 and v0.5.0 (in revision 115).

The idea of this change was to bring Mocha into line with jMock
v1 [1] so that it stops matching expectations when the maximum
number of expected invocations is reached. You are suffering
from "gotcha" number 2 in that jMock reference: "if
you create a stub and then an expectation for the same method,
the expectation will match, and when it stops matching the stub
will be used instead, possibly masking test failures". In
your example, the "never" expectation will never match
because it has already reached its maximum number of expected
invocations, but Mocha will then find the stub which does match.

Personally I prefer not to use setup methods, I prefer to make
tests as self-contained as possible [2]. I find this makes tests
more maintainable (albeit at the expense of some duplication).
So in this case I would prefer to *explicitly* put the stub
expectation in every test that needed it. If setting up the stub
expectation was complex or multiple stub expectations were needed
I would extract that into a descriptive method and call it explicitly
from each test that needed it.

If you don't like this approach, you do have another alternative.
Using "edge" Mocha (i.e. revision 233 or later), you
should be able to use the state-machine feature to achieve your
goal [2]. This should be released as a gem soon.

Thanks, James.
http://blog.floehopper.org

[1] "Stubs, Expectations and the Dispatch of Mocked Methods"
http://www.jmock.org/jmock1-dispatch.html
[2] "Testing: Inline Setup"
http://blog.jayfields.com/2007/06/testing-inline-setup.html
[3] OverrideExpectationsFromSetUpAcceptanceTest
http://pastie.caboo.se/141188
Date: 2008-01-17 13:34
Sender: James Mead

Just wanted to let you know I am looking into this. Thanks for
providing example tests and code.

Attached Files:

Name Description Download
No Files Currently Attached

Changes:

No Changes Have Been Made to This Item