[rspec-users] Mocks? Really?

Scott Taylor mailing_lists at railsnewbie.com
Sun Dec 16 22:43:58 EST 2007


Francis and Pat probably know my thoughts on this, already, but as  
far as I can see it, mocks (at least the message based ones) are  
popular for one reason in the rails / active-record world:

Speed.  Mocks are extremely  fast.  I don't think it's uncommon for  
those who write specs for rails projects to have a full test suite  
running in under 20 seconds if they are mocking all dependencies.   
Primarily, this means using mocks for all associations on a Rails  
model, and using only mocks for controller specs.

The issue of speed seems secondary, but I can already tell how costly  
a long build cycle is.  At work our test suite has about 1800 specs,  
and takes around 3 minutes to run (and hits the database in all model  
specs).  My coworker actually released something into production on  
Friday before he left which failed the build.  Obviously - this has a  
serious effect on the end user, who is currently receiving errors.   
If the test suite took 20 seconds to run, he would be running it all  
the time, and this error would have never occurred.  The fact that  
they don't run quickly means that he isn't going to run them all the  
time, and will need to rely on a CI server to beep or do something  
obnoxious like that (which isn't an option in the shared office space  
in which we are currently working).

Plus - let's be honest:  We use tests as our feedback loop.  The  
tighter we can get this, the closer we can stay to the code. When the  
specs take over a few minutes to run, we no longer have the luxury of  
running the whole suite every time we make a little one line change.   
We are forced to run specs from one file, or a subset of the specs  
from one file, and we loose a certain amount of feedback on how our  
code is integrating.

Yes - there are other reasons for using mocks - like defining  
interfaces that don't exist (or may not exist for a long time); So  
far the main reason I've seen them used is for speed.  I think this  
is a major problem with activerecord - and one which only be solved  
only by moving to an ORM with a different design pattern (like a true  
DataMapper).  Lafcadio would probably be in the running for this sort  
of thing - a library which can isolate the database and the API with  
a middle layer, which can easily be mocked either by a mock library  
which could be a drop in replacement for the database, or a middle  
layer which could easily be stubbed out with message based stubs.

As always, it's good to hear this sort of discussion going on.   
(Francis:  It was exactly this sort of discussion that got me  
involved with RSpec in the first place)

Regards,

Scott




On Dec 16, 2007, at 6:22 PM, Francis Hwang wrote:

> Coming to this thread a bit late:
>
> I think I'm pretty close to Dan, in practice: I'm not a big fan of
> fine-grained isolation in writing your tests. The practice seems to
> me like it would just bug you down. When I'm writing a behavior for a
> particular thing, such as a controller, I don't want to have to worry
> about the precise messages that are passed to its collaborators. I
> try to think in a fairly "black box" manner about it: Presupposing
> that there's a given document in a database table, when I make an
> HTTP request that's looking for that document, I should get that
> document in such-and-such a format. Ideally I wouldn't specify too
> much whether the controller hits Document.find or
> Document.find_by_sql or gets it out of some disk cache or gets the
> data by doing a magical little dance in a faerie circle off in the
> forest. It's really not my test's problem.
> On the other hand, I do think mocking is extremely useful when you're
> dealing with very externalized services with narrow, rigid interfaces
> that you can't implicitly test all the time. At work I have to write
> lots of complex tests around a specific web service, but I don't have
> a lot of control over it, so I wrote a fairly complex mock for that
> service. But even then it's a different sort of mock: It's more state-
> aware than surface-aware, which is part of the point as I see it. Of
> course, writing those sorts of mocks is much more time-consuming.
>
> If you haven't seen it before, Martin Fowler has a pretty good
> article about the differences in styles: http://martinfowler.com/
> articles/mocksArentStubs.html
>
> Francis Hwang
> http://fhwang.net/
>
>
>
> On Dec 16, 2007, at 5:59 PM, Dan North wrote:
>
>> Pat.
>>
>> I'm going to reply by promising to reply. You've asked a ton of
>> really useful and insightful questions. I can't do them justice
>> without sitting down and spending a bunch of time thinking about  
>> them.
>>
>> I'm going to be off the radar for a bit over Christmas - I've had
>> an insane year and I've promised myself (and my wife) some quiet
>> time. Your questions have a little star next to them in my gmail
>> inbox, which means at the very least they'll be ignored less than
>> the other mail I have to respond to :)
>>
>> The one sentence response, though, is that I honestly don't know
>> (which is why I need to think about it). I can tell you I think I
>> isolate services from their dependencies using mocks, I think I
>> never stub domain objects (I definitely never mock them, but
>> stubbing them is different), I can't say how I test layers because
>> I think we have a different definition of layers.
>>
>> The reason I'm being being so vague is that I usually specify
>> behaviour from the outside in, starting with the "outermost"
>> objects (the ones that appear in the scenario steps) and working
>> inwards as I implement each bit of behaviour. That way I discover
>> service dependencies that I introduce as mocks, and other domain
>> objects that become, well, domain objects. Then there are other
>> little classes that fall out of the mix that seem to make sense as
>> I go along. I don't usually start out with much more of a strategy
>> than that. I can't speak as a tester because I'm not one, so I
>> can't really give you a sensible answer for how isolated my tests
>> are. I simply don't have tests at that level. At an acceptance
>> level my scenarios only ever use real objects wired together doing
>> full end-to-end testing. Sometimes I'll swap in a lighter-weight
>> implementation (say using an in-memory database rather than a
>> remote one, or an in-thread Java web container like Jetty rather
>> than firing up Tomcat), but all the wiring is still the same (say
>> JDBC or HTTP-over-the-wire). I'm still not entirely sure how this
>> maps to Rails, but in Java MVC web apps I would want the controller
>> examples failing if the model's behaviour changed in a particular
>> way, so I can't think of a reason why I would want fake domain
>> objects.
>>
>> Like I said, I'll have a proper think and get back to you.
>>
>> Cheers,
>> Dan
>>
>> On Dec 15, 2007 7:17 AM, Pat Maddox < pergesu at gmail.com> wrote:
>> On Dec 8, 2007 4:06 AM, Dan North < tastapod at gmail.com> wrote:
>>> I prefer the mantra "mock roles, not objects", in other words,
>> mock things
>>> that have behaviour (services, components, resources, whatever your
>>> preferred term is) rather than stubbing out domain objects
>> themselves. If
>>> you have to mock domain objects it's usually a smell that your
>> domain
>>> implementation is too tightly coupled to some infrastructure.
>>
>> Assuming you could easily write Rails specs using the real domain
>> objects, but not hit the database, would you "never" mock domain
>> objects (where "never" means you deviate only in extraordinary
>> circumstances)?  I'm mostly curious in the interaction between
>> controller and model...if you use real models, then changes to the
>> model code could very well lead to failing controller specs, even
>> though the controller's logic is still correct.
>>
>> What is your opinion on isolating tests?  Do you try to test each
>> class in complete isolation, mocking its collaborators?  When you use
>> interaction-based tests, do you always leave those in place, or do  
>> you
>> substitute real objects as you implement them (and if so, do your
>> changes move to a more state-based style)?  How do you approach
>> specifying different layers within an app?  Do you use real
>> implementations if there are lightweight ones available, or do you
>> mock everything out?
>>
>> I realize that's a ton of questions...I'd be very grateful (and
>> impressed!) if you took the time to answer any of them.  Also I'd  
>> love
>> to hear input from other people as well.
>>
>> Pat
>>
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users



More information about the rspec-users mailing list