[rspec-users] Smart mocks for ActiveRecord to speed up tests

Rodrigo Rosenfeld Rosas lbocseg at yahoo.com.br
Mon Apr 16 17:17:30 UTC 2012


Em 16-04-2012 09:27, Rodrigo Rosenfeld Rosas escreveu:
> Thank you David and Matt for your comments.
>
> Indeed I had considered using FakeWeb but I thought that maybe there 
> could be an easier way just to mock 'open' directly since I guess it 
> would be faster to run.
>
> With regards to the work around to overcome the issue with mocking AR 
> so that I could speed up my tests I think that creating helpers in the 
> model that will restrict the use of the AR API sounds like a good 
> balance, but that wouldn't make my specific tests faster.
>
> Here is why: my Model class shouldn't be created or updated by any 
> other code in my application. So I'd just have to add a new method to 
> the Model class that would be specific to this action and I'd need to 
> write a "unit" test for this new method. I quoted "unit" because it is 
> actually an integration test as it will touch the database or 
> otherwise I wouldn't be able to upgrade Rails to a new version if I'm 
> mocking some methods from AR and they change in a future version.
>
> So I'd need to touch the database anyway and my 9 specs would still 
> take almost a full second to run while this specific controller spec 
> takes about 0.7s (up to 2s depending on my system load).
>
> Well, actually, I guess RSpec has changed the way it measure spec runs 
> since I last used it. I'm saying that because it will report me from 
> 0.7 to 2s when I run "rspec path/to/my_controller_spec.rb", but if I 
> run "rspec --profile" it will run the full suite about the same time 
> and this specific controller action will take 0.38s.
>
> Anyway, I don't see how I would create hundreds of specs and expect 
> them to run in a few seconds and still have a reliable test suite. I 
> mean, I want to feel safe if I'm upgrading to a newer Rails version 
> even if it changed its AR API.
>
> The only real improvements I can see for my tests running fasts 
> without changing anything in the specs is to use something like Hydra 
> so that I could use the full power of my 6-core processor.
>
> Does RSpec has a built-in threaded option? I guess not because I don't 
> think it would be much effective on CRuby because it would probably 
> lock on database access (or maybe not, I don't know how the pg driver 
> is implemented). For my particular application I suspect it is not 
> cpu-heavy, but io-expensive instead. So maybe a multi-thread spec 
> runner could improve its performance when running multiple specs.
>
> Best,
> Rodrigo.
>

After some more investigation I could find out why it is taking so much 
time for the spec to run. Even for a simple model, calling a single 
"Model.create! name: 'Model name'" will take almost 100ms. So, here is 
usually what happens:

root = FactoryGirl.create :model # 2 creates:
# section = Section.create! name: 'Section'
# root = Model.create! section: section, label: 'Label'
post 'action', parent_id: root.id # another Model creation in the 
controller alongside with 2 other creations of a has_many association

So, without counting any assertions just for creating all records so far 
about 400ms were already spent. There will also be about 2 more updates 
and 2 "delete" statements in this spec and that will complete about 
540ms of total spec run time.

Another interesting fact is that for measuring this I've mocked the user 
and controller authentication related methods, but if I actually create 
the user the time for completing the spec will be about the same, even 
if in the logs it is reported to spend about 70ms for executing the 
insert in the user's table.

But I don't think it should take so much time for creating/updating 
those records. See the output of the logging, for example:

Completed 200 OK in 83ms (Views: 0.6ms | ActiveRecord: 14.9ms)

This is consistent with the individual timing for each query/insert 
statement including validations. Even so, the first 'post' method call 
in the spec takes about 100ms.

So, my question is, if AR and Views are taking about 16ms what else 
could be taking about 70ms?

This is also hard to profile since PostgreSQL will need more time for 
the first queries and will be much faster for subsequent ones... This 
means that running all specs will take much less time than if I run each 
spec separately and compare to the total time.

But I don't really think I'll be able to test this action faster. I need 
to check for all created records including associations and fields set 
by the before_validation and removed by after_save so that I can get 
confidence in the code. And since all of this should only happen through 
this action, it wouldn't matter if I moved lots of those checks to the 
"unit" test of my Model as the total time would remain the same. It 
would only make my specs harder to read with lots of mocks and stubbed 
methods in my controller spec.

I mean, that is ok, I don't aim in having single assertions per unit 
test. I just need to make sure things won't break in a future 
refactoring or requirement changes. And of course I'd like my tests to 
run faster, but my current understanding is that I can't get this test 
to run faster because it is testing a lot of code. Of course that an 
in-memory AR adapter could help me speeding up my tests but using mocks 
here won't give me any confidence on my code base.

Usually slow tests are not a problem when I'm working in some controller 
as I don't need to run the full suite. But when I change some model or 
library it would be great to run the full suite and I'm a bit worried if 
that will become super slow some time in the future.

But I think I'll start to worry about this when this future becomes 
present because there is no easy replacement that will allow my specs to 
run faster. I'll still be able to try Hydra before having to resort to 
mocks if things get really unsustainable.

But thank you very much for your input on the subject. I was just 
worried if I was doing something fundamentally wrong.

Cheers,
Rodrigo.


More information about the rspec-users mailing list