[rspec-users] Smart mocks for ActiveRecord to speed up tests
Rodrigo Rosenfeld Rosas
lbocseg at yahoo.com.br
Mon Apr 16 17:17:30 UTC 2012
Em 16-04-2012 09:27, Rodrigo Rosenfeld Rosas escreveu:
> Thank you David and Matt for your comments.
> Indeed I had considered using FakeWeb but I thought that maybe there
> could be an easier way just to mock 'open' directly since I guess it
> would be faster to run.
> With regards to the work around to overcome the issue with mocking AR
> so that I could speed up my tests I think that creating helpers in the
> model that will restrict the use of the AR API sounds like a good
> balance, but that wouldn't make my specific tests faster.
> Here is why: my Model class shouldn't be created or updated by any
> other code in my application. So I'd just have to add a new method to
> the Model class that would be specific to this action and I'd need to
> write a "unit" test for this new method. I quoted "unit" because it is
> actually an integration test as it will touch the database or
> otherwise I wouldn't be able to upgrade Rails to a new version if I'm
> mocking some methods from AR and they change in a future version.
> So I'd need to touch the database anyway and my 9 specs would still
> take almost a full second to run while this specific controller spec
> takes about 0.7s (up to 2s depending on my system load).
> Well, actually, I guess RSpec has changed the way it measure spec runs
> since I last used it. I'm saying that because it will report me from
> 0.7 to 2s when I run "rspec path/to/my_controller_spec.rb", but if I
> run "rspec --profile" it will run the full suite about the same time
> and this specific controller action will take 0.38s.
> Anyway, I don't see how I would create hundreds of specs and expect
> them to run in a few seconds and still have a reliable test suite. I
> mean, I want to feel safe if I'm upgrading to a newer Rails version
> even if it changed its AR API.
> The only real improvements I can see for my tests running fasts
> without changing anything in the specs is to use something like Hydra
> so that I could use the full power of my 6-core processor.
> Does RSpec has a built-in threaded option? I guess not because I don't
> think it would be much effective on CRuby because it would probably
> lock on database access (or maybe not, I don't know how the pg driver
> is implemented). For my particular application I suspect it is not
> cpu-heavy, but io-expensive instead. So maybe a multi-thread spec
> runner could improve its performance when running multiple specs.
After some more investigation I could find out why it is taking so much
time for the spec to run. Even for a simple model, calling a single
"Model.create! name: 'Model name'" will take almost 100ms. So, here is
usually what happens:
root = FactoryGirl.create :model # 2 creates:
# section = Section.create! name: 'Section'
# root = Model.create! section: section, label: 'Label'
post 'action', parent_id: root.id # another Model creation in the
controller alongside with 2 other creations of a has_many association
So, without counting any assertions just for creating all records so far
about 400ms were already spent. There will also be about 2 more updates
and 2 "delete" statements in this spec and that will complete about
540ms of total spec run time.
Another interesting fact is that for measuring this I've mocked the user
and controller authentication related methods, but if I actually create
the user the time for completing the spec will be about the same, even
if in the logs it is reported to spend about 70ms for executing the
insert in the user's table.
But I don't think it should take so much time for creating/updating
those records. See the output of the logging, for example:
Completed 200 OK in 83ms (Views: 0.6ms | ActiveRecord: 14.9ms)
This is consistent with the individual timing for each query/insert
statement including validations. Even so, the first 'post' method call
in the spec takes about 100ms.
So, my question is, if AR and Views are taking about 16ms what else
could be taking about 70ms?
This is also hard to profile since PostgreSQL will need more time for
the first queries and will be much faster for subsequent ones... This
means that running all specs will take much less time than if I run each
spec separately and compare to the total time.
But I don't really think I'll be able to test this action faster. I need
to check for all created records including associations and fields set
by the before_validation and removed by after_save so that I can get
confidence in the code. And since all of this should only happen through
this action, it wouldn't matter if I moved lots of those checks to the
"unit" test of my Model as the total time would remain the same. It
would only make my specs harder to read with lots of mocks and stubbed
methods in my controller spec.
I mean, that is ok, I don't aim in having single assertions per unit
test. I just need to make sure things won't break in a future
refactoring or requirement changes. And of course I'd like my tests to
run faster, but my current understanding is that I can't get this test
to run faster because it is testing a lot of code. Of course that an
in-memory AR adapter could help me speeding up my tests but using mocks
here won't give me any confidence on my code base.
Usually slow tests are not a problem when I'm working in some controller
as I don't need to run the full suite. But when I change some model or
library it would be great to run the full suite and I'm a bit worried if
that will become super slow some time in the future.
But I think I'll start to worry about this when this future becomes
present because there is no easy replacement that will allow my specs to
run faster. I'll still be able to try Hydra before having to resort to
mocks if things get really unsustainable.
But thank you very much for your input on the subject. I was just
worried if I was doing something fundamentally wrong.
More information about the rspec-users