[rspec-users] Smart mocks for ActiveRecord to speed up tests
Rodrigo Rosenfeld Rosas
lbocseg at yahoo.com.br
Thu Apr 19 18:42:40 UTC 2012
Just in case someone is curious, I've replaced ActiveRecord models with
Sequel ones and my slowest example as reported by "rspec -d" got faster
from about 0.5s to 0.2s.
That is awesome! :)
It seems AR is bloated...
As if this wasn't already amazing enough, my models seem much cleaner
with Sequel than with AR.
Em 16-04-2012 14:17, Rodrigo Rosenfeld Rosas escreveu:
> Em 16-04-2012 09:27, Rodrigo Rosenfeld Rosas escreveu:
>> Thank you David and Matt for your comments.
>> Indeed I had considered using FakeWeb but I thought that maybe there
>> could be an easier way just to mock 'open' directly since I guess it
>> would be faster to run.
>> With regards to the work around to overcome the issue with mocking AR
>> so that I could speed up my tests I think that creating helpers in
>> the model that will restrict the use of the AR API sounds like a good
>> balance, but that wouldn't make my specific tests faster.
>> Here is why: my Model class shouldn't be created or updated by any
>> other code in my application. So I'd just have to add a new method to
>> the Model class that would be specific to this action and I'd need to
>> write a "unit" test for this new method. I quoted "unit" because it
>> is actually an integration test as it will touch the database or
>> otherwise I wouldn't be able to upgrade Rails to a new version if I'm
>> mocking some methods from AR and they change in a future version.
>> So I'd need to touch the database anyway and my 9 specs would still
>> take almost a full second to run while this specific controller spec
>> takes about 0.7s (up to 2s depending on my system load).
>> Well, actually, I guess RSpec has changed the way it measure spec
>> runs since I last used it. I'm saying that because it will report me
>> from 0.7 to 2s when I run "rspec path/to/my_controller_spec.rb", but
>> if I run "rspec --profile" it will run the full suite about the same
>> time and this specific controller action will take 0.38s.
>> Anyway, I don't see how I would create hundreds of specs and expect
>> them to run in a few seconds and still have a reliable test suite. I
>> mean, I want to feel safe if I'm upgrading to a newer Rails version
>> even if it changed its AR API.
>> The only real improvements I can see for my tests running fasts
>> without changing anything in the specs is to use something like Hydra
>> so that I could use the full power of my 6-core processor.
>> Does RSpec has a built-in threaded option? I guess not because I
>> don't think it would be much effective on CRuby because it would
>> probably lock on database access (or maybe not, I don't know how the
>> pg driver is implemented). For my particular application I suspect it
>> is not cpu-heavy, but io-expensive instead. So maybe a multi-thread
>> spec runner could improve its performance when running multiple specs.
> After some more investigation I could find out why it is taking so
> much time for the spec to run. Even for a simple model, calling a
> single "Model.create! name: 'Model name'" will take almost 100ms. So,
> here is usually what happens:
> root = FactoryGirl.create :model # 2 creates:
> # section = Section.create! name: 'Section'
> # root = Model.create! section: section, label: 'Label'
> post 'action', parent_id: root.id # another Model creation in the
> controller alongside with 2 other creations of a has_many association
> So, without counting any assertions just for creating all records so
> far about 400ms were already spent. There will also be about 2 more
> updates and 2 "delete" statements in this spec and that will complete
> about 540ms of total spec run time.
> Another interesting fact is that for measuring this I've mocked the
> user and controller authentication related methods, but if I actually
> create the user the time for completing the spec will be about the
> same, even if in the logs it is reported to spend about 70ms for
> executing the insert in the user's table.
> But I don't think it should take so much time for creating/updating
> those records. See the output of the logging, for example:
> Completed 200 OK in 83ms (Views: 0.6ms | ActiveRecord: 14.9ms)
> This is consistent with the individual timing for each query/insert
> statement including validations. Even so, the first 'post' method call
> in the spec takes about 100ms.
> So, my question is, if AR and Views are taking about 16ms what else
> could be taking about 70ms?
> This is also hard to profile since PostgreSQL will need more time for
> the first queries and will be much faster for subsequent ones... This
> means that running all specs will take much less time than if I run
> each spec separately and compare to the total time.
> But I don't really think I'll be able to test this action faster. I
> need to check for all created records including associations and
> fields set by the before_validation and removed by after_save so that
> I can get confidence in the code. And since all of this should only
> happen through this action, it wouldn't matter if I moved lots of
> those checks to the "unit" test of my Model as the total time would
> remain the same. It would only make my specs harder to read with lots
> of mocks and stubbed methods in my controller spec.
> I mean, that is ok, I don't aim in having single assertions per unit
> test. I just need to make sure things won't break in a future
> refactoring or requirement changes. And of course I'd like my tests to
> run faster, but my current understanding is that I can't get this test
> to run faster because it is testing a lot of code. Of course that an
> in-memory AR adapter could help me speeding up my tests but using
> mocks here won't give me any confidence on my code base.
> Usually slow tests are not a problem when I'm working in some
> controller as I don't need to run the full suite. But when I change
> some model or library it would be great to run the full suite and I'm
> a bit worried if that will become super slow some time in the future.
> But I think I'll start to worry about this when this future becomes
> present because there is no easy replacement that will allow my specs
> to run faster. I'll still be able to try Hydra before having to resort
> to mocks if things get really unsustainable.
> But thank you very much for your input on the subject. I was just
> worried if I was doing something fundamentally wrong.
> rspec-users mailing list
> rspec-users at rubyforge.org
More information about the rspec-users