[rspec-users] Smart mocks for ActiveRecord to speed up tests

Rodrigo Rosenfeld Rosas lbocseg at yahoo.com.br
Thu Apr 19 18:42:40 UTC 2012


Just in case someone is curious, I've replaced ActiveRecord models with 
Sequel ones and my slowest example as reported by "rspec -d" got faster 
from about 0.5s to 0.2s.

That is awesome! :)

It seems AR is bloated...

As if this wasn't already amazing enough, my models seem much cleaner 
with Sequel than with AR.

Cheers,
Rodrigo.

Em 16-04-2012 14:17, Rodrigo Rosenfeld Rosas escreveu:
> Em 16-04-2012 09:27, Rodrigo Rosenfeld Rosas escreveu:
>> Thank you David and Matt for your comments.
>>
>> Indeed I had considered using FakeWeb but I thought that maybe there 
>> could be an easier way just to mock 'open' directly since I guess it 
>> would be faster to run.
>>
>> With regards to the work around to overcome the issue with mocking AR 
>> so that I could speed up my tests I think that creating helpers in 
>> the model that will restrict the use of the AR API sounds like a good 
>> balance, but that wouldn't make my specific tests faster.
>>
>> Here is why: my Model class shouldn't be created or updated by any 
>> other code in my application. So I'd just have to add a new method to 
>> the Model class that would be specific to this action and I'd need to 
>> write a "unit" test for this new method. I quoted "unit" because it 
>> is actually an integration test as it will touch the database or 
>> otherwise I wouldn't be able to upgrade Rails to a new version if I'm 
>> mocking some methods from AR and they change in a future version.
>>
>> So I'd need to touch the database anyway and my 9 specs would still 
>> take almost a full second to run while this specific controller spec 
>> takes about 0.7s (up to 2s depending on my system load).
>>
>> Well, actually, I guess RSpec has changed the way it measure spec 
>> runs since I last used it. I'm saying that because it will report me 
>> from 0.7 to 2s when I run "rspec path/to/my_controller_spec.rb", but 
>> if I run "rspec --profile" it will run the full suite about the same 
>> time and this specific controller action will take 0.38s.
>>
>> Anyway, I don't see how I would create hundreds of specs and expect 
>> them to run in a few seconds and still have a reliable test suite. I 
>> mean, I want to feel safe if I'm upgrading to a newer Rails version 
>> even if it changed its AR API.
>>
>> The only real improvements I can see for my tests running fasts 
>> without changing anything in the specs is to use something like Hydra 
>> so that I could use the full power of my 6-core processor.
>>
>> Does RSpec has a built-in threaded option? I guess not because I 
>> don't think it would be much effective on CRuby because it would 
>> probably lock on database access (or maybe not, I don't know how the 
>> pg driver is implemented). For my particular application I suspect it 
>> is not cpu-heavy, but io-expensive instead. So maybe a multi-thread 
>> spec runner could improve its performance when running multiple specs.
>>
>> Best,
>> Rodrigo.
>>
>
> After some more investigation I could find out why it is taking so 
> much time for the spec to run. Even for a simple model, calling a 
> single "Model.create! name: 'Model name'" will take almost 100ms. So, 
> here is usually what happens:
>
> root = FactoryGirl.create :model # 2 creates:
> # section = Section.create! name: 'Section'
> # root = Model.create! section: section, label: 'Label'
> post 'action', parent_id: root.id # another Model creation in the 
> controller alongside with 2 other creations of a has_many association
>
> So, without counting any assertions just for creating all records so 
> far about 400ms were already spent. There will also be about 2 more 
> updates and 2 "delete" statements in this spec and that will complete 
> about 540ms of total spec run time.
>
> Another interesting fact is that for measuring this I've mocked the 
> user and controller authentication related methods, but if I actually 
> create the user the time for completing the spec will be about the 
> same, even if in the logs it is reported to spend about 70ms for 
> executing the insert in the user's table.
>
> But I don't think it should take so much time for creating/updating 
> those records. See the output of the logging, for example:
>
> Completed 200 OK in 83ms (Views: 0.6ms | ActiveRecord: 14.9ms)
>
> This is consistent with the individual timing for each query/insert 
> statement including validations. Even so, the first 'post' method call 
> in the spec takes about 100ms.
>
> So, my question is, if AR and Views are taking about 16ms what else 
> could be taking about 70ms?
>
> This is also hard to profile since PostgreSQL will need more time for 
> the first queries and will be much faster for subsequent ones... This 
> means that running all specs will take much less time than if I run 
> each spec separately and compare to the total time.
>
> But I don't really think I'll be able to test this action faster. I 
> need to check for all created records including associations and 
> fields set by the before_validation and removed by after_save so that 
> I can get confidence in the code. And since all of this should only 
> happen through this action, it wouldn't matter if I moved lots of 
> those checks to the "unit" test of my Model as the total time would 
> remain the same. It would only make my specs harder to read with lots 
> of mocks and stubbed methods in my controller spec.
>
> I mean, that is ok, I don't aim in having single assertions per unit 
> test. I just need to make sure things won't break in a future 
> refactoring or requirement changes. And of course I'd like my tests to 
> run faster, but my current understanding is that I can't get this test 
> to run faster because it is testing a lot of code. Of course that an 
> in-memory AR adapter could help me speeding up my tests but using 
> mocks here won't give me any confidence on my code base.
>
> Usually slow tests are not a problem when I'm working in some 
> controller as I don't need to run the full suite. But when I change 
> some model or library it would be great to run the full suite and I'm 
> a bit worried if that will become super slow some time in the future.
>
> But I think I'll start to worry about this when this future becomes 
> present because there is no easy replacement that will allow my specs 
> to run faster. I'll still be able to try Hydra before having to resort 
> to mocks if things get really unsustainable.
>
> But thank you very much for your input on the subject. I was just 
> worried if I was doing something fundamentally wrong.
>
> Cheers,
> Rodrigo.
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>



More information about the rspec-users mailing list