[rspec-users] NullDb makes rake spec take (much) longer

Ben Mabey ben at benmabey.com
Mon Jan 12 16:07:19 EST 2009


On 1/12/09 1:05 PM, Matt Wynne wrote:
>
> On 12 Jan 2009, at 17:36, Ben Mabey wrote:
>
>> On 1/12/09 7:09 AM, Matt Wynne wrote:
>>> Hi all,
>>>
>>> We did a spike last week to de-couple our view and controller tests 
>>> from the database, using NullDb. It didn't go too well. I realise 
>>> that this plugin isn't part of RSpec, but I thought others on this 
>>> list might have experiences to share.
>>>
>>> Here's a summary of my colleague's investigations:
>>>
>>>> I installed the plugin from 
>>>> http://github.com/jakehow/nulldb/tree/master
>>>>
>>>> Changed spec/spec_helper to set
>>>> "ActiveRecord::Base.establish_connection(:adapter => :nulldb)" by
>>>> default.
>>>>
>>>> The specs that complained of the lack of db, I included a before(all)
>>>> that changed the connection to the test database (include
>>>> NeedsDatabase - copied from Ben Mabey's Functional module)
>>>> By default, I also included the db for all models (config.include
>>>> NeedsDatabase, :type => :model)
>>>>
>>>> Running spec spec/views with or without nulldb takes about the same
>>>> time (a couple of seconds less with nulldb). However, running "rake
>>>> spec" with nulldb takes 10 times longer!!
>>>>
>>>> Another weird thing was that three tests failed in the controllers
>>>> (venues and concerts) in a very strange way, even when I included the
>>>> NeedsDatabase in the tests that needed it. The weird bit is, when I
>>>> run: "spec spec/controllers/venues_controller_spec.rb" it doesn't
>>>> fail. But when I run: "spec spec/controllers/users_controller_spec.rb
>>>> spec/controllers/venues_controller_spec.rb" It does fail... That is,
>>>> the user_controller test is influencing the results of the
>>>> venue_controller test!
>>>> The same weird behaviour happens in lib/sk/find
>>>>
>>>> This made me had to include NeedsDatabase for all lib and controllers
>>>> tests as well.
>>>
>>> The barrier for us was the shockingly poor performance of 'rake 
>>> spec' on the view specs - it really means we just can't use it, and 
>>> actually only barely improved the performance of the specs at all.
>>>
>>> I was disappointed that the view specs didn't get any faster. My 
>>> guess is that stub_model is the problem - as it has to do quite a 
>>> bit of work to set up the attributes on the models.
>>>
>>> So, can anyone tell us what we might have been doing wrong? Or did I 
>>> just have unrealistic expectations of how this might help?
>>>
>>> Matt Wynne
>>> http://blog.mattwynne.net
>>> http://www.songkick.com
>>>
>>> _______________________________________________
>>> rspec-users mailing list
>>> rspec-users at rubyforge.org
>>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
>> Hey Matt,
>> I had similar experiences when I started using NullDB.  I think that 
>> the reason why the overall spec suite runs slow is AR is switching 
>> from one adapter to another, and in the case of the mysql one it 
>> probably is relatively expensive to set up a new connection.  
>> Switching over to using NullDB on an existing project of any sizable 
>> size is going to be large task IMO.  On my last project we started 
>> using NullDB by default and the result has been very worthwhile and 
>> noticeable.  In this project DB connectivity was the exception, not 
>> the norm (even on our model specs) and so that is why I created a 
>> module that would turn on the DB temporarily.  In your case the 
>> opposite approach may be needed.
>>
>> Going back to the problem at hand... Based on the email you are using 
>> a module like this:
>>  unless Object.const_defined?(:NeedsDatabase)
>>    share_as :NeedsDatabase do
>>
>>      before :all do
>>        ActiveRecord::Base.establish_connection(:test)
>>      end
>>
>>      after :all do
>>        ActiveRecord::Base.establish_connection(:adapter => :nulldb)
>>      end
>>    end
>>  end
>>
>> Since you are including this on all of your model specs (and more) 
>> you are incurring the setup and teardown cost of the DB connection 
>> repeatedly.  Like I said, you may want to adopt the opposite policy 
>> of turning it off temporarily... You could also make the the changing 
>> smarter by only changing it when you need to.. something like:
>>
>> share_as :NeedsDatabase do
>>    before :all do
>>        ActiveRecord::Base.establish_connection(:test) if 
>> ActiveRecord::Base.connection.class.to_s =~ /NullDB/
>>    end
>> end
>> share_as :DontNeedDatabase do
>>    before :all do
>>        ActiveRecord::Base.establish_connection(:adapter => :nulldb) 
>> unless ActiveRecord::Base.connection.class.to_s =~ /NullDB/
>>    end
>> end
>>
>> With this approach you would need to specify whether or not to use 
>> the DB for each example group... just a thought.  Of course, an even 
>> better solution would be to somehow modify AR to maintain an active 
>> connection to the real DB even when it isn't the active adapter.
>>
>> re: the view specs
>> I wouldn't expect to see too much gain here.   The big win for nullDB 
>> is in models IMO.  Of course, having it on the view and controller 
>> specs helps prevent accidental DB calls.
>>
>> re: the random controller spec failures
>> I only guess is that you are somehow relying on the DB in your 
>> controller specs.  I did this too in my controllers when I called 
>> class methods on models that would get a list from the DB... I can't 
>> really help without seeing any specs though.
>>
>> I hope this helps and you can find a suitable policy on how to manage 
>> the connections/adapters.  Just ask if you have any other questions.
>>
>> -Ben
>
> Thanks Ben. I had a brief look at the AR code and it did appear to be 
> caching the adapters so I'd assumed the changeover wasn't expensive, 
> but it did feel like something like that, so that's probably it.

Oh really?  Hmm.. well, I never did any real testing to verify my 
assumption about the the changeover.  I just noticed that suites that 
had to switch adapters a lot ran very slowly.
>
> Ho hum. No quick wins for speeding up our specs then!
Unfortunately not. :(

>
>
> Matt Wynne
> http://blog.mattwynne.net
> http://www.songkick.com
>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users



More information about the rspec-users mailing list