[rspec-users] Spec/Test Speed

Scott Taylor mailing_lists at railsnewbie.com
Sun Oct 7 02:14:30 EDT 2007

On Oct 7, 2007, at 1:47 AM, Pat Maddox wrote:

> On 10/6/07, Scott Taylor <mailing_lists at railsnewbie.com> wrote:
>> On Oct 7, 2007, at 12:31 AM, Chad Humphries wrote:
>>> Scott,
>>> I don't really have a lot to contribute on how to make it faster,
>>> other than to outline what we've been doing on our projects.
>>> On one of our current projects we have the following 2570 examples
>>> that run in ~70 seconds on our pairing stations (mac minis, 1.83
>> I assume your "pairing stations" are two separate mac-minis, in which
>> you practice pair programming?  Or is this a cluster of two mac- 
>> minis?
>> But this sounds great - 70 seconds for 2500 specs.  How many of those
>> are model specs (that hit the database)?
>>> c2d).  In general across our various machines is at or a little more
>>> than a minute for specs for controllers, models, helpers, lib,  
>>> views,
>>> and plugins.   Our Story suite takes longer, but it's still under
>>> development so I don't really count it at this point.   We have Ruby
>>> 1.8.6 installed from MacPorts on all machines, as well as MySQL 5.
>>> (current as of a month ago) from macports.
>>> We make good use of mocking and stubbing through our controller
>>> tests, and little use of fixtures.  We primarily use the
>>> spec_attribute_helper (or factory method) as Luke Redpath and Dan
>>> Manges have outlined in their respective blog articles.     I've  
>>> been
>>> looking at deep test, or possible spec_distributed as a way to speed
>>> things up more.   Our main issue is our precommit task (rake cruise
>> The factory method (or attribute_helper) still hits the database.  I
>> don't see it as any sort of performance gain. In fact, I've even
>> developed a plugin around the Factory idea, and it was only when I
>> started using it in all of my tests that the speed really started to
>> affect me (I was using mocking/stubbing, with much frustration prior
>> to that point).  But to me it's pretty clear the plugin (or the
>> factory) is not the problem - the hit is the database.  DHH saw this
>> hit, and since they were using fixtures,he found that creating the
>> fixtures, and then wrapping each test in a transaction was a huge
>> performance gain.  I wonder if the same would be true with setups/
>> before(:each)...
>> The obvious thing to do to solve the performance problem is to remove
>> the hit to the database.  The question is: At what level of
>> abstraction should this be done?  The one camp (which would include
>> fellows like Jay Fields), would mock/stub everything they don't
>> write.  For me, I see testing as more than testing - it's the
>> documentation to my code which never lies to me (this documentation
>> is so good, that I can give it to my boss, who is not a programmer).
> I think you'd be far better served by user stories in this case.  The
> user story is the tool we use to communicate with non-geeks, be it
> customers or our boss.  Low-level specs fall squarely in the developer
> realm, as far as I'm concerned.  They're easy to read because we don't
> like to torture ourselves.  I think that they're fantastic for
> communicating intent with other developers, but that's where it ends.
> So, when you move documentation as a goal from specs over to stories,
> that frees specs to take advantage of techniques like mocks to help
> with your design.  Documentation is a pleasant side effect of BDD, not
> a goal.

Documentation is certainly a goal of BDD (Just not necessarily for  
the business user - as you pointed out).  I don't write my specs to  
be understandable by the business user, although it just often  
happens that they are (at least in this rails context, since even  
most of the fields of the database are more-or-less user facing).   
The point that I was trying to hammer home is that mocking/stubbing  
adds a lot of extra noise to a rails test, where it doesn't in the  
other projects that I've worked on.

But whether or not you call the specs "stories" or "integration  
specs" (it is, after all, all a continuum of what is considered an  
"integration" test), I still have no idea how to run them any faster.


More information about the rspec-users mailing list