[rspec-users] Spec/Test Speed

Chad Humphries chad at spicycode.com
Sun Oct 7 11:34:59 EDT 2007

The Stories (ie Story Runner full integration tests that hit the db)  
are fairly slow by comparison, I agree.  We have those run on our CI  
server, and only locally when we modify them.   That's our approach  
for handling integration testing.

Re: The mac mini's, yeah they were pairing stations (one per pair of  
users, no distributed spec running yet).  Our main specs are built  
around heavier use of mocking/stubbing.

- Chad

On Oct 7, 2007, at 2:14 AM, Scott Taylor wrote:

> On Oct 7, 2007, at 1:47 AM, Pat Maddox wrote:
>> On 10/6/07, Scott Taylor <mailing_lists at railsnewbie.com> wrote:
>>> On Oct 7, 2007, at 12:31 AM, Chad Humphries wrote:
>>>> Scott,
>>>> I don't really have a lot to contribute on how to make it faster,
>>>> other than to outline what we've been doing on our projects.
>>>> On one of our current projects we have the following 2570 examples
>>>> that run in ~70 seconds on our pairing stations (mac minis, 1.83
>>> I assume your "pairing stations" are two separate mac-minis, in  
>>> which
>>> you practice pair programming?  Or is this a cluster of two mac-
>>> minis?
>>> But this sounds great - 70 seconds for 2500 specs.  How many of  
>>> those
>>> are model specs (that hit the database)?
>>>> c2d).  In general across our various machines is at or a little  
>>>> more
>>>> than a minute for specs for controllers, models, helpers, lib,
>>>> views,
>>>> and plugins.   Our Story suite takes longer, but it's still under
>>>> development so I don't really count it at this point.   We have  
>>>> Ruby
>>>> 1.8.6 installed from MacPorts on all machines, as well as MySQL 5.
>>>> (current as of a month ago) from macports.
>>>> We make good use of mocking and stubbing through our controller
>>>> tests, and little use of fixtures.  We primarily use the
>>>> spec_attribute_helper (or factory method) as Luke Redpath and Dan
>>>> Manges have outlined in their respective blog articles.     I've
>>>> been
>>>> looking at deep test, or possible spec_distributed as a way to  
>>>> speed
>>>> things up more.   Our main issue is our precommit task (rake cruise
>>> The factory method (or attribute_helper) still hits the database.  I
>>> don't see it as any sort of performance gain. In fact, I've even
>>> developed a plugin around the Factory idea, and it was only when I
>>> started using it in all of my tests that the speed really started to
>>> affect me (I was using mocking/stubbing, with much frustration prior
>>> to that point).  But to me it's pretty clear the plugin (or the
>>> factory) is not the problem - the hit is the database.  DHH saw this
>>> hit, and since they were using fixtures,he found that creating the
>>> fixtures, and then wrapping each test in a transaction was a huge
>>> performance gain.  I wonder if the same would be true with setups/
>>> before(:each)...
>>> The obvious thing to do to solve the performance problem is to  
>>> remove
>>> the hit to the database.  The question is: At what level of
>>> abstraction should this be done?  The one camp (which would include
>>> fellows like Jay Fields), would mock/stub everything they don't
>>> write.  For me, I see testing as more than testing - it's the
>>> documentation to my code which never lies to me (this documentation
>>> is so good, that I can give it to my boss, who is not a programmer).
>> I think you'd be far better served by user stories in this case.  The
>> user story is the tool we use to communicate with non-geeks, be it
>> customers or our boss.  Low-level specs fall squarely in the  
>> developer
>> realm, as far as I'm concerned.  They're easy to read because we  
>> don't
>> like to torture ourselves.  I think that they're fantastic for
>> communicating intent with other developers, but that's where it ends.
>> So, when you move documentation as a goal from specs over to stories,
>> that frees specs to take advantage of techniques like mocks to help
>> with your design.  Documentation is a pleasant side effect of BDD,  
>> not
>> a goal.
> Documentation is certainly a goal of BDD (Just not necessarily for
> the business user - as you pointed out).  I don't write my specs to
> be understandable by the business user, although it just often
> happens that they are (at least in this rails context, since even
> most of the fields of the database are more-or-less user facing).
> The point that I was trying to hammer home is that mocking/stubbing
> adds a lot of extra noise to a rails test, where it doesn't in the
> other projects that I've worked on.
> But whether or not you call the specs "stories" or "integration
> specs" (it is, after all, all a continuum of what is considered an
> "integration" test), I still have no idea how to run them any faster.
> Scott
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users

More information about the rspec-users mailing list