[rspec-users] Spec/Test Speed
chad at spicycode.com
Sun Oct 7 11:34:59 EDT 2007
The Stories (ie Story Runner full integration tests that hit the db)
are fairly slow by comparison, I agree. We have those run on our CI
server, and only locally when we modify them. That's our approach
for handling integration testing.
Re: The mac mini's, yeah they were pairing stations (one per pair of
users, no distributed spec running yet). Our main specs are built
around heavier use of mocking/stubbing.
On Oct 7, 2007, at 2:14 AM, Scott Taylor wrote:
> On Oct 7, 2007, at 1:47 AM, Pat Maddox wrote:
>> On 10/6/07, Scott Taylor <mailing_lists at railsnewbie.com> wrote:
>>> On Oct 7, 2007, at 12:31 AM, Chad Humphries wrote:
>>>> I don't really have a lot to contribute on how to make it faster,
>>>> other than to outline what we've been doing on our projects.
>>>> On one of our current projects we have the following 2570 examples
>>>> that run in ~70 seconds on our pairing stations (mac minis, 1.83
>>> I assume your "pairing stations" are two separate mac-minis, in
>>> you practice pair programming? Or is this a cluster of two mac-
>>> But this sounds great - 70 seconds for 2500 specs. How many of
>>> are model specs (that hit the database)?
>>>> c2d). In general across our various machines is at or a little
>>>> than a minute for specs for controllers, models, helpers, lib,
>>>> and plugins. Our Story suite takes longer, but it's still under
>>>> development so I don't really count it at this point. We have
>>>> 1.8.6 installed from MacPorts on all machines, as well as MySQL 5.
>>>> (current as of a month ago) from macports.
>>>> We make good use of mocking and stubbing through our controller
>>>> tests, and little use of fixtures. We primarily use the
>>>> spec_attribute_helper (or factory method) as Luke Redpath and Dan
>>>> Manges have outlined in their respective blog articles. I've
>>>> looking at deep test, or possible spec_distributed as a way to
>>>> things up more. Our main issue is our precommit task (rake cruise
>>> The factory method (or attribute_helper) still hits the database. I
>>> don't see it as any sort of performance gain. In fact, I've even
>>> developed a plugin around the Factory idea, and it was only when I
>>> started using it in all of my tests that the speed really started to
>>> affect me (I was using mocking/stubbing, with much frustration prior
>>> to that point). But to me it's pretty clear the plugin (or the
>>> factory) is not the problem - the hit is the database. DHH saw this
>>> hit, and since they were using fixtures,he found that creating the
>>> fixtures, and then wrapping each test in a transaction was a huge
>>> performance gain. I wonder if the same would be true with setups/
>>> The obvious thing to do to solve the performance problem is to
>>> the hit to the database. The question is: At what level of
>>> abstraction should this be done? The one camp (which would include
>>> fellows like Jay Fields), would mock/stub everything they don't
>>> write. For me, I see testing as more than testing - it's the
>>> documentation to my code which never lies to me (this documentation
>>> is so good, that I can give it to my boss, who is not a programmer).
>> I think you'd be far better served by user stories in this case. The
>> user story is the tool we use to communicate with non-geeks, be it
>> customers or our boss. Low-level specs fall squarely in the
>> realm, as far as I'm concerned. They're easy to read because we
>> like to torture ourselves. I think that they're fantastic for
>> communicating intent with other developers, but that's where it ends.
>> So, when you move documentation as a goal from specs over to stories,
>> that frees specs to take advantage of techniques like mocks to help
>> with your design. Documentation is a pleasant side effect of BDD,
>> a goal.
> Documentation is certainly a goal of BDD (Just not necessarily for
> the business user - as you pointed out). I don't write my specs to
> be understandable by the business user, although it just often
> happens that they are (at least in this rails context, since even
> most of the fields of the database are more-or-less user facing).
> The point that I was trying to hammer home is that mocking/stubbing
> adds a lot of extra noise to a rails test, where it doesn't in the
> other projects that I've worked on.
> But whether or not you call the specs "stories" or "integration
> specs" (it is, after all, all a continuum of what is considered an
> "integration" test), I still have no idea how to run them any faster.
> rspec-users mailing list
> rspec-users at rubyforge.org
More information about the rspec-users