[rspec-users] Testing with activerecord at ThoughtWorks (Stubbing vs real DB)
ben at benmabey.com
Sun Jun 28 02:34:26 EDT 2009
> I am reading the post by Fowler, 'Ruby at ThoughtWorks'.
> He talks about testing with activerecord:
> "Right at the beginning of our use of Ruby, there was a debate on how
> best to organize testing in the presence of the Active Record database
> layer in Rails. The basic problem is that most of the time,
> performance of enterprise applications is dominated by database
> access. We've found that by using a Test Double we can greatly speed
> up our tests. Having fast tests is crucial to our test-intensive
> development process. Kent Beck recommends a basic commit build of
> under ten minutes. Most of our projects manage this these days, and
> using a database double is a vital part of achieving it.
> The problem with Active Record is that by combining database access
> code with business logic, it's rather harder to create a database
> double. The Mingle team's reaction to this was to accept that Rails
> binds the database tightly and thus run all the commit tests against a
> real database.
> The contrary view was advocated most firmly by the Atlanta and Jersey
> teams. Ruby has a powerful feature that allows you to redefine methods
> at run-time. You can use this to take an active record class, and
> redefine the the database access methods in that class as stubs. The
> team started the gem unitrecord to help with this.
> In the three years, we've not seen a generally accepted victor in this
> debate. The Mingle team run a couple of thousand tests against a real
> postgres database in around 8 minutes. (They parallelize the tests to
> make use of multiple cores.) The Atlanta and Jersey teams consider it
> valuable that their commit test runs in 2 minutes with stubs versus 8
> minutes without. The trade-off is the simplicity of the direct
> database tests versus the faster commit build of the stubbed tests.
> While both teams are broadly happy with their positions in this
> debate, the use of stubbing has led to another issue for the Atlanta/
> Jersey teams. As the teams became familiar with using method stubbing,
> they used it more and more - falling into the inevitable over-usage
> where unit tests would stub out every method other than the one being
> tested. The problem here, as often with using doubles, is brittle
> tests. As you change the behavior of the application, you also have to
> change lots of doubles that are mimicking the old behavior. This over-
> usage has led both teams to move away from stubbed unit tests and to
> use more rails-style functional tests with direct database access."
> What do you think of the two tactics?
I have done both on past projects. I did not use any of the TW gems to
do it. I used Avdi Grimm's nice NullDB plugin to do it. IMO it is a
much better and cleaner approach than the ones talked about in the
article. I gave a lightning talk about the tradeoffs of both approaches
and how I used NullDB in my project. Here are the slides for the talk:
The slides go through the benefits and tradeoffs of both approaches.
Let me know if you want me to elaborate on anything else. I should
point out that another, but less extreme, solution to stubbing out the
DB is to use an in-memory DB (like sqllite) to get a speed boost. With
this approach you may still need to switch to your real DB adapter for
DB specific SQL but it all the basic AR stuff with work just fine.
> Also, What does it mean and where can I see examples of "running all
> the commit tests against a real database" ?
Sounds like they have configured there project to to allow you to hot
swap which strategy to use. So, for development you can use the
stubbing approach and get faster feedback and then swap in the real
database on the CI server to make sure there aren't any surprises when
it is actually executed against the DB. This is a good approach and can
be done quite easily with NullDB.
> Can I find it in "the rspec book" ?
I don't think so. To be honest, I don't think it is done that much. I
think most teams just let there model tests hit the DB. (And thus are
not "real" unit tests by some people's definition....)
More information about the rspec-users