[rspec-users] How do I best setup data for Story Runner?

Pat Maddox pergesu at gmail.com
Fri Sep 28 04:35:22 EDT 2007


On 9/28/07, Scott Taylor <mailing_lists at railsnewbie.com> wrote:
>
> On Sep 27, 2007, at 6:11 PM, Carl Porth wrote:
>
> > On Sep 27, 2007, at 12:41 PM, Pat Maddox wrote:
> >
> >> On 9/26/07, Simon Peter Nicholls <simon at mintsource.org> wrote:
> >>> Just started looking at the Story Runner integration, and am
> >>> converting a few Rails integration tests to get a feel for it.
> >>>
> >>> My integration tests relied on fixtures, and since my models have a
> >>> significant amount of validation (and hence need valid data unless I
> >>> save without validation), this has made setup easy.
> >>>
> >>> With stories however, I'm wondering what the recommended approach
> >>> is.
> >>> Mocks are out I assume, since the point is testing the stack, but
> >>> are
> >>> fixtures out too? Should we be meeting our more complex or
> >>> repetitive
> >>> setup needs by treating it as plain ruby code that needs refactoring
> >>> via regular Ruby/OO techniques (Object Mother et al)?
> >>>
> >>> A typical scenario for me might involve a couple of users with
> >>> different roles, plus a collection of other collaborating objects.
> >>>
> >>> Thanks
> >>> _______________________________________________
> >>> rspec-users mailing list
> >>> rspec-users at rubyforge.org
> >>> http://rubyforge.org/mailman/listinfo/rspec-users
> >>>
> >>>
> >>
> >> I don't use fixtures (the yaml type, anyway) in story runner stories.
> >> I construct objects in the given blocks.  I find that that does two
> >> things for me.  First, it clearly expresses what a given really
> >> means.
> >>  So when I say Given "an activated user," I don't have to go dig
> >> around some YAML files.  If I need to understand it I can just
> >> look at
> >> the code and see
> >> u = User.create! :login => "pat"
> >> u.activate
> >>
> >> Secondly, constructing your object graph instead of using fixtures
> >> means you're actually exercising your code.  With fixtures you just
> >> instantiate some objects and fill them with data, which may not
> >> necessarily be valid (they require maintenance).  Also if you're
> >> using
> >> stuff like before/after create hooks to create child objects, using
> >> fixtures bypasses that.
> >>
> >> I look at fixtures as a weird kind of mock.  You're not using the
> >> full
> >> implementation, so what's the point really?  I'd rather use a real
> >> object or a proper mock object.  In stories I don't use mocks at all,
> >> so obviously I'll go for the full implementation.
> >>
> >> Pat
> >> _______________________________________________
> >> rspec-users mailing list
> >> rspec-users at rubyforge.org
> >> http://rubyforge.org/mailman/listinfo/rspec-users
> >
> > +1
> >
> > I've adopted a factory pattern for organizing my tests inspired by:
> > http://www.dcmanges.com/blog/38
> >
>
> Yeah, that post was great, and inspired me to create a plugin, which
> has been essential to my workflow.  Here is a screencast (and a
> shameless plug) for it:
>
> http://railsnewbie.com/files/fixture_replacement_demo.mov
>
> I was/am quite tired (it's about 3 AM), so just forward through my
> stupidity.
>
> Regarding Pat's post: fixtures aren't a weird kind of mock, they are
> real data.

I had a feeling that comment would get some attention :)  I hoped I
had clarified it in my post, but apparently I didn't.

When I say it's a weird kind of mock, I mean that you're basically
stubbing data without going through the entire business logic.  For
example, if you have a User model such as

class User < ActiveRecord::Base
  validates_uniqueness_of :login
end

and a fixture file such as

user1:
  id: 1
  login: pat
user2:
  id: 2
  login: pat

Then the fixtures will still load!  Of course that's because there are
no unique constraints in the db...but what bothers me is that Rails
just loads the fixtures like they're existing records, instead of
something like
User.create attributes_from_fixture_file

So, what I meant was, user stories are intended to exercise the entire
application stack.  Fixtures don't do this, because even though they
preload data, they skip an important part of business logic.  In that
sense, they return data without executing business logic, which is why
I consider them a strange kind of stubbed object.

> The problem is their large overhead in scaling to a
> project of any size (especially the typical one with lots of
> associations). Plus, fighting with YAML just sucks.  Am I putting in
> invalid data, or are those tabs instead of spaces?  The truth is that
> they are *PAINFUL* to use, and that's why no one uses them (and why
> "testing" sucks, and is done after a project instead of before).
>
>   Personally, I'd love to use mocks in my model specs, but I just
> can't bring myself to do it.  It, too, is painful, because I
> essentially need to understand the details of how activerecord
> works.  Why spend an hour figuring out that
> User.find_or_create_by_username 'scott' should be stubbed out as
> User.find("username" => "scott") and not User.find(:username =>
> "scott")?  Have you read Jay Field's blog recently?  Stubbing
> ActiveRecord::Connection::Column ?  Give me a break.  That's going to
> break the second you upgrade rails.

You're right.  I'd say this is mostly a consequence of the Active
Record pattern.  Basically your AR objects have two primary
responsibilities - business logic and persistence - and any time your
objects have more than one responsibility it becomes difficult to unit
test.  One example that I see crop up time and again is automatically
creating associations using callbacks.  Eventually you get to a point
where you save an object in a spec, and it blows up because there's a
null association.  We're saving an object that would normally be
created by the parent, because we need to exercise some behavior that
occurs when the record is saved.

That wouldn't be an issue if we had separate objects handling the
persistence and domain logic.  When you spec your PROs (plain ruby
objects.  I thought POROs sucked and I'm really digging PROs) you
might initially use mocks and then swap in real implementations.  You
could do this because (a) your specs will still be super quick because
it's all in memory and (b) you won't have stuff blow up because you're
trying to save stuff to the db when you didn't really want to.

As far as I can tell, there are a couple solutions to this problem:

1. Change the production code to handle errors that only crop up in
testing.  This sucks for what should be obvious reasons.  But it's
actually pretty pragmatic when your logic and assocations still aren't
very complex.  Just be sure to write a comment that says "this sucks
but what're ya gonna do"

2. Refactor your code to partially separate domain logic from
persistence logic.  For example, if you have to create associated
objects like we discussed above, move all of that into its own method.
 If I had code like this:

class Company < ActiveRecord::Base
  has_many :sites
  after_create :create_sites

  def create_sites
    %w(Production Development Testing).each { |n| sites <<
Site.new(:name => n) }
  end
end

I would change it to

class Company < ActiveRecord::Base
  has_many :sites

  def self.create_company(options = {})
    c = create options
    %w(Production Development Testing).each { |n| c.sites <<
Site.new(:name => n) }
    c
  end
end

This way you can call Company.create in your model specs and not have
it build all the associations.  For such a simple example it doesn't
really matter much, but once you start getting more complex your specs
will get slow and brittle.

3. Refactor your code to completely separate the domain logic from
persistence logic.  I've never had to do this in the 2.5 years I've
been coding Rails on a daily basis.  Until very, very recently, that
is.  I definitely believe in designing code to be testable, but the
first two options have made it so that I can stick with AR with a
little bit of smelliness.  Now that we're using the same business
logic on top of two different persistence mechanisms, it's time to
make decouple the business logic and make the persistence logic
configurable.

Story runner gives us one more option now.  You can write a story that
uses the real objects, and then you can mock out associations in your
model specs.  You get the benefits of knowing that your objects
integrate well and also enjoy speedy, decoupled unit tests.
Personally, despite how bright I've made it sound, I'm not a fan of
this option.  I want my unit tests to exercise the interactions
between objects, so I just deal with it.  The exception of course is
when I'm using objects from a lower layer, such as controllers
interacting with the model, in which case I always use mocks.  Code
from one layer should depend only on the interface of objects in the
lower layer.  When you're dealing with objects in the same layer, it's
useful to use real implementations because it aids refactoring a great
deal.

> Mocks are great for external libraries (or objects which you
> control), but with rails everything is so tightly woven together that
> mocking/stubbing seems to be more pain then it's worth.  The only
> positive thing the stub approach has going for it is the speed factor.

So, you're right that mocking can be difficult when it comes to AR
associations.  As I pointed out, it's a consequence of coupling
business and persistence logic.  There are some things that you can do
to offset that a bit.  AR makes it more difficult to write clean
specs, but the tradeoff is that the conceptual weight of AR is
extremely low.

> I'd be very interested in hearing how you are using mocks
> successfully without the overhead of learning tons of plumbing.

If you could be a bit more specific about what you're interested in, I
could perhaps shed some light.  I heavily rely on mocks in a way that
I would consider successful.

Wow that became a lot longer than I expected.  I hope there's some
value in it instead of it being a steaming pile.

Pat


More information about the rspec-users mailing list