[rspec-devel] Back to one repository?
dchelimsky at gmail.com
Thu May 1 09:50:37 EDT 2008
Wincent - thank you for taking the time to share your thoughts so
thoroughly. Comments in-line.
On May 1, 2008, at 5:06 AM, Wincent Colaiuta wrote:
> Disclaimer: I've submitted a few patches to RSpec but I can't claim
> to be a heavyweight participant in the development process. So bear
> in mind that my perspective might not might be as well-informed as
> that of others, but here are my thoughts anyway.
> Let's think for a moment about who our target audiences are:
> 1. Vanilla RSpec users
> These users want to be able to checkout just the RSpec code, so it
> makes sense that it live in its own repository.
> 2. Rails RSpec users
> These users want to be able to checkout this and the RSpec code, so
> again it makes sense that it live in its own repository.
> 3. People hacking on RSpec itself
> It's for this group that rspec-dev.git exists, and it exists (or
> _should_ exist) for the sole purpose of providing a convenient way
> of doing the "meta" tasks involved in RSpec development: ie. testing
> the three main parts of RSpec, setting up the example Rails app,
> packaging releases and so forth. If something about rspec-dev.git is
> starting to feel like a PITA then it's obviously not fulfilling its
> purpose of "providing a _convenient_ way..."
> 4. Users of the TextMate bundle
> Obviously, these are members of groups 1, 2 and 3, and need this to
> live in a separate repo for installation purposes.
> As I see it, rspec-dev.git should be almost static: once it's set up
> right there should be very few commits to it and there should be
> very little ongoing maintenance work on it.
> The optimal workflow as I envisage it is the following.
> - Have the developer clone the three or four repos, all in the same
> directory; ie.
> git clone ../path/to/rspec-dev.git
> git clone ../path/to/rspec.git
> git clone ../path/to/rspec-rails.git
> git clone ../path/to/rspec-tmbundle.git
> This is a once-off set-up and doesn't really justify having a Rake
> task IMO.
> The idea of having multiple git repos nested inside a master repo
> makes me feel decidedly uncomfortable. Submodules is the only
> "supported" way of doing that, and it was tried and rejected because
> the submodule system is really designed for a different type of
> problem than the one RSpec is trying to solve. Nesting repos inside
> one another sets off alarm bells: if they really are _that_
> interdependent then why do we have separate repos in the first
> place? But we want separate repos for ease of end-user consumption,
> so doesn't that mean that we haven't really achieved a sufficient
> degree of independence?
This is more or less how we had it set up in svn. One repo, yes, but
rspec and rspec_on_rails up at the root. The workflow problem with
this setup is that you have to have the rspec and (now) rspec-rails
directories in example_rails_app/vendor/plugins in order to run the
specs for rspec-rails. The way we were doing that before was to
actually copy them down in the pre_commit task - a workflow nightmare.
We can't just symlink them for two reasons - developers using windows
AND for some reason unbeknownst to me, rails won't see the generators
in rspec-rails if rspec-rails is symlinked from vendor/plugins. Crazy!
If someone out there can solve that problem it would be a great help.
> You should be able to work in these repositories independently of
> one another without having to worry about too many meta issues. ie.
> if you have a bug to solve in "rspec" you should be able to work
> inside that repository and not have to think about the others. Only
> at the end of the process do you worry about the "meta" issue of
> running the granddaddy pre_commit task in rspec-dev.
This is actually the case for the rspec repo.
> - Running the spec suite
> You should be able to run the spec suite for each of the
> repositories independently of the others. ie. you should be able to
> cd into the rspec clone and do a "rake spec" or "spec spec" (why is
> it "spec spec/spec?").
You can run "rake spec" or "bin/spec spec" - the reason for spec/spec
(which is not necessary to include on the command line when running
the specs) is to keep it parallel with the lib directory. We have spec/
spec/matchers, for example, which has specs for lib/spec/matchers.
> Likewise, you should be able to cd into the rspec-rails clone and do
> "rake spec". Where should the example Rails app go? Evidently inside
> rspec-rails, ignored appropriately using .gitignore, and with
> symlinks to the relevant bits. Putting the rspec-rails repo _inside_
> the Rails app just seems wrong. If the symlink thing doesn't work
> (as David indicates) then it's a bug in Rails, I suspect, and should
> be fixed, but in the meantime a simple Rake task should suffice to
> do a temporary copy instead of a symlink.
That's an interesting idea that I'm open to exploring.
> - "Meta" tasks
> So what's left? The "meta" tasks for stuff you do in rspec-dev, like
> the pre_commit and such. As I said, once rspec-dev is set up "right"
> it really shouldn't be a maintenance burden and if it is we've done
> something wrong.
> If you keep your repos up-to-date before you start working on an
> issue then there's less time for merge conflicts to arise. If the
> independence of the repositories is sufficient then you won't want
> or need to make changes to multiple repos at once, so you should be
> able to "git pull" in each and have a basically instant fast-forward
> merge. If the repositories really are isolated and you structure
> your commits with adequate separation then you should rarely/never
> need to do a set of commits that span multiple repositories, and if
> you do you should be doing them in the right order anyway (ie. while
> adding a new feature to rspec-rails you expose a latent bug in rspec
> causing failure in rspec rails? obviously the fix to rspec should be
> independent and go into rspec beforehand anyway).
One thing to keep in mind here is that rspec-rails is very closely
bound to rspec. I think rspec has improved in terms of logical
extension points, but if we really want to decouple rspec-rails from
rspec's internals, we're going to have to first make a concerted
effort to define and commit to a formalized extension API and use that
API in rspec-rails. Until that is fully thought out and enacted, rspec-
rails will continue to depend on internals of rspec and will need to
move with every turn rspec takes.
> So what does the workflow look like?
> - you always work on topic branches
> - ideally, your topic branches should be confined to _one_ of the
> - we can take advantage of the structure of the repositories to
> provide some convenience scripts for doing things like updating/
> rebasing all the repos at once, but the working patterns should be
> such that these rebases should be "fast forwards" 90% of the time
> I don't know if I've explained my thinking very well here, but the
> basic message I was trying to communicate was that:
> 1. Having multiple repos is a good thing because it makes
> consumption easy for end users; and RSpec exists for end users.
> 2. If having multiple repos starts to feel like hard work then its
> kind of like code smell; it's "maintenance smell" that indicates
> that something isn't right, but we should be able to fix it.
I agree with almost all of this in principle. I think, however, that
the current burden is not that much to bear and we have over 200 open
tickets in lighthouse. Admittedly, a lower barrier to entry for
developers might encourage more people to be taking on those tickets,
but I rarely see people in the community submitting patches to solve
existing tickets - they usually contribute patches to solve their own
We should keep this on the radar, and I welcome thoughts on how to
best approach it.
More information about the rspec-devel