[Nitro] Nitro Development

Sun Oct 22 10:16:18 EDT 2006

Nitro Development
==================

As the meeting on IRC didn't work out (like I expected..) I agreed with  
George
to discuss this on the ML.

So, here it goes.  (Document best viewed in a Markdown aware environment.)

First, problems:
----------------

### Broken Darcs repositories

We had this quite a few times now.  Symptoms:

* Merging problems on untouched files
* When trying to revert, reverts to inconsistent state

Possible reasons this happens:

* Moving files without tellings darcs (`mv`).
* Touching too many lines in a file (indending many lines) plus moving  
chunks
   lines to other places in the same file.
* Munching too many unrelated chunks into a single patch.
* Too long sideways development, merges take different paths.
* Patches, only marginally reworked getting `record`'ed anew.
* Patches which get revoked on a central repository, but still on dev  
repos.

In effect, Darcs fails to produce a valid repository state when too many  
and
too complex changes are made.

### Only one central authority

George can only do so much, he's only a single guy after all.
The problem here is, that George is the main developer as well as the  
project
maintainer.  I guess noone in the Nitro community can fill either position
atm.

### Detached community development

Many people (Alex, manv, Fabian, me, nusgnaf and others I don't remember)
worked on manverus repo.  We often tried to collect patches from George  
into
this repo, but we often faced the problem that either changes were already
done or something else was done at the same places => merge problems.
To resolve those, it's needed to make patches for patches, merge patches,  
which will never find their way back to George.  Bad, inconsistent state.

When trying to let the patches flow back to the head repo (George) we face  
the
exact same problem.  We need to ensure that we don't push crap to George.
This is either work intensive (make new patches, based on a pure George  
repo)
or messy (push also merge patches).

Despite Darcs being a very sophisticated tool, it is not perfect, as shown
above.

The current development style is two-fold:

The first side is George:  Work on new features, push them to own repo,
integrate occassional patch from outside (if it doesn't produce merge  
errors),
check with own applications.

Second is the community:  Work on whatever fails,  make it possible to run  
own
applications without errors, push to manv repo.  Many work goes into fixing
Unit Tests.  Finally try to push patches to George.

### Almost no communication between George and Community

George loves working on new features ("I got something great worked out!
You'll see it monday!").  The flow of information from George to the  
community
isn't quite as strong as it perhaps should be.
The other way round:  In #nitro on irc the 'core' community developers
coordinate their actions, nothing ever gets done twice, the feedback cycle  
is
much faster.  But this information doesn't flow back to George, as long as  
he
isn't online (which he rarely is for longer than 10 minutes and once or  
twice in the week, due to having no internet connection at home).  So  
information
(some of which perhaps should be posted to ML or to George) stays only in  
the
IRC community.

How to make it better?
----------------------

### Broken Darcs repositories

Two options:

1. Stay on Darcs, but create fresh repo
2. Use other revision control system

Problems with the first approach:  It could happen again, if it does, we  
have
to go for another rcs.

In case 1 we will lose all history, in the second case one can think of
transferring the history manually by "`record`'ing" the changes in the  
other
rcs.

When using another another rcs we have to consider a few things.

* Popularity of the rcs
   When popular enough (speak: Subversion) it can be used to get additional
   developers.
* Type of the rcs
   Distributed or central.  When central, George (or whoever the maintainer  
is)
   has to set up a central repository where everybody works on (possibly in
   branches).  Patches from outsiders (non core developers) have to be  
accepted
   by core-developers in normal diff/patch form.
   When using a distributed rcs, this is not needed and it is easier for
   non-core developers to create a branch, work on it and contribute to  
Nitro.
* Platform the rcs runs on
   Not everyone is on Linux or other posix-aware systems.  I think we have  
to
   support Windows users here as well.

Options:

http://en.wikipedia.org/wiki/Comparison_of_revision_control_software/

* Subversion
   Popularity: high
   Type: central
   Platform: any
   This would be my personal third choice.  It is perhaps the best option  
for
   Nitro.  No known problems with merging, used in many OSS projects.
* Bazaar (aka Bazaar NG aka Bazaar 2.0)
   Popularity: low
   Type: distributed
   Platform: any (Python)
   George brought that one up.  Created and maintained by Canonical.
   Apparently not really maintained actively.  I have heard of problems  
using
   it for bigger projects.
* Darcs
   Popularity: uh, low I guess
   Type: distributed
   Platform: any
   This would be my personal first choice.  Too bad it can't handle all  
kinds
   of development.
* SVK
   Popularity: no idea
   Type: distributed
   Platform: any
   Interesting?  Uses subversion as backend, dunno how it works.

(My personal second choice would be GNU Arch.  Git would also be very
interesting, but is Posix only.)  I would like sticking to Darcs, but I  
can't
keep working on broken repositories.  I don't like Subversion...

Lets make a survey, everyone please state his idea which revision control
software should be used.  Keep in mind that this will have influence on how
the Nitro development model will look like.

When going for a central repository, George (or the current maintainer)  
has to
set up a central repo, where _all_ current core-developers work on,  
including
George.  Under no circumstances will I shuffle normal `diff`s.  This has to
be quite dynamic, as the developers can change quite rapidly (I want bryan  
and
zimba back...).

### Only one central authority

Maybe the maintainer role of George can be partly taken over by someone  
else.
In 0.31 this already worked ok I think (this was devlab and release just  
like
that).  Many thanks to Zimba again to provide this place in the first  
place.

This someone would have to do following:

* Maintain the roadmap
* Check with the main developers what has been done, who is doing what
* Check the current 'stable to be' repo if it is on track to next roadmap
* Make sure that repo is in a consistent state

### Detached community development

This is more or less dependand on the communication between _all_
core-developers.

I hope _William_ and _Eivind Eklund_ can help here.

Basic layout of current community.

* A single main contributor (George)
* 3-4 core-developers (Manveru, Fabian, Jonathan)
* Other people with only occasional patches (Nusgnaf, Alex and others)

As you both have more experience with organizing (I for example have none,
and probably never will have) I really hope you can give some pointers,
observe how we do stuff, bring forth ideas which we haven't thought of.

### Almost no communication between George and Community

This is perhaps the biggest problem right now (right after darcs merging
problems).
Any ideas?

### Development in general

A quote from Tom Lord:

> **Using commit Well ? The Idea of a Clean Changeset**
>When you commit a set of changes, it is generally "best practice" to make 
> sure you are creating a clean changeset.A clean changeset is one that  
> contains only changes that are all related andfor a single purpose. For  
> example, if you have several bugs to fix, orseveral features to add, try  
> not to mix those changes up in a single commit.There are many advantages  
> to clean changesets but foremost among them are:**Easier Review** It is  
> easy for someone to understand a changeset if it isonly trying to do one  
> thing.**Easier Merging** As we'll learn in later chapters, there are  
> circumstancesin which you'll want to look at a collection of changesets  
> in an archive andpick-and-choose among them. Perhaps you want to grab  
> "bug fix A" but not"new feature B". If each changeset has only one  
> purpose, that kind ofcherrypicking is much more practical.

So, my proposal for patches:

* Clean Changesets
* Unit Tests **always** have to run

The second point is only valid, when working on small changes and or on the
stable repo.  When doing big structural changes to the whole source tree  
(like
the adapter changes) this can't work.

### Over and Out

Keep in mind, this is only my personal meaning and only meant to be a base  
for
discussion.

Things to think about:

* Which revision control software to use
* How to organize releases
* How to organize quality control
* How to organize the work to be done

Jonathan

-- 
Feel the love
http://pinkjuice.com/pics/ruby.png