The Build Process is central to the business of delivering software.  In my opinion, the Build Process IS your business.  It is crucial to the development of high quality software - from both a developer and end-user perspective.

In our organisation, we have a build pipeline which can effectively be separated into 3 distinct stages:

Stage 1 - Development

When the Developer starts the build the code is first run through our automated source analysis tools.  If there are no breaking changes here (the policy can be defined) then the source is built.  If this successfully builds then the required unit testing suites are run.  It is important to realise here that we do not build THE PRODUCT, every time.  We could, but most of the time we will be building only one specific component of the product.  These can be chained together in the CI system. 

Note: Most CI processes will commit the source FIRST, then perform Unit Testing.  While this still has the advantage of catching build errors early if performed regularly, it still results in broken builds.  The CI system we use, TeamCity, allows us to perform "Remote Personal Builds", which merges the source to be committed to a *unique* location.  If there is a build error or unit test failure then the defective source is NOT committed to VCS.  Therefore there will never be a broken build.

Of course, what this means is that you must set up your build scripts to work with RELATIVE PATHS ONLY, which could be a considerable change for legacy systems.  If using SVN, the Externals feature is crucial to this.  I hope you would agree with me that the result here more than justifies the cost in setting it up. 

Stage 2 - Automated Testing and Deployment

If Stage 1 has been successful, the Installation is built.  Note that we are talking about A PRODUCT DELIVERABLE in this case (although there can be several of these that comprise THE PRODUCT).

Virtualisation and Automation are two key foundation stones that a CI system is built on.  It is always worthwhile to remember that the majority cost of developing software isn't in the coding, but in the testing.  EVERY test that has to be performed manually, because it has to be repeated again and again by someone who needs to be paid, is throwing your money away.  Therefore it is much more prudent to develop automate as much of the test effort as you can.

The Build Process now deploys the installation to the designated virtual machines.  These comprise a number of different operating systems with different configurations.  Again, with the number of different versions of OS on the market, we can get as much coverage as we desire here with a fraction of the cost involved in doing this manually.

Once this has been deployed, the designated auto-tests are run on each configuration.  The results log of these tests are linked back into the appropriate build in the CI system, so it is fully traceable.  The final task of this stage is to roll back the virtual machines to their pre-test state - this allows anyone who wants to look at that particular install do so without having to install it themselves.

Stage 3 - QA

Because we have automated so much of the 'grunt work' we can concentrate on the elements of the system which cannot easily be automated, if at all. 

Any manual QA is performed here, along with Design Reviews of the product.  This can give the goal donor and other stakeholders insight into the product at every stage in its delivery, and of course the earlier that change is managed the easier it is to inject into the product.

There are so many benefits from setting up a successful build pipeline, and it really must be done at the start, not half-way, and certainly not at the end, to get the most out of it. 

When it's time to start implementing a new project, where do you begin?  Once the requirements have been gathered and the first Sprint planned, it's time to start coding, right? 

I'm not so sure.

Something that has dawned on me over a number of projects, is the seemingly back-to-front mantra of start at the end.

By this, I mean that you should always start at your build process.  Examine it, and make sure that before Feature One has been committed to source control, you have a fully functioning build system in operation.  That is, there should be a fully automated system in place that will build the latest sources, create any necessary installation files and deploy to pre-designated release locations, complete with versioning.  And this should be in place before you've got anything more than a single form in your application.

The reason for this is simple - the build process is critical to the success of your project.  It is too easy to underestimate the amount of work that has to go in to making a release of software.  You can lose days tangled up in build scripts when someone adds a new component, or worse - maybe there is a wrong version of an assembly being referenced.  A number of years ago, I remember we used to do product release builds inside the IDE, and then manually copy the files to a location on the network.  Thankfully those days are well behind us!

Of course, the other reason is that the build scripts can get very complicated - very quickly.  Like other, non-functional requirements of software such as exception management, localization and documentation - if you don't keep on top of them you will lose control. 

Key to the build process is the concept of Continuous Integration.  This is the idea that changes are integrated into the main build continously - we no longer wait on kicking off builds manually.  Builds are usually triggered as a result of source files being committed to the repository, or on hourly schedules.  The thinking behind this is that the earlier we catch an error, the easier it will be to fix.  Like most good ideas regarding software development - this is just common sense.

However, you don't get it for free.  But there are tools which take on a lot of this burden, and it's really up to you how complex you want your build process to be.  Do you want automatic code reviews to be run (such as FxCop), Unit Test coverage reports, automated deployment to virtual machines for automated testing?  Or else your build process could simply consist of a source control fetch followed by a build, followed by a copy to a location on the network entitled 'Releases'.  Still better than doing it yourself!

Over the next few weeks I'll be documenting my ideal approach that I believe would lead to the successful and timely delivery of software projects.