Reaching Continuum – My journey with continuous integration, delivery and testing

Reaching Continuum – My journey with continuous integration, delivery and testing

What is Continuum? Simply put, when an organization has implemented continuous Integration, continuous testing, continuous delivery, and continuous deployment – I call this reaching Continuum.  You are continually executing unit tests, continually integrating code, and continually deploying your tested software to test environments and production.

The key to reaching Continuum is to successfully implement each phase of being continuous. I call each of these phases “states of being”.

State of Being #1: Integrate Continually

From Wikipedia , “Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.”

To start your journey in reaching Continuum, you will need a software version and release management system and Continuous Integration software. You will store and manage code versions using the software version and release management system. More specifically, you will create a repository on your software version software. A repository is like a bank. A bank safely stores your money. A repository is where you will safely store your code.

Your Continuous Integration (CI) software will monitor any changes to your repository. If there are changes, your CI software will run your tests automatically against latest version of your code. If your tests fail then you probably broke something. This is the value that continuous integration provides. It can provide you fast feedback on the code that you checked in. The more tests you have, the more confident you will be in the quality of your code if the tests pass. If the tests fail, you know that your code that you checked in broke something and you should review your code and fix the issue. When you check-in your newly fixed code, the tests will automatically run again to validate your fix.

There are many software version and release management systems out there. One is Subversion. Subversion is open source so it is free. It is widely used and popular. It has a large community of followers. You can find out more about Subversion by going to their site here. Along with Subversion, you can use TortoiseSVN which is a Subversion client that allows you integrate Subversion as part of your Windows file explorer. With TortoiseSVN, you can commit and update your code within the Windows file explorer. TortoiseSVN can also integrate with MS Visual Studio. You can go to the TortoiseSVN web site for more information. Git is another popular free version control system. Git has an advantage for providing local branching support.

For Continuous Integration, like software version and release management systems there are many out there. Some CI systems are Jenkins, Atlassian Bamboo and CruiseControl. Like Subversion, Jenkins is open source and widely popular and widely used. It can be installed on many different platforms and it can be extended with lots of plug-ins. Learn more Jenkins by clicking here.

State of Being #2: Test Continuously

Now that you have a software version system and CI system, you will need to create tests that can run on the CI system. In Jenkins, you can create a build step to run a test runner like nUnit and provide an nUnit file containing your tests. In my company, we use nUnit for unit testing and Specflow, a plugin to Visual Studio, for functional testing. Specflow allows us to create test specifications using a simple system of Given-When-Then statements (Gherkin Language). You bind your Given-When-Then statements to some code. Our testers write their test cases or scenarios using Gherkin and directly through Visual Studio. Our developers then create the bindings to the Gherkin. Once a Gherkin statement has been bound to code, you don’t need to rebind it. We have found that using Gherkin and Specflow has quickly grown our automated tests.

To reach Continuum, you should test continually at all testing levels:

  • Unit testing
  • Smoke testing
  • Regression functional testing
  • End-to-end system testing
  • Performance Testing

Each level of testing may require separate infrastructure. You could also create a ‘build’ for each type of test. Each test build monitors changes to the code repository and executes the tests.

Initially, at my company, test automation was an afterthought. The bulk of our testing was manual and the test cases themselves were maintained in MS Excel sheets or MS Word documents. We were purely a Waterfall shop. For a given project, we allocated several months for requirements and design work, several months for development, several months for documentation and several months for testing. Most people did not favor test automation. They felt no need to do so because we can run all of our manual tests just fine in the time allocated. Besides, our folks do not have the skills to learn or understand automation is what they also said. With our Waterfall model, we released 1 or 2 times a year to our customers. Typically, our customers would tell us that what we released they wanted several months ago and now they want something else. We were always trying to catch-up to our customer’s demands. Our customers were increasingly unsatisfied with our slow delivery of functionality that they wanted and needed. Things started to change when half the company wanted to try something new and you can call radical at the time. They wanted to try Agile Software Development. Agile Software Development is defined by Wikipedia as “...  a group of software development methods in which requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development, early delivery, continuous improvement and encourages rapid and flexible response to change.“.

One of the challenges with moving to Agile is the shorter cycles to develop, to test, and to deliver. This short cycle forces you to look at methods to automate anything that is manual. Anything that is done manually will eventually slow down the Agile team.

State of Being #3: Deploy Continuously

Continuous deployment is deploying changes after the changes pass automated tests to a production-like environment. This builds confidence that you can deploy these changes to production when the business is ready to accept the changes to production.

State of Being #4: Deliver Continuously

Continuous delivery is when you deploy every change after it passes automated tests to production. There might be business cases where delivering continuously may not be practical. If you have customers that impose ‘black-out’ dates where they do not want changes in their environments, continuous delivery may be impractical. You also need to look at downtime that this needed and impacts to customers. What would happen if a customer was using your product and changes were implemented? Would it disrupt the customer?

One thought is to batch your changes from #3 above and deploy these changes on some schedule such as weekly at a set time. If you have set maintenance windows, batching changing and deploying changes at a known set time and day will be more acceptable.

What a Complete Continuum Environment Might Look Like


Alden Mallare


  1. Hello! I could have sworn I’ve been to this site before but after browsing through some of the post I realized it’s new to me. Anyways, I’m definitely glad I found it and I’ll be bookmarking and checking back often!

  2. I have been gone for some time, but now I remember why I used to love this web site. Thanks, I

  3. I have recently started a blog, and the info you offer on this web site has helped me greatly. Thank you for all of your time & work.