Deployment Automation

This post is partly a definition and partly describes ideals that have served me well – it is not meant to be prescriptive – please provide constructive feedback!

Useful deployment automation describes EVERY change required to install and configure a complete service.

A separation of concerns is required to describe the essentials here. I have used the word “scripts” but flat file input to any deployment automation tool could be used.

Ideals:

The following ideals have served me well:

• NO ENVIRONMENT specific information should be embedded in the scripts. This enables the same script to be tested thoroughly and, when appropriate, used from development, build, staging and in production see Environment Configuration.
• Scripts have to be stored in source control.
• Where possible, ALL script dependencies should be stored in the same repository and use RELATIVE paths.
• When dependencies are not stored (e.g. deployable artifacts) – they should be referenced unambiguously (CRC’s serve well here).
• Any script language or flat file input to a tool that is appropriate to the task should be used.
• Any automation should check every input and fail if ANY step is unsuccessful (problem solving is easy when detected at the point of failure).
• All automation should reuse the same building blocks by breaking down the entire system requirements into deployment

The separation of configuration from automation is the most important point here to enforce repeatable testable deployments. Doing this properly is at the heart of Release Management.

Tools:

The simplest tools with the least system dependencies are the best fit for some of the ideals I express above.

At its simplest, environment management can be easily implemented by even Batch and Shell scripts with a little help from run XSLT processors. XML configuration can easily provide all the environment specific input.

Other scripting languages that are equally applicable to system management and provide better support for XML configuration input include Perl, Ruby and on Windows; PowerShell.

More recently, tools which try to holistically solve deployment automation include RPath, Nolio and build servers like AnthillPro. These tools introduce very specific ways to working which need a high commitment to a single vendor without the ability to easily share the solutions.

Orchestration

In the spirit of separating concerns it seems best to separate the actual release that can take place on one host from the steps that coordinate the release across multiple computers. The same orchestration steps can then be carried out on a single node (e.g. development) or systems with varying number of nodes (e.g. test, staging or even separate production installations for separate customers).

The specific host names and the mapping of roles that they carry out should be stored separately from actual orchestration steps (which should be concerned with roles only). This information forms part of the Environment Configuration.

Open "Ecosystems"

How to make hardware and software that works together?

Build open "ecosystems" of hardware and software designed to work together at the point of sale.

How to build the ecosystem? By publishing the automation that configures hardware and software for use.

Softservice Distro

...What?

The term Distro, in relation to software, effectively boils down to a 'single package'. This is ultimately what allows a program (or set of many programs) to be shared, tested and ultimately improved in a repeatable way. A good package is one that makes this distribution process easy and accurate (read fully automated - think app stores).

I've coined the phrase 'Softservice' to describe all the software and configuration required to provision a high level user service on as many computing devices as required. The term 'Softservice Distro' would define how the softservice could be easily distributed and therefore shared and tested (just add hardware / virtual or otherwise).

In summary a single packaged distro of distros such that the entire configuration forms a single, testable, end user service or"Softservice Distro"!

...Why?

I think the cost, reliability and the effective security of computing services is held back as the complexity of distributed systems grows. I believe these issues could be largely mitigated by removing the duplication of effort that exists in maintaining software services at the level of a single vendors software package.

If the unit of a distro were to be extended to encompass an entire computing environment, required to provide services, then the costs and reliability could be effortlessly shared between providers or even end users.

...How?

DevOps are already solving these problems for their own clients. I'm just advocating that the solutions are shared so that a maintainable 'Softservice Distro' can be provided direct to end users for use and to other DevOps to improve. This isn't a cloud provided service as such - more a provisioning service from the cloud to be owned and run as end users require. This would effectively recycle the knowledge used by the DevOps community so that it becomes accessible to a broader base of users. See my previous post on the Deployment Automation Solution for a few more details.

The result could be a collaborative effort to provide incrementally improving computing services to benefit end users directly. The idea would be to avoid the disparately provided and maintained services we experience today - again, see my other post Solution Store not Hobby Kit.

Devops Discovered

I've just discovered that I do 'devops':

"I've been treading the line between development and operations my whole career. When in support roles I've striven to develop automation as it was always more fun. When working on large development projects I've been heavily involved in test driven development, agile methods and generally automating the development lifecycle."

The other day I googled deployment, automation or clouds or some such thing, as I often do, and out popped the devops movement!

A good definition of 'devops' is on the dev2ops blog as is a take on how this is relevant to deployment.

Investing in Cloud Computing and Automated Infrastructure

Was searching for "automated cloud building" - as you do - and this message seems about right:

Alpha Guy: Investing in Cloud Computing and Automated Infrastructure

My two pennies: I think that a shared and open tool set for automated infrastructure would be a great way for all companies to share the much needed investment in this area!