In my latest post “git repository and deployment procedures for CFEngine policies” I explained how we structured our git repository for CFEngine policies, and how we built a deployment procedure, based on GNU make, to easily deploy different projects and branches from the same repository to the policy hubs. Please read that post if you haven’t yet, as this one is not going to make much sense without it.

The make-based deployment procedure worked pretty well and was functional, but still had annoyances. Let’s name a few:

• the make command line was a bit long and ugly; usually it was something like:
  make -C /var/cfengine/git/common/tools/deploy deploy PROJECT=projX BRANCH=dev-projX-foo SERVER=projX-testhub
• the Makefile was not optimized to deploy on more than one server at a time. To deploy the same files on several hubs, the only solution was to run make in a cycle several times, as in
  for SERVER in projX-hub{1..10} ; do make -C /var/cfengine/git/common/tools/deploy deploy PROJECT=projX BRANCH=dev-projX-foo SERVER=$SERVER ; done
• deploying a project on all the policy hubs related to that project required one to remember all of the addresses/hostnames; forget one or more of them, and they would simply, hopelessly left behind.

At the same time, there were a few more people that were interested in making tiny changes to the configurations via ENC and deploy, and that long command line was a bit discouraging. All this taken together meant: I needed to add a multi-hub deployment target to the Makefile, and I needed a wrapper for the deployment process to hide that ugly command line.

For first, I added to the Makefile the functionality needed to deploy on more than one hub without having to re-create the temporary directory at every run: it would prepare the files once, deploy them as many times as needed, and then wipe the temporary directory. That was nice and, indeed, needed. But the wrapper couldn’t wait any longer, and I started working on it immediately after. That’s where cf-deploy was born.

Meet cf-deploy

cf-deploy is a bash script that reads the settings from the command line and configuration files and runs make for you. Each project is associated to a configuration file (actually more, but let’s make believe it’s just one for now), for example projX.proj, that specifies: 1) if the project is remote (must rsync to a remote server to deploy) or local (must rsync to a local filesystem); 2) the name of one or more files containing lists of hubs to deploy to; 3) which branch should be deployed by default; 4) which subdirectory should be deployed together with common/.

An example of such a file is:

PROJECT_TYPE=remote
HUB_LISTS=projectX.hubs
BRANCH=master
PROJECT=projX

To deploy this project in production one would just run:

cf-deploy deploy projX

However, since deploy is probably the most common action one will use this script for, the command line above can be shortened to:

cf-deploy projX

The script reads the settings from the project file and runs make with the appropriate parameters to deploy the master branch (set in BRANCH) on all the hubs listed in the file projectX.hubs (set in HUB_LISTS). Notice that HUB_LISTS could list more than one file, e.g.:

HUB_LISTS="projectX-location1.hubs projectX-location2.hubs projectX-location3.hubs"

If we want to preview the changes instead, that is: seeing which files would be changed on the policy hub when we deploy, we run

cf-deploy preview projX

Checking the diff between the current files on a hub and the ones we are about to deploy on an hub (e.g.: projX-hub1) is as easy as running:

cf-deploy diff projX hub projX-hub1

All the command lines above will deploy the branch that was declared as default for that project with the BRANCH setting. If we want to override that setting and deploy a different branch, we can do that specifying a branch argument in the command line:

cf-deploy deploy projX branch dev-projX-foo

This approach of having all project details specified in external files comes handy in many situations.

Deploying in a test environment

Suppose, for example, that you want to deploy in a test environment instead of production: in the same way we had a file called projX.proj we can also have a projX-test.proj file that is a bit different:

PROJECT_TYPE=remote
HUB_LISTS=projectX-test.hubs
BRANCH=merge-projX
PROJECT=projX

With this, you could use cf-deploy projX-test to deploy your integration branch on projX’s test environment: it is no more complicated than deploying in production! (It could be trickier by using the makefile directly).

Deploying on pre-production hubs only

In the same way, you can have files listing (for example) all pre-production hubs for a project, and make a projX-preprod.proj file with a HUB_LISTS setting that will help you deploy on those hubs only when you want to test a special feature on a small set of guinea pig nodes.

Is there more?

cf-deploy also has a show action and I’ll leave it to you to read through the code of cf-deploy and understand what it does. Talking of which…

The script

Update, February 16th, 2015: the current code for cf-deploy is now on github.

#!/bin/bash

### SCRIPT CONFIGURATION ###############################################
GITDIR=/var/cfengine/git
TOOLDIR=common/tools/deploy
### YOU SHOULD NOT CHANGE ANYTHING BELOW THIS LINE #####################

MAKEDIR="${GITDIR}/${TOOLDIR}"
MAKECMD="make -e -C $MAKEDIR"

function usage() {
    echo ""
    echo "Usage:"
    echo "  $0 PROJECT_NAME"
    echo "    deploys project PROJECT_NAME on all hubs. It's a shortcut for"
    echo "    $0 deploy PROJECT_NAME"
    echo ""
    echo "  $0 deploy PROJECT_NAME [ branch BRANCH ] [ hub SERVER ]"
    echo "    Deploys PROJECT_NAME with the specified options"
    echo ""
    echo "  $0 preview PROJECT_NAME [ branch BRANCH ] [ hub SERVER ]"
    echo "    preview the changes that would be applied"
    echo ""
    echo "  $0 diff PROJECT_NAME [ branch BRANCH ] [ hub SERVER ]"
    echo "    runs a diff for a project (hub is mandatory)"
    echo ""
    echo "  $0 show PROJECT_NAME"
    echo "    Describes the project PROJECT_NAME, other options are ignored"
    echo ""
    echo "  $0 show list"
    echo "    Lists all defined projects, other options are ignored"
    echo ""
    echo "  You can override the default branch of a project by using the"
    echo "  keyword branch. You can also override the project's hub list"
    echo "  by specifying an hub with the hub keyword -- notice that"
    echo "  specifying a hub for a local project is pointless"
    echo ""
    exit $1
}


function source_project() {
    PROJECT_NAME=$1
    
    # Source the project file, bail out if we can't find it
    source "$MAKEDIR/$PROJECT_NAME.proj"
    if [ $? -ne 0 ]
    then
	echo "Cannot load project $PROJECT_NAME, bailing out"
	exit 32
    fi
}

# No arguments: show usage info and get out
if [ $# -eq 0 ]
then
    usage 1 ;
fi

# One argument, we assume we must deploy this project with default
# settings
if [ $# -eq 1 ]
then
    ACTION="deploy"
    PROJECT_NAME=$1
    shift

else

# More than one argument. The first is an action: deploy preview diff
# The second argument is the project name
# More arguments can follow
    ACTION=$1
    case "$ACTION" in
	deploy|preview|diff|show)
	    shift ;;

	*)
	    echo "What do you want to do?"
	    usage 1
    esac

    PROJECT_NAME=$1
    shift ;

    # Parse the remaining arguments, check that they are known and
    # act accordingly
    while (( "$#" ))
    do
	CMD=$1
	shift

	case "$CMD" in
	    branch)
		# We override the branch set in the project file with
		# the one specified after the keyword "branch"
		BRANCH_OVERRIDE=$1
		shift
		;;

	    hub)
		# We override the hub list in the project file with the
		# policy hub specified after the keyword "hub"
		SERVER=$1
		shift
		;;

	    *)
		# Watcha talkin'bout, Willis?
		usage 2
		;;
	esac
    done
fi

# Bail out if the project name is null
if [ -z "$PROJECT_NAME" ]
then
    echo "Which project do you want to deploy?"
    usage 4
fi

# The action project needs to be handled here so that:
# * it doesn't get polluted by BRANCH_OVERRIDE
# * it doesn't try to source list.proj when a project list is requested
if [ "$ACTION" == "show" ]
then
    if [ "$PROJECT_NAME" == "list" ]
    then
	echo "PROJECTS DEFINED IN $MAKEDIR:"
	( cd "$MAKEDIR" && ls -1 *.proj ) | sed -e 's/\.proj$//'

    else
	source_project "$PROJECT_NAME"
	echo "Description for project $PROJECT_NAME"
	echo "Project type:   ${PROJECT_TYPE}"
	echo "Default branch: ${BRANCH}"
	echo "Git project ID: ${PROJECT}"
	echo "Hub lists:      ${HUB_LISTS}"
	echo "Defined in:     $MAKEDIR/$PROJECT_NAME.proj"
    fi

    exit 0
fi

source_project "$PROJECT_NAME"

# Override the default branch if so asked
if [ -n "$BRANCH_OVERRIDE" ]
then
    BRANCH="$BRANCH_OVERRIDE"
fi

# Define the action to execute depending on the project being a local
# or a remote one
case "${PROJECT_TYPE}" in
    remote)
	DEPLOY_ACTION=deploy
	PREVIEW_ACTION=preview
	DIFF_ACTION=diff
	DEPLOY_MULTI_ACTION=deploy_multi
	PREVIEW_MULTI_ACTION=preview_multi
	;;

    local)
	DEPLOY_ACTION=deploy_local
	PREVIEW_ACTION=preview_local
	DIFF_ACTION=diff_local
	DEPLOY_MULTI_ACTION=$DEPLOY_ACTION
	PREVIEW_MULTI_ACTION=$PREVIEW_ACTION
	;;

    *)
	echo "project type ${PROJECT_TYPE} not supported"
	exit 5
	;;

esac

# Make these variables visible to "make", so that they override the
# makefile defaults and the makefile actually does what we want
export PROJECT BRANCH SERVER LOCALDIR MASTERDIR HUB_LISTS

# If the action requested is deploy, deploy on all the hubs defined in
# the project, unless we specified a specific hub on the command line
if [ "$ACTION" == "deploy" ]
then
    if [ -n "$SERVER" ]
    then
	$MAKECMD $DEPLOY_ACTION

    else
	$MAKECMD $DEPLOY_MULTI_ACTION
    fi

# If the action requested is preview, preview on all the hubs defined in
# the project, unless we specified a specific hub on the command line
elif [ "$ACTION" == "preview" ]
then
    if [ -n "$SERVER" ]
    then
	$MAKECMD $PREVIEW_ACTION

    else
	$MAKECMD $PREVIEW_MULTI_ACTION
    fi

# See the comment above the "preview" action for the logic of this.
elif [ "$ACTION" == "diff" ]
then
    case "${PROJECT_TYPE}" in
	local)
	    $MAKECMD $DIFF_ACTION 2>&1
	    ;;

	remote)
	    if [ -z "$SERVER" ]
	    then
		echo "diff in remote projects requires a hub"
		exit 4
	    fi
	    $MAKECMD $DIFF_ACTION 2>&1
	    ;;

	*)
	    echo "What the fuck do you do here?!"
	    exit 64
	    ;;
    esac
fi

Disclaimer: bash is not my favorite language and the script may well need improvements. If you have suggestions, please speak!

Where do we go from here?

As you can see, cf-deploy gives you plenty of possibilities to make your life easier and, once it was released, it took me a very little time to become addicted to it! But yet another area was calling for improvement: agent runs. To get the latest version of the policies and run them immediately on a node, one would usually run something like:

cf-agent -KIf update.cf && cf-agent -KI

Quite OK for any CFEngineer, but kind of weird for anyone else. That’s when a new tool was born, and it will be the subject of my next post. Watch out!