I've been doing a lot on cfEngine recently. At $PREVIOUS_EMPLOYER I used cfEngine 2, which I have liked a lot. Here at $WORK I did quite an extensive job on Puppet, but I always longed to try cfEngine again, this time with version 3. And, finally, I did. …cfEngine 3
cfEngine is one of the most popular Configuration Management Tool. Puppet is another one, and you have Chef, and LCFG, and then more.

As expected, starting was hard. With cfE2, you had to assimilate a lot of knowledge before you could actually do something useful: cfE3 is no different. You have to connect a lot of pieces together, including documentation.

cfEngine3's documentation deserves its own section. In "essential", it is:

• a bare-bones tutorial
• a complete reference (a bit less than 600 pages…)
• a standard library reference (i.e.: the "Cfengine Open Promise Body Library" reference)
• a best practices guide
• a number of "Special Topics" guides (more than 30)

Guess yourself how easy is to find the right path through all these documents…

The good side is that, once you connect enough pieces together, things start going the right way and you get a feeling of what cfEngine can offer: power, coherence, robustness.

My earlier experiments, you can find them in the cfEngine help forum. My latest one, which I am going to talk about, was an attempt to downsize an already working, but huge, policy.

The ntp policy
In the past weeks I put together a policy to configure ntp on all our clients and servers at work. That worked, but it was something like 800 lines. I didn't like that, because I felt it was not as manageable as I wanted, and it contained a lot of stuff which wasn't needed at every location.

I started to study and experiment on how to downsize it. The goal was to have all location specific settings in location specific files (let's call them ntp_x.cf), plus a file with the common stuff in (ntp.cf). The location specific policies would then set their variables and classes, and then call the common part, passing the relevant bits and pieces through.

Badly, that proved to be a challenge. But it was doable, and finally, it worked.

The first problem was how to pass an associative array of settings to a parametric bundle. A user suggested this approach:

        usebundle => action_user_promise("caller_N.user_a");

which worked, and I used it. But I don't like it. Basically, you are passing a string to a bundle, which contains the fully qualified name of an associative array (in the form: bundle_name.array_name). Think of it as calling a method or a subroutine providing a symbolic reference to a somehow "global" array. The subroutine would then dereference that array, and use their values.

An approach I never liked. But still, I had to use it.

At this point, the hurdle was: how to dereference this symbolic reference correctly. After some trial and error I realized I could dereference single values in the associative array, e.g.:

  vars:
      # dereferencing a scalar value:
      "tstring" string => "$($(atest)[string])" ;

      # dereferencing a list value:
      "tlist"   slist  => { "@($(atest)[slist])" } ;

but I couldn't dereference the whole associative array itself. There doesn't even exist an explicit data type for associative arrays. OK, have to live with it…

Now I had a way to pass values around with associative arrays. I couldn't pass classes unfortunately, and I didn't feel like defining global classes — using global objects into shared pieces of codes is just looking for trouble.

But why did I need to pass classes, in the first place? Well, the plan was to define in each location-specific ntp_x.cf file which machine was a server or a client (is_client and is_server classes) and which machine would use unicast or multicast (is_ucast and is_mcast). These classes would then be combined in other non-location-specific ones, hence in ntp.cf, like in:

  classes:
    any::
      "ucastserver"     and => { "is_ucast", "is_server" } ;
      "ucastclient"     and => { "is_ucast", "is_client" } ;
      "mcastclient"     and => { "is_mcast", "is_client" } ;
      "mcastserver"     and => { "is_mcast", "is_server" } ;

      # These classes will help us when editing the file, to decide
      # what we should put into that
      "has_keys"        expression => "is_mcast" ;
      "has_serverlist"  or         => { "ucastclient", "is_server" } ;
      "has_peerlist"    expression => "is_server" ;

For this to work, I needed a way to "propagate" the local, location-specific class definitions to the called bundle. This is how I did it.

In the location-specific file I define:

  vars:
    is_client::
      "ntpconf[role]" string => "client" ;

    is_ucast::
      "ntpconf[casting]" string => "ucast" ;

and then in ntp.cf I use this:

  vars:
    any::
      "role"            string => "$($(ntpconf)[role])" ;
      "casting"         string => "$($(ntpconf)[casting])" ;

  classes:
    any::
      "is_$(role)"      expression => "any" ;
      "is_$(casting)"   expression => "any" ;

The pieces were finally in place. But something still didn't work as expected. It took a number of tests to understand that the join() function had problems to dereference, e.g., this.serverlist correctly. I changed all "this." with "ntp.", and that finally did the trick.

My first location-specific policy file is 28 lines long (but it's a really minimal one), and the shared one is 200, much smaller and manageable than the previous version (~800 lines). Too big to put them here, but you can still have a look in the cfEngine forum and mailing list archives.

Enjoy!