cfengine

hENC now on github

/ bronto / Leave a comment

github-logo hENC is a radically simple external node classificator for CFEngine that I developed as part of my work in Opera starting from 2013. To my surprise and despite its simplicity, it was so much appreciated by the community that I was encouraged to present it at FOSDEM and Configuration Management Camp in 2014, which I did with much to my satisfaction.

This year I presented an updated version of the same talk at the Software conference in Oslo and I was asked if the code was open source and available. It wasn’t yet, and the main reason was that I knew it would take an effort to generalize it, to abstract it from our environment and make it suitable to be used anywhere. And it did: if you look at the history it took 16 days and 24 commits before I was happy enough with the result to publish it. But I finally did and it’s now available on github. Now it’s your turn.

You can do a lot to help these contribution coming. Give them a chance. If they seem to solve your problem, try them; if they actually help you, contribute to them if you have a chance; if you don’t have a chance to contribute you can at least promote them.

The CFEngine community is rather small and doesn’t have the large ecosystem of tools that other CM communities can boast about. Despite that, pearls like EvolveThinking’s EFL, Delta Reporting and Delta Hardening or Normation’s NCF (to name only a very few!) are there: help them grow, help more tools come, support developers to keep up the good work, share and encourage sharing, contribute back if you can. More than anything else, please let’s stop reinventing the wheel every time and become good at sharing.

If you do, everybody wins: you win because you get more tools, and we win because we see that our contributions are useful and appreciated.

Thanks in advance
-- bronto

Safer package installations with APT and CFEngine

/ bronto / Leave a comment

CFEngineAgentPackage installation can be tricky sometimes when using configuration management tools, as the order in which package operations are performed can have an impact on the final result, sometimes a disastrous impact. Months ago I had been looking for a way to make apt-get a bit less proactive when trying to solve dependencies and removing packages and came up with the following package method that we now have in our library.

To use it, just put it in your policies and use apt_get_safe in your policies instead of apt_get wherever you want a more prudential approach to package installations.

I’m putting it here in the hope that it may be useful for everyone. I have used it successfully in Debian 5, 6, 7, 8 and on CFEngine 3.4.4 (where I borrowed parts of the masterfiles from 3.5 and 3.6 like, e.g., the debian_knowledge bundle) and 3.6.x. Enjoy!

Continue reading →

My round of conferences in February

/ bronto / Leave a comment

cfgmgmtcamp-logoThe first half of February is going to be quite busy to me with two conferences just a few weeks away now.

On February 2nd and 3rd I’ll be at the Configuration Management Camp in Ghent, Belgium, where I’ll hold the seminar “Many projects, one code” on the 3rd. A little more than a week later, the 12th of February, I’ll then be at Software 2015 in Oslo, where I’ll hold the seminar “The classification problem: challenges and solutions” in the Continuous delivery and DevOps track. And yes, the slides will be available on my SpeakerDeck account as soon as a seminar is done.

Continue reading →

A small leap for a clock, a giant leap for mankind

/ bronto / Leave a comment

TurnBackTimeIt’s going to happen again: IERS’ Bulletin C was published yesterday announcing that we’ll have a leap second at the end of June this year. It’s time to get ready, but we this time we are luckier than in 2012: leapocalypse’s scars were deep and still hurt, and only inexperienced sysadmins or experienced idiots will deny the need of preparing a strategy to mitigate the unavoidable bugs that will be triggered by the leap second insertion. Or even by the leap second announcement, like it happened in 2012.

Which bugs? Nobody knows. We have six months left during which you could review the source code of your OS’ and software. Or maybe not, but you can at least test what happens on your systems when the leap second is announced and when it is inserted.

Like in 2012, I’ll soon start testing the behavior of both ntpd and Linux. The test framework will be the same as last time, but with the added wisdom gained fighting against leapocalypse. This is what I plan to add to the picture:

  • more than one NTP test server: either three or four, to measure the convergence speed more accurately;
  • test under heavy load;
  • test on at least some of the software that we run;
  • test with the kernel discipline disabled to compensate for kernel bugs;
  • test on both stock ntpd and the recently released 4.2.8;
  • test on Debian Jessie, since it will be surely released before the leap second takes place;
  • prepare CFEngine policies to handle the leap second and test those as well.

Like last time I’ll share my results. If you’ll be doing similar tests in your environment, I’ll be more than glad to see your findings. This is going to affect us all, the more information we share, the more chances we have to overcome the bugs.

Image from http://writing.wikinut.com/img/3bguk3_7i7al8s9b/If-We-Could-Turn-Back-Time

How we shaved the poodle

/ bronto / 2 Comments

CFEngineAgentIn this post I’ll describe how we used CFEngine to apply fixes to apache and nginx to defuse the infamous poodle bug. The post is a bit rushed, in the hope it may still be useful to someone. The policies use bundles and bodies from either the standard library or from our own. The libraries are not shown here but the names speak for themselves… hopefully 🙂

As you’ll probably know, the “trick” on the server side is not to allow secure (erm…) connections to use anything older than TLSv1. In order to do that, we decided to

  • deploy a conf.d snippet to set the appropriate protocol versions as a default;
  • disable the same directive in existing configuration files to avoid weaker directives take priority;
  • restart the server if/when the configuration gets fixed.

Continue reading →

Test dummies on sale!

/ bronto / Leave a comment

DummySaleSomeone in the CFEngine community said that configuration management is a big hammer: you can manage a zillion of systems with ease. Or wreck them, with the same ease. Deploying configuration changes is a risky business. It takes responsibility. And testing, lots of testing.

The biggest shops in town™ have wonderful continuous integration systems, where every commit spawns a number of virtual machines where the changes are tested and won’t make it into the master branch unless they work correctly. In smaller shops most of the testing phase is done by hand, and it’s absolutely key to have the ability to destroy or create VMs in a snap, well configured and with all the necessary software already on board. We are that kind of small shop, we made that kind of system and we did it on the cheap. If you want to buy yourself one, just read on.

Continue reading →

cfe: agent runs made easier

/ bronto / Leave a comment

This is the third and final installment of a three-post series. In the first post, “git repository and deployment procedures for CFEngine policies”, I explained how we have structured our repository to efficiently manage many projects in one branch and share common policies across projects. In the second post, “cf-deploy: easier deployment of CFEngine policies“, I illustrated the script that we use to deploy the policies on our policy hubs and to preview the changes that would be applied to the policies themselves. In this final post I’ll illustrate yet another tool that helped us simplify the way we run the agent by hand on a node. Although one doesn’t need very often to run the agent by hand on production nodes, that is much more needed on test nodes and when debugging policies, and we do that often enough that it made sense to optimize the process. The principle is the same as the other script, cf-deploy: a makefile does the hard work, and we provide a convenient front-end to run make.

Continue reading →

cf-deploy: easier deployment of CFEngine policies

/ bronto / 2 Comments

Update: this article refers to the very first version of cf-deploy. For the latest release, check the github repository.

GitRepoStructureIn my latest post “git repository and deployment procedures for CFEngine policies” I explained how we structured our git repository for CFEngine policies, and how we built a deployment procedure, based on GNU make, to easily deploy different projects and branches from the same repository to the policy hubs. Please read that post if you haven’t yet, as this one is not going to make much sense without it.

The make-based deployment procedure worked pretty well and was functional, but still had annoyances. Let’s name a few:

  • the make command line was a bit long and ugly; usually it was something like:
    make -C /var/cfengine/git/common/tools/deploy deploy PROJECT=projX BRANCH=dev-projX-foo SERVER=projX-testhub
  • the Makefile was not optimized to deploy on more than one server at a time. To deploy the same files on several hubs, the only solution was to run make in a cycle several times, as in
    for SERVER in projX-hub{1..10} ; do make -C /var/cfengine/git/common/tools/deploy deploy PROJECT=projX BRANCH=dev-projX-foo SERVER=$SERVER ; done
  • deploying a project on all the policy hubs related to that project required one to remember all of the addresses/hostnames; forget one or more of them, and they would simply, hopelessly left behind.

At the same time, there were a few more people that were interested in making tiny changes to the configurations via ENC and deploy, and that long command line was a bit discouraging. All this taken together meant: I needed to add a multi-hub deployment target to the Makefile, and I needed a wrapper for the deployment process to hide that ugly command line.

For first, I added to the Makefile the functionality needed to deploy on more than one hub without having to re-create the temporary directory at every run: it would prepare the files once, deploy them as many times as needed, and then wipe the temporary directory. That was nice and, indeed, needed. But the wrapper couldn’t wait any longer, and I started working on it immediately after. That’s where cf-deploy was born.

Continue reading →

Git repository and deployment procedures for CFEngine policies

/ bronto / 4 Comments

This is the first installment of three, where I’ll talk about how we structured the git repository for our CFEngine policies together with the deployment policies. This first episode will be about how the repository was (badly) structured before and how we redid it (better), and it will introduce our deployment procedures based on GNU Make. The second installment will talk about how we built upon the deployment procedure and we made it easier. The third installment will be about how we greatly simplified how we manage agent runs by hand on our nodes, so that even the non-CFEngine-savvy can do the right thing with little to no knowledge of CFEngine.

Continue reading →

The classification problem: challenges and solutions

/ bronto / 2 Comments

Update March 1st, 2015: the latest version of the code for hENC is now on github

It’s been about a month since I came back from FOSDEM and cfgmgmtcamp, a month where I gradually recovered from the the backlog both in the office and at home. It’s been a wonderful experience, especially at cfgmgmtcamp, and I really want to thank all those that helped make it special — more details at the end of this article.

But promise is debt (no pun intended with promise theory here), and I promised to write a long blog post with some (or all) the details from my talks. It’s time to keep that promise. So, without any further ado…

Continue reading →