From AWS instance IDs to private DNS names

github-logoJust a small bash snippet for those cases where, for example, a command returns AWS instance IDs but not the matching DNS names or an IP addresses. The function id2dns, that you can add to your .bashrc file, will do the translation for you. In order to use the function you will:

  • ensure you have the aws CLI installed and functional;
  • ensure you have jq command available;
  • ensure you have valid AWS credentials set, so that your aws CLI will work.


Update 2020-08-14: jq not needed any more


A quick guide to encrypting an external drive

luks-logoI am guilty for not having considered encrypting my hard drives for too long, I confess. As soon as I joined Telenor Digital (or, actually,ย early in the process but a bit too late…) I was commanded to encrypt my data and I couldn’t delay any more. To my utter surprise, the process was surprisingly simple in my Debian jessie! Here is a short checklist for your convenience.

Continue reading

cfe: agent runs made easier

This is the third and final installment of a three-post series. In the first post, โ€œgit repository and deployment procedures for CFEngine policiesโ€, I explained how we have structured our repository to efficiently manage many projects in one branch and share common policies across projects. In the second post, “cf-deploy: easier deployment of CFEngine policies“, I illustrated the script that we use to deploy the policies on our policy hubs and to preview the changes that would be applied to the policies themselves. In this final post I’ll illustrate yet another tool that helped us simplify the way we run the agent by hand on a node. Although one doesn’t need very often to run the agent by hand on production nodes, that is much more needed on test nodes and when debugging policies, and we do that often enough that it made sense to optimize the process. The principle is the same as the other script, cf-deploy: a makefile does the hard work, and we provide a convenient front-end to run make.

Continue reading

cf-deploy: easier deployment of CFEngine policies

Update: this article refers to the very first version of cf-deploy. For the latest release, check the github repository.

GitRepoStructureIn my latest post “git repository and deployment procedures for CFEngine policies” I explained how we structured our git repository for CFEngine policies, and how we built a deployment procedure, based on GNU make, to easily deploy different projects and branches from the same repository to the policy hubs. Please read that post if you haven’t yet, as this one is not going to make much sense without it.

The make-based deployment procedure worked pretty well and was functional, but still had annoyances. Let’s name a few:

  • the make command line was a bit long and ugly; usually it was something like:
    make -C /var/cfengine/git/common/tools/deploy deploy PROJECT=projX BRANCH=dev-projX-foo SERVER=projX-testhub
  • the Makefile was not optimized to deploy on more than one server at a time. To deploy the same files on several hubs, the only solution was to run make in a cycle several times, as in
    for SERVER in projX-hub{1..10} ; do make -C /var/cfengine/git/common/tools/deploy deploy PROJECT=projX BRANCH=dev-projX-foo SERVER=$SERVER ; done
  • deploying a project on all the policy hubs related to that project required one to remember all of the addresses/hostnames; forget one or more of them, and they would simply, hopelessly left behind.

At the same time, there were a few more people that were interested in making tiny changes to the configurations via ENC and deploy, and that long command line was a bit discouraging. All this taken together meant: I needed to add a multi-hub deployment target to the Makefile, and I needed a wrapper for the deployment process to hide that ugly command line.

For first, I added to the Makefile the functionality needed to deploy on more than one hub without having to re-create the temporary directory at every run: it would prepare the files once, deploy them as many times as needed, and then wipe the temporary directory. That was nice and, indeed, needed. But the wrapper couldn’t wait any longer, and I started working on it immediately after. That’s where cf-deploy was born.

Continue reading

Compact, plain-text output for tree

I needed to remove all fancy ANSI graphics and colors from the output of tree, and just print something that fit a plain text file. This did the trick:

tree -L 1 --noreport -S -n --charset=US-ASCII


bronto@brabham:/var/cfengine/git/common$ tree -L 1 --noreport -S -n --charset=US-ASCII
|-- controls
|-- libraries
|-- modules
|-- services
|-- sources
|-- templates
|-- tools
|-- unit

…and promise_summary.log met Solarized

Inspired by the blog post at, I’ve spent a few days hacking on promise_summary.log and rrdtool, and I have finally something to show.

Let’s recap quickly: a file promise_summary.log in cfengine’s workdir (usually /var/cfengine) contains a summary of every agent run: when the run started and finished, which version of the policy has run, and how many promises (in percentage) were kept, repaired, and not repaired. The first thing I wanted were graphs for these metrics; a medium-term goal is to bring these metrics into some well known tool like Munin or Cacti — that will come later.

I chose to use RRDtool for many reasons; among all:

  • it takes care for both saving the data and making the graphs;
  • it saves the data at different resolutions, automatically;
  • all aspects of a graph are customizable, and different type of graphs can be embedded in the same picture

I had previous experience with RRDtool, and I knew the downsides of course, mainly: the odd, cryptic syntax. What I had forgotten since such a long time was that it’s actually easier than it looks ๐Ÿ™‚ … Continue reading

How to count the occurrences of “something” in a file

This is something we need to do from time to time: filter some useful information from a file (e.g., a log file) and count the instances of each "object". While filtering out the relevant information may be tricky and there is no general rule for it, counting the instances and sorting them is pretty simple.

Let's call "filter" the program, chain of pipes, or whatever you used to extract the information you are longing for. Then, the count and sort process is just a chain of three commands:

filter | sort | uniq -c | sort -nr

The first sort is used to group equal instances together; then this is passed to uniq -c, which will "compress" each group to a single line, prefixed by the number of times that instance appeared in the output; then we feed all this to sort again, this time with -n (numeric sort) and -r (reverse sort, so that we have the most frequent instance listed first).

Nice, isn't it? ๐Ÿ˜‰