In the last few months I have been working, together with a colleague, on an API client for several well-known systems and cloud providers. When we started, I was a novice in the Go programming language, I had no experience in programming API clients, and I trusted the makers of the APIs enough to have great expectations at them.
Today, a few months later, several hours programming later and a bunch of lines of code later, I am a better novice Go programmer, I have some experience in interfacing with APIs, and my level of trust in the API makers is well beneath my feet.
This article will be a not-so-short summary of the reasons why we started this journey and all the unexpected bad surprises we got along the way. Unfortunately, I will be able to show only snippets of the code we have written because I didn’t get the authorisation to make it public. I will make the effort to ensure that the snippets are good enough to help you get a better understanding of the topics.
OK, enough preface. It’s time to tell the story.
Errata corrige: it’s actually v3! This is what happens when you don’t publish updates for your software for too long…
I took some time this weekend to release an update for cf-deploy. You have now the option to override the configuration hardcoded in the script by means of environment variables. Check the README for the details.
If you don’t know what cf-deploy is, that’s fair 😉 In two words, it’s a Makefile and a Perl front-end to it that makes it easier to pack together a set of files for a configuration management tools and send them to a distribution server. Designed with git and CFEngine in mind, it’s general enough that you can easily adapt it to any version control system and any configuration management tool by simply modifying the Makefile. If it sounds interesting, you are welcome to read Git repository and deployment procedures for CFEngine policies on this same blog. Enjoy!
Back from the holiday season, I have finally found the time to publish a small library on GitHub. It’s called cfengine-tap and can help you writing TAP-compatible tests for your CFEngine policies.
TAP is the test anything protocol. It is a simple text format that test scripts can use to print out the results and test suites can consume. Originally born in the Perl world, it is now supported in many other languages.
Using this library it’s easier to write test suites for your CFEngine policies. Since it’s publicly available on GitHub and published under a GPL license, you are free to use it and welcome to contribute and make it better (please do).
Here’s another quick post about docker, sorry again if it will come out a bit raw.
In my previous post I talked about my first experiments with docker. There was a number of unanswered questions at first, which got an answer through updates to the blog post during the following days. All but one. When talking about a containerized process that needs to log through syslog to an external server, the post concluded:
if the dockerized process itself needs to communicate with a syslog service “on board”, this may not be enough…
Learning more of systemd has been on my agenda since the release of Debian 8 “Jessie”. With the new year I decided that I had procrastinated enough, I made a plan and started to study according to the plan. Today it was time for action: to verify my understanding of the documentation I read up to now, I decided to put together unit files for CFEngine. It was an almost complete success and the result is now on GitHub for everyone to enjoy. I would appreciate if you’d give them a shot and report back.
Main goals achieved:
- I successfully created three service unit files, one for each of CFEngine’s daemons: cf-serverd, cf-execd and cf-monitord; the units are designed so that if any of the daemon is killed for any reason, systemd will bring it back immediately.
- I successfully created a target unit file that puts together the three service units. When the cfengine3 target is started, the three daemons are requested to start; when the cfengine3 target is stopped, the three daemons are stopped. The cfengine3 target completely replaces the init script functionality.
Goal not achieved: I’ve given a shot at socket activation, so that the activation of cf-serverd was delayed until a connection was initiated to port 5308/TCP. That didn’t work properly: systemd tried to start cf-serverd but it died immediately, and systemd tried and tried again until it was too much. I’ll have to investigate if cf-serverd needs to support socket activation explicitly or if I was doing something wrong. The socket unit is not part of the distribution on GitHub but its content are reported here below. In case you spot any problem please let me know.
Today I am releasing the version 3 of hENC, the radically simple hierarchical External Node Classifier (ENC) for CFEngine (version 2 was released at the end of May and added support for data containers).
This version adds new features and bug fixes, namely:
! primitive is added to specify commands; three commands exist currenty:
!RESET_ACTIVE_CLASSES to make hENC forget about any class that was activated up to that point,
!RESET_CANCELLED_CLASSES ditto for cancelled classes, and
!RESET_ALL_CLASSES that makes hENC forget about any class that was activated or cancelled;
- fixed enc.cf, so that it is possible to run the henc module more than once during the same agent run;
- added a Changelog;
- improved tests: tests have been added for the new features and the whole test suite has been improved to support the TAP protocol; for example, it’s now it’s possible to use the
prove utility to verify if hENC works correctly on your system before trying the installation.
See the README and Changelog for more information.
hENC is a radically simple external node classificator for CFEngine that I developed as part of my work in Opera starting from 2013. To my surprise and despite its simplicity, it was so much appreciated by the community that I was encouraged to present it at FOSDEM and Configuration Management Camp in 2014, which I did with much to my satisfaction.
This year I presented an updated version of the same talk at the Software conference in Oslo and I was asked if the code was open source and available. It wasn’t yet, and the main reason was that I knew it would take an effort to generalize it, to abstract it from our environment and make it suitable to be used anywhere. And it did: if you look at the history it took 16 days and 24 commits before I was happy enough with the result to publish it. But I finally did and it’s now available on github. Now it’s your turn.
You can do a lot to help these contribution coming. Give them a chance. If they seem to solve your problem, try them; if they actually help you, contribute to them if you have a chance; if you don’t have a chance to contribute you can at least promote them.
The CFEngine community is rather small and doesn’t have the large ecosystem of tools that other CM communities can boast about. Despite that, pearls like EvolveThinking’s EFL, Delta Reporting and Delta Hardening or Normation’s NCF (to name only a very few!) are there: help them grow, help more tools come, support developers to keep up the good work, share and encourage sharing, contribute back if you can. More than anything else, please let’s stop reinventing the wheel every time and become good at sharing.
If you do, everybody wins: you win because you get more tools, and we win because we see that our contributions are useful and appreciated.
Thanks in advance
As the title says: the code for cf-deploy is now on github. Please ensure you read the README to understand the current limitations, and please help improving the tool.
The thing it’s lacking most is an external configuration file. Other useful additions could be Makefiles to support different version control systems other than git, and tools other than rsync for deployments.
During the past years I’ve published a few Perl modules of mine to CPAN. Nothing big, nothing special, just some small, simple modules that I published in the hope that they would be useful to more people than just me. That code lived, or rather slept, in my hard disk and was not shared anywhere than in CPAN.
At the end of May, a bug was opened against the Net::LDAP::Express module and I decided it was time to bring that code to year 2014. Now, and since a few days ago, you can find the code of all my modules in github. With the code shared on github I was able to share a fix, have it tested by the person who submitted the bug, and confirm the bug was solved. Since one hour ago, the bugfix release 0.12 of Net::LDAP::Express is available on CPAN (on metaCPAN only for now, will hit all the archives in the next few hours).
You are welcome to clone the code from github, fork, branch, open pull requests… Just share the code, make it better, help people, and don’t forget to have fun in the process!