Managing apt_preferences with CFEngine

October 4th, 2013 - 22:45October 5th, 2013 - 16:48 / bronto / Leave a comment

Since when I started using Linux Mint Debian Edition (LMDE), it has become increasingly important to have my apt_preferences set up correctly, so that I could pull packages from Debian testing (currently codenamed “jessie”) if I wanted, but use LMDE packages by default. It took me some time and research to find the right settings.

Once found the right settings, and discovered that I needed to “hardcode” the hostname of the LMDE mirror I was using, there came the problem to have the apt sources and apt preferences in sync. It seemed a nice task for CFEngine, and I started to work on managing my apt preferences. The first solution I had in mind didn’t work properly due to a bug in CFEngine that wasn’t yet fixed in version 3.4.4, the one I’m using. The solution below works. It’s a bit more cumbersome than the one that I had in mind, but it should be useful nonetheless 🙂 It’s an edit_line bundle:

# This bundle is used to compile an apt-preferences file (usually
# /etc/apt/preferences, or another file in /etc/apt/preferences.d.
# It receives in input the name of an array with the following
# structure:
#
# pin[id][Package] string => "package_identifier"
# pin[id][Pin]     string => "pin for this package"
# pin[id][Prio]    string => "priority for this package"
#
# We use preserve_block in insert_lines, so all the three items must
# be present (Package, Prio, and Pin). Besides, it is advisable that
# the whole file is managed from a blank state.

bundle edit_line apt_preferences(pins) {
  vars:
      "n" string => "$(const.n)" ;
      "ids" slist => getindices("$(pins)") ;

  insert_lines:
      "Package: $($(pins)[$(ids)][Package])$(n)Pin: $($(pins)[$(ids)][Pin])$(n)Pin-Priority: $($(pins)[$(ids)][Prio])$(n)",
	  insert_type => "preserve_block" ;

}

Put the bundle in one of your libraries, import it, and then you can use it like this:

bundle agent test {
  vars:
      "ids" slist => {"lmde","mint","jessie","wheezy"} ;

      "pin[lmde][Prio]"   int => "700" ;
      "pin[mint][Prio]"   int => "700" ;
      "pin[jessie][Prio]" int => "-1"  ;
      "pin[wheezy][Prio]" int => "1"   ;

      "pin[lmde][Pin]"   string => "origin debian.lth.se" ;
      "pin[mint][Pin]"   string => "origin packages.linuxmint.com" ;
      "pin[jessie][Pin]" string => "release n=jessie"  ;
      "pin[wheezy][Pin]" string => "release n=wheezy"   ;

      "pin[$(ids)][Package]" string => "*" ;

  files:
      "/tmp/prefs"
	  create        => "yes",
	  edit_defaults => empty,
	  perms         => owner("bronto"),
	  edit_line     => apt_preferences("test.pin") ;

}

The test bundle above will create a file like this:

Package: *
Pin: origin debian.lth.se
Pin-Priority: 700

Package: *
Pin: origin packages.linuxmint.com
Pin-Priority: 700

Package: *
Pin: release n=jessie
Pin-Priority: -1

Package: *
Pin: release n=wheezy
Pin-Priority: 1

Enjoy!

Making Skype work in LMDE update 7

September 29th, 2013 - 00:54September 29th, 2013 - 12:50 / bronto / 2 Comments

Even if my machines are all on a 64 bit architecture, I am using the 32 bit Skype version, as suggested. Recently, my laptops and workstation accidentally initiated the installation of the update 7 of Linux Mint Debian Edition. As a result, Skype audio was broken (again, sigh…). It took me an amount of research and trial-and-error to find the right combination of fixes that made things work (eh yes, one fix only was not enough 🙂

For starters, the package libasound2-plugins:i386 must be installed. And, as reported in several places (including Mint forums) you’ll need to fix PulseAudio’s configuration as follows:

in the file /etc/pulse/default.pa the line

load-module module-udev-detect

must be changed in

load-module module-udev-detect tsched=0

You’ll then need to restart pulseaudio. Try to kill it with:

pulseaudio -k

and verify that it’s actually down with:

pgrep -lf pulseaudio

(or pgrep -u your_username -lf pulseaudio in case you’re not the only active user of the system).

If it’s not gone, try to kill it again until it goes away. Then run

pulseaudio --start

and test: Skype should be working fine now.

Is it possible to teach CFEngine to apply this fix? Of course it is! This bundle does the job:

bundle agent fix_skype_audio {
  files:
      "/etc/pulse/default.pa"
      classes   => if_repaired("pulseaudio_skype_fix_applied"),
      edit_line => replace_line("\s*load-module module-udev-detect\s*",
                    "load-module module-udev-detect tsched=0") ;

  packages:
      "libasound2-plugins:i386"
    package_policy => "add",
    package_method => apt ;

  reports:
    pulseaudio_skype_fix_applied::
      "Pulseaudio fix applied, please restart pulseaudio!" ;
}

# Replace the content of a line matching the regex (anchored)
bundle edit_line replace_line(regex,content)
{
  replace_patterns:
      "^($(regex))$"
          replace_with => value("$(content)"),
          comment => "Search and replace string";
}

You’ll need to have the CFEngine standard libraries included in the inputs to make this work. Note that I preferred to just get a warning instead of letting CFEngine restart my pulseaudio. I prefer to choose myself the right moment to do that, so a warning is enough.

Enjoy!

Managing hundreds of Linux servers with CFEngine and Git: full control over configuration

September 5th, 2013 - 15:31September 7th, 2013 - 12:11 / bronto / Leave a comment

Remi Bergsma's blog

Back in June, just before I went off for holiday, I attended a CFEngine training in Amsterdam. When I returned from holiday a few weeks later, me and my team started making plans to implement CFEngine in our environment. After two months of hard work, I’m proud to say we manage about 350 out of our 400 Linux servers with CFEngine!

The ride has been fun, although not always easy. In this post I’ll give a quick overview of our CFEngine implementation, where I found useful info, etc.

CFEngine is different
To start, let me tell you that one of the most difficult parts of learning CFEngine is to get used to the terminology and to ‘think’ CFEngine. For example, a ‘class’ in CFEngine is not what you think it is. It has nothing to do with object oriented programming. It’s more like a ‘context’ that you can use…

View original post 1,269 more words

Who’s using my Perl modules?

August 1st, 2013 - 22:00August 2nd, 2013 - 19:26 / bronto / Leave a comment

blog-tpf_logo_150x79 It was ten years ago, when the version 0.01 of my module Net::LDAP::Express hit CPAN. And it was early December 2004 when the current version of it, 0.11, landed in the same place. It was my first public module, it was a simple one, it was hit by some criticism in the Perl community and in the perl-ldap community in particular. In short, I always thought that it was just a little, simple module that a very few people would use, and that would never be included in any mainstream product.

Today, for no special reason, I decided to run a search on Google, and I found out I was wrong. Delightfully wrong.

Net::LDAP::Express is used when installing Bugzilla on Windows
it’s part of the Netops toolkit
it’s used in the code samples in the book “Nagios 3 enterprise monitoring“
it’s used in ELSA (Enterprise log search and archive), and thus in Security Onion, a Linux distribution for IDS (Intrusion Detection) and NSM (Network Security Monitoring) — ELSA is included in Security Onion

It was strange to see how wrong I was, but nice!

What about you? Are you using any of my Perl modules in your software? If so, it would be so nice if you’d let me know! Continue reading →

How we upgraded a CFEngine cluster

July 23rd, 2013 - 20:38 / bronto / Leave a comment

This post describes how we performed, on a cluster of ours, an upgrade of both the CFEngine policies, and of the CFEngine software itself from 3.3.5 to 3.4.4. The tools we used for the job were an editor and meld to edit the policies and merge changes from different branches, and cfengine itself (in particular cf-runagent), and clusterssh to apply the changes. I don’t claim this is a perfect procedure, and it wouldn’t scale to very big clusters; but in our case, the cluster was small enough to allow for some manual operations, and we took the opportunity to do things step by step, ensuring that nothing bad could happen to the production nodes at any time. On bigger clusters, with a better test environment, and with more “instrumentation” to monitor the changes, handwork could be reduced further, and the whole update would take less time. In our case, the bigger part was done between the workday start and the lunch break (about 3 hours). Continue reading →

No leap second in December

July 8th, 2013 - 14:39 / bronto / Leave a comment

I’ve just got the Bulletin C from the IERS, and with good news: there won’t be a leap second in December. The next leapocalypse will have to wait at least until June 2014.

For your reference, here’s the full bulletin:

     INTERNATIONAL EARTH ROTATION AND REFERENCE SYSTEMS SERVICE (IERS) 

SERVICE INTERNATIONAL DE LA ROTATION TERRESTRE ET DES SYSTEMES DE REFERENCE

SERVICE DE LA ROTATION TERRESTRE
OBSERVATOIRE DE PARIS
61, Av. de l'Observatoire 75014 PARIS (France)
Tel.      : 33 (0) 1 40 51 22 29
FAX       : 33 (0) 1 40 51 22 91
Internet  : services.iers@obspm.fr

                                             Paris, 8 July 2013

                                             Bulletin C 46

                                             To authorities responsible
                                             for the measurement and 
                                             distribution of time             

                          INFORMATION ON UTC - TAI

 NO leap second will be introduced at the end of December 2013.
 The difference between Coordinated Universal Time UTC and the 
 International Atomic Time TAI is :		

     from 2012 July 1, 0h UTC, until further notice : UTC-TAI = -35 s

 Leap seconds can be introduced in UTC at the end of the months of December 
 or June,  depending on the evolution of UT1-TAI. Bulletin C is mailed every  
 six months, either to announce a time step in UTC, or to confirm that there 
 will be no time step at the next possible date.
                                            Daniel GAMBIS
                                            Director			
                                            Earth Orientation Center of IERS
					    Observatoire de Paris, France
__________________________________________

Sorry for the silence, but…

June 28th, 2013 - 00:00 / bronto / Leave a comment

You may have already read, either from Opera’s official statement or from third party news, that we had a security breach. As you can imagine, that has been keeping me (and my colleagues) very busy, and still is.

I had a post planned, explaining how we successfully executed both a reorganization of our policy structure, and, at the same time, an upgrade of CFEngine from version 3.3.5 to 3.4.4, but I’ll have to put it on hold until the dust settles. In the meanwhile, you can still follow me on twitter, where I find a few seconds each day to post some updates and suggestions for your sysadmin/devops life.

Thanks for your patience.

Classes don’t blink

June 6th, 2013 - 22:28June 8th, 2013 - 17:13 / bronto / 1 Comment

This post is about a CFEngine feature that is a bit counter-intuitive at first, and may leave you scratching your head for a while if you don’t know about that. And, seemingly, many don’t. I’ll talk about how CFEngine classe change their state, but especially I’ll talk about how they don’t change their state. Continue reading →

CFEngine to the masses

June 2nd, 2013 - 19:47June 3rd, 2013 - 14:57 / bronto / Leave a comment

With the aim of suggesting a way to bring CFEngine to the wider audience it deserves, I am launching a project called Dumbed-down CFEngine (or DDCFE for short, or DDC for shorter). It is a raw draft specification of a simple configuration management language, to be compiled into native CFEngine policies. There is no software yet, rather: it’s a call for action to build one because, to be honest, I have neither the resources nor the qualifications to build it all alone in a reasonable amount of time.

Continue reading →

The value of the promisee in cfengine promises

May 21st, 2013 - 14:58May 21st, 2013 - 15:02 / bronto / 2 Comments

A lesson I learnt today is: use promisees in cfengine promises, and you won’t regret it. In fact, I had a proof today that specifying a promisee is a very useful thing.

Let’s take a step back: a promisee is introduced by a -> sign, and according to the documentation the promisee is the abstract object to whom the promise is made. When CFEngine made its way in our trouble ticket, I started including a ticket number as a promisee every time a promise was created to address a reported issue.

Today I found out that, every time I deployed a change that caused firewall rules to be reloaded, ntpd was also restarted. I couldn’t understand why, so I checked the policies and found this:

      "/etc/init.d/ntp" -> "CLO-2313"
	  args       => "restart",
          ifvarclass => "ntp",
	  comment    => "Restart ntpd when the firewall is restarted" ;

CLO-2313 was a resolved ticket, and there I found out why I made such a decision more than one month ago, and why it was a good thing.

So… use promisees!

A sysadmin's logbook

in every challenge there is an opportunity

Sysadmin