Compact, plain-text output for tree

I needed to remove all fancy ANSI graphics and colors from the output of tree, and just print something that fit a plain text file. This did the trick:

tree -L 1 --noreport -S -n --charset=US-ASCII

E.g.:

bronto@brabham:/var/cfengine/git/common$ tree -L 1 --noreport -S -n --charset=US-ASCII
.
|-- controls
|-- libraries
|-- modules
|-- services
|-- sources
|-- templates
|-- tools
|-- unit
`-- update.cf

Tracking down memory hogs on Linux

I am working hard on my talk for FOSDEM, and don’t have much time to write detailed posts, so you’ll have to put up with me and be contempt with this “one liners” series 😉

In a memory shortage situation, it’s handy to understand which processes are memory hogs. Not only the ps command can tell you that: it can even return the process list sorted by memory occupation. At that point, getting the “top N” is a piece of cake. The following example shows the top 3 on my machine this morning, after shutting down and restarting a few of them:

bronto@brabham:~$ ps -e -o pid,user,rss,vsz,args --sort -vsz | head -n 4
  PID USER       RSS    VSZ COMMAND
27971 bronto   213896 3099640 /usr/bin/python /usr/bin/hotot
27884 bronto   44572 2306604 liferea
24470 bronto   202724 2164244 /usr/bin/gnome-shell

Hope it helps!

Managing apt_preferences with CFEngine

Linux Mint Debian EditionSince when I started using Linux Mint Debian Edition (LMDE), it has become increasingly important to have my apt_preferences set up correctly, so that I could pull packages from Debian testing (currently codenamed “jessie”) if I wanted, but use LMDE packages by default. It took me some time and research to find the right settings.

Once found the right settings, and discovered that I needed to “hardcode” the hostname of the LMDE mirror I was using, there came the problem to have the apt sources and apt preferences in sync. It seemed a nice task for CFEngine, and I started to work on managing my apt preferences. The first solution I had in mind didn’t work properly due to a bug in CFEngine that wasn’t yet fixed in version 3.4.4, the one I’m using. The solution below works. It’s a bit more cumbersome than the one that I had in mind, but it should be useful nonetheless 🙂 It’s an edit_line bundle:

# This bundle is used to compile an apt-preferences file (usually
# /etc/apt/preferences, or another file in /etc/apt/preferences.d.
# It receives in input the name of an array with the following
# structure:
#
# pin[id][Package] string => "package_identifier"
# pin[id][Pin]     string => "pin for this package"
# pin[id][Prio]    string => "priority for this package"
#
# We use preserve_block in insert_lines, so all the three items must
# be present (Package, Prio, and Pin). Besides, it is advisable that
# the whole file is managed from a blank state.

bundle edit_line apt_preferences(pins) {
  vars:
      "n" string => "$(const.n)" ;
      "ids" slist => getindices("$(pins)") ;

  insert_lines:
      "Package: $($(pins)[$(ids)][Package])$(n)Pin: $($(pins)[$(ids)][Pin])$(n)Pin-Priority: $($(pins)[$(ids)][Prio])$(n)",
	  insert_type => "preserve_block" ;

}

Put the bundle in one of your libraries, import it, and then you can use it like this:

bundle agent test {
  vars:
      "ids" slist => {"lmde","mint","jessie","wheezy"} ;

      "pin[lmde][Prio]"   int => "700" ;
      "pin[mint][Prio]"   int => "700" ;
      "pin[jessie][Prio]" int => "-1"  ;
      "pin[wheezy][Prio]" int => "1"   ;

      "pin[lmde][Pin]"   string => "origin debian.lth.se" ;
      "pin[mint][Pin]"   string => "origin packages.linuxmint.com" ;
      "pin[jessie][Pin]" string => "release n=jessie"  ;
      "pin[wheezy][Pin]" string => "release n=wheezy"   ;

      "pin[$(ids)][Package]" string => "*" ;

  files:
      "/tmp/prefs"
	  create        => "yes",
	  edit_defaults => empty,
	  perms         => owner("bronto"),
	  edit_line     => apt_preferences("test.pin") ;

}

The test bundle above will create a file like this:

Package: *
Pin: origin debian.lth.se
Pin-Priority: 700

Package: *
Pin: origin packages.linuxmint.com
Pin-Priority: 700

Package: *
Pin: release n=jessie
Pin-Priority: -1

Package: *
Pin: release n=wheezy
Pin-Priority: 1

Enjoy!

Making Skype work in LMDE update 7

Linux Mint Debian EditionEven if my machines are all on a 64 bit architecture, I am using the 32 bit Skype version, as suggested. Recently, my laptops and workstation accidentally initiated the installation of the update 7 of Linux Mint Debian Edition. As a result, Skype audio was broken (again, sigh…). It took me an amount of research and trial-and-error to find the right combination of fixes that made things work (eh yes, one fix only was not enough 🙂

For starters, the package libasound2-plugins:i386 must be installed. And, as reported in several places (including Mint forums) you’ll need to fix PulseAudio’s configuration as follows:

in the file /etc/pulse/default.pa the line

load-module module-udev-detect

must be changed in

load-module module-udev-detect tsched=0

You’ll then need to restart pulseaudio. Try to kill it with:

pulseaudio -k

and verify that it’s actually down with:

pgrep -lf pulseaudio

(or pgrep -u your_username -lf pulseaudio in case you’re not the only active user of the system).

If it’s not gone, try to kill it again until it goes away. Then run

pulseaudio --start

and test: Skype should be working fine now.

Is it possible to teach CFEngine to apply this fix? Of course it is! This bundle does the job:

bundle agent fix_skype_audio {
  files:
      "/etc/pulse/default.pa"
      classes   => if_repaired("pulseaudio_skype_fix_applied"),
      edit_line => replace_line("\s*load-module module-udev-detect\s*",
                    "load-module module-udev-detect tsched=0") ;

  packages:
      "libasound2-plugins:i386"
    package_policy => "add",
    package_method => apt ;

  reports:
    pulseaudio_skype_fix_applied::
      "Pulseaudio fix applied, please restart pulseaudio!" ;
}

# Replace the content of a line matching the regex (anchored)
bundle edit_line replace_line(regex,content)
{
  replace_patterns:
      "^($(regex))$"
          replace_with => value("$(content)"),
          comment => "Search and replace string";
}

You’ll need to have the CFEngine standard libraries included in the inputs to make this work. Note that I preferred to just get a warning instead of letting CFEngine restart my pulseaudio. I prefer to choose myself the right moment to do that, so a warning is enough.

Enjoy!

A story of faulty hard disks, kernel panics, and broken pins

When I got a new laptop from my employer, I decided I wanted to use a pure GNOME 3 desktop. The only distribution to offer it out of the box was Fedora, which I didn't really want to use, so I took a completely different path: install Linux Mint Debian Edition with Cinammon, and install GNOME 3 on top of it.

And it is working great! On "my" Dell Latitude 6430u everything works but the Bluetooth adapter. I had some freezes from time to time, which I suspect are due to some bug in the video drivers, but it happened so rarely that it wasn't really a problem. Until yesterday. … Continue reading

How to create a debian package for Riak on Wheezy

Riak is a distributed database that we use in a project of ours. The project is being migrated from a difficult setup on Debian Squeeze to a easier one on Debian Wheezy; unfortunately, while an official Debian package for squeeze is available, one for Wheezy is not. None of the ubuntu packages works on Wheezy either, due to dependencies.

I have searched a lot to see if there was a way to create a Debian package for Riak 1.3.0 on Wheezy. I found a few references, but not a complete one. I finally succeeded in creating the package, and I am writing a four step guide here.

Step 1: install an handful of prerequisites:
As root (or via sudo, if it's already installed) run:

apt-get install libc6-dev build-essential debhelper git sudo devscripts erlang-nox erlang-reltool erlang-dev

Step 2: clone the git repository and checkout 1.3:

cd /usr/local/src
git clone git://github.com/basho/riak.git
cd riak
git checkout 1.3

Step 3: compile

make package RELEASE=1

Step 4: fetch the package
You'll find the package in package/packages. Now you can install it using dpkg.

Enjoy!

An eye on the clock

Recently I was trying to make sane a system clock that, for some reason, suddenly slowed down to a crawl. I started to fiddle with adjtimex, and I needed a way to verify the reaction of the clock itself, and ntpd’s. On one window, I had a watch ntpq -c pe -c as running. On another, there was an ntpdate -q in a loop. In a third one, I wanted to monitor the changes in frequency (set by ntpd) and ticks (set by me). I found this one-liner pretty useful:

# while true ; do adjtimex -p | perl -alne '/frequency/ and $f=sprintf("%3.2f%%",100*$F[1]/32768000) ; /tick/ and $t=$F[1]-10000 ; END { print scalar(localtime),"\tf=$f\tt=$t" }' ; sleep 30 ; done

This snippet assumes that the frequency tolerance of the clock is 32768000, and that the normal value for ticks is 10000. Check the output of adjtimex -p and the man page to verify this fits your system, too.

Know your tools: adjtimex

Cosimo was one of the speakers at Velocity Europe 2012 last week. He gave a nice talk, and he even dared to show my face in his presentation (slide 32 if you're curious).

However, what caught my attention was the very next slide, where it was evident that he was blindly installing adjtimex on all his puppet-managed nodes. I warned him that it could be a bad idea, and he seemed to be quite surprised. Sadly, I was proved to be correct just a few hours later: they had an handful of virtual machines whose clocks were going crazy in all possible ways, and the root cause was tracked down to be the installation of the adjtimex package.

But let's take a step back: what is adjtimex, and why it can switch from a useful tool to an evil rapist of system clocks?
Continue reading

Turning the wireless ESSID into a class

I think it could be handy to have a class in cfengine that tells us something about the ESSID of the wireless network we are connected to. Unfortunately, cfengine doesn't define one by default, and I thought it could be a good idea to make a module for that. It has been buzzing in my head for a while, and tonight I took some 30 minutes to make it happen.

On Linux, we have the command iwgetid that tells us the ESSID of the wireless networks we are connected to. However, ESSIDs may contain characters that are not allowed in a class name, so we need to "escape" them. The following, tiny bash script reads the output of iwgetid and returns the corresponding class names to be defined:

#!/bin/bash

PATH="/sbin:/usr/sbin:/usr/bin:/bin"

for ESSID in $( iwgetid --raw )
do
  CESSID=$( echo $ESSID | tr -c "a-zA-Z0-9_" "_" | sed -e 's/_*$//' )
  CLASS="essid_${CESSID}"

  echo "+${CLASS}"
done

exit 0

On the laptop I am currently using, it returns:

root@cooper:/var/cfengine/inputs# modules/essid.sh 
+essid_WiFiOslo

And here it is! When we use this script as a module in cfengine, it will define a class named essid_WiFiOslo.

For more information about cfengine's module protocol, see the reference.