My first puppet facts

An apparently simple problem: have puppet manage an host's own entries: the localhost one, and the hostname's one. Well, we have plenty of facts to help us with this: we have ipaddress, and hostname, and fqdn. It should be simple, right?

But think. What happens if the host is multihomed? What if one of the interfaces is a bond interface? Does facter choose a value for ipaddress in a smart way?

Unfortunately, it doesn't. And to make it smart, you have to feed it the right thing. And to feed it, you need to write code in Ruby. Oh oh…

I decided to give it a try, and to pull in some shell script to actually do the job (bad thing, I know, but I really don't have time to learn ruby; and when I'll use some time to learn another programming language, I'd go through python and perl 6 first). … Continue reading

Too late to fix it

We installed a new datacenter right before I went on vacation, and of course we set up an NTP synchronization subnet there. As always, we configured four NTP multicast servers, and the rest as clients (we are talking about several hundreds of servers). The servers were running Debian Linux "Squeeze", while the vast majority of the clients was running Debian Linux "Lenny". For reasons I am not going to discuss here, using Squeeze instead of Lenny is not an option.

Right after the configuration was done, I noticed a really odd thing: all clients displayed a poll interval of 1024 seconds for one of the servers. This is just nonsense, as each server sends an NTP packet every 64 seconds. Anyway, the clients were in good sync so I decided I would investigate this after my vacation. And so I did. … Continue reading

Using rrdgraph for better NTP monitoring

Munin is a great tool, and it seems quite easy to monitor how your NTP service is going overall. E.g.: it's easy to put a web page together with all the offset graphs for your servers.

Unfortunately, this is far from optimal. In fact, the graphs will have different scales on the y axis, so a glance is not enough to check how they are doing overall. You'll actually need to check which values are displayed at the left side of the graph. This is annoying, because if you don't pain enough attention, you could miss bad things happening.

That's why I threw a reluctant eye to rrdtool's graph stuff. I've always been scared by the apparent complexity of the syntax, but I found out that what I needed was easy indeed. … Continue reading

Testing Oracle Solaris 11 Express

I've been testing Oracle Solaris 11 Express recently. For those who don't remember it, Oracle acquired Sun Microsystems :rip: and killed OpenSolaris :rip: with no official statement, the only information about the process was a leaked internal note (I leave it to you to decide whether that that leakage was real, and if it was intentional or not).

Solaris 11 Express is what remains of OpenSolaris after Oracle decided how they should move forward with it.

The immediate change you may notice in case you want to download and test it is that the license has changed, and that you are not allowed to download it unless you explicitly accept the license. Up to my knowledge, the license allows you to use it for free for personal use, otherwise you need to buy some sort of support; I didn't investigate this further because, well, I am interested in it for personal use at the moment. Why? Well, many reasons. … Continue reading

Building a Xen firewall with Firewall Builder

Original post date: March 30th, 2011
Updated: April 4th, 2011 (missing rule in prolog)

It's a kind of a problem to manage a firewall for a Xen dom0 with firewall builder. Xen itself adds forwarding rules when starting a virtual machines, and these rules are wiped away when fwbuilder scripts install theirs, which is unfortunate.

Summing up everything, the final plan was to install a firewall on dom0, which should a) forward to the VM the packets originating outside and directed to them (and back), and b) protect the dom0 itself.

It took me some time, experiments and advice to get it right, and here's how. … Continue reading

/proc/sys/net/ipv4/conf/all (and source routing in particular)

I was trying to find out if certain Linux machines allowed source routing or not. To do this, it should be enough to just take a peek into /proc/sys/net/ipv4/conf/*/accept_source_route. But, as it seems, the results are contradictory. Look at what I have on my workstation:

root@brabham:/proc/sys/net/ipv4/conf# for IF in * ; do echo -n "$IF: " ; cat $IF/accept_source_route ; done
all: 0
default: 1
eth0: 1
lo: 1
vboxnet0: 1

Now, if you look up how the settings in the "all" directory actually work, you'll see that a few talk about that, and those who do don't give much detail. This always rings a bell in my head, as it is a symptom that nobody actually knows exactly how it works, and those who do just don't write enough 🙂 E.g., a source writes:

The /proc/sys/net/ipv4/conf/ directory allows each system interface to be configured in different ways, including the use of default settings for unconfigured devices (in the /proc/sys/net/ipv4/conf/default/ subdirectory) and settings that override all special configurations (in the /proc/sys/net/ipv4/conf/all/ subdirectory).

Now, the last sentence does not make sense if you read it carefully, because that would make the per-interface subdirectories useless. In fact, if this was really the case, you could not, e.g., accept source routing on one interface and disable it on all the others, because the value in "all" would have priority.

So after some research I tried to ask to the most Linux informed person I know, one that I can safely mention without having to check the sources. And he told me that the "all" subdirectory was born as a shortcut to help changing the configuration on all interfaces simultaneously, but then evolved in an incoherent fashion, to the point that different settings work in different way (e.g.: one setting in all may override the per-interface setting, while others will do a boolean OR between all and the interface-specific value, while others again will do an AND). If you want to know how it goes in a specific case, you need to read the kernel code.

OK, now I have the confirmation that /proc/sys/net/ipv4/conf/*/accept_source_route is a minefield, but I did not make a step forward. In my specific case, that is the accept_source_route setting, how does it work?

Well, it turns out that it's a logical AND between "all" and the interface-specific value. Good to know.

Thanks Francesco 😉