programming

Bash scripting: using ‘read’ without a loop

/ bronto / Leave a comment

Another post in the “note to myself” style.

For anyone who does bash scripting, the command read is a well known tool. A usual task that we use read for it is to process the output of another command in a while loop, line by line, picking up a few fields and doing something with them. A stupid example:

sysctl -a 2> /dev/null | grep = | while read PARM DUMMY VALUE
do
  echo "Value for $PARM is $VALUE"
done

That is: we read the output of sysctl, line by line, selecting only the lines that contain a = sign, then read the name of the setting and its value in PARM and VALUE respectively, and do something with those values. So far so good.

Based on what we have just seen, it’s easy to expect that this:

echo foobar 42 | read PARM VALUE
echo "Value for $PARM is $VALUE"

would print “Value for foobar is 42“. But it doesn’t:

Value for  is

So, where did those values go? Did read work at all? In hindsight I can tell you: yes, it worked, but those values have disappeared as soon as read was done with them. To both parse them and use them you have to run both read and the commands using the variables in the same subshell. This works:

echo foobar 42 | ( read PARM VALUE ; echo "Value for $PARM is $VALUE" )

Or even

echo foobar 42 | (
    read PARM VALUE
    echo "Value for $PARM is $VALUE"
)

This will print “Value for foobar is 42”, as expected.

Advertisement

Experimenting with Python

/ bronto / Leave a comment
The SRE team at RiksTV, 2020-2021

In our team at RiksTV, the company I joined in March 2020, we use Python. I had never used Python before and I’m working as hard as I can to fill the gap.

During the Christmas break I assigned myself a small coding challenge, both to test what I have learned so far and to avoid that new knowledge to be washed away. I decided to share that code, and will continue sharing as I keep learning and whenever I make something that could be useful to more people than just myself. Head to github if you are interested.

Enjoy!

cf-deploy v4 released

/ bronto / Leave a comment

After five years after the release of cf-deploy v3, I have just released cf-deploy v4. This version of cf-deploy fixes a number of shortcomings that made their way up to this point and that I wasn’t able to see until recently. It is now more flexible and easier to configure than it ever was. In particular, the documentation is way more comprehensive, covering installation, configuration and usage. The documentation also covers some of the internals, that will allow the hardcore user to fine tune the tool to better suit their needs.

You will find cf-deploy on github, as always. Enjoy!

Continue reading →

A multi-platform API client in Go

/ bronto / Leave a comment

In the last few months I have been working, together with a colleague, on an API client for several well-known systems and cloud providers. When we started, I was a novice in the Go programming language, I had no experience in programming API clients, and I trusted the makers of the APIs enough to have great expectations at them.

Today, a few months later, several hours programming later and a bunch of lines of code later, I am a better novice Go programmer, I have some experience in interfacing with APIs, and my level of trust in the API makers is well beneath my feet.

This article will be a not-so-short summary of the reasons why we started this journey and all the unexpected bad surprises we got along the way. Unfortunately, I will be able to show only snippets of the code we have written because I didn’t get the authorisation to make it public. I will make the effort to ensure that the snippets are good enough to help you get a better understanding of the topics.

OK, enough preface. It’s time to tell the story.

Continue reading →

The things I wish I knew before I started using Golang with JSON

/ bronto / 4 Comments

A sign sold on EbayThis is not an article about how you can work with JSON in Go: you can easily learn that from the articles and web pages in the bibliography. Rather, this post is about the concepts that you must understand clearly before you set yourself for the task. Don’t sweat, it’s just two concepts two, and I’ve tried to explain them here.

In the last few weeks I have worked together with a colleague to write some automation with Golang and the Atlassian Crowd API. With several separate user databases (and, at the current state, no hope to unify them in a smart way) it would be very handy to take advantage of the APIs offered by, say, G Suite to fetch all the email addresses related to a user and use that information to automatically deactivate that user from all systems.

Coming from a Perl 5 background, I was hoping that decoding and encoding JSON in Go was as simple as it is in Perl.  But it turns out that it wasn’t, and it’s obvious if you think about it: as Perl 5 is weakly typed, decoding any typed data into an “agnostic” data structure must be simple. Encoding a weakly typed data structure into a typed format may be a bit trickier, but as long as you don’t have too many fancy data (i.e., in this context: strings made of only digits or non-obvious boolean representations) this will also work well. But with strongly typed Go and struct field names having side effects depending on upper-/lowercase, that’s a different story.

As it often happens in cases like this, you will not find all the information you need in a single place. This is my attempt to collect it all and hand it to you, so that you won’t have to waste as much time as I did. You will still have to read through stuff though.

Continue reading →

Concurrency in Go

/ bronto / 2 Comments

In my quest to learn the Go language I am currently in the process of doing the Go Code Clinic. It’s taking me quite some time because instead of going through the solutions proposed in the course I try to implement a solution by myself; only when I have no idea whatsoever about how to proceed I peep into the solution to get some insight, and then work independently on my solution again.

Continue reading →

Perl to go

/ bronto / 3 Comments

I have been using Perl for more than 20 years now, seen Perl 4 bow out and Perl 5 come in and develop in that fantastic language that has helped me uncountable times in my professional life. During those years I’ve also considered learning another language, but I have been unable to take a stand for a long time.

And there came Go and the hype around Go, just like years ago there was a lot of hype around Java. But while whatever written in Java I came across was a big, heavy and slow memory eater, most of the tools I came across that were written in Go were actually good stuff — OK, still a bit bloated in size, but they actually worked. The opportunity came, and I finally gave Go a shot.

Continue reading →

New home for my code, new release

/ bronto / Leave a comment

Perl (onion)During the past years I’ve published a few Perl modules of mine to CPAN. Nothing big, nothing special, just some small, simple modules that I published in the hope that they would be useful to more people than just me. That code lived, or rather slept, in my hard disk and was not shared anywhere than in CPAN.

At the end of May, a bug was opened against the Net::LDAP::Express module and I decided it was time to bring that code to year 2014. Now, and since a few days ago, you can find the code of all my modules in github. With the code shared on github I was able to share a fix, have it tested by the person who submitted the bug, and confirm the bug was solved. Since one hour ago, the bugfix release 0.12 of Net::LDAP::Express is available on CPAN (on metaCPAN only for now, will hit all the archives in the next few hours).

You are welcome to clone the code from github, fork, branch, open pull requests… Just share the code, make it better, help people, and don’t forget to have fun in the process!