Concurrency in Go

In my quest to learn the Go language I am currently in the process of doing the Go Code Clinic. It’s taking me quite some time because instead of going through the solutions proposed in the course I try to implement a solution by myself; only when I have no idea whatsoever about how to proceed I peep into the solution to get some insight, and then work independently on my solution again.

The second problem in the Clinic is already at a non-trivial level: compare a number of  images with a bigger image to check if any of those is a “clipping” of the bigger one. I confess that I would have a lot to read and to work even if I was trying to solve it in Perl!

It took some banging of my head to the wall till I eventually solved the problem. Unfortunately my program is single-threaded and the process of matching images is very expensive. For example, it took more than two hours to match a clipping sized 967×562 pixels with it’s “base” image sized 2048×1536. And for the whole time only one CPU thread was running 100%, the others where barely used.If I really want to say that I solved the problem I must  adapt the program to the available computing power by starting a number of subprocesses/threads (in our case: goroutines) to distribute the search across several CPU threads.

Since this was completely new to me in golang, I decided to experiment with a much simpler program: generate up to 100 random integers (say) between 0 and 10000 and run 8 workers to find if any of these random numbers is a multiple of another number, for example 17. And of course the program must shut down gracefully, whether or not a multiple is found. This gave me a few problems to solve:

  • how do I start exactly 8 worker goroutines?
  • what’s the best way to pass them the numbers to check? what’s the best way for them to report back the result?
  • how do I tell them to stop when it’s time that they shut down?
  • how do I wait that they are actually shut down?

The result is the go program that you can find in this gist. Assuming that it is good enough, you can use it as a skeleton for a program of yours, re-implementing the worker part and maybe the reaper part if a boolean response is not enough. Enjoy!


2 thoughts on “Concurrency in Go

  1. Hi! I’m doing the same journey from perl to go, for mostly the same reasons. Added reasons is that I want a strictly typed language, partly because type checking at compile time. Other major reason is that I have been fighting with UTF8 and perl one time to many 😉

    I guess your journey have continued as this blogpost is almost a year old, but I want to inflict some thoughts about go routines.

    Go routines are not threads, (quick googling shows multiple resources telling why), so you might not need/want to start workers from number of cores. In this and other processor heavy programs it maybe correct, but if i/o or network waiting is a bottleneck it may be more effective starting a bigger pool of workers.

    But nice work, I learned a few tricks from this and some of your other blogposts.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.