Assembling my motley crew of Raspberry Pi systems into a cluster computing group.
The inspiration for this post (and this project) came from something that I recently read in the Raspberry Pi blog. I would like to start out by saying if you are interested in computers, programming, DIY electronics, space exploration, or just technology in general, you are very likely to find the Raspberry Pi blog interesting and entertaining.
The project I am starting here is based on OctaPi: Cluster Computing and Cryptology. It’s basically a way of connecting and configuring a number of Raspberry Pi systems into a cluster of servers with a single client to control them and dispatch jobs to them. Of course, the project as described in that blog post, and in the detailed Build an OctaPi instructions, is very professional looking, using nine identical Raspberry Pi 3 systems, each with an LED HAT (blinking lights always make a project look better) , and the whole thing is mounted on a colorful perspex board.
What I hope to build will be made up of the various Raspberry Pi systems I have scattered about on my desk, ranging from the original Model B to the latest Zero W, and no two are the same. Also, the original project uses a 10-port power hub — wouldn’t that be nice to have? I will be using whatever power supplies I can find around here, so it will be a challenge to get them all plugged in and keep the cables reasonably neat (and the circuit breakers in my house intact) .
Getting started
The first step in this project is simply getting together the necessary materials, and making sure the software is up to date.
It is not necessary to create an ‘OctaPi’ cluster with exactly eight Raspberry Pi servers. You can use pretty much as many as you like, although only one server couldn’t really be called a cluster, and by the time you get to eight or so you are likely to be struggling to power them all. I am going to use the following, which is determined by one simple criterion — it’s all that I’ve got:
Some of these are already running Raspbian, so I will just have to make sure they are updated. Others are running Fedora, PiCore, Manjaro, Kali, or openSUSE, so I have more work to do to prepare them.
I will be using a TP-Link WR802N Nano router, because it is small, simple and inexpensive (less than 30 CHF) .
I strongly (very strongly) recommend you do not use your home wi-fi network for this project. While it might well work for you, it is likely to expose you to some serious security risks. The instructions for this project use open SSH connections with default user names and passwords, for example. Not a good idea at all.
One of the more mundane aspects of this build is going to be powering all of the Pi systems. The original project mentions a very spiffy Universal 10 port USB charging Hub, which is capable of delivering 2.4A simultaneously to all ten ports. Zowie! That’s a nice piece of kit for a project like this — unfortunately, I can’t find anything like it here in Switzerland.
To be sure what capacity power supplies I will need, I checked the Raspberry Pi FAQs, where there is a very handy table listing the power requirements for each model. Now I just have to see if I can find some reasonable multi-port power supply, or if I will have to cobble together a bunch of separate power supplies.
Finally, each of the Raspberry Pis that I use will need a wireless network connection. The Pi 3 and Zero W have that built in, of course, but for the other five I will need USB WiFi dongles. I’m not sure that I have enough of those laying around, but if necessary I can get a very low-priced 802.11b/g/n dongle from my friends at the Pi-Shop.ch. One small piece of advice on this — when buying a USB wi-fi dongle, make sure that it has already been tested with Raspbian.
First step – the client
The first Pi system I will set up is the one which will be used to manage the cluster — the project description refers to this one as the OctaPi Client. It is the only system which will have a keyboard, mouse, and display in the final setup, and it will be used to monitor the cluster servers, dispatch and manage jobs to them, and to reboot or shut them down.
I am going to use my oldest Raspberry Pi Model B for this task, because there isn’t much work to be done by this system. I want to keep the systems which have faster CPUs and more memory for use as servers in the cluster.
This one is already running Raspbian, so I just have to make sure that it has all the latest updates installed. That requires an internet connection (duh) , then I just use this:
sudo sh -c « apt-get update && apt-get dist-upgrade && apt-get autoremove »
Because this is the oldest/slowest of my Raspberry Pi systems, and I tend not to use it all that often, there were a lot of updates to be installed. Then those updates took a long time to download and install, so this was a time-consuming process.
Next I have to install the utilities and application software needed for this project. This is kind of interesting, because there are three different things to be installed, and they are downloaded and installed in three different ways.
The cluster operation in this project is done using a small group of Python packages. These are installed using the Python pip3 installer:
sudo pip3 install dispy==4.7.1
Note the double-equals notation — that is not a typo; also, note the version specification, without it you will get a later version of the package, which does not yet work on Raspbian.
Next, it needs the nmap utility, which will be used to find the cluster members: it is installed using the apt-get utility that we are already familiar with:
sudo apt-get install nmap
Finally, I need the OctaPi software, which is available on the github development platform, so it is downloaded using the git clone command:
This actually gets both the client and server files. We only need the client side on this system, so I can move them to my home directory:
mv octapi-setup/client/* /home/pi
Believe it or not, that’s all the client needs at this time, So I can shut it down and set it aside, and turn my attention to the first of the cluster servers.
Second step – a cluster server
This is where I start to see the first consequences of my decision about which Raspberry Pi systems to use for cluster servers. In the project description they use eight identical Pi 3 systems. Besides the aesthetic appeal of that, there is a huge benefit in the fact that the setup procedure for every server is identical. So identical, in fact, that you actually only have to go through the setup and configuration once, then you can simply clone the SD card for each of the other servers. Nice.
My systems have all sorts of different SD cards, and some of them are currently not even running Raspbian — I have PiCore, Manjaro, Kali, Fedora and openSUSE loaded on various of them. So I am going to have to check each one, update those which are already running Raspbian, and on those running something else I will have to either prepare a new SD card or overwrite their existing card — either way, those will require a fresh installation.
I am going to use my Raspberry Pi 3 for the first cluster server system, mostly because the Pi 3 has built-in wireless networking, so that’s one less thing to have to fiddle with during this first setup.
While preparing the first SD card, I remembered something I had written above — the server systems will not have a monitor, keyboard, or mouse. That made me consider using Raspbian Jessie Lite rather than the full Jessie with PIXEL, because it is considerably smaller, so it is faster to download and copy to the SD card.
There are a few other things to consider when deciding between Jessie Lite and Jessie with PIXEL, though. First, some of the operating system configuration that has to be made is a lot easier to do on the GUI than from the CLI — things like wi-fi connection, for example. Second, the PIXEL GUI is not the only thing that is not included in Jessie Lite — python3 is not included, so you would have to install that as well.
In the end, even though I am a pretty dedicated and experienced CLI user, I would say that it is better (or at least easier) to install the full Raspbian with PIXEL on all of the Raspberry Pi systems used in this project.
Even if you made a fresh installation from the latest image, there probably have been updates released since that image was created, so you have to make sure you’re up to date:
sudo sh -c « apt-get update && apt-get dist-upgrade && apt-get autoremove »
The server will also need the Python dispy package:
sudo pip3 install dispy==4.7.1
It also needs the psutil package, which is used to report server load to the client, so the client can decide how to distribute the work:
sudo pip3 install psutil
This time we don’t need to specify the version, because whatever the latest is will work.
Next you need to add dispy to the system startup process. I’m not going to try to teach you how to use a text editor here, I will just say add these three lines to the end of /etc/rc.local, just before the « exit 0 » line:
Before a bunch of people jump in here to point out how lame those three lines are, let me say first that I agree with you. To be fair, we have to keep in mind that this is a home/hobby project, and the intent is to set up a Pi cluster, not to produce a production-ready system. But still, I just have to comment on each one of those lines:
In most cases, these are not serious problems. But you should be aware of them, just in case you run into some situation where the client doesn’t use one or more of the cluster servers in the following tests.
Next, you need to make sure that SSH is enabled on the server systems.
IMPORTANT! (It’s soapbox time again, sorry.) Enabling SSH on a system with the default user name and password is such a bad idea that I can’t even think of words to describe it. The latest versions of Raspbian will actually complain when it is set up this way, telling you over and over again that you need to change the password for the pi user.
I don’t care that this project is supposed to be implemented on a dedicated router that does not have internet access. I don’t care about any other « special circumstances » that are supposed to make this OK ‘just in this case’. It’s not OK. Period. At least change the password — and if you have any sense, use a different login name as well, and either disable or delete the ‘pi’ login.
Soapbox end. Thank you for your attention. Now back to your regularly scheduled project.
If you are running a full Jessie with Desktop system, you can enable SSH by going to the PIXEL GUI menu, Preferences, Raspberry Pi Configuration, Interfaces. If you have a text-only system (Raspbian Lite, no PIXEL) , you can get a text-mode version of the Pi Configuration utility with sudo raspi-config, then go to Interfacing Options, SSH.
There is one other way you can get SSH enabled, by putting an empty file named ssh in the boot directory, and then rebooting.
