This project is an educational, interactive demonstration of fault-tolerant Apache Kafka data streaming technology as used in astronomy by NASA's General Coordinates Network (GCN). It is made for exhibition the NASA booth at conferences and trade shows. It was first shown at the 243rd Meeting of the American Astronomical Society in New Orleans, LA, January 7-11, 2024.
The General Coordinates Network (GCN) is a public collaboration platform run by NASA for the astronomy research community to share alerts and rapid communications about high-energy, multimessenger, and transient phenomena. Idistributes alerts between space- and ground-based observatories, physics experiments, and thousands of astronomers around the world.
One of GCN's core services is a public astronomy notification service built on a Confluent Kafka deployed in the cloud on Amazon Web Services. In this demonstration, the Kafka cluster consists of three brokers running on Raspberry Pi single-board computers, configured in a manner that is similar to the production GCN cluster. Three more Raspberry Pis are configured as Kafka clients; each client produces alerts on one topic and consumes alerts on the two other topics.
The 3-broker Kafka cluster is running in a fully replicated configuration: each topic has three copies, one stored on each broker. The cluster acknowledges records produced by the clients if it is stored on at least two in sync replicas. Thus, in normal operation, the cluster is tolerant to outages of any one broker.
Each Raspberry Pi has an LCD display. On the brokers, the display shows a table that indicates which topics are in sync. On the clients, the displays show a scrolling record of messages produced and consumed.
The visitor can interact with this demonstration in two ways:
- Toggle the on/off switches to cut the network connection to any combinations of brokers and clients.
- Press any of the buttons next to the clients to produce a message on that client's topic. Press and hold to produce messages repeatedly.
The CAD model is available online in Autodesk Fusion 360 at https://a360.co/3TFbETm.
| Qty | Image | Item | Part |
|---|---|---|---|
| Electronic Components | |||
| 6 |  |
Single Board Computer - Raspberry Pi 4B 8 GB | DigiKey 2648-SC0195(9)-ND |
| 6 |  |
SanDisk 32GB Extreme UHS-I microSDHC Memory Card | B&H Photo SAEMSD32A1G3 |
| 6 |  |
LCD Display - Adafruit Mini PiTFT 1.3"/> | DigiKey 1528-4484-ND |
| 1 |  |
8-Port Ethernet Switch - Brainboxes SW-008 | DigiKey 2265-SW-008-ND |
| 6 |  |
DPST On/Off Rocker Switch | DigiKey EG5600-ND |
| 1 |  |
Arcade Style SPDT Momentary Pushbutton - Red | DigiKey 1568-1476-ND |
| 1 |  |
Arcade Style SPDT Momentary Pushbutton - Green | DigiKey 1568-1481-ND |
| 1 |  |
Arcade Style SPDT Momentary Pushbutton - Blue | DigiKey 1568-1477-ND |
| 6 |  |
USB-A Male to USB-C Male Right Angle Cable | DigiKey 189-3021108-01M-ND |
| 1 |  |
DC 5V 5W Power Adapter | DigiKey 102-4120-ND |
| 18 |  |
Insulated Female Quick Connector 24-26 AWG | DigiKey WM18235-ND |
| 12 |  |
RJ45 8P8C Ethernet Plug | DigiKey 1742-VDV826-704-ND |
| 1 |  |
Anker 360 Charger (60W) | Anker A2123 |
| - | Spool of Cat5 cable | ||
| - | 24 AWG Insulated Wire | ||
| - | Female Breadboard Pin Headers | ||
| Spacers | |||
| 24 |  |
Brass Spacer M2.5 x 6mm+6mm* | DigiKey 732-12901-ND |
| 6 |  |
Brass Spacer M2.5 x 10mm+6mm* | DigiKey 732-12917-ND |
| 18 |  |
Brass Spacer M2.5 x 20mm+6mm* | DigiKey 732-12949-ND |
| Fasteners | |||
| 24 |  |
M2.5 Hex Nut* | McMaster 91828A113 |
| 4 |  |
M2.5 x 10mm Phillips Flat Head Bolt | McMaster 92010A020 |
| 6 |  |
M2.5 x 6mm Phillips Pan Head Bolt | McMaster 92000A104 |
| 18 |  |
M2.5 x 8mm Phillips Pan Head Bolt | McMaster 92000A105 |
| 4 |  |
M3.5 x 10mm Phillips Pan Head Bolt | McMaster 92000A155 |
| 4 |  |
M3.5 Hex Nut | McMaster 91828A220 |
| 24 |  |
M2.5 Lock Washer | McMaster 92148A070 |
| Machined Parts | |||
| 1 |  |
Aluminum Project Board | Fusion 360 CAD model |
| 4 |  |
Aluminum Project Board Foot | Fusion 360 CAD model |
| 6 |  |
Transparent Acrylic Raspberry Pi Cover | Fusion 360 CAD model |
| Miscellaneous | |||
| 1 |  |
Pelican V525 Vault Rolling Case | Manufacturer product page |
* Rather than ordering these items individually, it is more cost-effective to purchase Qty. 3 M2.5 Spacer Kit available from Amazon.
The Brainboxes SW-008 Ethernet switch has a terminal block plug. To connect the DC 5V 5W power adapter to the Ethernet switch:
- Snip off the adapter's barrel connector. Separate a few centimeters of the wire and strip a few mm of insulation.
- Identify the ground and +5V wires of the power adapter using a multimeter.
- Consult the wiring diagram on the data sheet for the SW-008. Note that the power adapter has two conductors but the SW-008 terminal block has three conductors. Insert the power adapter's ground wire into the terminal block's -V receptacle and the +5V wire into the +V receptacle. Connect the terminal block's -V and Functional Earth receptacles with a jumper wire.
Follow these instructions for N = 1 to 6 to prepare each of the 6 Raspberry Pis. On your workstation, do the following:
-
Download, install, and launch the Raspberry Pi Imager application.
-
Mount a MicroSD card.
-
In the Raspberry Pi Imager application, select the OS
Raspberry Pi OS (64-bit). -
Click the gear icon to select advanced options.
a. Set hostname to
gcndemoNwhere N is an integer.b. Check
Enable SSH. CheckAllow public-key authentication onlyand select your authorized keys.c. Check
Set username and password. Set the username togcndemo; choose (and save in a secure location) a random password.d. Check
Set locale settingsand select your time zone and keyboard layout.e. Click
Save. -
Click
Write. -
Remove the MicroSD card from your computer
Then, on the Raspberry Pi:
-
Insert the MicroSD card into the Raspberry Pi. Connect a monitor, keyboard, and mouse, and boot it.
-
Open a terminal and make the following changes:
a. Write the following to /etc/network/interfaces.d/eth0:
auto eth0 allow-hotplug eth0 iface eth0 inet static address 10.0.42.N/16b. Write the following to /etc/ssh/sshd_config.d/linklocal.conf:
ListenAddress 10.0.42.N
c. Run the following:
sudo apt-get update sudo apt-get update -
Reboot the Raspberry Pi.
Make sure that you can ssh to the Raspberry Pi. Then unplug the monitor, keyboard, and mouse. Then proceed to set up the next Raspberry Pi.
We use Ansible to automate the rest of the setup of the Raspberry Pis.
-
Connect all 6 of the Raspberry Pis and your workstation to a single Ethernet switch. Boot all of the Raspberry Pis.
-
Install Ansible on your workstation by running the command:
pip install ansible -
Run each of our playbooks in the following order:
ansible-playbook -i inventory.yml playbooks/proxy.yml ansible-playbook -i inventory.yml playbooks/pitft.yml ansible-playbook -i inventory.yml playbooks/reboot.yml ansible-playbook -i inventory.yml playbooks/setup.yml ansible-playbook -i inventory.yml playbooks/clients.yml ansible-playbook -i inventory.yml playbooks/pitft_buttons.yml
-
Ensure that all rocker switches are in the "On" position.
-
Connect the power cable for the Ethernet switch to a wall outlet.
-
Connect the power cable for the USB hub to a wall outlet.
-
For each Raspberry Pi, carefully reach under its transparent acrylic cover with an insulating object and press and hold for five seconds the uppermost of the two buttons on the Mini PiTFT.
-
What until the Mini PiTFT displays the message,
It is now safe to turn off your computer. -
Unplug the power supplies for the USB power hub and the Ethernet switch.
When Raspberry Pi OS is shut down and power is still suplied to the Raspberry Pi, the Mini PiTFT display will show the last image that was sent to it. If the green ACT light in the corner of the Raspberry Pi is off and the link lights on the Raspberry Pi's Ethernet port are off, then Raspebrry Pi OS is halted. If this is the case, then unplug and re-plug the power to the Raspberry Pi to restart it.
If Raspberry Pi is on and the display is frozen, then you can restart the program that drives the dispaly. To do this, carefully reach under the Raspberry Pi's cover with an insulating object and press the uppermost of the two buttons on the Mini PiTFT to restart the display program.
Sometimes the Kafka broker fails to start when the system boots up. To forcibly restart the Kafka broker, carefully reach under the Raspberry Pi's cover with an insulating object and press the lowermost of the two buttons on the Mini PiTFT to restart the display program.
The Kafka client prints many useful diagnostic messages when it obtains or loses connections to brokers. These messages are useful for tuning performance and fault tolerance. You can view these messages live by following these steps:
-
Connect your workstation to the Ethernet switch.
-
Start an SSH session on the Raspberry Pi that you want to watch by running this command:
ssh gcndemo@10.0.42.N
-
In the SSH session, run the following command:
journalctl -f -u gcndemo.service(Note that if you cut off that Raspberry Pi's network connection, then the live display will stop, but it should resume and fill in everything that you have missed as soon as you restore the connection.)
-
We built the chassis by CNC-machining a sheet of aluminum because our machine shop is most suited for precision machining of metals and plastics for spacecraft parts. However, next time it might be cheaper and faster to build it out of plywood if we can get access to a laser cutting machine or a CNC wood router.
-
We added the artwork to the chassis by printing on paper, gluing the paper to the aluminum with spray adhesive, and cutting out the pockets and holes with a hobby knife. It might be less labor-intensive and result in a more durable finish if we instead printed it on a vinyl plotter/cutter and adhered it to the chassis using heat transfer.
-
It was difficult to get accurate dimensions for the right-angle USB power connectors. As a result, they just barely fit through the holes that we made for them in the chassis, and each connector butts up against the adjacent Raspberry Pi's PCB.