Settle in and get ready for a ride because this tutorial covers a lot of the technical buzzwords around IoT. We’ll be making an IoT Gateway with a Raspberry Pi Zero W. To achieve this we’ll put the Pi’s built-in WiFi antenna into AP Mode (Access Point), or in other words, the Pi will be broadcasting its own wifi network for devices to connect.
There will be some NodeMCUs that will connect to the Pi’s WiFi. To make communication between the nodes and Pi extremely simple we’ll use MQTT. The Pi will act as the MQTT Broker (server) which the nodes will publish data.
Lastly, we’ll consume the node’s data on the Pi through MQTT Subscribe and perform some data manipulation (edge computing). We'll then send the modified data through Hologram's Global Cellular Network to the cloud using Hologram's Nova, Python SDK, and Data Engine.
I like to access my Pi through SSH over my local wifi network, but since we’ll be turning this Pi into a router, we’ll need to find another way to access the Pi. Below are a few other connection options and my thoughts:
-
Ethernet Cable: Unfortunately the Pi Zero does not have a network jack.
-
Keyboard and Monitor: I rarely consider this option since I prefer working in the development environment of my main computer, but if there is no other option available and you have the extra peripherals this may be chosen.
-
TTL Serial Cable: This is a great option, and I would have picked this. Unfortunately, I did not have a USB Console cable available to me while writing this tutorial. Adafruit sells a popular TTL Serial Cable compatible with the Pi.
-
USB Serial Console: There is a way to turn the single USB port on the Pi into a network interface, giving us SSH access over USB cable, SWEET!
We’ll initially access the Pi through the USB>Serial interface then later we’ll connect to the Pi’s own WiFi network.
(modified from https://www.youtube.com/watch?v=aL1pWI2K60w)
Install the latest Raspbian (either Lite or Desktop will work).
Insert SD card into your main computer.
Open a terminal and change directory into the BOOT drive. On Mac OSX:
➜ cd /Volumes/boot/Modify the config file:
➜ nano /boot/config.txtAt the bottom of the config file comment out the audio setting and add the three additional settings. dwc2 allows ethernet over usb and the other two settings disable the onboard LED to conserve energy.
# Enable audio (loads snd_bcm2835)
#dtparam=audio=on
dtoverlay=dwc2
dtparam=act_led_trigger=none
dtparam=act_led_activelow=on
Save the file by typing in Control-X then Y then return.
Modify the command line file:
➜ nano cmdline.txtAdd modules-load=dwc2,g_ether (after rootwait , before quiet ). Save the file by typing in Control-X then Y then return.
Lastly, enable SSH. SSH is not enabled by default. Adding an empty file named SSH will enable it during the next boot.
➜ touch sshEject SD card and insert into the Pi. Connect the Pi micro USB port to your computer’s USB. Make sure it is the USB port on the Pi Zero and not the PWR port. The Pi should be powered through the USB port.
OSX:
After a minute or two - open a terminal app on your computer and SSH into the Pi (user “pi”, password “raspberry” - no quotes).
➜ ssh pi@raspberrypi.localWindows:
On Windows we'll need to setup the RNDS driver to gain the SSH functionality.
- Goto Device Manager and find RNDS/Ethernet Gadget under Other devices. Right-click on the device and select update driver software.
- Browse my computer
- Pick from a list of drivers
- Find Microsoft > on the right select Remote NDS Compatible Device
- The Pi should now show up and able to be SSH with Putty
Your Pi is accessible through SSH, but it still does not have access to the internet. Without the internet, we're not able to update the Pi or install any additional libraries required by this tutorial.
➜ ping www.google.comYou should not be able to ping the server (unless you already have internet sharing enabled).
Close the SSH session by typing exit before going forward.
OSX:
Open System Preferences, select Sharing. 
Configure RNDIS Gadget to share your WiFi connection. 
Connect through SSH and reboot the Pi
➜ ssh pi@raspberrypi.local
➜ sudo rebootAfter reboot SSH back into the Pi. It should now have internet.
➜ ping www.google.comWindows:
Open Network Connections, right click on the connection you want to share and select Properties. 
Select the Sharing tab, check Allow other network users…, from the drop-down select the connection of the Pi RNDIS Gadget. 
Putty into the Pi and reboot.
➜ sudo rebootAfter reboot SSH back into the Pi. It should now have internet.
➜ ping www.google.comAlright, we should have an SSH session going. In the terminal run the following to bring up the Raspbian Stretch configuration. These config changes will require a Pi reboot. Just start another SSH session once the Pi becomes available again.
➜ sudo raspi-config
We're going to set a few options:
-
3 Boot Options > B1 Desktop/CLI > B1 Console
-
7 Advanced Options > A1 Expand Filesystem
-
4 Localisation Options > I2 Change Timezone
-
4 Localisation Options > I3 Change WiFi Country
-
8 Update
-
Optional but recommended
- 1 Change User Password
- 2 Hostname
NOTE: to conserve energy we are disabling Raspbian's UI (Pixel) and only allowing the console.
To conserve more energy we will also disable the HDMI output
➜ sudo nano /etc/rc.local
At the bottom, but before exit 0 add this line:
/usr/bin/tvservice -o
(modified from https://learn.adafruit.com/setting-up-a-raspberry-pi-as-a-wifi-access-point)
To work as an access point, the Raspberry Pi will need to have access point software installed, along with DHCP server software to provide connecting devices with a network address.
➜ sudo apt-get install hostapd isc-dhcp-server Ignore any DHCP errors
IPTables will give us the tools we need ot manage multiple connections. The iptables-persistent package ensures the right settings are set during startup.
➜ sudo apt-get install iptables-persistentTwo config screens will prompt you, say Yes to both.
Configure DHCP
Next, we need to setup DHCP since the Pi will be managing local IP addresses of the nodes.
➜ sudo nano /etc/dhcp/dhcpd.confComment out these two lines
#optiondomain-name "example.org";
#optiondomain-name-serversns1.example.org, ns2.example.org;Uncomment
authoritativeAdd these lines at the bottom of the file.
subnet 192.168.42.0 netmask 255.255.255.0 {
range 192.168.42.10192.168.42.50;
option broadcast-address 192.168.42.255;
option routers 192.168.42.1;
default-lease-time 600;
max-lease-time 7200;
option domain-name "local";
option domain-name-servers 8.8.8.8, 8.8.4.4;
}Save the file by typing in Control-X then Y then return.
➜ sudo nano /etc/default/isc-dhcp-serverChange the interface(s) to wlan0:
INTERFACESv4=“wlan0”
INTERFACESv6=“wlan0”
Save the file by typing in Control-X then Y then return.
Configure the Network Interface
Bring down the local wifi in case you have it up right now.
➜ sudo ifdown wlan0➜ sudo nano /etc/network/interfacesMake sure your file looks like this:
auto lo
iface lo inet loopback
iface eth0 inet dhcp
allow-hotplug usb0
iface usb0 inet dhcp
allow-hotplug wlan0
iface wlan0 inet static
address 192.168.42.1
netmask 255.255.255.0
Save the file by typing in Control-X then Y then return.
Set the static IP for wifi right now:
➜ sudo ifconfig wlan0 192.168.42.1Configure the Access Point
➜ sudo nano /etc/hostapd/hostapd.confThese are the settings for the WiFi we’ll be broadcasting frome the Pi. Make sure the file looks exactly this (you can modify the ssid and passphrase):
interface=wlan0
ssid=friendly-raspberry
country_code=US
hw_mode=g
channel=6
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2
wpa_passphrase=hologram
wpa_key_mgmt=WPA-PSK
wpa_pairwise=CCMP
wpa_group_rekey=86400
ieee80211n=1
wme_enabled=1
Save the file by typing in Control-X then Y then return.
Now point the startup Daemon to use this configuration file.
➜ sudo nano /etc/default/hostapdAdd the file location AND make sure to remove the '#' at the beginning of this line:
DAEMON_CONF="/etc/hostapd/hostapd.conf"
And one more time.
➜ sudo nano /etc/init.d/hostapdChange DAEMON_CONF= to:
DAEMON_CONF=/etc/hostapd/hostapd.conf
Configure Network Address Translation (NAT)
The NAT determines how to move data between the network interfaces and allow multiple clients to connect to the WiFi.
➜ sudo nano /etc/sysctl.confFind net.ipv4.ip_forward=1 and uncomment it. Save the file by typing in Control-X then Y then return.
Activate the change immediately:
➜ sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"Run the following commands to configure wlan0 and usb0:
➜ sudo iptables -t nat -A POSTROUTING -o usb0 -j MASQUERADE➜ sudo iptables -A FORWARD -i usb0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT
➜ sudo iptables -A FORWARD -i wlan0 -o usb0 -j ACCEPT➜ sudo sh -c "iptables-save > /etc/iptables/rules.v4"Run the following two commands to verify things got set.
➜ sudo iptables -t nat -S
➜ sudo iptables -SThe response should look exactly like this:
Let's reboot to allow everything to execute. Start a new SSH session after reboot.
➜ sudo rebootFinishing Touches
Let's log back into the Pi and fire up the Access Point.
➜ sudo service hostapd start
➜ sudo service isc-dhcp-server startNow, from another device, you should see a new WiFi network and be able to log-in. If not then go back to double check your work.
Devices within the vicinity of the Pi can connect to it through WiFi.
To make the AP come up during boot run the following:
➜ sudo update-rc.d hostapd enable
➜ sudo update-rc.d isc-dhcp-server enableYou can monitor activity by running:
➜ tail -f /var/log/syslogOn the Raspberry install Mosquitto - An open-source and popular MQTT Broker.
➜ sudo apt-get install mosquitto mosquitto-clients
➜ sudo pip install paho-mqttInitially, Mosquitto is installed with no security, meaning anyone on the network can publish and subscribe without a password. Since the Pi is running a private password protected network, we’ll be fine keeping the default MQTT settings.
Create Python script to gather all data
➜ sudo nano gateway.pyimport time
# python library for publishing and subscribing to MQTT data
import paho.mqtt.client as mqtt
# This script is running on the same machine the MQTT Broker (server)
# is running on. Localhost will work fine, otherwise use the Broker IP.
Broker = "localhost"
# Topics are how MQTT organize data.
# Clients publish data to a specific topic(s)
# Clients also can subscribe to a topic(s) and wild cards
# Topics and subtopics are organized in a folders-ish structure
sub_topic = "node/value"
# when connecting to mqtt do this;
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
client.subscribe(sub_topic)
# when receiving a mqtt message do this;
def on_message(client, userdata, msg):
message = str(msg.payload)
print(client)
print(msg.topic+" "+message)
# Istantiate a new Client
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
# Connect to the broker, by default broker runs on port 1883
client.connect(Broker, 1883, 60)
# Paho's loop that keeps data streaming
client.loop_start()
# We create our own loop to keep this script running
while True:
time.sleep(5)Start Pi Script Start the script by running
➜ sudo python gateway.pyTo terminate the script press CTRL+C
Have the script running before moving to the next step.
Learn More about MQTT
We'll be using the Arduino IDE to flash NodeMCU development boards. NodeMCU uses the affordable Espressif ESP-8266 WiFi module and is capible of running Lua, MicroPython, Javascript, and C/C++. We'll use C/C++ to program these devices.
Install an IDE to flash micro-controllers. Here are two options:
Install the board and code library dependencies:
-
PubSubClient by Nick O’Leary
- PlatformIO Installation
- ArduinoIDE Installation - Under drop-down Sketch > Include Library > Manage Libraries... then search and install for PubSubClient
-
Install NodeMCU Board support
- PlatformIO Installation - support already built in :)
- ArduinoIDE Installation - follow these instructions
Code
https://github.com/benstr/TUT-raspi-cellular-gateway/blob/master/client/client.ino
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
// Pi's access point name
const char* ssid = "friendly-raspberry";
// Pi's access point password
const char* password = "hologram";
// broker IP address
const char* mqtt_server = "192.168.42.1";
WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[50];
int value = 0;
void connect_wifi() {
delay(10);
// We start by connecting to a WiFi network
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
if (WiFi.status() == WL_NO_SSID_AVAIL) {
Serial.println();
break;
} else {
delay(1000);
Serial.print(".");
}
}
if(WiFi.status() == WL_CONNECTED) {
randomSeed(micros());
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
} else {
Serial.println("Network not found, trying again...");
}
}
void setup_wifi() {
while (WiFi.status() != WL_CONNECTED) {
connect_wifi();
delay(2000);
}
}
void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
// Switch on the LED if an 1 was received as first character
if ((char)payload[0] == '1') {
digitalWrite(BUILTIN_LED, LOW);
// Turn the LED on (Note that LOW is the voltage level
// but actually the LED is on; this is because
// it is acive low on the ESP-01)
} else {
// Turn the LED off by making the voltage HIGH
digitalWrite(BUILTIN_LED, HIGH);
}
}
void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
// First check for wifi connectivity
if(WiFi.status() == WL_DISCONNECTED){
Serial.print("Lost WiFi connection, attempting new connection..");
setup_wifi();
}
Serial.print("Attempting MQTT connection...");
// Create a random client ID
String clientId = "ESP8266Client-";
clientId += String(random(0xffff), HEX);
// Attempt to connect
if (client.connect(clientId.c_str())) {
Serial.println("connected");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}
void setup() {
pinMode(BUILTIN_LED, OUTPUT);
Serial.begin(115200);
setup_wifi();
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
}
void loop() {
long now = millis();
// verify device is connected to the Broker
if (!client.connected()) {
reconnect();
}
// MQTT client library loops through checking the queue
client.loop();
// publish a message every 2 seconds
if (now - lastMsg > 2000) {
lastMsg = now;
++value;
snprintf (msg, 75, "hello hologram #%ld", value);
Serial.print("Publish message: ");
Serial.println(msg);
// publish msg to the topic "node/value"
client.publish("node/value", msg);
}
}note: This example is a modified version of a PubSubClient example.
Flash this code to the device(s) with the following settings:
Open the serial monitor and set the baud to 115200. You should see information about the device connecting to the wifi network, MQTT broker, and publishing data.
Over on your Pi, you should see data coming in too!
In an SSH session with your Pi (plugged into your computer’s USB) install the Hologram SDK. This may take a few minutes.
➜ curl -L hologram.io/python-install | bashIncorporate Cellular into Python Script
Copy existing gateway script.
➜ cp gateway.py gateway_cell.pyEdit the newly created file.
➜ sudo nano gateway_cell.pyNear the top add an import for the Hologram SDK and instantiate.
import time
import paho.mqtt.client as mqtt
from Hologram.HologramCloud import HologramCloud
hologram = HologramCloud(dict(), network='cellular')Add a Hologram SDK sendMessage() to the bottom of the on_message() function.
# when receiving a mqtt message do this;
def on_message(client, userdata, msg):
message = str(msg.payload)
print(client)
print(msg.topic+" "+message)
response = hologram.sendMessage(message, topics=[msg.topic])
print responseSee the gateway_cell.py file in the GitHub repository to check your work.
Moar Power!
We’re about to plug in the cellular modem (do not do it yet). Up until now, we’ve been running the Pi from USB power. Now with the modem attached we need more juice!
Find a 5.1v 2.5A power supply and connect it to the power outlet on the Pi.
Note: ensure you’re using a large enough power adaptor. Review the Raspberry Pi Foundation's guidelines.*
New SSH Connection
Up until now, we’ve been accessing the Pi through SSH using USB. Soon we’ll need to use the USB port for the Nova. So how will we SSH into the Pi? We’ll use the Pi’s WiFi network!
Close the current SSH session by typing exit and disconnect the USB cable.
Now connect your development computer to the Pi’s WiFi. We’ll lose internet connectivity and for the next few moments that will be ok.
Being connected to the friendly-raspberry WWiFi network, you can now reconnect through SSH the same as you did before.
➜ ssh pi@raspberrypi.localPlug-in the Hologram Nova
Time to connect the Hologram Nova. If you have not done so already, disconnect your computer from the USB port on the Pi and in its place plug in the Hologram Nova with a USB-A to USB Micro adaptor.
Now wait for the Nova to show one solid LED, and one blinking LED. This means the Nova has detected an available network. If you never get a blinking LED then disconnect the Nova and double check the SIM is inserted correctly, and the antenna is attached.
With the Nova plugged into the Pi run the new script we made above.
➜ sudo python gateway_cell.pyThe previous Nodes should still be running and will automatically connect once the script starts running. You should be able to see serial debug messages. If you see the following, then chances are we have done everything correctly!
node/value hello hologram #4
0
P.S. While the script is running the blinking light on the Nova should become solid. This is to show we are connected to a network and have established a TCP session.
Data in the Cloud
Close the SSH session by typing exit. This will also kill the script.
Disconnect your main computer form the Pi's WiFi and connect to your home network.
Go to the Hologram web console to see your data in the cloud!
With the skills you've learned above lets build a network of environmental sensors (light, temperature, and humidity).
You'll need the following components for each node:
- DHT11 Sensor
- Photoresistor
- 2 10k Resistors
- 2 LEDs (preferrably green and red)
- ~10 Jumper Wires
In PlatformIO or Arduino IDE install the following libraries:
- Adafruit DHT
- Adafruit Unified Sensor
Schematic:
Code: The code below expands from the example we used in step 5.0. We add logic to read sensors and format their results plus sprinkle in some logic to control two LEDs (red and green).
The LEDs will help us determine the connection status of the device without needing access to the serial console.
#include <ESP8266WiFi.h>
#include <PubSubClient.h>
#include <DHT.h>
#define DHT_PIN D3
#define DHT_TYPE DHT11
#define LUM_PIN A0
#define GREEN_LED D1
#define RED_LED D2
DHT dht(DHT_PIN, DHT_TYPE);
// Update these with values suitable for your network.
const char* ssid = "friendly-raspberry";
const char* password = "hologram";
const char* mqtt_server = "192.168.42.1";
WiFiClient espClient;
PubSubClient client(espClient);
long lastMsg = 0;
char msg[120];
int msgNum = 0;
void connect_wifi() {
delay(10);
// We start by connecting to a WiFi network
Serial.println();
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
if (WiFi.status() == WL_NO_SSID_AVAIL) {
Serial.println();
break;
} else {
delay(1000);
Serial.print(".");
}
}
if(WiFi.status() == WL_CONNECTED) {
randomSeed(micros());
Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
} else {
Serial.println("Network not found, trying again...");
}
}
void setup_wifi() {
digitalWrite(GREEN_LED, LOW);
digitalWrite(RED_LED, HIGH);
while (WiFi.status() != WL_CONNECTED) {
connect_wifi();
delay(2000);
}
}
void callback(char* topic, byte* payload, unsigned int length) {
Serial.print("Message arrived [");
Serial.print(topic);
Serial.print("] ");
for (int i = 0; i < length; i++) {
Serial.print((char)payload[i]);
}
Serial.println();
// Switch on the LED if an 1 was received as first character
if ((char)payload[0] == '1') {
digitalWrite(BUILTIN_LED, LOW); // Turn the LED on (Note that LOW is the voltage level
// but actually the LED is on; this is because
// it is acive low on the ESP-01)
} else {
digitalWrite(BUILTIN_LED, HIGH); // Turn the LED off by making the voltage HIGH
}
}
void reconnect() {
// Loop until we're reconnected
while (!client.connected()) {
if(WiFi.status() == WL_DISCONNECTED){
Serial.print("Lost WiFi connection, attempting new connection..");
setup_wifi();
}
Serial.print("Attempting MQTT connection...");
// Create a random client ID
String clientId = "ESP8266Client-";
clientId += String(random(0xffff), HEX);
// Attempt to connect
if (client.connect(clientId.c_str())) {
digitalWrite(GREEN_LED, HIGH);
digitalWrite(RED_LED, LOW);
Serial.println("connected");
} else {
Serial.print("failed, rc=");
Serial.print(client.state());
Serial.println(" try again in 5 seconds");
// Wait 5 seconds before retrying
delay(5000);
}
}
}
void setup() {
Serial.begin(115200);
pinMode(BUILTIN_LED, OUTPUT);
pinMode(GREEN_LED, OUTPUT);
pinMode(RED_LED, OUTPUT);
dht.begin();
setup_wifi();
client.setServer(mqtt_server, 1883);
client.setCallback(callback);
}
void loop() {
if (!client.connected()) {
digitalWrite(GREEN_LED, LOW);
digitalWrite(RED_LED, HIGH);
reconnect();
}
client.loop();
long now = millis();
if (now - lastMsg > 10000) {
lastMsg = now;
++msgNum;
int iLum = analogRead(LUM_PIN); // read photoresistor value (brightness)
float fHum = dht.readHumidity(); // Read humidity
float fTemp = dht.readTemperature(true); // Read temperature as Fahrenheit (isFahrenheit = true)
// Check if any reads failed and exit early (to try again).
if (isnan(fHum) || isnan(fTemp)) {
Serial.println("Failed to read from DHT sensor!");
return;
}
// convert sensor readings to strings
char lum[8];
char temp[8];
char hum[8];
dtostrf(iLum, 1, 0, lum);
dtostrf(fTemp, 1, 2, temp);
dtostrf(fHum, 1, 2, hum);
// format data into a JSON string
snprintf (msg, 120, "{\"msgNum\": %ld, \"temp\": %s, \"hum\": %s, \"lum\": %s}", msgNum, temp, hum, lum);
Serial.print("Publish message: ");
Serial.println(msg);
client.publish("environment/studio", msg);
}
}Gateway Code
The gateway python code is nearly untouched. we just make a few tweaks to the MQTT topic.
https://github.com/benstr/TUT-raspi-cellular-gateway/blob/master/gateway_cell_sense.py
import time
import paho.mqtt.client as mqtt
from Hologram.HologramCloud import HologramCloud
hologram = HologramCloud(dict(), network='cellular')
Broker = "localhost"
sub_topic = "environment/#"
# when connecting to mqtt do this;
def on_connect(node, userdata, flags, rc):
print "Connected with result code "+str(rc)
client.subscribe(sub_topic)
# when receiving a mqtt message do this;
def on_message(node, userdata, msg):
message = str(msg.payload)
print msg.topic+" "+message
response = hologram.sendMessage(message, topics=[msg.topic, sub_topic])
print response
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect(Broker, 1883, 60)
client.loop_start()
while True:
time.sleep(10)Run the code as we did before and check the Hologram console to see the code in the cloud. If everything works then I'd recommend setting your python script to automatically run after the Pi boots. You can learn the different ways to accomplish that from this great blog post.
Enjoy!