An example web application to configure and view sensor data from Blues Sparrow devices.
- 🐦 Sparrow Reference Web App
- Development Overview
- Cloud Deployment
- Security
- Testing
- Support
To get started you need to:
- Create a Notehub account if you don't already have one.
- Create a Notehub project for your Sparrow devices.
- Set up a Sparrow Gateway and one or more sensors.
- Clone this repository to your local development environment.
- Install the project’s development dependencies.
- Configure this starter web app via environment variables.
- Start up a local database.
- Set up event routing.
- Launch the Sparrow Reference Web App app in development mode.
To start using the Sparrow Reference Web App you must clone this repository to
your local development machine. You can do this with git clone.
git clone https://github.com/blues/sparrow-reference-web-app.git
With your local project downloaded, you’ll next want to open up the
sparrow-reference-web-app folder in your text editor or IDE of choice. Once
you have the project open in your editor you’re ready to configure the project’s
environment variables.
Although this project is designed for development on Linux, Visual Studio Code (VS Code) can quickly create a Linux "Dev Container" on Windows, Mac, or Linux. To use this workflow you must install both VS Code and Docker, if you haven’t already.
Before continuing, additionally make sure Docker is running, which you can do by checking the following.
- Windows: Check for the docker (whale) icon in the system tray.
- Linux/Mac: Run the command
docker run hello-worldfrom your terminal. If everything is working correctly you’ll see a confirmation message.
When you open the folder containing this README in VS Code you will see boxes that prompt you to install the extension Remote - Containers, and then to “Reopen in Container”. Do both.
The Dev Container will automatically install Linux and the project dependencies, no matter which kind of operating system your development machine uses.
As a final step, open a Linux terminal in VS Code, as you’ll need it to run commands throughout the rest of this guide:
- VS Code > Menus > Terminal > New Terminal
If you choose not to use a Dev Container in VS Code, you can install the project dependencies as follows.
The Sparrow Reference Web App uses Node.js as a runtime, Yarn as a package manager, and Volta as a way of enforcing consistent versions of all JavaScript-based tools. You can install these dependencies by following the steps below.
- Install Volta by following its installation instructions.
- Run the command below in a terminal to install the appropriate versions of
both Node.js and Yarn.
volta install node yarn - Navigate to the root of the Sparrow Reference Web App in your terminal or
command prompt and run
yarn install, which installs the starter’s npm dependencies.yarn install - Install the PostgreSQL database engine.
The Sparrow Reference Web App uses a series of environment variables to store project-specific configuration. You must define your own values for these variables for the Sparrow Reference Web App to run. You can complete the following steps to do so.
- Create a new
.envfile in the root folder of your project. - Copy the contents of this repo’s .env.example file, and paste it in your new
.envfile. - Change the required values in your
.envto your own values using the steps below.
The Sparrow Reference Web App needs access to your Notehub project in order to show the gateway and sensor nodes in your project. An access token is used to authenticate the app.
To find retrieve an authentication token, put this in your command line (VS Code
Menu > Terminal > New Terminal), replacing YOUR_NOTEHUB_EMAIL &
NOTEHUB_PASSWORD with your own:
curl -X POST -L 'https://api.notefile.net/auth/login' \
-d '{"username":"YOUR_NOTEHUB_EMAIL", "password": "NOTEHUB_PASSWORD"}'
When successful, you will see a response like
{"session_token":"BYj0bhMJwd3JucXE18f14Y3zMjQIoRfD"}
Copy the value after the colon to set the environment variable in .env, e.g.
HUB_AUTH_TOKEN=BYj0bhMJwd3JucXE18f14Y3zMjQIoRfD
This is the unique identifier for your project in Notehub, and has the prefix app:. You can find this by going to your Notehub project, clicking the Settings menu, and finding the Project Information heading which contains Project UID. Click the copy icon to copy this to the clipboard.
HUB_PROJECTUID=app:245dc5d9-f910-433d-a8ca-c66b35475689
The default for these variables are fine for development purposes. In a production environment you'll set them to point to your production database.
The Web App receives data from Notehub through a Route created on Notehub.io and targeting the Web App. To set up your own route you’ll need to complete the following two steps.
- Create a tunnel to a server running the reference app.
- Running the tunnel allows your local copy of the reference app to be accessible on the public internet. This is necessary for Notehub to route events to your local setup.
- Set up a Notehub route to your tunnel.
- When Notehub receives an event it can optionally route that event to other servers. In this step, you’ll have Notehub route events to your local setup via the tunnel you created in step #1.
The Sparrow reference app contains logic to process incoming Notehub events. But in order for Notehub to forward data to your local app for processing, your local app must be accessible from the public internet.
To make your local environment accessible you must set up a tunnel. You’re welcome to use any tunneling setup you’re comfortable using, but we recommend localtunnel or ngrok.
localtunnel is a simple free tunnel that you can run as follows. Replace acme
with the name of your choice.
$ npx localtunnel --port 4000 --subdomain acme
Need to install the following packages:
localtunnel
Ok to proceed? (y) y
your url is: https://acme.loca.ltYou can close the tunnel with ctrl+c.
Ngrok is a freemium alternative to localtunnel, which requires an e-mail signup, a modicum of setup, and in free-mode does not give you a tunnel with a consistent domain name. The inconsistent domain name will require you to update your route each time you start your tunnel. To use ngrok you’ll first need to:
- Sign up for ngrok. (It’s free to start.)
- Install ngrok. (
brew install ngrok/ngrok/ngrokworks well for macOS users—and yes,ngrok/ngrok/ngrokis the package name recommended by Ngrok itself.) - Set up your ngrok auth token.
Next, open a new terminal outside VS Code and run ngrok http 4000, which
creates the tunnel itself.
ngrok http 4000
If all went well, you should see a screen in your terminal that looks like the image below. ngrok is now forwarding all requests to https://<your-id>.ngrok.io to http://localhost:4000. Copy the forwarding address (shown in the red box below) to your clipboard, as you’ll need it in the next step.
NOTE: Your
ngrokterminal needs to stay running for the tunnel to remain active. If you close and restartngrokyour URL will change.
To verify everything worked correctly, you can try loading the URL you just copied in a web browser; you should see your reference app’s home page.
With a tunnel in place, your next step is to create a route in Notehub that forwards events to your local app.
To set up the route complete the following steps:
- Visit Notehub and open the project you’re using for your Sparrow app.
- Select Routes in the navigation on the left-hand side of the screen.
- Click the Create Route link in the top right of the screen.
- Find the General HTTP/HTTPS Request/Response route type, and click its Select button.
- Give your route a name.
- For the route URL, paste the localtunnel or ngrok URL you copied earlier, and append
/api/datastore/ingest. For example your route should look something likehttps://bb18-217-180-218-163.ngrok.io/api/datastore/ingest. - Scroll down, and click the blue Create new Route button at the bottom right of the page.
And with that your route is now complete. When Notehub receives an event it should automatically route that event to your tunnel, and ultimately to your local app.
NOTE Event routing only happens when Notehub receives an event, therefore your Sparrow hardware needs to generate new data and send it to Notehub for Notehub to invoke your route OR you can manually route an existing event from Notehub by clicking the "Route Event" button in the righthand corner of an individual event. Learn more about manually routing events here.
- Visit Notehub and open the project you’re using for your Sparrow app.
- Select Settings in the navigation on the left-hand side of the screen.
- Scroll down to Device Dashboard URL.
- Click the pencil icon to edit the URL, and paste the localtunnel or ngrok URL you copied earlier followed by
api/go?gateway=[device]&sensor=[sensor]&pin=[pin]. For example,https://bb18-217-180-218-163.ngrok.io/api/go?gateway=[device]&sensor=[sensor]&pin=[pin]. - Click the checkmark on the right to save the changes.
Now that you have both a tunnel, route and Device dashboard URL in place, your last step to get up and running is to create the database itself.
The Sparrow reference web app uses PostgreSQL to store data, including the data it receives from Notehub events. To set up your own PostgreSQL instance you need to complete the following steps.
There are many different ways you might want to create a Postgres database. If you’re unsure how to start, we recommend running Postgres through Docker as follows.
Open a terminal (VS Code > Terminal Menu > New Terminal) and run one of the following commands.
# Choose one:
./dev.db.ephemeral.sh # Start a database which will delete its data when stopped
./dev.db.persistent.sh # Start a database with data that persists after stopping and starting again.On Windows you may need to Allow access when the Windows Defender Firewall
asks you to allow com.docker.backend.exe.
$ ./dev.db.persistent.sh
... elided ...
Done in 2.65s.
Database is now running the background. Use ./dev.db.stop.sh to stop it.This creates a PostgreSQL database running in Docker. You can ensure the database is
running as expected with the ./dev.db.status.sh command.
$ ./dev.db.status.sh
398940737bae postgres "docker-entrypoint.s…" 2 hours ago Up 2 hours 0.0.0.0:5432->5432/tcp, :::5432->5432/tcp sparrow-postgresql-containerTo stop the database you can use ./dev.db.stop.sh.
./dev.db.stop.sh # Stop the database (and delete the ephemeral data if any)And you can delete the database’s data with ./dev.db.delete.sh.
./dev.db.status.sh # Show whether the database is running
./dev.db.delete.sh # to remove the persistent database dataIf you'd like to connect to your locally running Postgres instance to ensure new Notehub events are being added, you can use the Prisma Studio database management tool to easily explore.
./dev.db.manage.sh # Open a webpage at http://localhost:5555 that lets you explore the databaseThis will open up a new browser window at http://localhost:5555 where you can see your Prisma DB, its tables, and any data that currently resides therein— which was routed in by Notehub.
And just like any other database GUI, you can click into models to view data, manipulate data, filter, query, etc.
If you manually created your PostgreSQL database instead of using one of the
scripts above to automatically create one in Docker you will want to configure
the DATABASE_* environment variables in .env and then use the
./prod.db.init.sh script to initialize the database.
$ ./prod.db.init.sh
This script will clear your database and reinitialize it.
Continue (y/n)?y
yes
Done in 10.36s.
Database has been reinitialized.
There are a number of gotchas that could prevent your Notehub data from making it to Postgres and your Sparrow app. If you’re having issues try the following things:
- Is Docker running the local Postgres instance on your machine?
- Currently there's no error message thrown if the Postgres Docker container's not running.
- Does your Ngrok endpoint match what's in Notehub and have the suffix
/api/datastore/ingest?- Be aware, every time the Ngrok connection is shut down and restarted, it will be started up with a brand new URL, so you'll need to update the route accordingly in Notehub to ensure data keeps flowing to it
- Have you added the correct Postgres URL and Notehub project API environment variables to
your
.envfile?
The Sparrow Reference Web App uses the Next.js web
framework to serve React-powered web pages and HTTP JSON APIs. You can start a
Next.js development server using yarn dev.
yarn dev
With the development server running, open http://localhost:4000 in your web browser to see your application.
Next.js automatically watches your project’s files, and updates your application
as you make changes. To try it, open your app’s src/pages/index.tsx file, make
a change, save the file, and notice how your browser automatically updates with
the change. Changes to .env, not being automatically reloaded require you
to stop the yarn dev with ctrl+c and to start yarn dev back up.
The project’s src/pages/api directory are
APIs as opposed to
React-powered HTML pages.
The Sparrow Reference Web App uses these routes in several places to access the
Notehub API
server without triggering a full-page reload.
NOTE: If you’re new to Next.js, the Next.js official interactive tutorial is a great way to learn the basics, and understand how the Sparrow Reference Web App works.
The Sparrow Reference Web App is a Next.js project, and is therefore easily deployable to any platform that supports Next.js applications. Below are specific instructions to deploy to a handful of common platforms.
NOTE: For all deployment options we recommend creating a fork of this repository, and performing all deployment steps on that fork.
A PostgreSQL database will be required for any of the deployment methods below. Setting up a production PostgreSQL database is beyond the scope of this guide but, for a price, many cloud services such as Azure Database Service, Heroku or Amazon AWS RDS can host a PostgreSQL database for you.
This repo contains Netlify configuration that allows you to deploy to Netlify with a simple button click! Click the button below to automatically fork this repo, set environment variables, and immediately to deploy to Netlify.
Read our step-by-step guide to deploying the Sparrow Reference Web App app on Netlify for more information.
The next easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.
Read our step-by-step guide to deploying the Sparrow Reference Web App app on Vercel for more information.
NOTE: Running this site as Azure Container Instances will cost about $30/mo.
Follow the steps below to deploy to Microsoft Azure Cloud. If you need more details on any of the steps, see Docker’s documentation on deploying Docker containers on Azure.
Build Machine and Cloud Setup
- Sign up for Azure.
- Sign up for Docker Hub.
- Install Docker.
- Install docker-compose.
- Install the confusingly named Compose
CLI, which adds cloud-specific
compose-like support to
dockervia a wrapper of the standarddockercli. - Check that Compose CLI is working.
docker version | grep 'Cloud integration' && echo Yay || echo Boo. - Sign into Azure using
docker login azure. See https://docs.docker.com/cloud/aci-integration/. - Create a docker context on Azure named however you like. For example,
docker context create aci myacicontext.
Configure the sparrow-reference-web-app environment
cp .env.example .env.production.local- Configure the Required variables and the Azure variables.
Build and Deploy
./deploy.sh
...
[+] Running 3/3
⠿ Group sparrow-reference-web-app Created 6.3s
⠿ sparrowreferencewebeapp-https-reverse-proxy Created 113.8s
⠿ sparrowreferencewebeapp Created 113.8s
[deploy.sh] 🚀 Successful deployment.
[deploy.sh] 🔃 To deploy new changes, simply run this script again.
[deploy.sh] 🚮 To delete the deployment or see cloud details, visit the Azure Portal: https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.ContainerInstance%2FcontainerGroups
[deploy.sh] ⏰ In a few minutes the site should be visible here:
[deploy.sh] 🔜 https://mysparrowstarer.eastus.azurecontainer.io
Authentication and authorization are beyond the scope of this reference project. If you add basic auth or other HTTP-header-based auth, you can add those headers to your Notehub route to authorize it to route data to your web app.
The Sparrow Reference Web App contains both unit and end-to-end tests to ensure the project continues to work as intended.
This repo contains a unit testing setup that utilizes Jest and React Testing Library. No additional installation is necessary to use these tools (yarn install already installed them), but there is some additional setup you must perform to run the tests.
Unit Test Setup
The Sparrow Reference Web App’s testing setup requires a test-specific environment variable file. Follow the steps below to create that file.
- Create a
.env.test.localfile in the root of your project. - Copy the contents of the repo’s
.env.test.local.examplefile and paste it into your.env.test.localfile. - Change the values in your
.env.test.localfile to your own values. (You can likely copy and paste them from your.envfile.)
Running Unit Tests
This repo’s tests live in its __tests__ folder, and you can run the full test suite using the command below.
yarn testCode Coverage from Unit Tests
To see code coverage for the entire project, run the following command.
yarn test:coverageWhen the command finishes, you can open the coverage report by opening the repo’s coverage/lcov-report/index.html file in your web browser. You can do so on macOS using the command below.
open coverage/lcov-report/index.htmlThe Sparrow Reference Web App uses Cypress for automated UI & API testing.
Cypress Setup
To run the project’s Cypress tests you first need to perform the following setup.
- Create a
cypress.env.jsonfile in the root of your project. - Copy and paste the code below into your newly created file.
{
"gatewayUID": "dev:###############"
}
- Change the placeholder
"dev:###############"value to a valid gateway UID.
Running Cypress Tests
You can run the Cypress test suite with the yarn cypress command. yarn cypress:run runs the tests in your terminal.
yarn cypress:runAnd yarn cypress:open launches the tests in the Cypress GUI.
yarn cypress:openIf you run into any issues using this repo, feel free to create an issue on this repository, or to reach out on our developer forum.