Back in
the first part of this series, we detailed the process of building a simple LoRa-connected weather station to monitor temperature, humidity, and pressure, and send the readings to The Things Stack (TTS). In this second part, we’re going to take a look at how you can capture that data, record it and produce a dashboard and graphs to visualize the data.
Note: 28th June 2021: The Things Network (TTN) is moving officially to The Things Stack (TTS). TTN is no longer available to create LoRaWAN gateways, so we’ve reflected this guide to represent those changes.
Introduction
When our weather station transmits data, the gateway receives the transmission and forwards it to TTS. If it isn’t able to be forwarded to some other system for storage, it’s discarded. Obviously, this is no use if you want to build up a history of weather data and start to maintain records. So, what do we do about it?
The answer is to use the ‘Integrations’ section within your TTS application to forward the data on. There are numerous cloud services available that will allow you to do this. A couple of examples are
Ubidots and
Datacake; both of these services are able to connect with TTS and receive your data.
However, if you wish to store an extensive history, you’ll have to pay a small amount for the privilege. This is fair, of course, as in that case we’re asking these services to hold our data indefinitely which depends on storage space on systems that have associated costs.
If you’re willing to set up your own device though, you can take responsibility for your own data, so let’s take a look at that; we’re going to set up a Raspberry Pi and configure TTS to send data to it via an HTTP integration.
We’re going to set up the Raspberry Pi using
balenaBlocks;
ready-to-use services that we can use to get up and running quickly with zero configuration. This is going to be a little different from our usual guides in that we haven’t built this project and hosted it on GitHub. Instead, we’re going to show
you how to create it!
You’ll need to be familiar with code editors and the command line interface on your chosen operating system; this isn’t a super difficult process and is a great next step if you’ve already deployed some of our other projects in the past, but isn’t aimed at the complete beginner.
Hardware required
I’m going to be using a Raspberry Pi 3 to receive the data for this project and so I’ve listed these components below, but you could also use many other device types as what we’re doing isn’t Pi-specific. Additionally there are also cloud providers you can use that won’t require any hardware to receive the data; it depends if you want to have total control of your data, or not.
- Raspberry Pi (3, or 4)
- An SD card (we always recommend SanDisk Extreme Pro SD cards)
- Power supply
Software required
App structure
In order to receive data from TTS, we need a listening HTTP server. If we set up a HTTP integration on TTS it will make a post request with the data from our sensor to the URL we specify.
First, we need something that listens and receives these requests. Next, we’ll need somewhere to store the data after we receive it, so we’ll need a database. Then, we’ll need some way to visualize the data and make it easy to read, identify trends, maintain record values, etc.
That’s three core functions that our device needs to perform in order to carry out this task. Fortunately, these are very common pieces of functionality that are present in a lot of different applications, and so there are balenaBlocks available for all three! Yaaaas!
Building a balena app with balenaBlocks
OK so let’s build something. You’re going to need to familiarize yourself with the balena basics here. If you’ve already built one of our other projects, you should know what to do. If this is your first balena project, I recommend taking a look at our
getting started guides first.
Step 0: Install the prerequisites!
Before we get started, make sure you have all the items from the
software required section installed and working. This means your code editor, the balena CLI tools and Etcher.
Step 1: Set up an application and device within balenaCloud
This is the same as any other balenaCloud project; create an account if you don’t have one already, create an application being sure to select the correct device type for the device you’re using (we’re using a Raspberry Pi 3 (not 64-bit)), and then add a device to that application. Flash the SD card with the downloaded image and boot the device.
For more information, take a look at the
getting started guide and follow it up until adding a release. Stop there as we’ll make our own release here.
When you’ve finished step 1, you should be at the point where you have an application set up with a device added within balenaCloud. You’ll notice that device is not doing anything yet (as indicated by ‘Factory build’ being shown under ‘Current release’. Onward!
Step 2: Create a project folder and compose file
Create a folder on your computer, create a new file called docker-compose.yml
, and open this in your code editor (I’m using VS Code).
Edit the empty file so that it includes the following code:
version: '2.1'
volumes:
database:
dashboard:
services:
The version
is the version of compose file we’re using, and the volumes
entry sets up a data volume called data
, which is persistent storage for us to store our database.
Step 3: Add the connector block
Next, we’re going to expand upon this by adding
connector. This block forms the HTTP listener we were looking for and receives the data from TTS.
We add this by adding the following to our compose file under services:
connector:
image: balenablocks/connector:raspberrypi3
restart: always
labels:
io.balena.features.balena-api: '1'
privileged: true
environment:
- 'ENABLE_EXTERNAL_HTTP_LISTENER=1'
ports:
- "8080"
Find out more about the connector block and the options it offers on
the GitHub repo.
Step 4: Add InfluxDB
After connector
, we need a database to store our data. For this we’re going to use InfluxDB. In the same way, add the following to the compose file, directly beneath the connector block. It’s important to ensure that the compose file is indented correctly.
influxdb:
image: influxdb@sha256:b4529fd53e9022bd1e83aaf04c6c6a1ebfffdfa112c31367d69da1354983a24b
container_name: influxdb
restart: always
volumes:
- 'database:/var/lib/influxdb'
Step 5: Add the dashboard block
Now we’ve got
connector
and a database provided by
influxdb
, we need something to visualize the data and allow us to see it with a web browser. Enter
dashboard! This block provides Grafana in an automatically configured way, and we add this in a similar way two the other two services.
dashboard:
image: balenablocks/dashboard:raspberrypi3
restart: always
ports:
- "80"
volumes:
- 'dashboard:/data'
Find out more about the dashboard block and the options it offers on
the GitHub repo.
Step 6: Review and push
After combining all the above code blocks, you should be left with something like this:
version: '2.1'
volumes:
database:
dashboard:
services:
connector:
image: balenablocks/connector:raspberrypi3
restart: always
labels:
io.balena.features.balena-api: '1'
privileged: true
environment:
- 'ENABLE_EXTERNAL_HTTP_LISTENER=1'
ports:
- "8080"
influxdb:
image: influxdb@sha256:b4529fd53e9022bd1e83aaf04c6c6a1ebfffdfa112c31367d69da1354983a24b
container_name: influxdb
restart: always
volumes:
- 'database:/var/lib/influxdb'
dashboard:
image: balenablocks/dashboard:raspberrypi3
restart: always
ports:
- "80"
volumes:
- 'dashboard:/data'
Open a command line or terminal and navigate to your project directory where you created docker-compose.yml
. From here, and as long as you have set up the balena CLI tools, you should be able to run balena push <appName>
where <appName>
is the name of the application you created within the balenaCloud dashboard. This will combine the 3 blocks into an application which will then be downloaded by the Raspberry Pi.
Configure TTS
Once your device is up and running, we have to tell TTS where to send the data. First, enable the public URL on your device within the dashboard:
Open the public URL and you should see the Grafana dashboard provided by the dashboard block. Copy this URL from the address bar, append :8080
to the end after .com
, and importantly change from HTTPS to HTTP, so that you have something that looks like this:
https://025c14d375b36dae49fd71771e83.balena-devices.com:8080
Head over to the
TTS console, find your application and go to the ‘Integrations’ screen. From here you will be able to add a new integration and find the ‘Webhook’ option. Now add a
Custom webhook
.
Create the Custom webhook with the balenaCloud public URL on http
pointing to the port 8080
.
With this integration added to the TTS application, every time your node sends a reading, TTS will now forward it to your Raspberry Pi where the connector block is listening out for the data.
Access and customize the dashboard
Access the dashboard by accessing the device either via clicking the public URL link in the dashboard or by browsing to the local IP address of the device.
By default, the dashboard block adds all the data it receives from TTS to a dashboard called External Http Listener
; it’s shown above under ‘Manage dashboards’ accessible via the dashboards menu on the left-hand menu.
Looking under fields called ‘payload fields’, you should see the data from your sensor!
Until next time
Thanks for following along with the process and reaching the end of this series. If you had success with the process I’d love to hear about it!
Tweet us, tag us on
Instagram or hit us up
on Facebook. Similarly if you get stuck, or didn’t understand anything, let us know as well.
The best place to start is in the
forums where I and other members of the balena team hang out and will be happy to chat balenaBlocks!