In this post, we’ll look at how Edge Impulse and balena work together and what led me down the path to improvise a solution using Bluetooth Low Energy (BLE) so the two could communicate with each other.
I’ve been fascinated by the work of Edge Impulse
, which is a platform that allows devices such as microcontrollers and single board computers to natively run machine learning (ML) models on the edge. “On the edge” means that these devices actually perform ML on their own, rather than relying on a more powerful device nearby or in the cloud.
It also means no internet connection is necessary to run the ML models and get a result. The fact that they can cram everything needed to perform these operations into a small edge device is amazing to me!
Combining Edge Impulse and balena
Edge Impulse and balena are actually a complementary combination. While Edge Impulse handles data collection and machine learning, balena takes care of device updates and fleet management. Edge Impulse recently added official support for the Raspberry Pi 4, which you can try out in one of our previously released projects on simple classification using edge AI
. Subsequently, a group of developers created a project that makes it easy to deploy that new support on the Pi 4 via the balena platform.
For instance, you could easily manage a fleet of Pi 4s and push updated versions of the Edge Impulse software to them. Another great way the two work together is through the use of environment variables. You could change one variable on the balenaCloud dashboard and switch one or more of the Pis in your fleet from data collection mode to inference mode. For more details check out this post on Hackster
or the project repo
That’s a great solution for a single board computer such as the Pi 4, but what about microcontrollers? Edge Impulse supports a variety of them but balenaOS does not. (Yet!) Sometimes it makes more sense to use a microcontroller for a particular task due to power requirements, space requirements and/or cost.
Exploring my use case: AI-driven preventative maintenance
Edge Impulse has a great example project on their site called “continuous motion recognition.
” It allows you to collect motion/vibration data from sensors, train a model, and deploy the final model back to a device to run inferencing. The key element of this project for me was anomaly detection
- the ability to detect when a motion is far outside any of the trained classes.
This project sounded like a perfect way to predict potential issues with my central air conditioning unit. It’s a large mechanical device that sits in the basement mostly unseen and unheard. I could train a model with the vibration characteristics of each normal mode of operation (startup, running, high, idle) and using anomaly detection get an alert whenever the vibrations are outside of these patterns. This could allow me to obtain preventative maintenance before a potential problem gets out of hand.
These graphs display the difference in motion (mostly in the z axis) from two operating modes of my AC unit. Dozens of samples such as these were used to train a neural network using Edge Impulse.
What’s very inspiring is that while my setup will monitor the performance of an average household HVAC unit, it shares a lot of commonality with how one might potentially set up a larger-scale industrial solution.
I decided that the Arduino Nano 33 BLE Sense was the best device for the job. I easily attached it to my air conditioner with double-sided tape - no drilling required. The accelerometer is built-in and the footprint is tiny.
The Arduino is full featured, but does not support WiFi or have an ethernet connection, so getting the inference data off the device was going to be a challenge. There’s a serial output, but I did not want to use a wire or try to parse data in that format. That leaves us with Bluetooth Low Energy (known as BLE).
The Ardunio Nano 33 BLE Sense attached to the side of my AC unit. The connected cord is for power only.
About Bluetooth Low Energy (BLE)
BLE is a technology for short distance wireless data communications between devices. It is independent of “classic” Bluetooth (such as that used with wireless headsets, etc...) and has no compatibility with it. It was designed primarily for use with devices such as fitness trackers, medical equipment and beacons. In fact, BLE defines several profiles - or communication format specifications - for classes of low energy devices such as these.
Now that I decided on BLE for the Arduino side of the system, I had to make a decision about the other (receiving) side. It definitely needed to have BLE support, the ability to run balenaOS, and did not need to be too powerful. For Raspberry Pis, the options are: Zero W, 3A+, 3B, 3B+, 4B, 400. I chose a model 3A+ since it was more powerful than the Zero, but not overkill for the task. The Pi would need to receive the anomaly data from the Arduino via BLE, process it, and then publish it via MQTT into my home automation system. I’m using Home Assistant for home automation, which handles any necessary alerts but you could easily program alerts or some other form of communication directly into a script running on the Pi.
Now that the system was designed, all that was left to do was actually write the software. Unfortunately, straightforward working examples of communication between an Arduino and a Raspberry Pi are few and far between. I managed to piece together the few examples I could find into a working prototype, and the results are available in this repository
. (Part of my motivation for creating this post was to provide a simple example others could build upon.) All of the detailed steps for reproducing my setup are in that repository, so rather than duplicate them here, I’ll explain some of the high-level concepts that are required to enable this type of communication.
Earlier we mentioned that BLE profiles (or services) already exist for certain low energy devices such as fitness trackers. The BLE specification also allows you to create your own service, which is essentially what I did since one does not exist for air conditioner anomaly monitoring! Taking the Arduino sketch produced from the continuous motion project, I modified it to add a BLE service named “HVACanomaly” that contains two values: (called “characteristics” in BLE) AnomalyScore and TopClass. AnomalyScore is a boolean value that is True if the anomaly score is above 0.3 and TopClass is an integer between 1 and 4 indicating the currently inferred mode of operation based on my model.
One challenge was to determine that when you make your own service, it needs its own 128-bit UUID. In addition, each characteristic needs its own 16-bit UUID. These UUIDs are what the client (the Pi in my case) will use to refer to these services/characteristics, so they need to be consistent on both sides. Another challenge was to obtain the BLE address of the Arduino. This is required by the client code on the Pi, which is written in Python and uses the bluepy
library to communicate over Bluetooth LE.
Even though my use case involved interfacing one Arduino to one Raspberry Pi, balenaOS was still my platform of choice for the Pi side of the project. I wanted to run multiple projects on the Pi since the BLE service would not be using too many device resources. I added the balena sensor block
to publish temperature and humidity from an attached sensor since I was already sending MQTT to Home Assistant. Not to mention that balenaOS makes it easier to push code changes to the device without requiring physical access to it, which is great for a basement location.
If you have multiple Arduinos in your basement, this solution is easily expandable by simply adjusting the code to read from multiple BLE microcontrollers. If you have multiple basements that need monitoring, the fleet aspect of balena becomes very attractive in this situation. You can have remote ssh access to each installation as well as the ability to push software and OS updates to each Pi. You could then send all of the collected data to a project such as balenaSense
and view anomalies over time on a grafana dashboard.
Shortly after installing this setup, my air conditioning contractor visited for a regular maintenance appointment. He is accustomed to my habit of adding various DIY sensors to the unit but was perplexed by the attached Arduino. After explaining this new functionality of anomaly detection and remote access there were many questions and excited talk of how to duplicate it on other customer’s units. (One suggestion was to add the WiFi connect
block to the project’s docker-compose to make it simple to establish wifi access in a customer’s location.) I could practically hear the gears spinning in his head thinking of the possibilities.
What ideas will you come up with?