15 May 2023 / Last updated: 15 May 2023

Use the MING stack to accelerate your IoT application development

Last week I received my 100W solar panel, a charge controller, and a power inverter. My goal was to get the information coming from the charge controller into a Raspberry Pi. My engineer's mind was telling me that I should create a service that could pull the data from the controller and visualize it on a nodeJS dashboard. However, some time ago the balena Ambassador, Alex Lennon, contacted me to show me some of the work he was doing and what template he was using on balena in order to avoid starting every IoT project from scratch. Luckily I followed the steps inspired by Alex and a few minutes later I had my solar panel controller reporting data on a dashboard thanks to the template he shared with me.

Back in 2019 Alex first posted about the “MING” (MQTT, InfuxDB, Node Red, Grafana) stack on the balena forums. The idea was that embedded developers could benefit from starting their projects with these already-built technologies and not reinvent the wheel. This would also make it easier for those just starting to learn about IoT application development. You can still find Alex’s original Dynamic Devices MING repository here. This reminded me of another glorious stack from the past, the LAMP stack.

Remember the LAMP stack (Linux, Apache, MySQL and PHP) from the late 90’s when the World Wide Web was exploding into the public consciousness? As you may recall, LAMP came about as an answer to a set of challenges that the Web development community were facing. At that moment, open source came to the rescue to enable new developers.

Time has moved on, however embedded systems and Internet of Things (IoT) developers still face parallel challenges to those faced by the creators of the LAMP stack. This is why we think that the open source MING stack can enable the next generation of IoT Application developers to be more productive.

There are a set of core generic requirements to most, if not all, of these types of IoT projects such as assured data-communications, data storage, data business logic and finally data visualization via a flexible tool.

In this blogpost, we will cover how each of these open-source technologies can be managed and deployed using container platforms such as balena.io with balena “blocks” to componentize each of the functional blocks within MING, to easily control, configure, and update the stack.

MQTT, InfluxDB, NodeRED and Grafana

The MING stack is a powerful combination of technologies that can be used on the edge to collect, process, and visualize real-time data from IoT devices such as Industrial PLCs or LoRaWAN sensors sending data to the LNS (LoRa Network Server). Together, these technologies create a seamless end-to-end solution for real-time data analytics and visualization on the edge to enable you, or any developer, to start creating innovation and value.

Can the IoT industry benefit from a MING stack? There is a fair amount of complexity building IoT systems. Therefore, having defined architecture patterns might help reduce some of the confusion. Another important feature of MING is that it can be deployed on the edge or in the cloud. IoT systems are inherently distributed so having the same technology available at different tiers is useful for lower barriers to adoption.

In fact, the MING stack does bring together the key open source components of an IoT system:

MING stack diagram running in the edge with balena

Let’s briefly describe each one of the services running on the MING stack:

Mosquitto is a lightweight MQTT messaging broker that is ideal for sending and receiving data from Internet of Things (IoT) devices. MQTT is a highly scalable publish and subscribe protocol that can handle millions of messages per second, making it ideal for use in large-scale IoT applications. Nowadays MQTT is getting more and more popular in the Industrial IoT space. With MQTT, data can be sent in real-time, allowing for rapid decision-making and action.
InfluxDB is a high-performance time-series database that is optimized for storing and querying large amounts of data. It is designed to handle large volumes of data at high speeds, making it ideal for use with MQTT. InfluxDB is highly scalable, so it can be used to store data from a single IoT device or from a large network of devices.
NodeRED is a powerful open-source tool that can be used to create flows for IoT devices. It allows you to connect devices, data sources, and services together, and automate the flow of data between them. With NodeRED, you can quickly build custom IoT applications that can process and analyze data in real-time.
Last but not least, Grafana is a powerful open-source platform for data visualization and monitoring. It allows you to create dashboards that can display real-time data from IoT devices, and to create alerts that can notify you of important events or anomalies. With Grafana, you can create custom dashboards that are tailored to your specific needs, and that can help you make informed decisions based on real-time data.

Deploy MING stack on a device

Each of the MING stack services is built and runs in its own container on an embedded Linux target supporting balena.io, however the MING stack can run on any Linux machine (cloud or edge) with Docker.

Hardware

Raspberry Pi 0/2/3/4 (the MING stack might work on x86 devices, however the InfluxDB version available here is built for arm32 devices to have cross-compatibility among all the arm devices, for x86 devices you can try this MING application).
SD card in case of the RPi 0/2/3/4 or USB drive in case of x86
Power supply and (optionally) ethernet cable

Software

A balenaCloud account (sign up here) and get the first 10 devices free.
balenaEtcher

Deploy

You have two options here. Use the deploy with balena button from below.

Or, you can create an application in your balenaCloud dashboard and balena push this code to it the traditional way and deploy via balena CLI.

If you are a balena CLI expert, feel free to use balena CLI. This option lets you configure in detail some aspects, like adding new services to your deploy or configure the DNS Server to use.

Sign up on balena.io
Create a new fleet on balenaCloud.
Add a new device and download the image of the balenaOS it creates.
Burn an SD card (if using a Pi), connect it to the device and boot it up.
While the device boots (it will eventually show up shortly in the balena dashboard) we will prepare the services:

cd ~/workspace
git clone https://github.com/mpous/ming
cd ming

Using balena CLI, push the code with balena push <fleet-name>
See the magic happening, your device is getting updated 🌟Over-The-Air🌟!

Mosquitto (MQTT)

Mosquitto MQTT broker is available on the port 1883 to publish messages or get subscribed from other services.

InfluxDB

InfluxDB time-series database is available on the port 8086. On MING we are running the arm32 version which used to be the version 1.7 of this InfluxData product. You can also run InfluxDB on the latest version if you want.

NodeRED

The nodeRED service is available on the port 80, however you can manage the port of the service through the balena Device Variables. We exposed nodeRED on the port 80 to enable anyone to access remotely to the nodeRED using the Device Public URL from the balenaCloud dashboard. This is useful when you try to access remotely to the device.

Device Variables

Go to the Device Variables section on the menu and add / modify these variables.

Variable Name	Default	Description
PORT	`80`	the port that exposes the Node-RED UI
USERNAME	`balena`	the Node-RED admin username
PASSWORD	`balena`	the Node-RED admin password
ENCRIPTION_KEY	`balena`	the encryption key used to store your credentials files

You must set the USERNAME and PASSWORD environment variables to be able to save or run programs in Node-RED.

To access Node-RED, use the local IP address on port 80, if you are on the same network as your device. You also can use the Public Device URL via balena to access the Node-RED UI.

nodeRED flow using the MING stack with balena

Grafana

Grafana visualization tool is available on the port 8080 (the default port is the 3000) as we would like to provide access through the Device Public URL from the balenaCloud dashboard.

What can you build with the MING stack?

Real-world examples and success stories using the MING stack with balena are already present in the balena blog. Here are a few examples:

Industrial IoT

Industrial IoT (IIoT) usage is growing fast, including the use of containers for application life management at the edge. With balena and the MING stack it is possible to rapidly develop an IIoT project with Modbus and OPC UA machinery in the factory floor as shown in this image below.

Here is an example to convert a legacy factory into a digital system by taking advantage of the MING stack.

Furthermore we built another Industrial IoT edge gateway reference architecture based on the MING stack using HiveMQ MQTT broker instead of Mosquitto. You can read more here.

The Things Stack network server

This project uses The Things Stack LoRaWAN Network Server Open Source Edition along with the MING stack. to enable the complete LoRaWAN stack (gateway + network server) running together on a single Raspberry Pi with NodeRED, InfluxDB, Grafana along with the LoRaWAN stack.

The Things Stack with the MING stack on balena

What’s next for the MING stack?

After sharing information about the MING stack with the developer community, I found that many were already using it (or subsets of it) without even knowing the name for it! One example is Alejandro, the developer from the balena community who is using the MING stack along with the Thingsboard platform. Read more here.

MQTT brokers

Mosquitto is just one of the many MQTT brokers available in the market, but there are alternatives. There are other MQTT brokers with community editions such as HiveMQ or EMQX which are currently the most popular. If you like Mosquitto, there is Cedalo, which is the company who maintains the Mosquitto MQTT broker and offers the professional or managed edition of the Mosquitto broker.

You can read more about HiveMQ integrations with balena on these Industrial IoT posts. And find here a repository to run EMQX MQTT broker with balena.

NodeRED

NodeRED simplifies the collection, processing and sending of data among services. However there are alternatives if you want to change the N of the MING stack.

One possibility is to use FlowForge locally in the device or as a cloud instance if you would like to use NodeRED as a Professional tool.

A similar approach without the no-code/low-code interface of NodeRED is the InfluxData tool Telegraf. For even greater simplicity you can use the Connector block which is a block built by balena to reduce the need to configure the Telegraf inputs and outputs. You can find an example of the MING stack with the Connector block being used in the balenair product or this Internet Speed Test application.

In the Industrial environment space, NodeRED has other competition such as N3uron or Ignition Edge.

Databases

InfluxDB is a great time series database that works perfectly with Grafana as a default data source. However there are other time series (or not) databases that can fit on your MING stack. Currently I am testing MongoDB and Redis on the MING stack. I am happy to get insights from anyone who would like to contribute to the database space for the MING stack.

Data visualization

Finally on data visualization, Grafana is a great tool. You can read more about Grafana running on balena here. However we do recommend that instead of installing vanilla Grafana, use the balena Dashboard block that makes the automatic dashboard and data sources configurations simpler.

We also recommend for small projects the IoT platforms DataCake and Ubidots which work really well to store and visualize data in a cloud solution.

Finally Streamsheets which is an Eclipse project, looks like a promising solution to explore to visualize data and create actions and relationships among data.

Other services

Alex Lennon always mentioned that he would like to see Traefik service running with the MING stack to have a reverse proxy.

I also strongly recommend adding the WiFi connect block to your project.

Feedback

If you are interested in the MING stack please feel free to contribute to the MING stack repository. Our ambition is to convert the MING stack as an open source LAMP-like stack for the Internet of Things, Edge Computing and Embedded.

The balena platform is all about reducing friction to address the challenges of our times for fleet owners. As always, let us know if you run into any issues when following this guide or getting into the MING stack. Feel free to contact us at the balena forums and share any error logs you have encountered!