Introduction
The announcement for the end of Google Cloud Platform’s IoT Core came earlier than expected, and it’s a huge challenge for the world of IoT service development and maintenance. GCP IoT Core will be discontinued on August 16th, 2023.
If you are affected by the retirement of the GCP IoT Core this is the right time to examine what it is and how you have to migrate from it.
How does GCP IoT Core Work?
GCP IoT Core is a device registry, a device configuration, and a device communication system. Every device gets registered with a unique identifier and one or more private/public key pairs. To communicate with IoT Core from a device, each device creates a JWT that is signed with the private key of the device key pair.
GCP IoT Core uses the MQTT protocol to transfer data from the device and send data to the device. Devices may also communicate with the IoT Core via HTTP request to publish a message. A business logic can manage the IoT Core through an API or directly utilizing provided SDKs. Besides this, the GCP cloud console provides a web interface to manage the IoT Core configuration, devices, key pair refreshment and PubSub linkage.
Device registry
A device registry in GCP IoT Core represents a management unit for a group of devices that are registered to this single device registry. Each registry is identified by a unique identifier. Each device is registered with a unique identifier to one and only one registry. A registered device gets a public/private key pair assigned and the device holds the private key. Communication between the registry and the device takes place by MQTT protocol or an HTTP API, always authenticated with a JWT (JSON Web Token) that is signed by the device’s private key. The business logic running in the customer’s backend will communicate to a registry and the contained devices by giving the registry identifier and the specific device identifier. Ingoing data coming from the devices in one registry are collected in a GCP PubSub topic. For this single topic per registry, subscriptions can be configured and these subscriptions are listened to in the business logic.
Device authentication
The GCP IoT Core can check and prove the authenticity of a device by using the key pair registered for every device. A device can only communicate with the GCP IoT Core when providing a valid private key signed JWT. When there is one private key exploit only the communication for this single device is insecure, all other devices remain on a secure communication.
Device Connectivity State
The GCP IoT Core can check if a device is subscribed to the device-specific MQTT topic. Thus commands sent to the device will fail early when the device is not connected. Then the business logic can react early on a failed command send, without waiting for an acknowledgement from the device or a timeout.
Device configuration
Every device has a versioned configuration that is sent on a specific topic on the MQTT transfer layer and is a retained message. Hence, a device can pick up the latest configuration from the retained topic without needing to request the latest configuration. The configuration is transferred base64 encoded. Setting the configuration in the IoT Core also takes a base64 encoded configuration.
The GCP IoT Core console user interface provides a way to review older versions or set new versions as a string field. A configuration is per device and has no support for inheritance, moreover the registry cannot hold generic configuration parameters, such as they are inherited from the device and overwritten by device-specific values.
Device communication
A device can communicate with an MQTT client to the single device telemetry topic. Therefore the device has to create a JWT with a specific payload and sign this token with the device’s private key. A device can receive configuration and commands from the IoT Core and sends event and state messages to the IoT Core. For each device, in a registry, a unique topic is created on the GCP IoT Core MQTT broker and the device can only subscribe and publish this specific topic and its under-topics.
Working with the GCP IoT Core – some opinionated field experience
Before working for balena I have worked for a smart fridge and micro market company that used the GCP IoT Core and gained a good amount of experience using it. From this experience, using the GCP IoT Core for a fleet of multiservice devices, I would recall it is convenient to use but also challenging. The usage and implementation on the cloud layer of the IoT service have been straightforward.
Sending commands to a device is conveniently solved as the IoT Core is aware of the online status of a device and the request to send a command to a device will fail early. Thus, the business logic can react early on the device status without implementing other checks for the device’s online status.
Updating a configuration is as convenient as sending a command but will not fail on an offline device. The IoT Core retains the configuration and the device coming back online will receive the latest configuration. In the case of a multi-service application running on a device, the configuration received may need to be distributed between services and implementations on the device.
Limitations that I’ve seen were about the incoming data stream from a device mapped to a PubSub topic that didn’t allow multiple topics to be collected on one subscription. Thus, for multiple fleets represented by multiple device registries the business logic had to implement multiple active subscriptions to receive data for all fleets. That added a layer of business logic to manage fleet subscriptions.
For the data coming from the device, there is no automated way of an acknowledgment from the device to business logic, so this was added in the business logic as an application layer handshake for single transactions. For example, when the business logic decided to start a session on a device, the device had to acknowledge that the transaction was created on the device. After ending a session on the device and sending a transaction end to the business logic, the business logic had to finally acknowledge by an IoT command that it received the session and ended the session on the cloud site as well.
Conclusion
Depending if you have fully leveraged the GCP IoT Core features you now have to find replacements for all components of the IoT Core or only parts. The key components can be split into the device registration and configuration and the device communication.
Thus, you now need to rethink how you want to achieve a device registration and configuration service besides a secure device communication that can ingest data from your device to your business logic and send messages to the device.
After figuring out how to solve the device registration and configuration and rebuild a secure communication between your devices and your business logic, you have to re-provision all these changes to each of your devices in the field. This may include cycling private keys, reconfiguring cloud service endpoints, re-requesting allowlisting changes for client site firewalls.
Next steps
Balena as an IoT fleet management tool which provides you all you need to run your IoT devices in the field. From the balenaOS up to the balenaCloud convenience fleet management dashboard, our goal is to reduce your friction as a fleet owner. With our experience of running and enabling our customers to manage thousands of fleets containing thousands of devices, we resonate with frustration and resignation on these huge changes to be implemented.
Already now you are able to migrate your devices to balena.io using the balenaOS which is available for +75 different device types. This will allow you to gradually move your devices to an IoT focussed cloud service as a fleet and device management platform.
As we understand that an IoT stack is mostly highly sophisticated and does not only involve the central management of devices but a secure and reliable configuration and communication we are lining up a series of posts that will cover:
– Mapping of GCP IoT Core features to balenaCloud,
– Migration procedure using balena blocks to replace the GCP IoT Core configuration
– Migration to a third party IoT Core replacement using our provider agnostic cloud relay block.
In the meantime, sign up to balena, create a fleet and provision one of the devices on your desk. You will experience a frictionless provisioning of your IoT devices with balena and learn how all devices can be managed centrally. By doing this, you may already understand that balena covers parts of the GCP IoT Core and you may only need to rethink and implement the messaging between the device and your business logic.
The balena platform is all about reducing friction for fleet owners. If you use GCP IoT Core in different ways, tell us about your IoT stack definition as we would like to help you. And feel free to add a comment below or contact us on our forums.
