Today, our concept for IoT app enablement graduates from a concept to a first-class entity in the balenaCloud dashboard, with plenty of exciting roads still left to travel.
This post is about what you can do with blocks today, why we needed to correct course, and where we are heading with Blocks in the future. Buckle up– here we go!
What’s a Block again?
Blocks have been around for a while now at balena, and I’ve written about what they are and how to make your own in previous posts. In a nutshell, they’re a container you can drop into your fleet to provide functionality to handle common tasks.
My go-to example is always the Browser Block which provides a Chromium browser on any connected display and automatically displays any HTTP output also running on the device. A developer creating a Point-of-Sale system can focus on their payment application, and let the browser Block take care of putting it on the screen.
You can find lots of other Blocks on balenaHub.
What’s new?
Today we are enabling users to add and manage Blocks in the balenaCloud dashboard, and make them visible to others in balenaHub should they choose to. Blocks are now a first-class entity in balenaCloud, and so can be found at the same level as fleets within a user’s organization:
Users can add and manage Blocks using all of the same tools they are familiar with for managing their fleets. This includes managing releases…
…and changing the Block’s settings, including whether it should be made public and have it appear in balenaHub. Here I’ve toggled the browser Block to be visible:
Here is the Block appearing in balenaHub for everyone to find and use:
Couldn’t we self-submit blocks at one point?
You’re right. There was a time, not long after we first created balenaHub where you could add Blocks yourself. However, for the same reasons we renamed applications to fleets we removed the ability to add Blocks, because our initial implementation was confusing and not in line with the direction we want to take.
Back then developers had to add their Block to the balenaCloud dashboard as an application, and then use a setting to tell balenaHub that it was actually a Block. That led to confusion and friction, so we removed it and worked on this implementation, which is much more intuitive and unlocks a host more functionality we’re now working on.
Let me take you on a tour of the roadmap.
Where are we headed next? (roadmap time)
The changes we have made definitely make it more intuitive how to add and manage your Blocks today, but we’re not stopping there. We don’t like to stand still at balena, and we think the concept of Blocks has a lot of potential growth, so I’d like to share with you some high-level elements of the roadmap.
Block image management
The intention of Blocks has and always will be to reduce the friction for edge developers creating their fleets. The best experience for using a Block is when you can simply add an image reference to your docker-compose file, since that saves you having to clone any source code and build it yourself:
yml
services:
browser:
image: balenablocks/browser
privileged: true
network_mode: host
ports:
- '5011' # management API (optional)
- '35173' # Chromium debugging port (optional)
To enable this user flow, however, the developer of the Block currently has to do all the heavy lifting: building the Block, pushing the resulting image to a container registry such as Docker Hub, and then maintaining those images and ensuring they stay in sync with the source code. None of that is hard, but it adds friction and could conceivably stop the developer from making the Block public, in order to avoid that maintenance burden.
By enabling them in the balenaCloud dashboard, we’re already providing the tools for developers to build and manage releases of their Blocks. The built images will already be stored in balena’s own container registry, and we are now working on a way to give each Block a friendly image name that can then be used directly in a docker-compose file.
In the future, after deploying the Block to their balenaCloud dashboard, the developer will be able to see the Block’s image name on the summary screen. They can then add this name to a Fleet’s docker-compose, and when the Fleet is deployed balena’s builder will fetch the image from balena’s registry, just like pulling any other docker image. If the developer makes their Block public, this image name will also be shown in the Block’s card in balenaHub, so that other users can use it in their own Fleets.
Watch this space, as we’ll be letting you know more details about this feature as it gets closer to being available in production.
Managing Block Deployments
Another area we are focusing on at the moment, is helping developers to manage the deployment of their fleets and Blocks to balenaCloud. To this point the workflow has included doing manual deployments via the balena CLI in order to push changes to their Fleets and Blocks. Although this isn’t complex, it does add another job for the developer and requires context switching in order to initiate and monitor progress. What we want to enable is an experience much more integrated with the developer’s normal workflow.
Our approach is to release the Deploy to Balena Github Action. This will have a dedicated blog post about it coming your way soon – but here’s a sneak preview:
Github actions are a way to create automated workflows which act upon the code in your github repository. Our Deploy to Balena action can be added to the repository where you are developing your fleet or block. When you push a new version of the code, the action runs to create either a draft or final release on balenaCloud, depending on the workflow you follow.
Combined with the block image management explained above, this gives Block developers a full deployment management workflow, from development to a released image, all automated and based on tools they already use.
Automated Testing
The final piece in the developer workflow is how to ensure that the block that worked while it was being developed, still works once it has been deployed to balenaCloud. Perhaps the block interacts with other blocks, custom services or external resources, and the developer needs to test that this all still works after making a change.
We are very early in our thinking about how we want to enable automated testing in balenaCloud, but I’m keen to work in the open here, communicate our early ideas and encourage feedback straight away.
The development experience we want to enable for blocks is one where a deployment to balenaCloud then sets off automated tests. These simulate the block operating as part of a fleet, and record the success or failure of the new release. A failure would cause the release to be marked as such, and not affect the block image or any fleets using it.
We don’t want the tests themselves becoming part of the block, and so we are looking into methods such as “System Under Test” or SUTs. This method allows a developer to create a testing specific docker compose file which references images to pull in addition to the block itself. These additional services are used to test the block. A successful result would then cause the release of the block to continue, and a failure would halt the process and report the state back to the developer.
What we want to avoid is having to go back to treating a block like a Fleet, again, by necessitating that developers have to add other services alongside the block, and provision test devices, just to make sure the block functions correctly.
If you have any ideas or feedback about our thoughts here, we’d love to hear them!
Blocks User Experience
Finally, we are also thinking about how we want to improve the workflow for users of blocks. Currently, a user has to find the image name of the block, and then add this and any other necessary lines to their Docker Compose file.
The current experience works, as developers have done a good job of providing snippets of Docker Compose that users can copy and paste into their fleet, but it could be much better. Copying and pasting YAML is not what Edge developers should be spending their time doing. Again, we are quite early in our thoughts here, but a vision of what we want to build is beginning to form (sound off in the comments below if you have any ideas– we welcome it).
Ideally, we want developers to be able to find blocks, select them, and add them to the fleet they are building, all in the same UI. We want that UI to handle any configuration where at all possible, and enable the developer to simply handle any the UI can’t do automatically. For instance adding a block should automatically add all of those pesky Docker Compose lines, such as adding labels and volumes. But the user should be able to change the default restart policy and deconflict blocks wanting to use the same network port.
We also want to make sure that the developer workflow allows them to create a fleet by combining blocks, but then take the result and add their own custom services, without that feeling disjointed or clunky.
Whether this UI looks and feels like Node-RED, happens inside balenaHub or GitHub, is browser-based or a plugin for an IDE like VSCode, or all of those, we’re not sure yet and fully open to hearing any ideas you may have. Feel free to drop them into the comments below this post.
Blocks self-submissions are back!
We think that the best blocks come from practical, grounded everyday use cases from our community. Yes, our first self-submission attempt was bumpy, but we think we have a much better solution in place now.
Hackers, makers, edge developers, and everyone of all skill levels are encouraged to submit blocks to balenaHub. If after reading all of this, you’re burning to add your awesome block, go for it.
NOTE: Blocks submissions work from the balenaCloud dashboard if you follow the instructions above. We’re working quickly to freshen up the balenaHub site to reflect this change!
If you have a seedling of an idea that you want to grow, come hang out with us on the balenaForums. Everyone there is warm, welcoming, and interested in helping you succeed.
If you have any thoughts, questions, and observations about our big moves here, sound off in the comments below.
Any questions, comments, suggestions – let me know, I want your feedback to help me make a better product.