Question

Somewhat notoriously, the standard OpenWRT firmware update process is not atomic. I.e. if there's a failure during an update, the device is likely to end up bricked, and requiring a manual reset. This makes it risky to push OTA updates to devices in the field.

Does anyone have experience with implementing atomic updates for Teltonika devices? There are tools like SWupdate, Mender, and RAUC that can essentially be layered into the OS image in order to achieve atomic updates, but it's not clear how to potentially apply these in the context of RutOS/OpenWRT. The closest I got is the discussion here, though it seems far from conclusive.

If Yocto or Buildroot were used to build an OpenWRT/RutOS image, then one of the above could potentially be incorporated. But based on my experience, those do not play a role in the existing firmware SDK.

Any pointers on whether such tools can be incorporated, or any other ideas regarding atomic upgrades?

Answer 1

Yes, sysupgrade is not atomic. But there is only a very small chance, to be caught with a bricked device, i.e. when having a power fail during writing new image to flash.

Having large fleets of openwrt devices, I am even doing fw upgrades over LTE. Can not remember of any problems.

Using custom firmware, based on "official" openwrt, for RUT955 and other openwrt devices.

Comments

Thanks @augustus_meyer, it's great to hear about your positive experience! My concern is that we're looking at deploying many devices in remote areas, some in locations with poor power quality. So the possibility of a power outage during an update is unfortunately quite real - or at least real enough that I'd rather not risk it. But indeed, devices like the RUT955 don't necessarily even have enough flash to enable something like recovery or A/B booting. So might need to forget about atomic updates for now. Maybe finding a good UPS would be more worthwhile...

---

You will have statistics regarding power outages on your devices. These ones compared to may be 5 minutes of critical time, when writing new image to flash. And how often you need to write a new image ? May be, once in a year, I guess. Of course, a small battery backup (buffer battery)  would be 100% solution, having the additional benefit of taking care of power fluctuations.