I've just had exactly the same problem, and for others looking for this, heres where I think it went wrong.
I didnt have a Wired WAN cable plugged in, so downloaded the modem firmware via cellular. Also, our config has a 'ping reboot' enabled in case the RUTX11 loses contact with the internet for longer than x. After the modem firmware update failed, I checked the logs and noticed that there had been a 'ping reboot' of the modem, so I suspect that the modem firmware update took long enough for the 'ping reboot' to trigger, rebooting the router mid update, and bricking the modem.
So the lesson would be to ensure that you have a wired WAN connection to the RUTX11 before doing a modem firmware update, or make sure you remember to disable the 'ping reboot'.
Thanks for PMing me the manual firmware update instructions Vykintas.