Over-the-air runtime updates of IoT gateways

Over-the-air runtime updates of IoT gateways
Author: Henryk Konsek

Adding new components at runtime Apache Camel relies on the components to provide connectors which can be used to consume messages from the various endpoints as well as send messages to the endpoints. For example, a gateway based on Apache Camel could use a/the Paho MQTT component to consume control commands from the

Adding new components at runtime

Apache Camel relies on the components to provide connectors which can be used to consume messages from the various endpoints as well as send messages to the endpoints. For example, a gateway based on Apache Camel could use a/the Paho MQTT component to consume control commands from the data center and at the same time it might use the Netty component to communicate with backend REST services.

Fabric Docker Png

Now imagine that at some point we decide to add the Salesforce component to our gateway, so that the copy of the message we sent to the backend REST service can also be sent directly to our Salesforce account.

Fabric Docker 2 B 25281 2529 Png

Can we, somehow, dynamically add a jar containing the Salesforce component to all our production gateway devices deployed in the field? Since the gateways are constantly processing messages from sensors, we would like to avoid any restarts of the devices.

Dynamic updates using Camel Grape endpoints

Camel 2.16 comes with a new Groovy-based Grape component. Grape is part of the Groovy language – it is a library that can download other libraries and add them to the classpath of the current JVM. The Grape component downloads jar libraries from file repositories (including Maven repos) and adds these jars dynamically to the application classpath. As soon as the library is downloaded, Camel can take advantage of new components and data formats provided by the jar files. For example, the following Camel route will download and install the Camel Saleforce component:

from("direct:start").
  to("grape:org.apache.camel/camel-salesforce/2.15.2");

After the Saleforce jar has been loaded, you can use it in the Camel routes; it all happens without a restart of the gateway application:

ProducerTemplate template = camelContext.createProducerTemplate();
template.sendBody("direct:dynamicEndpoint", "salesforce:createSObject");
...
from("direct:dynamicEndpoint").
  recipientList().body();

Over-the-air updates

How can we dynamically update many gateways at once using over-the-air technology? As the gateways usually run in environments with unreliable network connection, I recommend that the gateways should listen to MQTT messages sent from the data center. An MQTT client is capable of handling unstable network connectivity. An MQTT client installed on the gateway should connect to a given “update” topic (let’s refer to it as over-the-air topic). Whenever we want to load the component dynamically, we can simply use an MQTT over-the-air topic and send the Maven coordinates of the artifact we want to be downloaded to the devices. We can use the Camel Paho component for this purpose:

// This code is executed on the data center side.
ProducerTemplate template = camelContext.createProducerTemplate();
// Tell all our devices connected to the over-the-air MQTT topic
// to download Camel Salesforce connector.
template.sendBody("paho:over-the-air", "org.apache.camel/camel-salesforce/2.15.2");

The following Camel route deployed on gateway field devices can be used to receive the update requests from the MQTT broker and send them to the Grape component:

from("paho:over-the-air"). to("grape:over-the-air");

That’s all you need to do to fetch and load a new library into all of your gateway devices.

Fabric Docker 2 B 25282 2529 Png

Dealing with the download failures

You may wonder what happens when the Grape component cannot download a requested artifact, for example, because of network malfunctioning, which is, by the way, a common occurrence in field environments. The is the moment when Camel and its Camel error handler redelivery policy kicks in. You can easily instruct Camel as to what retry strategy should be used to eventually fetch and load the required dependency. All you have to do is to use a bit of the Camel DSL:

errorHandler(defaultErrorHandler().maximumRedeliveries(10).useExponentialBackOff());

When using this approach, you can be sure that Camel will keep trying to download the requested library. After a certain number of attempts to download the artifact, you may want to send back the error MQTT message back to the data center, so that your operations team can be notified about the upgrade problems at a given device.

Loading patches on the gateway restart

Another interesting question is if the patches loaded by the Grape component will be persisted after the gateway restarts. Yes, indeed – the Camel Grape component keeps track of the patches it loaded – by default the list of the patches deployed is stored in the device file system using a plain text file. If you would like to tell Camel to load the installed patches, add the GrapeEndpoint.loadPatches method call into your route definition:

import static org.apache.camel.component.grape.GrapeEndpoint.loadPatches;
...
camelContext.addRoutes(new RouteBuilder() {
  @Override
  public void configure() throws Exception {
    loadPatches(camelContext);
  }
});

Listing patches installed on the gateway

You might also be interested in listing the patches installed on a particular gateway device. The following route demonstrates how to connect embedded HTTP server based on the Netty with the Grape component:

import static org.apache.camel.component.grape.GrapeEndpoint.loadPatches;
...
camelContext.addRoutes(new RouteBuilder() {
  @Override
  public void configure() throws Exception {
    loadPatches(camelContext);

    from("netty4-http:http://0.0.0.0:80/patches").
      setHeader(GrapeConstants.GRAPE\_COMMAND, GrapeCommand.listPatches).
      to("grape:listPatches");
  }
});

… you can now open the following URL in the web browser and see the list of the installed artifacts:

$ curl http://your.gateway.com/patches
org.apache.camel/camel-salesforce/2.15.2
org.apache.camel/camel-ftp/2.15.2

Summary

As you can see, Camel in combination with Groovy class loading capabilities provides hot updates features, which are extremely useful for IoT systems. Camel’s capability to provide runtime updates over-the-air is an interesting alternative for OSGi deployments. With the Camel Grape component you can take advantage of dynamic class loading while still keeping all the benefits of the flat classpath.