Measuring Throughput Of IoT Gateways
The key part of the process of tailoring a perfect IoT solution is choosing proper hardware for the gateway device. In general, the more expensive the gateway hardware is, the more messages per second you can process. On the other end, the more expensive the gateway device is, the less affordable your IoT solution
The key part of the process of tailoring a perfect IoT solution is choosing proper hardware for the gateway device. In general, the more expensive the gateway hardware is, the more messages per second you can process. On the other end, the more expensive the gateway device is, the less affordable your IoT solution becomes for the end client. This trade-off is the main reason why you might like to have a proper tool for measuring IoT message flow scenarios across multiple devices in the unified way.
Rhiot Performance Testing Framework
Rhiot comes with a Performance Testing Framework that can be used to define the hardware profiles and test scenarios. The performance framework takes care of detecting the devices connected to your local network, deploying the test application into the devices, executing the tests and generating results in the form of a human-readable chart. For example, sample output for MQTT QOS testing could generate the following diagram:
When to use the Performance Testing Framework
The Performance Testing Framework is an excellent solution when you need to figure out how the different field hardware setups perform against a given task. In order to find out, you just need to connect your devices to the local network, execute the performance testing application and compare the generated diagrams.
This section covers hardware profiles for the performance tests. The profiles are used to describe a particular hardware configuration that can be used as a target device for the performance benchmark. Every performance test definition can be executed on a particular hardware profile.
An example of a hardware profile can be Raspberry PI 2 B+ (aka RPI2). The RPI2 hardware profile is just the Raspberry Pi 2 B+ model equipped with a network connector, e.g. WiFi adapter or the ethernet cable. Let us assume that the device is running the Raspbian operating system (version 2015-05-05).
The other profile we defined for Rhiot is the Raspberry is the PI 2 B+ with BU353 (aka RPI2_BU353). The RPI2_BU353 hardware profile is the same as the RPI2 profile, but it is additionally equipped with the BU353 GPS receiver plugged into the USB port.
Running the performance tester
The easiest way to run the performance benchmark is to connect the target device – for example, Rapsberry Pi – into your local network via WiFi or Ethernet cable and start the tester as a Docker container using the following command:
docker run -v=/tmp/gateway-performance:/tmp/gateway-performance --net=host camellabs/performance-of RPI2
Keep in mind that RPI2 can be replaced with some other supported hardware profile, e.g. like RPI2_BU353. The performance tester detects the tests that can be executed for the given hardware profile, deploys the gateway software onto the target device, executes the tests and collects the results. When the execution of the benchmark comes to an end, the result diagrams are located in the /tmp/gateway-performance directory or any other directory you specified when executing the command above. A sample diagram may look like this:
Keep in mind that we currently assume that your Raspberry Pi has a default Raspbian SSH account available (username: pi / password: raspberry).
The future of the framework
I intend to further develop the gateway Performance Testing Framework to make it the tool for IoT gateways sizing. In particular, I will add more benchmarks and supported hardware profiles. The performance framework is also a great tool to perform fully automated end-to-end integration tests on real hardware. I would definitely like to see the PTF as the tool for running Jenkins-based integration tests executed on real devices.