This article is part of IoT Architecture Series – https://navveenbalani.dev/index.php/articles/internet-of-things-architecture-components-and-stack-view/
In this article, we would look at how to realize the IoT use case using IBM IoT platform. IBM provides the Watson IoT Platform, which is a fully managed, cloud-hosted service that makes it simple to derive value from Internet of Things (IoT) devices. Together with IBM PaaS platform (Bluemix), you can rapidly build IoT applications from the catalog of services available in Bluemix. The catalog of services for an IoT application typically includes – storage services, rules, analytics services, stream analytics and machine learning. We went through this earlier when we talked about Microsoft IoT stack in detail. The strategy adopted by Microsoft and IBM is pretty much the same.
Note – IBM Bluemix is an implementation of IBM’s Open Cloud Architecture based on Cloud Foundry. Cloud Foundry is an open source industry standard platform for building cloud applications in a vendor neutral way.
In the first article, we had discussed about a generic Enterprise IoT stack. The following shows our representation on how the IBM IoT services can be mapped to our generic Enterprise IoT stack.
Let’s go through the components in details
Watson IoT Platform Device Client Library
These are client libraries, which make it easier to connect devices to the Watson IoT Platform. The Watson IoT Platform provides client library in Python, Java, Embedded C, Node.js, and C#. For details on the client library API, please look at https://docs.internetofthings.ibmcloud.com/index.html
IBM Watson IoT Platform also offers device recipes, which provide code and detail steps to connect various devices to IBM Watson IoT Platform using MQTT protocol. (IBM Watson IoT Platform currently provides support for only MQTT protocol.). The device recipe is contributed by IBM, third party vendors, and the community.
You can also build your own device recipe using the open source Eclipse-based Paho library (http://www.eclipse.org/paho/), which provides client implementations of MQTT and MQTT-SN protocol for required devices. You need to pick up a supported Paho library, like a Net for Windows 10 devices or C client for devices supporting C programming language.
Following is the code snippet that can be used by devices to connect to IBM Watson IoT Platform using Watson IoT Platform Node.js library:
var Client = require("ibmIoTF").IoTFDevice;
var config ={
"org":"organization",
"id":"deviceId",
"type":"deviceType",
"auth-method":"token",
"auth-token":"authToken"
};
var deviceClient =new Client(config);
deviceClient.connect();
deviceClient.on("connect",function(){
var myQosLevel=2// QOS, message is delivered exactly once
client.publish("status","json",'{"d" : { "cpu" : 60, "mem" : 50 }}', myQosLevel);
});
Watson IoT Platform
IBM Watson IoT Platform lets you connect a wide variety of devices securely and collect and manage data in time series. Watson IoT Platform follows the pub-sub model, where you create topics for communication between publishers (like devices) and subscribers. Watson IoT Platform also addresses security by providing a secure channel for devices to connect to Watson IoT Platform using MQTT over TLS. It provides device management infrastructure like web dashboards for managing and monitoring the data usage, providing a set of APIs for registering the devices, getting a list of connected devices, events generated by devices and device diagnostic logs. The APIs can be accessed at: https://docs.internetofthings.ibmcloud.com/devices/mqtt.html
To get started with Watson IoT Platform, you select the Internet of Things Platform service (i.e. available at https://console.ng.bluemix.net/catalog/services/internet-of-things-platform/) in Bluemix and create an instance. This will assign an organization unit and ID, which is a unique ID provided by the Watson IoT Platform. Organizations act like a domain where data is only accessible from devices and applications in that domain. Once devices are registered, devices and API keys are bound to that organization. When an application connects to the Watson IoT Platform service using an API key, it registers with the organization that “owns” the API key. Using this combination, it makes it impossible for other devices and applications which are not part of the same organization to communicate with each other. Watson IoT Platform also provides storage of device data based on terms specified (in months). The following image shows a snapshot of Watson IoT Platform dashboard, depicting one registered device.
Once you register the device, you get an authorization code, which is used, for connecting the device to the Watson IoT Platform. As part of device registration process, you can also store the device metadata like serial number, firmware or custom attributes in JSON, which can be used further for device management activities.
Watson IoT Platform allows for firmware updates, reboot, and factory reset commands, which need to be supported and executed by the device. Think of this as a set of device commands, which a device agent runs on the device when it receives the command input from Watson IoT Platform dashboard. You specify the command actions through the Action tab in the dashboard or the device command APIs. The APIs supported by Watson IoT Platform and sample message formats are listed at the URL – https://docs.internetofthings.ibmcloud.com/devices/device_mgmt/requests.html.
The IBM Watson IoT Platform platform is powered by IBM’s core products and services which includes MessageSight (rapid bi-directional messaging), Informix TimeSeries (time series database), Data Power Gateway (secure gateway and integration), Cloudant (NoSQL for storing device metadata), WebSphere Application Server Liberty Core (runtime for Watson IoT Platform dashboard/web apps) and a host of other open source projects.
Tip – To try a one-minute quick demo to see MQTT accelerometer events from your phone to Watson IoT Platform, visit http://www.ibm.com/cloud-computing/bluemix/internet-of-things/(look for play now button in ‘Try it out now with our cool sample app’ section.)
Storage
Watson IoT Platform by default provides data storage capability and time series view of the data from devices. But it’s better to use a separate storage for storing, accessing and analyzing volumes of historical data, which can be used by services and applications that need to consume or augment the data with other data sources. For the choice of storage, Bluemix offers a variety of scalable database storage options like MongoDB, PostgreSQL, Cloudant NoSQL, IBM DB2, ObjectStorage and Informix Time Series data. The choice of using a specific database depends on the use case requirements, the volume of data that needs to be stored and what kind of processing needs to be executed. Typically MongoDB is widely used, and you can define the schema (document design), which is tuned for better performance, especially when you are dealing with time series domain.
Node-RED
Node-RED is a tool for wiring together hardware devices, APIs, and online services in new and interesting ways. Bluemix provides a Node-RED visual editor for creating a flow for your IoT application. For instance, you can create a flow, which would listen to messages (topics from Watson IoT Platform), create rules in JavaScript, execute various other Bluemix services or invoke external services. Every functionality is available as a node and can be dragged in the visual editor to create flows by assembling various services and rules for triggering actions. You can even add a custom node to provide custom functionality. This feature is very useful when you want to quickly build a small proof-of-concept (POC) of an IoT application.
You can use the starter template – ‘Watson IoT Platform Starter’ in the Bluemix catalog to get started quickly with Watson IoT Platform and Node-RED. The starter template includes a sample flow that processes temperature readings from a simulated device. The image below shows the Watson IoT Platform Starter Kit in the catalog:
The following image shows the sample Node-RED workflow that receives data from the device through the input connector and throws an alert if the temperature is greater than some value.
IoT Real-Time Insights The IoT Real-time Insights is a recent addition to IBM IoT offerings, which works in conjunction with Watson IoT Platform to monitor and contextualize data from devices, and apply rules to trigger action. The offering allows you to combine existing master data available from asset management products (like Maximo from IBM) with runtime data from devices for triggering automated action (like an SMS or creating a work order in Maximo). We had discussed the use of asset management and how the data can be leveraged, in the manufacturing use case in earlier article.
Note – The problem of integrating asset management solutions, IoT devices, workflow, etc. needs to be looked at holistically starting from how you design a connected product to the realization of the end product. In future, we could see a tight end to end integration between asset management products and IoT platforms.
IBM Streaming Analytics
IBM Streaming Analytics service provides analyzing in-flight data in real-time. For those who are familiar with IBM InfoSphere Stream product, the IBM Streaming Analytics is the same technology now available on Bluemix. IBM Streaming Analytics service can analyze millions of events per second and perform analysis with predictable low latency, which is a key requirement for IoT application dealing with millions of connected devices. The service allows you to ingest, analyze and correlate information coming from thousands of real-time sources. In order to build stream applications, you need to download Streams Studio (an eclipse based environment) separately to build streams and deploy via the Streaming Analytics dashboard. At the time of writing this book, we could not find seamless integration of IBM Streaming Analytics with Watson IoT Platform. One approach is to use compatible toolkit and adapters (MQTTSource from com.ibm.streamsx.messaging toolkit) in Streams Studio to integrate with Watson IoT Platform. With this approach, the stream flow gets invoked when a message arrives at the MQTT topic defined in Watson IoT Platform, and you can execute the required logic using streamflow components. Once the stream is deployed in Bluemix, you can invoke the stream as part of the Node-RED flow or in any application using the Stream REST API.
Note – When we talk about seamless integration in the above context, what we really mean is the ability of components like Streaming Analytics or any components like Apache Spark Streaming to consume data directly from Watson IoT Platform without the need to build custom adapters or Node-RED flows. We assume this could be simplified through declarative configurations between Watson IoT Platform and streaming (or any other) components that need to consume data from Watson IoT Platform.
Apache Spark and Apache Spark Streaming
Apache Spark is a fast and general purpose engine for big data processing, with built-in modules for streaming, SQL, machine learning and graph processing. Bluemix provides Apache Spark cluster service. Apache Spark is used for aggregating and transforming large volumes of data, building analytics solution using Scala and Python, streaming and analyzing data in real-time and even building machine learning models using Spark-MLlib module.
The Apache Spark streaming component provides analyzing streaming data in real-time. This provides one more option for real-time streaming other than IBM Streaming Analytics. The choice of picking up one technology as opposed to other depends on use case requirements, flexibility, the maturity of the product, availability of skills and various other factors (and having doubts, you can reach out to us through the forum at http://enterpriseiotbook.com).
Currently, there is no integration provided for Apache Spark streaming service to consume data directly from Watson IoT Platform, but it could be just a matter of building connectivity adapters, and it’s a point in time statement. There are alternate options, like storing the data from Watson IoT Platform into one of the storage database (like ObjectStorage or MongoDB) and apply the Spark programming model. For example, using Python or Scala to create RDD (Resilient Data sources, abstract data model) to aggregate, transform and analyze data. Please note that this is just our approach or one way to integrate Apache Spark service with Watson IoT Platform. We may see an integrated standard solution in the future.
Machine Learning
IBM provides the predictive analytics service on Bluemix. To build predictive models, you use IBM SPSS Modeler and deploy the model using the administration dashboard of the predictive analytics service. Currently, SPSS Modeler is not available with Bluemix; you need to purchase it separately or get an evaluation copy. The IBM SPSS Modeler is used to build and test the predictive model and then the model needs to be exported and deployed as part of predictive analytics service. To access the model at runtime, you can use the REST API from your application (from the Node-RED flow) or web application.
Custom Solutions
These are end-to-end IoT solutions developed using Watson IoT Platform and Bluemix services. Apart from the above services, Bluemix provides a bunch of other services that can be used for building IoT applications like – Geospatial Analytics to track when devices enter or leave the designated boundaries or mobile push services to send an alert based on the analysis and Embeddable Reporting service to include dashboards in your applications. You can find all the services in the catalog view of Bluemix at https://console.ng.Bluemix.net/catalog/.
As part of Watson IoT Platform Recipes page, you could also find Application Recipes, which are end-to-end examples that can be used as a starting point for building IoT applications. The example application recipes include machine condition monitoring, geospatial analysis, track and trace, real-time dashboards for wearable and many more. These application recipes use Watson IoT Platform and various Bluemix services and are contributed by IBM and the open source community.
In the next article, we will implement the the connected elevator solution that we had discussed in earlier article.
