The necessary evil: Enterprise IoT needs a new middleware

IoT is rapidly becoming a foundational element of the next generation of enterprise software solutions. As IoT evolves in the enterprise organizations are encountering new challenges integrating data between sensors, smart devices and enterprises. Quickly, enterprises are discovering a new reality: IoT requires a new type of middleware.

internet of things laptop

The Internet of Things (IoT) has become one of the fastest growing technology trends in the enterprise in the last few years. From productivity wearables to extremely sophisticated industrial deployments of sensors, enterprise IoT solutions are dominating the technology agenda of modern enterprises. The emergence of enterprise IoT has brought together a new set of integration challenges connecting the new world of smart devices with existing line of business systems.

The integration challenges created by enterprise IoT topologies have been unprecedented in the enterprise. Never before have companies encountered integration scenarios involving such a large number of endpoints, such large volumes of data, and such heterogeneous environments. Quickly, enterprises are discovering a new reality: IoT requires a new type of middleware

Really? A new middleware?

With decades of history implementing expensive integration technologies, enterprises are sure to be hesitant about embracing a new middleware stack for IoT solutions. However, after examining the characteristics of enterprise IoT solutions, organizations quickly realize that they require very unique integration capabilities that are not available in traditional middleware solutions.

  • Thousands of endpoints: Enterprise IoT middleware technologies need to be optimized to support tens of thousands of devices exchanging data concurrently. This model highly contrasts with middleware integration platforms that operate integrating a small number of line of business systems.
  • Stream messaging: Collecting streams of data from smart devices and sensors is a fundamental element of IoT solutions. Traditional middleware solutions have been designed to operate in batched or message-based modes that are not a very good fit for enterprise IoT topologies.
  • Dynamic number of endpoints: Enterprise IoT topologies should be designed to operate with variable numbers of sensors and smart devices. This model drastically contrasts with traditional enterprise integration platforms which are typically designed to operate with a fixed number of endpoints. 
  • Message volume: The ability to process large volumes of concurrent messages is another key feature of enterprise IoT solutions. In order to process those messages, enterprises require middleware solutions that support elastic scalability models which are not present in traditional integration brokers.
  • New protocols: IoT solutions are notorious for requiring smart devices using protocols that are completely unfamiliar to traditional enterprise integration solution. In order to address this challenge, enterprises should rely on middleware technologies that provide seamless integration with IoT protocols and standards.
  • Real time integration: Many of the integration patterns in enterprise IoT solutions require processing decisions and executing actions in fractions of a second. This requirement proves impossible for most traditional enterprise middleware platforms.
  • Embedded and brokered integration: We tend to associate middleware technologies with brokered integration models in which a centralized entity processes the data generated by different endpoints. While this integration model is certainly valid in enterprise IoT solutions, many other scenarios require embedded models in which the middleware capabilities are included in smart devices. Consider scenarios such as vehicle telematics in which thousands of sensors are exchanging information inside a vehicle. An IoT middleware platform used in this scenario needs to be flexible enough to run within the vehicle hardware.

 Enterprise IoT middleware platforms

The argument that enterprise IoT requires a new type of middleware is far from being just a theoretic exercise. In the last few years, the industry has produced a series of technologies that can be considered the first generation of enterprise IoT middleware platforms.

Apache NiFi

Developed by the National Security Agency (NSA), Apache NiFi is a platform for real time data integration. Many of the capabilities of NiFi such as real time multi-directional communication, data provenance, seamless scalability, and support for IoT protocols has quickly made it a favorite for IoT data integration solutions.

Hortonworks DataFlow

In August 2012, Hortonworks announced the acquisition of Onyara, the company behind Apache Nifi. Shortly after, Hortonworks announced a commercial distribution of Apache NiFi known as Hortonworks DataFlow (HDF). From a functional standpoint, DataFlow expands the Hortonworks platforms with stream data processing and integration. The addition of the DataFlow platform puts Hortonworks in a great position to be the first big data platform vendor to make a smooth transition into the IoT space. 


Kaa is one of the most popular open-source IoT platforms focused on providing a complete set of backend capabilities for building and managing IoT solutions. Integration is a key element of the Kaa platform providing seamless communication between smart devices and backend systems.


One of the most decorated IoT projects in the market, OpenRemote offers an open source IoT middleware that supports most of the relevant protocols used by smart devices. OpenRemote also includes very sophisticated integration management capabilities.


microServiceBus is a new and exciting entrant in the IoT integration space. Built on the Microsoft Azure platform, microServiceBus provides bi-directional integration capabilities between smart devices and enterprise systems. The platform seems to be designed for enterprise IoT from the ground up and includes sophisticated capabilities such as management, monitoring, security, heterogenous deployment topologies, and other fundamental elements of enterprise IoT solutions.

The 4th generation enterprise middleware

Integration is called on to become one of the pillars of enterprise IoT solutions. While still early, we can already see the first generation of platforms optimized to address the integration challenges in enterprise IoT topologies. As a technology trend, IoT middleware will become the 4th generation enterprise integration solutions.

The first generation of mainstream enterprise integration platforms was based on extract-transform-load (ETL) solutions. The service oriented architecture (SOA) movement brought us the enterprise integration servers and the enterprise service bus (ESB) integration platforms. The cloud triggered the integration platform as a service (iPaaS) trend. Now IoT is redefining the enterprise integration landscape.

We just need a better acronym …

Copyright © 2016 IDG Communications, Inc.

7 inconvenient truths about the hybrid work trend
Shop Tech Products at Amazon