Kevin Ashton captured the imagination with his concept of everything networked together in his 2009 paper[i] where he used the term “Internet of Things,” computers gathering data about everything around us automatically. This vision came from the realization that technology and processing power will become cheap enough to deploy on massive scale. The original concept, described as “Device 2 Device” communication by Bill Joy in 1991,[ii] envisaged the use of RFID tags, and this concept has evolved into more general interpretations of physical objects that have virtual identities and which can communicate over networks.
IoT is a vast topic covering everything from sensor hardware through to data lakes, illustrated by the “Internet of Things Landscape 2018” from Firstmark.[iii]
As architects, we need to understand the capabilities required in IoT and have an overview of the technology options before considering any of the bewildering number of solutions from the vendors in this space.
This article is intended to provide the reader with a high-level understanding of IoT, including insights from the application of some of these technologies. This is an update to an article first published on LinkedIn. It is the first of three; I intend to follow up with two new papers, one on “AI and IoT” and another on “Industrial IoT (IIoT).”
To help set the scene, let’s start with a definition of IoT:
“Interconnection of sensing and actuating devices providing the ability to share information across platforms through a unified framework, developing a common operating picture for enabling innovative applications.”
Gubbi et al. 2013, S. 4
While I think this is a good definition, anyone who has seen this quote before will notice that I have (purposely) left out the latter part of the quote which says, “This is achieved by seamless large-scale sensing, data analytics, and information representation using cutting-edge, ubiquitous sensing and cloud computing.” This is more a measure of IoT maturity that we would expect to see of a large-scale implementation than a definition of IoT.
Capgemini[i] has a five-layer architecture model for IoT:
- Acquisition: Data measurement from sensors
- Aggregation: Distribute, collate and enrich data from multiple sources
- Analysis: Create dashboards, reports, predicative and prescriptive analytics
- Assignment: Routing
- Action and actuation: Automation and recommendation.
These layers may need to be applied at different scales: from the small scale of home automation or a car, through to large-scale sensing and data analytics. A typical home automation is likely to have edge devices that provide all these layers within the home. The home will become a data acquisition point, feeding into cloud-based aggregation systems. Some of this data is likely to be sold, for example, or data mining.
The challenge is to create low cost, easy-to-setup and use systems, that are secure now and into the future, across the various domains of IoT: Smart Home, Smart City, Smart Factory, Logistics/Transport, E-Health/Wearables, Smart Enterprise, and Retail. Primary considerations for delivering IoT systems include cost, power consumption, the ability to implement, upgradeability, scalability, and security.
For data acquisition and device actuation, the sensors and activators are typically dumb, wired devices requiring a microcontroller. There are a lot of options for prototyping using one of the popular microcontroller boards such as the Arduino, the Mbed-compatible ST Nucleo boards or even the BBC Microbit. These wired connections can be analog, but are typically digital, and based on standards such as I2C and whose driver complexities are encapsulated by easy-to-use libraries. Microcontrollers are a popular approach, easy to program and to create prototypes; production boards can be readily designed for production runs. The downside of the microcontroller approach is that they are “constrained devices” with limited capabilities, and the difficulty in upgrading can lead to security concerns. They usually need to be installed into a “safe,” wired or Wi-Fi environment and connected to a “hub,” sometimes called an “IoT gateway,” which is responsible for collecting and distributing data.
One recent innovation for sensors is the ESP8266 and the ESP32 with the NodeMCU open-source firmware. These are low-cost Wi-Fi modules with a full TCP/IP stack and a microcontroller capability with digital and analog inputs, allowing a wide range of digital and analog sensors to be Wi-Fi-enabled and programmed using a Lua interpreter. (Lua is an efficient embeddable scripting language; the interpreter and library are less than 1MB in size, and it uses a register-based byte code, making it a good choice for constrained devices). From the production angle, these modules can be easily incorporated into board designs, allowing for a wide range of devices to be Wi-Fi enabled at very low cost. These boards also have a “deep sleep” mode, which makes battery use a practical option for periodic measurements and allows Wi-Fi to be a realistic consideration for even low- power consumption applications.
Over The Air (OTA) updates is a key capability that is often overlooked for microcontrollers. Particle[i] offers ESP32 based devices paired with a low-power microprocessor, that can receive OTA updates with WI-FI, mobile, and mesh-enabled variants. Their devices are designed for surface mounting onto custom boards (the company also offers a custom board design service). Another capability that can be important for some types of IoT, is the inclusion of an ARM TrustZone CryptoCell.[ii]
Where more compute power is needed, the alternative approach is a microprocessor board such as the Raspberry Pi. While the consumer version is sometimes used as-is by start-ups, there are better options for production use, such as the compute model which can be incorporated into custom boards, and Element14 which licenses the design and provides a custom-built service for customers with large productions runs. Techbase[vi] provides industrialized versions of both the Pi and the ESP32 microcontroller mentioned above.
Over The Air (OTA) updates is a key security and maintenance capability. Since the Pi is a Linux board, there are numerous options for providing security updates and upgrades. The Raspian OS derivative of Debian comes with Docker support; Hypriot[vii] provides Docker images for this and other similar devices. Resin.io[viii] provides an OTA update service with Docker support.
There is a wide variety of other devices that can be used for IoT: System on Modules (SOM), such as those produced by Variscite. Toolchains such as Yocto and OpenEmbedded can be used to create a custom Linux OS to run on these devices, tailored to your own needs.
Moving onto more powerful multi-core CPU platforms, the LattePanda[ix] is one of the more powerful CPU boards available at present, using an Intel Atom processor and capable of running Windows 10, as well as Linux or Android. It is designed with an Arduino compatible co-processor for peripherals, WiFi and Bluetooth.
There are some use cases where considerable edge processing power is needed, and the microcontroller and microprocessor approaches can’t easily meet the interface complexities or processing requirements, such as local machine learning, robotics, AI, and local image processing. In the past, ASICS (application-specific integrated circuits) would need to be designed, which is expensive and only practical where there are high volumes. FPGA (field-programmable gate arrays) technologies have become a practical alternative to ASICS, where the gates on the chip are configured at boot time. The key enabler for its use in IoT is the embedding of FPGAs into the microprocessor chips; examples are the ARM-based processors from Xilinx and Intel (Intel has also recently announced Zeon processors with FPGAs). These chips allow complex interfacing or computations to be offloaded to the custom circuits created in the FPGA, and the less time-critical tasks to be done by the slower CPU. Another approach is to offload to a stream processor or GPU. The Parallella[x] board from Adapteva is an example of a relatively low-cost board with CPU, FPGA, and a 16 core (non-GPU) stream processor.
There are numerous other vendors providing hardware and software IoT eco systems, such as the Intel Edison, Brillo, and the ARM Mbed environment which have similar capabilities to those outlined above. Texas Instruments’ SimpleLink[xi] sensor tag takes an alternative approach by combining a set of low-power sensors with Bluetooth (with variants supporting other connectivity options) and a battery that lasts a year into a single small tag.
The purpose is to outline the capabilities to be considered for “Edge” processing, rather than list a set of vendors that will get outdated relatively quickly.
IoT overlaps with another set of identification and acquisition technologies – Radio Frequency Identification (RFID). These are established, widely adopted technologies ranging from passive RFID tags, optical tags (barcodes), short range contactless tags, active devices such as beacons, NFC, and Bluetooth, and longer range smart meters.
Once we have our data-acquisition capabilities we need to consider data distribution to the aggregation points.
For wireless connections, there are a lot of transmission protocol options; it is important to understand the IP stacks together with non-functional requirements such as security, availability, range, power consumption, and cost requirements. The physics behind this dictate that the higher the frequency, the higher the bandwidth, the shorter the range, and less penetration through physical obstacles such as walls. There is also a direct relationship between the bandwidth of the protocol and the power requirements. While Wi-Fi might be considered the default approach for a WAN (wide area network), it is high bandwidth and high power and not suited for all applications.
Bluetooth is often used in the wearables BAN (body area network) and audio markets because of the wide spread support of Bluetooth by mobiles and the low-power requirements; for other market segments there is limited availability of Bluetooth sensors and actuators. It uses the same 2.4GHz-frequency band as Wi-Fi. (Wi-Fi can also be on the 5GHz band.)
Google/Nest have collaborated with a number of manufacturers including Samsung to create the “Thread” IP stack. This is essentially a secure, low-power, mesh UDP network using Zigbee (and in the future Bluetooth) as the underlying transport layer. The advantage over standard Wi-Fi is lower power consumption, and the advantages over “Smart Bluetooth” are range and that it is not limited to 100 devices. Disadvantages include cost and availability, something that is likely to quickly change as adoption increases.
Whether we are dealing with a house, factory, or smart city, these protocols all use the same frequency bands and have similar range limitations when using a standard hub and spoke architecture. All the above technologies support peer-to-peer or mesh networks.
Wi-fi range can also be extended using mesh technologies; there are numerous examples of companies and communities linking Wi-Fi routers to create public mesh networks, often based on open source Linux such as OpenWRT. The Byzantium project has taken this a stage further, providing a mesh network and additional server-based collaboration tools into a standard Linux distribution, which can run on any low-cost, single board computer hardware. The Commotion Wireless is a similar and compatible open-source project providing interoperability for mesh networks with a range of computing, routing, and cellular hardware. These use the OLSR and batman-adv mesh routing embedded into the Linux kernel. The Smart Bluetooth and Thread stacks have these mesh capabilities “baked in.”
There are also a wide range of radio-controlled devices, for measuring electricity, gas, and water meters, and for consumer devices such as weather monitoring, and controlling mains equipment. The technology is cheap and readily available, using the 433MHz and 868MHz bands, with a range that for some modules is up to a few hundred meters. For smart meters this range is sufficient to allow for data collection over fixed networks or drive-by. Radian is commonly used in the UK for first-generation smart meters, where data is collected, and not shared directly with consumers. Many of these use older standards lack modern security measures.
Where a longer range is required (the VWAN, very wide area network), there are several LPWAN (low-power WAN) technologies including LoRaWAN, Symphony Link, and SigFox using the 868MHz and similar bands. These technologies are low power and use a low bandwidth to gain the long range (15km in a suburban environment); a base station can typically serve many thousands of devices. SigFox devices transmit up to 12 bytes and no more than 140 messages per day; it is claimed that two AA batteries could power a car-theft location tracker for 10 years. Arquiva is building a low-power, long-range, low- bandwidth IoT network in the UK using SIGFOX; some other European counties already have SIGFOX networks. Hardware costs can be less than US$2 per device and in the US US$1 per connection per year.[i] Digital Catapult has announced a London LoRaWAN network with 50 base stations which is free to use. LPWAN enabled hardware is becoming more accessible, e.g. Techbase.[iii]
These technologies are lower-power and lower-cost alternatives to the various mobile technologies, which in the US, typically charge US$1 per month. Mobile phones and mobile-enabled controller boards such as the Electron from Particle can provide low-cost hardware and with low-contract price implementations. With the potential closure of 2G mobile networks, the mobile operators need an alternative low-cost VWAN technology. There are two related technologies to address this, building on the existing LTE mobile networks: LTE-M (eMTC) and NB-IoT (narrowband Internet of Things). They provide higher bandwidth and lower latency, and have higher power requirements compared to LPWAN. The primary difference between them is that LTE-M has a higher bandwidth and NB-IOT has lower processing and memory requirements. Both are expected to be able to support 10 year battery life[i] using power down modes. Vodafone has started its roll-out of NB-IoT in the EU.[ii]
Another disadvantage of using mobile networks was the necessity to use physical pre-provisioned Sim cards, with no easy way to setup or change the network operator. These issues are being addressed by the introduction of eSim cards, which can be incorporated into the electronics and remotely provisioned and managed.
These technologies are all VWAN but are quite different in the trade-offs they offer between bandwidth, power consumption, the number of devices supported by a base station, bidirectional communications, and OTA updates. They should be seen as complimentary technologies; there is a need to consider all the non-functional requirements such as security, cost, power consumption, bandwidth requirements and latency and availability.
There are only a small number of vendors supporting low-cost implementations of these technologies. Pycom is one of these with low-cost, ESP32-based boards that support Wi-Fi, Bluetooth, LoRa, Sigfox, and LTE.
On top of the transmission protocol we need a transport protocol, and to consider whether messages need to be synchronous or asynchronous, real-time or near-real-time and whether message delivery is “at-most-once,” “at-least-once” or “exactly-once.” For some scenarios, https with REST or SOAP will suffice; CoAP is similar, a UDP version of http that is more suited for resource constrained devices. Often a messaging protocol will be a better fit. MQTT is a lightweight and very popular publish subscribe messaging protocol. AMQP is the main, more capable and more complex alternative, more suited to server-to server than device to server communication. It is a peer to peer protocol providing a number of different messaging patterns and is proven in very large-scale deployments. It uses OSPF for routing allowing for best path delivery.
Another option is gRPC, a remote procedure call mechanism, promoted and used internally within Google. Its original use was for microservices. The drivers for use in IoT are its performance,[i] (with a potential for a 3x data throughput with ¼ the CPU resources compared to using REST) and its range of invocation patterns: synchronous calls, asynchronous calls, notifications, and also client, server, and bidirectional streaming.
The aggregation topology is a key consideration in these protocols – where is the data collected? For many applications, including home automation systems, home video surveillance, and smart meters, a home area network is essential, so that consumers can use the data they are generating. This is typically a star (or mesh) topology using Wi-Fi from a centralized base: the “hub” or “IoT gateway.” The IoT gateway can incorporate a number of capabilities including aggregation, forwarding, analytics, local decision-making, and coordinating and managing a potentially large number of local devices.
The lack of a common standard in this area is an impediment for IoT within the home environment and to home automation. The key part of the home infrastructure, the broadband router, which is the obvious device as a home aggregation point, typically lacks many of the transmission and transport protocols. I personally use a self-hacked router running the open source OpenWRT, and running an MQTT service as an aggregation point for IoT measurements; not a consumer plug-and-play option.
Smartphones are a crucial part of a typical system, providing control, administrative, and reporting capabilities for Wi-Fi and Bluetooth-based networks, and sometimes acting as the “IoT gateway.” For industrial scenarios there are a range of vendors including GE, Siemens, Dell, and Cisco. Industrial IoT will be covered in a future article.
There are some interesting recent developments for home automation. Samsung SmartThings is an IoT gateway that provides interoperability with all the different IoT technologies and uses the smartphone as the controller. Amazon Echo and Google Home provide an alternative, voice-based, control mechanism; when used with a “SmartThings,” it becomes an alternative controller to the smartphone. Apple has taken interoperability a step further with its HomeKit eco system. Rather than choosing particular technologies, it uses a combination of software libraries and physical bridge devices to provide connectivity and control to a large range of devices; vendors will need to integrate their devices into its eco system and either use Wifi or Bluetooth or integrate via a bridge device.
One side effect arising from the lack of an aggregation capability in the home is that it forces many current IoT devices to connect directly to cloud services, tunneling through the router using the UPnP protocol. This bypasses the NAT security offered by (IPV4) broadband routers and exposes devices directly to the internet where they can be exploited, e.g. by the Mirai and other botnets. This is possible because the UPnP protocol is typically enabled by default on broadband routers and the protocol opens ports when requested by devices on the local LAN without additional controls.
There are real architectural choices where the main compute capabilities are located; cloud providers drive one type of solution; SLAs and the practicalities of coping with large numbers of devices with connections and fluctuating network demands may necessitate using FOG techniques where caching, storage, and compute power reside in the edge. We discussed earlier some examples of advanced low-cost, high-compute power that can be used for this. One advanced pattern is the use of cloudlets, where hardware and capabilities on the local network (cloudlets) act as proxies for the cloud capabilities.
A recent Wikibon report[i] concludes that “for almost all IoT cases, some Edge pre-processing and data reduction will reduce the overall cost of IoT. For most IoT cases, moving the processing of the data will result in greater function and lower costs.”
An interesting technology for managing edge processing is Greengrass from Amazon. Greengrass runs on Linux and allows the edge device to run Lambda functions in a container-like environment. It also supports OTA updates and inference engines for AI.[ii]
The UK Smart Meter infrastructure is an example of how these different capabilities can be combined. The UKs first-generation smart meters were one-way devices, where data is collected remotely by the supplier, (e.g. using a dedicated mobile handheld device), and not directly shared with the consumer, e.g. using the Radian protocol.
Under the UKs SMETS2 standards, a home has a set of smart meters that communicate with a communications hub (IoT gateway) over Zigbee. In addition, a home can have one or more controllers (IHD, in-house displays) and a set of switches, thermostats, etc., all communicating over Zigbee. The Data Communications Company provides the infrastructure to collect the data from all homes in the country using a WAN. For the South and central England, this uses mobile (GSM) technologies. For the rest of the country, it uses an Arquiva radio network based on Flexnet[i] from Sensus.
The UK Smart Meter program has a number of design compromises, which illustrates the difficulties in creating large-scale IoT solutions:
- The standard doesn’t allow for integration with smartphones; instead, users are provided with dedicated control and display devices.
- While Zigbee is used by many home automation products including those by Philips, it is incompatible with the popular Google NEST devices.
- Zigbee uses the same frequencies as Wi-Fi, and just like Wi-Fi, its range can be too low for some houses.
- Although Zigbee supports mesh topologies, a mechanism to improve coverage, the mesh capability is disabled in the SMETS2 standards because of concerns about the setup complexity and that users could easily (accidentally or purposely) break the network.
- One of the WAN technologies uses the 2G mobile network; some UK network providers already are reducing the bandwidth allocated to 2G.[ii] The UK government doesn’t mandate provision for 2G,[iii] so there are risks that 2G could be phased out in the UK as early as 2020.
- The communication is bidirectional, and the supplier can turn off supplies to a home, e.g. for non-payment. If the network was compromised, there is a risk this could be exploited to turn off the supply to a large number of homes.
Summary of on-premises IoT
The diagram below is a summary of the discussions so far, illustrating the various topologies that have been discussed and highlighting that there is no single best practice or reference architecture for IoT; there are lots of options.
Aggregation at scale
So far, we have considered the on-premises options covering “edge” devices and the “IoT gateway.” The next capabilities are those to aggregate and process the data from a large number of edge devices.
The main capabilities to consider are:
- Aggregating and routing of data for post processing
- Device registration and management
- Data analysis both real time and for data mining
- Presentation for device management and the data analysis
- Automation and recommendation, and connected service experience (CSX); real- time analysis for next-best action to deliver a personalized response
- Systems of record, e.g. for billing purposes.
The questions we considered earlier are still relevant: synchronous or asynchronous, real-time, near-real-time, or batch, and whether message delivery is “at-most-once,” “at-least-once,” or “exactly-once,” One reference architecture I read recently aggregated data into files routed by ftp; unusual perhaps; its use was justified because batch processing was sufficient and provided easy integration into legacy systems of record. Other key questions are around data classification: what data needs real-time processing; what data needs to be stored for subsequent data mining; which data is critical; which data can be classified as personal data and covered by the data protection act; does any data need anonymizing.
The number of potential systems that could be discussed here are huge because of the overlap into big data, CRM, ERP, etc. Typically, you would be using cloud-based platforms such as AWS, Azure, and Bluemix where many of the capabilities have already been packaged using chosen solutions. There are many papers and articles on the offerings of these platforms which I won’t duplicate here.
One way of simplifying the many options is the use of accelerators, industry-tailored solutions that address the whole stack. Capgemini has an energy sector platform[i] based on the Intel IoT reference architecture, and a number of tailored solutions for sensor networks, Health, Automotive, and other industries. Architecture needs to be driven by the business case. An illustration of some of the business cases for IoT can be found here.[ii]
One final note, on security, which has been recurring theme in this paper; a good checklist on IoT security is published and maintained by OWASP: https://www.owasp.org/index.php/IoT_Security_Guidance.
Follow me on LinkedIn for my forthcoming papers on “AI for IoT” and “Industrial IoT”: https://uk.linkedin.com/in/drutter
David is a Digital Consultant in Capgemini’s Architecture and Technical Leadership Practice, which has developed a unique set of approaches to optimize your business transformation success. Capgemini is a leading IoT SI and has delivered 202 IoT projects[iii] since 2002 and has 23 million devices managed as a service. Find our latest thinking on IoT here: https://www.uk.capgemini.com/search/site/iot with other key topics and trends here: https://www.uk.capgemini.com/expert-connect .
[i] “Capgemini Energy Internet of Things” by Mike Malloni https://www.capgemini.com/resources/capgemini-energy-internet-of-things
[ii] “Thirty Measurable Business Cases for IoT with Connected Services” by Prof. Dr. Michael J. Capone https://www.uk.capgemini.com/resource-file-access/resource/pdf/thirty_measurable_business_cases_for_iot_with_connected_services.pdf
[iii] Microservices: The Future-Proof Framework for IoT by Prof. Dr. Michael J. Capone https://www.slideshare.net/capgemini/microservices-the-futureproof-framework-for-iot
[i] Smart meters in Scotland and northern England http://www.wireless-mag.com/News/31690/arqiva-lights-up-first-sigfox-internet-of-things-networks-in-10-uk-cities-.aspx
[i] Wikibon: The Vital Role of Edge Computing for IOT: 2016 Update
[i] Announcing gRPC Alpha for Google Cloud Pub/Sub
[i] LTE evolution for IoT connectivity http://resources.alcatel-lucent.com/asset/200178
[ii] Digital Catapult launches IoT network 19/20/16:
[i] IoT Portfolio, Prof. Dr. Michael J. Capone, 01.19.2016.
[ii] Pontin, Jason (Sept 29, 2005). “ETC: Bill Joy’s Six Webs.” MIT Technology Review. Retrieved Nov 17, 2013.
[iii] Matt Turuck VC at FirstMark, http://mattturck.com/iot2018/