Testing times

Consumer devices can, and often do, fail to connect to wireless or cellular networks. Whether you lose the contents of the document you’re working on, the show you’re streaming is suddenly interrupted, or you buy a new device and it just won’t link to the app, the results can be – at best – annoying. Imagine, then, taking this consumer technology into an industrial space. If devices fail to connect to a network, operations can grind to a halt, with substantial consequences for the business. And for the device manufacturer, a non-connecting device will quickly show up in non-sales.

It’s important, therefore, for smart product manufacturers to understand why their devices can fail to connect, and what they can do to minimize the risk of this happening.

Layers of connectivity

Connectivity relies on a number of different components. There’s the device itself, of course. Within that is the connectivity chipset, which contains the necessary protocols for Wi-Fi, Bluetooth, and other networking layers. At the other end of the chain is a gateway device, such as a router, after which lies a backhaul network, taking data to and from the cloud.

Even consumer devices can face problems. Sometimes the hardware chip won’t behave as intended. Likewise, a device’s firmware and drivers simply may not work as they should, or may work individually but not together, resulting in a failure to connect. Often users are required to download and sign-in to a corresponding app, and sometimes these are not well built, so that the app doesn’t recognize the device on the network. Any issues here, and people can just give up.

But things get even more complicated when smart devices are transposed into a factory setting, where the service requirements are very different. Guaranteed reliability and quality of service are essential, as a bare minimum. While Wi-Fi can handle these requirements at low densities such as the 40 to 50 devices you’d find in a typical home, 5G private networks are becoming an increasingly sought-after solution when there are thousands of devices on a network.

Both Wi-Fi and 5G have their own unique advantages. But from the device perspective, it needs to connect regardless of where it ends up. Getting this right means focusing on connectivity at the device level.

Product testing

The key to ensuring the device will connect when it reaches its point of use, is rigorous testing. Whether it’s end-to-end device testing, or functionality, performance, stress, interoperability, or user experience testing, the process is essential to identifying issues and analysing where problems lie so that they can be fixed before the product gets to the consumer. Doing so will save a lot of exasperation, time, and money, and will result in a better user experience and reviews. Importantly, testing must reflect real-world conditions, for example, by trying it out on a wide variety of different models of laptops and mobile phones including old ones – to see how the device connects and spot issues that may arise in the wild.

Interoperability testing, in particular, can be especially useful in this regard. Issues can be found due to incompatible implementations in devices from different vendors, which mean a device won’t connect to the network. A mobile device may connect perfectly well with an access point, for instance, but an attempt to stream video content from that mobile device to a smart TV could fail.

It’s here, where a combination of devices is in play, that interoperability testing is valuable. In a single home, one person can be browsing the internet, another can be watching a movie on Netflix, and someone else can be streaming music via Spotify. By taking all of these instances, and testing them to see how they function when they’re all working together, it’s possible to identify various issues.

Things are complicated by the addition of new devices or operating systems. Whenever a new iPhone or Android device is released, for example, it would have to be tested with other devices to make sure they still connect to the newer operating systems.

Ultimately, whether it’s a chip issue, a firmware issue, or a driver issue, the aim of the testing process is to ensure a product will achieve the best possible Wi-Fi connectivity before it ships to market.

Rigorous testing is needed to spot all possible issues. Ideally, this should be performed over a fixed duration, in which testers run thousands of test cases, using well-established problem scenarios, and historical bug reports. In the last stage of this, additional unplanned or ‘monkey’ testing is performed, where the testers just try to break the device’s connectivity by putting it through really challenging situations and seeing what comes up.

Positive reviews

Whilst rigorous testing at the start is critical, it must be considered over the product’s lifecycle. Sometimes products will display problems post-launch (especially if testing wasn’t rigorous), but many develop problems as products, software and drivers are updated, either in the device itself or in its connectivity ecosystem (phones, laptops, private or enterprise networks, cloud, etc.).

One example of how product testing works in practice is the recent case of a connected air purifier, which we at Capgemini worked on. While its main function was praised, when it was first released, its connectivity received several poor reviews. Quite simply, the device wouldn’t connect to the mobile application used to control it remotely. As a result, many people bypassed the app and used the purifier manually instead.

Following a series of robust tests, we were able to identify the issues with the device’s connectivity, and applied fixes to its firmware, improved its drivers, updated its software, and modified the chipset, allowing the device to better interact with its surroundings and more consistently communicate with the app.

Since these fixes were made, reviews to the air purifier, and its connectivity, have improved which, along with a corresponding increase in sales, is a strong indicator of success. In addition, by modifying its cloud connectivity architecture, we were able to reduce the manufacturer’s operating expense by more than 50 percent.

The air purifier is a relatively simple consumer product. This challenge of ensuring reliability and quality of service is scaled up in a much more diversified field, of, say, a smart manufacturing facility. There may not yet be a complete answer to this but, as with consumer connectivity, the answer will lie at the device level, and problems can only be found and addressed through rigorous testing.

Conclusion

How Capgemini Engineering can help

Capgemini’s Intelligent Devices Business Unit comprises about 1,400 engineers, with over 18 years’ experience in connectivity. It hosts automated connectivity product test labs, and an in-house training center. It focuses on everything that makes up ‘Intelligent Things’ from Chip-to-Cloud: Wi-Fi, Bluetooth and cellular connectivity, embedded devices, cloud and applications and, increasingly, embedded AI and Machine Learning. One of the group’s key functions is product testing, which accounts for around 60 percent of its work.