Are LLMs dangerous?

The short answer is sometimes. With Large Language Models having such a diverse range of applications, the potential risks are numerous. It is worth pointing out that there is no standard list of the risks but a selection is presented below.

Some of these dangers are linked to the model itself (or to the company developing it). The data in the model could contain all sorts of biases, the results might not be traceable, or user data or copyrights could have been used illegally, etc.  

Other dangers are linked to the use of these models. Users seek to bypass the security measures of templates and use them for malicious purposes, such as generating hateful or propagandic texts.  

Additionally, Large Language Models have social, environmental, and cultural consequences that can be harmful. They require enormous amounts of storage and energy. Moreover, their arrival in society has weakened employee power in many industries. For example, writers striking in Hollywood have complained about the use of LLMs. Finally, Large Language Models’ AI is challenging the boundaries of literary art, just as DALL-E did with graphic art.

How can you deal with these risks?

It often takes a while before the risks of emerging technology are fully understood. This is also true of the survival tactics. However, we are already beginning to see early strategies being deployed.  

LLM developers invest in safeguards

OpenAI invested six months of research to establish safeguards and secure the use of Generative Pre-trained Transformers (GPT) models. As a result, ChatGPT now refuses to respond to most risky requests. As for its responses, they now perform better on such benchmarks as veracity or toxicity. Furthermore, unlike previous models, the ChatGPT Large Language Models has improved since it was deployed. 

However, it is possible to circumvent these safeguards, with examples of such prompts being freely available on the Internet (Do Anything Now (DANs)). These DANs often capitalize on ChatGPT’s human-centric nature – the model seeks to satisfy the user, even if this means overstepping its ethical framework or creating a confirmation bias. Furthermore, the opacity of the model and its data creates copyright problems and uncontrolled bias. As for benchmark successes, suspicions of contamination with the training database undermine their objective value. Finally, despite announced efforts to reduce their size, OpenAI models consume a lot of resources. 

Some Large Language Models now claim to be more ethical or safer, but this is sometimes to the detriment of performance. None of the models are faultless, and there is currently no clear and reliable evaluation method on the subject.

GPT-4 safety in five steps

To go into more detail about implementing guardrails, let’s look at the 5 steps implemented by OpenAI for GPT models, as shown in Figure 2.

1. Adversarial testing: Experts from various fields have been hired to test the limits of GPT-4 and find its flaws.
2. Supervised policy: After training, annotators show the model examples of the desired responses to fine-tune it.
3. Rule-based Reward Model (RBRM) classifiers: The role of these classifiers is to decide whether a prompt and/or its response are “valid” (e.g., a classifier that invalidates toxic requests).
4. Reward model: Human annotators train a reward model by ranking four possible model responses from best to least aligned.
5. Reinforcement learning: Using reinforcement learning techniques, the model takes user feedback into account.

Governments and institutions worry about LLMs

Several countries have decided to ban ChatGPT (see Figure 3). Most of them (Russia, North Korea, Iran, etc.) have done so for reasons of data protection, information control, or concerns around their rivalry with the USA. Some Western countries, such as Italy, have banned it and then reauthorized it, while others are now considering a ban. For the latter, the reasons cited are cybersecurity, the protection of minors, and compliance with current laws (e.g., GDPR). 

Many tech companies (Apple, Amazon, Samsung, etc.) and financial institutions (J.P. Morgan, Bank of America, Deutsche Bank, etc.) have banned or restricted Large language models ChatGPT. They are all concerned about the protection of their data (e.g., a data leak occurred at Samsung). 

Scientific institutions, such as scientific publishers, forbid it for reasons surrounding trust – given the risk of articles being written surreptitiously by machines. Finally, some institutions are concerned about the possibility of cheating with such tools.  

European regulation changes

Many articles in Quantmetry’s blog have already mentioned the future EU AI Act, which will regulate artificial intelligence as soon as 2025. However, we should add here that this legislation has been amended following the rapid adoption of the ChatGPT LLM, and the consequences of this amendment are summarized in Figure 4. The European Union now defines the concept of General Purpose AI (GPAI). This is an AI system that can be used and adapted to a wide range of applications for which it was not specifically designed. The regulations on GPAIs therefore concern Large Language Models’ AI as well as all other types of Generative AI. 

GPAIs are affected by a whole range of restrictions, summarized here in three parts:

• Documentary transparency and administrative registration, should not be complicated to implement.
• Risk management and setting up evaluation protocols. These aspects are more complicated to implement but feasible for LLM providers, as outlined by OpenAI with ChatGPT Large Language Models.
• Data governance (RGPD and ethics) and respect for copyright. LLM providers are far from being able to guarantee these for now.

The European Union will therefore consider LLMs to be high-risk AIs, and LLM providers still have a lot of work to do before they reach the future compliance threshold. Nevertheless, some believe that this future law is, in some respects, too impractical and easy to circumvent. 

Assessing model compliance is one of Quantmetry’s core competencies, particularly in relation to the EU AI Act. Regarding LLMs specifically, Stanford researchers published a blog post evaluating the compliance of 10 LLMs with the future European law. The results are shown in Figure 5. To establish a compliance score, the researchers extracted 12 requirements from the draft legislation and developed a rating framework. Annotators were then tasked with conducting an evaluation based on publicly available information. The article identifies copyright, data ecosystem, risk management, and the lack of evaluation standards as the main current issues, aligning with our analysis above. The researchers estimate that 90% compliance is a realistic goal for LLM providers (the top performer currently achieves 75%, with an average of 42% across the evaluated 10 LLMs). 

A few tips

Faced with all these risks, it would be wise to take a few key precautions. Learning a few prompt engineering techniques to ensure that prompts provide reliable and high-quality responses could be a good way forward. It’s also worth watching out for data leaks via free chatbots (e.g., on the free version of ChatGPT). The paid version does not store your data a priori. Finally, Figure 6 illustrates how to use tools like ChatGPT with care.   

How do you audit such models?

There are three complementary approaches to auditing an LLM, summarized in Figure 9.

Organizational audit

An organizational audit can be carried out to check if the company developing the LLM is working responsibly, along with ensuring, for example, that its processes and management systems are compliant.   

It will be possible to do this for our clients who are not suppliers of LLMs but wish to specialize them further, to ensure that they are well employed.  

Audit of the foundation model

Auditing the foundation model is the current focus of scientific research. For such an audit, it would be necessary to be able to explore the dataset (which is inaccessible in reality), run test benches on recognized benchmarks and datasets (but face the problem of contamination), and implement adversarial strategies to detect the limits of the model. If we go into more detail, there is a multitude of possible tests for evaluating the following aspects of the model: 

• Responsibility: Understanding how risks materialize and finding the limits of the model (typically with adversarial strategies).
• Performance: This involves using datasets, test benches, or Turing tests to assess the quality of the language, the skills and knowledge of the model, and the veracity of its statements (see Figures 7 and 8).
• Robustness: The aim here is to assess the reliability of responses by means of calibration or stability measurements in the face of prompt engineering strategies.
• Fairness: Several methods exist to try to identify and quantify bias (even without access to the dataset) but remain limited. For example, a method could be counting biased word associations (man = survival, woman = pretty).
• Frugality: Some inference measurements can be made to estimate the environmental impact of the model, but they are also limited without access to supplier infrastructures.

Theoretically, an LLM can be assessed on five of the eight dimensions of a Trustworthy AI defined by Quantmetry. On the dimension of being explainable, the previously mentioned solution of the Chatbot citing its sources responds to this problem, to a certain degree. 

Use case audit

Quantmetry and Capgemini Invent are currently working together to define a framework that enables our clients to audit their AI systems based on LLMs. The primary aim of this audit is to check that the impact of the system on the user is controlled. To do this, a series of tests check compliance with regulations and the customer’s needs. We are currently developing methods for diagnosing the long-term social and environmental impact of their use within a company. Finally, we will create systems that can assess risks and biases, as well as operational, managerial, and feedback processes. The methods used are often inspired by but adapted from, those used to audit the foundation model.  

How can Capgemini Invent and Quantmetry help you capitalize on LLMs?

Amidst the media excitement surrounding the widespread adoption of ChatGPT, harnessing the full potential of Generative AI and LLMs while mitigating risks lies at the heart of an increasing number of our clients’ strategic agendas. Our clients must move quickly along a complex and risky path, and the direct connection between the technology and end-users makes any errors immediately visible – with direct impacts on user engagement and brand reputation.  

Drawing upon our experience in facilitating major transformations and our specific expertise in artificial intelligence, our ambition is to support our clients at every stage of their journey, from awareness to development and scalable deployment of measured-value use cases. Beyond our role in defining overall strategy and designing and implementing use cases, we also offer our clients the opportunity to benefit from our expertise in Trustworthy AI. We assist them in understanding, measuring, and mitigating the risks associated with this technology – ensuring safety and compliance with European regulations.  

In this regard, our teams are currently working on specific auditing methods categorized by use cases, drawing inspiration from the academic community’s model of auditing methods. We are committed to advancing concrete solutions in this field.  

What is Large Language Models (LLM)?

Artificial intelligence (AI) is a technological field that aims to give human intelligence capabilities to machines. A generative AI is an artificial intelligence that can generate content, such as text or images. Within generative AIs, foundation models are recent developments often described as the fundamental building blocks behind such applications as DALL-E or Midjourney. In the case of text-generating AI, these are referred to as Large Language Models (LLMs), of which the Generative Pre-trained Transformer (GPT) is one example made popular by ChatGPT. More complete definitions of these concepts are given in Figure 1 below.

The technological history of the ChatGPT LLM

In 2017, a team of researchers created a new type of model within Natural Language Processing (NLP) called Transformer. It achieved spectacular performance for sequential-data tasks, such as text or temporal data. By using a specific technology called ‘attention mechanism’, published in 2015, the Transformer model pushed the limits of previous models, particularly the length of texts processed and/or generated. 

In 2018, OpenAI created a model inspired by Transformer architecture (the decoder stack in particular). The main reason for this was that Transformer, with its properties of masked attention, excels in text generation. The result was the first Generative Pre-trained Transformer. The same year saw the release of BERT, a Google NLP model, which was also inspired by Transformers. Together, BERT and GPT launched the era of LLMs.  

Improving the performance of its model over BERT LLM variants, OpenAI released GPT-2 in 2019 and GPT-3 in 2020. These two models benefited from an important breakthrough: meta-learning models. Meta-learning is a paradigm of Machine Learning (ML) in which the model “learns how to learn.” For example, the model can respond to tasks other than those for which it has been trained.  

OpenAI’s aim is for GPT Large Language Models to be able to perform any NLP task with only an instruction and possibly a few examples. There would be no need for a specific database to train them for each task. OpenAI has succeeded in making meta-learning a strength, thanks to increasingly large architectures and databases massively retrieved from the Internet.  

To take its technology further, OpenAI moved beyond NLP by adapting its models for images. In 2021 and 2022, OpenAI published DALL-E 1 and DALL-E 2, two text-to-image generators.¹⁰ These generators enabled OpenAI to make GPT-4 a multi-modal model, one that can understand several types of data.  

Next, OpenAI released InstructGPT (GPT 3.5), which was designed to better meet user demands and mitigate risk. This was the version OpenAI launched in late 2022. But in March 2023, OpenAI released an even more powerful and secure version: the premium GPT-4. Unlike preceding versions, GPT-3.5 and GPT-4 gained strong commercial interest. OpenAI has now adopted a closed source ethos – no longer revealing how the models work – and become a lucrative company (it was originally a non-profit association). Looking to the future, we can expect OpenAI to try to push the idea of a prompt for all tasks and all types of data even further. 

Why is everyone talking about Large language models?

Only those currently living under a rock will not have heard something about ChatGPT in recent months. The fact that it made half the business world ecstatic and the other half anxious should tell you how popular it has become. But let’s take a closer look at the reasons why. 

OpenAI’s two remarkable feats­­

With the development of meta-learning, OpenAI created an ultra-versatile model capable of providing accurate responses to all kinds of requests – even those it has never encountered before. In fact, GPT-4 achieves better results on specific tasks than specialized models. 

In addition to the technological leaps, OpenAI has developed democratization. By deploying its technology in the form of an accessible chatbot (ChatGPT) with a simple interface, OpenAI has made it possible for everyone to utilize this powerful language model’s capabilities. This public access also enables OpenAI to collect more data and feedback used by the model.

Rapid adoption  

The rapid adoption of GPT technology via the ChatGPT LLM has been unprecedented. Never has an internet platform or technology been adopted so rapidly (see Figure 2). ChatGPT now boasts 200 million users and two billion visits per month.  

The number of Large Language Models is exploding, with competitors coming from Google (Bard), Meta (Llama), and HuggingFace (HuggingChat, a French open-source version). There is also a surge in new applications. For example, ChatGPT LLMs have been implemented in search engines and Auto-GPT, which latter turns GPT-4 into an autonomous agent. This remarkable progress is stimulating a new wave of research, with LLM publications growing exponentially (Figure 3).  

Opportunities, fantasies, and fears

The new standard established by GPT-4 has broadened the range of possible use cases. As a result, many institutions are looking to exploit them. For example, some hospitals are using them to improve and automate the extraction of medical conditions from patient records.  

On the other hand, these same breakthroughs in performance have given rise to a host of fears: job insecurity, exam cheating, privacy threats, etc. Many recent articles explore this growing anxiety, which now seems justified – Elon Musk and Geoffrey Hinton are just two of the many influential tech figures now raising the alarm, calling it a new ‘code red.’  

However, as is often the case with technological advances, people have trouble distinguishing between real risk and irrational fear (e.g., a world in which humans hide from robots like those in The Terminator). This example explores the creation of a model that rivals or surpasses the human brain. Of course, this is inextricably linked with the formation of consciousness. Here, it is worth noting that this latter fantasy is the ultimate goal of OpenAI, namely AGI (Artificial General Intelligence). 

Whether or not these events will remain fantasies or become realities, GPT-4 and the other Large Language Models’ AI are undoubtedly revolutionizing our society and represent a considerable technological milestone.

What can you do with an LLM?

Essentially, a ChatGPT LLM can:

1. Generate natural language content: Trained specifically for this purpose, this is where they excel. They strive to adhere to the given constraints as accurately as possible.
2. Reformulate content: This involves providing the LLM with a base text and instruction to perform tasks, such as summarizing, translating, substituting terms, or correcting errors.
3. Retrieve content: It is possible to request an LLM to search for and retrieve specific information based on a corpus of data.

How can you use an LLM?      

There are three possible applications of Large Language Models’ AI, summarized in Figure 4. The first one is direct application, where the LLM is only used for the tasks that it can perform. This is, a priori, the use case of a chatbot like ChatGPT, which directly implements GPT-4 technology. While this is one of the most common applications, it is also one of the riskiest. This is because the ChatGPT LLM often acts like a black box and is difficult to evaluate. 

One emerging use of LLMs is the auxiliary application. To limit risks, an LLM is implemented here as an auxiliary tool within a system. For example, in a search engine, an LLM can be used as an interface for presenting the results of a search. This use case was applied to the corpus of IPCC reports.¹⁹ The disadvantage here is that the LLM is far from being fully exploited.  

In the near future, the orchestral application of ChatGPT LLMs will consume much of the research budget for large organizations. In an orchestral application, the LLM is both the interface with the user and the brain of the system in which it is implemented. The LLM therefore understands the task, calls on auxiliary tools in its system (e.g., Wolfram Alpha for mathematical calculations), and then delivers the result. Here, the LLM acts less like a black box, but the risk assessment of such a system will also depend on the auxiliary tools. The best example to date is Auto-GPT.

Focusing on the use case of a Chatbot citing its sources

One specific use case that is emerging among our customers is that of a chatbot citing its sources. This is a response to the inability of Large Language Models’ AI to interpret results (i.e., the inability to understand which sources the LLM has used and why).

To delve into the technical details of the chatbot citing its sources (The relevant pattern – illustrated in Figure 5 – Is called Retrieval Augmented Generation or ‘RAG’), the model takes a user request as input, which the model then transforms into an embedding (i.e., a word or sentence vectorization that captures semantic and syntactic relationships). The model has a corpus of texts already transformed into embeddings. The goal is then to find the embeddings within the corpus that are closest to the query embedding. This usually involves techniques that find the nearest neighbours’ algorithms. Once we have identified the corpus elements that can help with the response, we can pass them to an LLM to synthesize the answer. Alongside the response, we can provide the elements that were used to generate it. The LLM then serves as an interface for presenting the search engine’s results. This ‘RAG’ approach therefore facilitates the decoupling of factual information provided by the sources from the semantic analysis provided by the LLM, leading to better auditability of the results provided by the Chatbot.  

Read more in Auditing ChatGPT – part II

Competition in the automotive industry is intensifying and brands are competing on more fronts than at any time in history. Of course, price, performance, brand, and residual values continue to be important. But as the industry gravitates toward electrification and software-defined vehicles, customers are looking at what else their vehicles can do for them. How well do they integrate with their lives and their digital ecosystems? Can and will the car evolve over time to add more value to daily life? And, for manufacturers, how do you build supply chain resilience and competitiveness to address these evolving demands, while ensuring availability and affordability? 

Automakers – especially at the luxury and premium end of the market – are also intensifying their focus on providing assisted and autonomous driving capabilities and new ways to add value with digital experiences, inside and outside the vehicle. In the face of increased competition, the speed with which automakers are able to innovate and the extent to which they can engage and satisfy their customers in new ways will be crucial to future success or failure.

Autonomous mobility at the crossroads

For years, tech and innovation events like CES have been dominated by autonomous vehicles of all shapes and sizes. The technology is always impressive … at the shows. But, in the real world, progress has been slower than expected. For every success, it seems like there’s been at least one story of a scaled-back or canceled investment, an unfulfilled promise, or a serious safety scare.

The pursuit of autonomous mobility is a double-edged sword. The cost of adding sensors for 20+ detection zones around the car is significant. And the volumes of data, the sophistication of algorithms, and the amount of computing power required to develop, test, and validate systems are eye-watering. And yet, the ability to offer customers safe and stress-free ways to travel; to give back quality time while getting from A to B, is a once-in-a-lifetime opportunity to build trust and open the door to a whole new world of services and revenue streams. It’s no wonder the pursuit of the various certification levels is so intense and why so many companies are taking different routes – from in-house development with tech partners to major alliances with tier-1 suppliers, and even acquisitions. Some companies are making more progress than others, but the race is still wide open.

The in-car experience is evolving

The transition to electric and the pursuit of autonomous-driving capabilities have major implications for the automotive customer experience, especially the in-car digital experience. With electric vehicles, we know that recharging away from home will involve idle time. And – though it may still be a way off – autonomous mobility will allow us to focus less on driving the car and leave us more time to do other things. Today, our first thought might be to reach for our smartphones or tablets, but this is a lost opportunity for vehicle manufacturers.

And so the question becomes: How can your car keep you entertained and engaged while it charges or self-drives?

The answers are emerging in the form of expansive screens, adaptive interfaces, the addition of extra screens for passengers, an increasing emphasis on in-car gaming, content consumption, subscription services, and almost unlimited ways to pass the time productively, recreationally, or relaxingly in a vehicle.

And then there’s the potential to have an AI-powered assistant, or companion, that connects all the different services and is capable of providing pretty much any information you need about your journey, your agenda, upcoming commitments, highlights from your inbox or social media feed, and much more.

All of these features represent potential points of differentiation, and many of them are revenue-generating opportunities (e.g. subscription-based services). Beyond direct revenue and new levels of customer intimacy, in-car digital interactions also create opportunities to generate new data and insights, which can (with the right levels of consent and anonymity, of course) be used to shape new products and services – inside and outside the vehicle – and new monetization opportunities.

Speed and satisfaction – why they matter more than ever

You could argue that the evolutions I’ve explored above are technology trends, much like many others. However, these trends are different in that if you can achieve the combination of safe autonomous or highly assisted mobility and engage customers with compelling in-car experiences, you can gain a level of trust, and access – and even companionship – that is unprecedented in the history of OEM-customer relationships. This brings with it the opportunity to develop deeper, longer, and more lucrative relationships.

But the race for the hearts and minds of customers is intense, with a raft of new players (many from China) to compete against, new demographics, and rapidly evolving customer expectations. In this climate of increased competition, it is imperative that automotive companies intensify their innovation efforts in a bid to deliver the integrated and connected customer experience that will soon be taken for granted. And if your brand isn’t able to provide it, you can assume that another one will. 

Balancing the need for speed and satisfaction with a ‘zero compromise’ approach to safety and security

Against this backdrop of ultra-intense competition and a relentless focus on innovation, OEMs must remain vigilant and understand that speed to market can never take priority over safety and security.

Assisted and autonomous mobility can offer comfortable, convenient, and stress-free travel. But they also mean taking a significant degree of responsibility for the safety of vehicle occupants. In short, ADAS and autonomous driving systems cannot fail. Failures will result in more than a few lost sales – they could lead to loss of life, high-profile court cases, and a complete loss of confidence in your brand.

And though it’s less likely to be a life-or-death matter, automotive brands need to be vigilant about ensuring the cybersecurity of their vehicles and data ecosystems. Digital assistance or companionship, subscriptions, services, integrated payment solutions and ecosystem services (e.g. via wearable health devices, smartphones, etc.) will typically require some degree of data sharing. This opens the door for personalization and seamlessly convenient experiences, but it’s not without its risks. No brand wants to be the next one to appear in a high-profile data leak story and risk losing the hard-earned trust of its customers.

Software is the key to safe, secure, and satisfying experiences

So what’s the key to accelerating innovation cycles and customer satisfaction without compromising on safety and data security?

The answer lies in your software strategy. After all, software is at the heart of assisted and autonomous driving systems, it drives immersive and engaging digital experiences through infotainment systems and more, and it can be the key to ensuring the security of personal data and the identification and elimination of sophisticated cybersecurity threats. The right software strategy and architecture (i.e. a simplified one) can also provide you with greater flexibility during times of supply chain instability, meaning you can maintain product availability while your competition potentially suffers. As many of us learned during the pandemic, simply making sure your cars are available to potential buyers can be the biggest advantage of all.

But the stakes are too high with software and the task of transforming into a software company is too big to go it alone. Here are three ways automotive companies can get their transformation right.

1. Partner up to boost software capabilities

Software-driven transformation is a broad and deep-reaching process, which can encompass upskilling your existing team, building new capabilities, and finding the right balance between maintaining your existing digital products and developing new ones. This is a huge undertaking, and so it makes sense to partner up with automotive software specialists and engineers who can share and instill industry best practices, build dedicated software factories for you, or support you in maintaining existing products or developing new ones.

2. Use cloud, virtualization, and AI to achieve more

Cloud and AI can be used to process and analyze the high volumes of data produced during autonomous driving system development and testing, to virtualize ECUs, and to support data spaces and service ecosystems. These technologies, combined with the suite of automotive-specific accelerators being built by hyperscalers today, can supercharge your innovation and product development cycles, enabling you to get to market faster with new products and services, while keeping your – and your customers’ – valuable data secure. 

3. Look for external inspiration

Automotive companies can’t be everything to everybody. It’s difficult (impossible?) to develop an infotainment UX that rivals that of smartphone makers like Apple and Google if it’s not your core business. Likewise, you won’t suddenly create ‘killer’ content and entertainment options if you’re just starting out. Instead, partner up with startups and niche players in differentiating domains and focus on the bigger picture.

The road ahead is filled with complexity and exciting developments. And yet, for all the focus on new technology, there are still large groups of customers who care little for new tech, and who continue to value practicality, build quality, and affordability above all else. How organizations address these oft-divergent customer desires within their product portfolio will be a challenge for many ‘traditional’ OEMs.

What we can say with confidence is that mobility experiences of the future – whether they’re autonomous or human-driven – must be satisfying, safe, and secure. Automotive companies must be quick to give their customers what they want. Check out our perspective on software in automotive to learn more. 

The semiconductor industry is standing at the edge of a profound transformation, thanks to the advent of a game-changing technology: chiplets.

Throughout its history, the semiconductor industry has pursued relentless integration and miniaturization. However, the escalating costs and complexities associated with cutting-edge Integrated Circuits (ICs) on advanced semiconductor technology have led to a revolutionary alternative approach: chiplets.

Most contemporary chips are designed as a monolithic SoC (System-on-Chip), integrating all essential functions—such as the processor cores, domain-specific hardware accelerator, memory, and interfaces — into a monolithic die, ie everything is built into an integrated circuit on a single piece of semiconductor.

Chiplets are a game-changer. They consist of a self-contained semiconductor die that, when combined with other dies through advanced packaging techniques, forms a complex integrated circuit similar to a monolithic integrated circuit. This modular approach enhances scalability, cost-efficiency, and performance. It also enables the integration of diverse functions, such as general-purpose processing, domain-specific processing, and memory into a single system, overcoming some limitations of traditional monolithic designs.

The chiplet approach not only addresses the challenges of rising costs and complexities but also unlocks unparalleled flexibility. Heterogeneous chiplet designs enable tailored solutions for specific applications or market segments. Solution providers can modify or add relevant chiplets without disrupting the overall system, resulting in reduced development costs and faster time-to-market, as redesign efforts only affect the package or additional domain-specific dies, not the entire chip.

There are still crucial challenges in the chiplet domain such as power and thermal management. Effective multi-vendor support is required to manage those aspects across all integrated chiplets seamlessly. And the standardization of interfaces and testing will be vital to ensure seamless integration, though, notably, organizations such as the Open Compute Project and UCIe (Universal Chiplet Interconnect Express) have already released specifications for open source chiplet interconnect characteristics.

Semiconductor giants such as Intel, Nvidia, and AMD have been quick to adopt chiplet technology, successfully demonstrating its viability in manufacturing, testing, and packaging as chiplet adoption gains momentum, the development of an ecosystem of suppliers is underway to serve its needs in areas such as packaging and thermal management. This will facilitate more widespread implementation across the industry, transcending adoption, reducing over-reliance on a few major players.

The growing popularity of chiplet designs has sparked interest across the entire semiconductor value chain, including Intellectual Property (IP) and Electronic Design Automation (EDA) vendors.

Beyond the leading semiconductor companies, the chiplet approach presents opportunities for design houses and semiconductor service providers like Capgemini. Collaboratively developed General-Purpose chiplet dies can cater to a range of vertical applications, for example serving a consortium of automotive companies pursuing in-car digital services. Additionally, Domain-Specific chiplets or custom dies can be tailored to meet specific requirements.

In conclusion, chiplets represent a flexible, adaptable, and cost-effective alternative to traditional monolithic designs. With its potential to revolutionize chip design, packaging, and integration, the chiplet paradigm is poised to redefine the semiconductor landscape, driving innovation and efficiency across the industry.

“I fight for perfection”

– Mike Tyson

“Do you achieve it?”

– Charlie Rose

“Nah! No one does, but we aim for it…”

– Mike Tyson

Put simply, deep learning is a type of machine learning that aims to mimic the way the human brain works to recognize patterns and make decisions. Deep learning on mobile and edge devices, like smart wearables with limited computational resources, can be challenging. However, running machine learning models on edge devices and keeping the data local has advantages for data privacy, sustainability and latency.

With one of our staff a keen boxer, we took on the challenge of working out what types of punches were being thrown with the constraints of:

• A need for real-time feedback
• Minimal hardware
• Low powered devices

How does it work?

Capgemini developed an end-to-end demonstrator, where data was collected using a custom iPhone and Apple Watch application, allowing sensor readings to be automatically labeled. A hybrid deep learning model was built to accurately classify each punch type. The model was converted to an optimal form to run efficiently on low powered edge devices.

The demo streams raw sensor data from the watch to display on a screen, along with the punch classifications, as close to real time as the network connectivity allows. In this case, we’re using a single Apple Watch on the left wrist to generate sensor data (specifically accelerometer, gyroscope and orientation data). This data is sent to an AI model that determines which types of punches are being thrown by either hand.

When we demonstrate this to the attendees at various events, the reaction is often “Wait – how is that even possible?!”. Indeed, how do you know what the right hand is doing with a watch strapped to the left wrist? While it’s certainly easier to track the lefts, boxing is a whole-body sport – there are small but characteristic movements that happen in response to throwing a punch. These movements are what the ML model detects, and uses to classify the type of punch.

In a world where seemingly everything is smart and sensors are everywhere, this may seem like an artificial constraint, but often you can’t put sensors right where you want them – for example, due a harsh industrial environment or because they would be too cumbersome or intrusive for the wearer.

To run on low powered edge devices and give near real time results, the machine learning model had to be as lightweight and efficient as possible. Through a careful selection of model architecture and the type of optimization techniques discussed in this blog post on model optimization, we produced an AI model that could interface in real time on a low end smartphone. 

Applications

Our technology has potential applications across many domains. For example, you could extend this application to give feedback to a novice boxer and help them avoid common mistakes – the minimal hardware would make this a very portable solution. For patient monitoring, perhaps when managing chronic illness, ensuring privacy is crucial and minimizing the number of sensors could help ensure compliance and guarantee that solutions are minimally invasive and cost effective.

In industrial settings, being able to classify in real time close to where data is being generated allows for rapid intervention. It can also help to reduce the cost and associated carbon emissions from the transfer of large volumes of data.

In disconnected applications, like drone operation in remote locations, this approach can be used to improve autonomy, for example allowing these drones to locate a safe place to land in an emergency, through on-board real-time video analysis.

Conclusion

Capgemini is exploring the applications of this technology in sports equipment, but we believe that this is just the beginning; this technology certainly isn’t limited to the complex movements of the ‘sweet science’. Whether based on incoming sensor, video or other data, being able to analyze inputs in real time on low powered devices has a crucial part to play in unleashing the potential of intelligent products and services.

Want to see in action? Watch our boxing demo video

Interested in finding out more? Take a look at our Intelligent Products and Services offer and follow David on LinkedIn.