Do all roads lead to AI?

Publish date:

Applying intelligent algorithms to make road traffic safer and smarter

By Pierre-Adrien Hanania and Michael Schimpke

Applying AI to road traffic can lead to a better and safer society. On the one hand, security services would be able to better detect anomalies and fight danger. On the other hand, road infrastructures and their users would clearly benefit from a more energy-efficient and regulated use of what are the veins of a city.

How AI enables a smarter road traffic

But before start thinking about what kind of city we could live in, we need to determine how AI can help to make road traffic smarter.

Machine learning algorithms require both high quantity and quality of data to efficiently gather information. But where to get it? Traffic neither keeps bills and receipts, nor does it generate sensor data as do production lines. The only source available is traffic cameras. Consequently, smart traffic management and surveillance rely mainly on advanced computer vision systems.

As traffic surveillance videos are highly complex data sources comprising a plethora of information, a model that is more sophisticated than simple image recognition must be built. A two-step model is needed, where all relevant shapes have to be identified first. This is important to filter crucial picture components and reduce computational efforts.


In a second step, each identified shape is classified. This procedure has already been implemented successfully based on convolutional neural networks showing high accuracies, such as Mask-R-CNN[1] by Facebook AI Research and R-FCN[2] by Microsoft Research.


Nevertheless, these networks are precise but rather slow. Therefore,  they are not applicable to real-time analysis of traffic camera input. New algorithms, including single shot detectors (SSDs), provide a solution to this problem. They substantially increase the processing speed of pictures, such that videos with frame rates over 30 fps can be analyzed instantly. Although the speed boost trades off with decreasing recognition accuracy, state-of-the-art techniques[3],[4],[5] manage to maintain high frame rates while keeping a decent precision level. Hence, these thoughts are clearly not a vision of the future but already a reality that is applied in several fields.

AI identifies, searches, and tracks specific elements

The model focus needs to be more detailed in order to influence security and surveillance matters. It might be not enough to just identify cars as cars or persons as persons. To find persons of interest or cars, the detection algorithms also need to disclose unique specifics. This implies detecting license plates or faces, capturing relevant information, and identifying it by matching it to databases. This procedure highlights three features of what AI can do in road traffic:

  • Identifying and recognizing wanted objects, such as stolen cars or missing persons
  • An accurate working identification system enables searching for characteristics or details

All classes of the classified model can be searched for in a connected traffic surveillance database, for example persons with red shirts, green cars, or blue bikes.

  • This search function can be extended into a tracking process using an integrated system where all traffic cameras share information on a central platform.

A certain person is identified by several characteristics, his profile gets enriched with detailed data describing clothes and other features. The profile is sent to the platform, where other traffic cameras can access and search for it. As the next camera identifies the person, a movement profile can be recorded.

From security cases to the bigger picture of a smart city

The huge increase in computational power as well as the continuous elaboration of data-driven AI systems has made the vision of smart roads a realistic one. Traffic cameras were first installed decades ago, but today their benefit no longer depends on human control capacities. Single objects can be recognized, analyzed, and matched with databases in real time. Automatic decision-making based on this gathered information opens up many opportunities.

For security cases, applying AI could help fight organized crime, which often uses roads as channels for transporting drugs or weapons. These criminal activities tend to follow common schemes regarding road behavior and suspicious car components, among other things.

The identification of punctual dangers, such as ghost drivers or objects lying on high-speed roads, might be quicker and more efficient with AI.

What serves security can obviously also serve other purposes; security being only one element of the bigger picture that is a smart city. Monitoring parking, for instance, is a good example of how AI can improve road traffic. By assessing the overall situation or recognizing free parking spots and communicating these to the driver, AI-based insights might help the driver decide whether he wants to drive or take public transportation, or it can direct that driver to the nearest available parking spot.[6] AI could also use cameras on street crossroads to connect with traffic lights in order to enable intelligent traffic flow.[7] By detecting license plates, AI-based cases can identify the need for toll charges and initiate the payment process. This is actually done in London with the Congestion Charge and has led to 30% less traffic.[8] Furthermore, replacing old streetlamps with smart lighting systems significantly lowers energy consumption while increasing the subjective sense of security.[9]

Potential use cases for AI in road traffic

AI ethics will need to be at the heart of it all

By dealing with roads as playground, AI developers will need to think about core elements of what has been conceptualized as AI ethics. Four dimensions must be at the center of the use cases mentioned above:

  • Reliability of AI

How “right” is AI – does it recognize false positive, such as a fake weapon, or will I be asked to pull over by the police when I take my child to a costume party dressed as knight with a sword?

Beyond the recognition of factitious objects also lies the question of the neutrality of AI – how can one guarantee that the AI will be able to identify the independent variable and avoid jumping to conclusions based on skin color or ethnicity?

  • Data privacy and processing

By dealing with very personal data, AI applications in road traffic will be subject to the EU’s General Data Protection Regulation, which regulates the processing of personal data. Many of the use cases mentioned will therefore need to address data privacy – with one of the possible solutions being anonymization.

  • Explainability of AI

When it comes to security services, we have to legitimize the use of AI in making decisions that affect our day-to-day lives. While the black-box issue unavoidable, substantial insights on how AI reaches its conclusions will be a crucial asset.

  • The role of the human

Who is accountable for AI-based decisions? Where does the human step in and how can misled AI be contradicted? Many studies have shown that the human tends to stop thinking when AI takes over – this is a danger that cannot happen in use cases involving road security. As in many fields, we have to think of AI as something that augments the human, rather than replacing him.

Do all roads lead to artificial intelligence?

In our ever-expanding cities, AI can help organize and optimize the way we use roads safely. The benefits of AI could have a valuable impact on better organizing our movements, mastering the urban veins, and making decisive environmental and security-related advances.

The industry will need to embrace this potential in a comprehensive way that respects privacy and builds on a robust technology that is able to deal with the huge amount of data needed.

One key component of it will be to involve the human in all processes in which AI should only add time-saving and accuracy-helping features. Only then, the trip toward AI will be a safe and successful one.


[1] Kaiming HeGeorgia GkioxariPiotr DollárRoss Girshick 2017, Mask-R-CNN, arXiv:1703.06870 [cs.CV].

[2] Jifeng DaiYi LiKaiming HeJian Sun 2016, R-FCN: Object Detection via Region-based Fully Convolutional Networks, arXiv:1605.06409 [cs.CV].

[3] Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi 2016, You Only Look Once: Unified, Real-Time Object Detection, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 779–788.

[4] Tsung-Yi LinPriya GoyalRoss GirshickKaiming HePiotr Dollár, Focal Loss for Dense Object Recognition, arXiv:1708.02002 [cs.CV].

[5] Wei Liu , Dragomir Anguelov , Dumitru Erhan , Christian Szegedy , Scott Reed , Cheng-Yang F , Alexander C. Berg 2016, SSD: Single Shot MultiBox Detector, arXiv:1512.02325v5 [cs.CV].

[6] See

[7] See

[8] Garcia, Irene 2019, $15 a Day to Drive? Londoners Say ‘Thanks, I’ll take the train,’ on

[9] See

Related Posts


Leading the way to a data-powered future

Ramana Bhandaru
Date icon August 27, 2021

Proud to announce that Capgemini has been named a Leader in Everest Group PEAK Matrix®...


How to become part of the data-powered magic

Zhiwei Jiang
Date icon April 1, 2021

Proud to announce that Capgemini has been named a Leader in 2021 Gartner Magic Quadrant for...


House of the rising data

Pierre-Adrien Hanania
Date icon March 31, 2021

How strong data governance will benefit scaled AI in the public sector