Artificial Intelligence in the Automotive Industry: 6 Key Applications for a Competitive Advantage

Artificial intelligence (AI) technologies have made the futuristic idea of self-driving cars a reality. Today, automotive industry leaders including BMW, Volvo, and Tesla leverage AI capabilities to improve manufacturing, increase supply chain efficiency, and make our driving experience safer, more comfortable, and more entertaining.

In this article, we discuss uses of AI and machine learning for the automotive industry. We go over some of the key tools you can use for building AI-powered automotive solutions and discuss the main challenges of AI adoption to expect along the way. We also overview interesting examples of automotive software development projects and the technologies used in them.

The role of artificial intelligence in today’s automotive industry

Car manufacturers are constantly looking for ways to speed up design, production, and manufacturing processes while improving vehicle quality. Customers want to see vehicles that offer pleasant, comfortable, and productive experiences rather than simply getting them from point A to point B.

Artificial intelligence (AI) may be the answer. AI technologies have enormous potential when applied both in production and manufacturing processes as well as within vehicles to power in-car functionality.

Gartner predicts that the total number of new vehicles equipped with autonomy-enabling hardware will rise from 137,129 units in 2018 up to 745,705 units by 2023. The size of the global market of autonomous vehicles is expected to reach as high as $37 billion.

Let’s see where exactly we can leverage the capabilities of AI and machine learning in the automotive industry:

1. Design and manufacturing

AI-powered solutions and ML algorithms help vehicle manufacturers improve production processes, speed up data classification during risk assessments and vehicle damage evaluations, and do many other things. AI systems and robotics solutions relying on such technologies as computer vision, natural language processing, and conversational interfaces are widely applied in vehicle manufacturing.

For example, Nvidia’s Quadro RTX graphics card [PDF] uses AI to significantly accelerate design workflows. Rethink Robotics makes collaborative robots for performing tedious tasks like handling heavy materials and inspecting produced parts.

2. Supply chain

It’s vital for vehicle manufacturers to be able to monitor every stage of a component’s journey and know exactly when to expect its arrival at the destination plant. That’s why modern supply chains often rely on cutting-edge IoT, blockchain, and AI technologies.

In particular, vehicle manufacturers can turn to solutions relying on different machine learning algorithms and AI-powered predictive analytics. With their help, manufacturers can estimate demand for components and predict possible changes in demand in a timely manner.

For example, Blue Yonder leverages AI technologies to increase inventory movement visibility and enable manufacturers to predict possible delivery disruptions.

3. Quality control

AI can enable timely detection of various technical issues. Based on data gathered by in-vehicle sensors, an AI system can inform a user that a certain component or system requires maintenance or needs to be replaced as early as the need arises. Manufacturers also use AI-powered quality control systems to detect possible flaws in parts before they get installed.

In-car quality control systems mostly rely on data processing and analysis methods, while solutions used in manufacturing leverage image recognition and sound processing AI solutions.

BMW uses AI-powered solutions for predictive maintenance of welding tongs and paintwork quality analysis, among other tasks. And Predii’s AI-based platform prescribes vehicle repairs based on analysis of sensor data.

4. Passenger experience

To make sure all passengers are safe and satisfied, manufacturers enhance their vehicles with all kinds of AI-powered applications meant to upgrade the passenger experience.

Some systems use face recognition and emotion recognition methods to evaluate the state of the driver and passengers. Others deploy natural language processing and natural language generation methods to enable passengers to watch movies, listen to music, and even order goods and services while on the road.

For example, Dentsu and Hyundai invested $10 million in the Audioburst project to create an AI-powered infotainment system. Using automatic speech recognition and natural language understanding, this system will enable passengers to search music/audio libraries, enjoy personalized music playlists and news briefs, and so on.

Amazon is working on enabling the use of their AI-powered Alexa voice assistant in vehicles of different brands. Integration with Alexa is already available for infotainment systems in BMW, Toyota, Ford, and Audi cars.

5. Driver assistance

Of course, let’s not forget about improvements to the driving experience offered by AI technologies. There are AI systems meant to assist drivers and ensure safety by warning them about traffic and weather changes, offering the most efficient routes, or paying for goods and services on the go. 

CarVi is an advanced driver assistance system (ADAS) that uses AI capabilities to analyze traffic data. It also notifies drivers in real time about possible dangers like driving conditions, lane departure, and forward collisions. Such solutions rely heavily on real-time image and video recognition, object detection, and action detection, but may also use speech recognition and natural language processing technologies.

Other systems aim to take on the driver’s role — either temporarily, as with the autopilot functionalities in some Tesla cars, or completely, as in Waymo’s driverless cars and Zoox’s autonomous vehicles for robotic ridesharing. These systems often combine complex computer vision capabilities with real-time analysis of big data and natural language processing.

6. Automotive insurance

AI-powered solutions have great potential in handling insurance claims. On the driver’s side, in-vehicle AI capabilities can be used for gathering incident data and filling out claims. Such a system would need to combine smart data analytics, speech recognition, natural language processing, and text processing and generation.

On the insurance provider’s side, AI systems leveraging image processing and object detection technologies can be of great help for improving the accuracy of vehicle damage analysis.

An example of using AI in car insurance is the Ping An Auto Owner application which uses AI capabilities to assess photos uploaded by users making insurance claims. Nauto’s intelligent fleet management system has an AI-powered collision detection feature that enables quicker and more accurate processing of insurance claims.

What to consider when building an AI solution for the automotive industry

While AI has promising potential in the automotive industry, the practical adoption of this technology in vehicles has certain limitations. In this section, we overview the key pitfalls to expect when using artificial intelligence for automotive applications.

Biased AI

As AI systems tend to be biased, you can try to solve your task using robust algorithms instead of an AI system. Just make sure to analyze and thoroughly check all algorithms responsible for safety-critical functions. You can evaluate the correctness of algorithms either manually or with the help of special mathematical correctness proofs.

But note that retrained algorithm parameters can’t be applied right away. You should first check if the algorithm remains correct and functions properly after receiving new data, and only then apply it in production.

Poor data quality

Data is the core of any AI system, so it must be of the highest quality. However, collecting a large enough dataset filled with high-quality, properly labeled and annotated data is a true challenge. AI models used for smart vehicles must be predictable, precise, and fast enough to enable safe and accurate responses to different events on the road in real time.

Some of the needed data can be collected either from smart systems and robots used at the manufacturing plant or from in-car sensors. Other data needs to be created and improved artificially. And no matter the source, all data needs to be thoroughly checked and tested to ensure both its quality and its completeness.

Sensor and device limitations

The quality of data heavily depends on the technical capabilities of the sensors and devices used to collect it. That’s why gathering lots of meaningful data is useless if you receive it from the wrong sensors.

For example, when deploying a machine learning model to process audio data received from microphones, it might be more effective to record audio using ultrasonic devices instead of regular devices, as they can filter out background noise.

Black-boxed AI

Many of today’s AI solutions are black-boxed, meaning even a system’s developers can’t tell how it came to a particular decision. But for the automotive industry, it’s critical to rely only on explainable AI (XAI) solutions. 

In contrast to black-box AI models, decisions made by XAI systems must be transparent and understandable for humans. This enables AI developers to detect possible defects in their algorithms and implement improvements in a timely manner.

Non-compliance with local regulations

The applications of AI and deep learning in the automotive industry progress faster than the implementation of respective laws and regulations. There are some legal gaps regarding the development of AI-powered solutions for vehicle manufacturing and transportation. However, there are still some requirements you should take into account during software development.

You can take into account requirements and recommendations from:

• The Society of Automotive Engineers (SAE) International
• The US Department of Transportation
• The European Automobile Manufacturers’ Association [PDF]
• And other entities

Useful tools and frameworks

Depending on the task at hand, you will need to use different datasets, libraries, and frameworks as well as pre-trained AI algorithms and models. Below, we list some common tools and frameworks that might be useful in your AI-powered automotive project.

Deep learning frameworks

Deep learning frameworks provide AI developers with the tools and capabilities needed for quickly creating and efficiently training AI models. There are several popular deep learning frameworks you can use for building computer vision and conversational AI solutions, including PyTorch, TensorFlow, and Keras.

Simulator platforms

Simulators are widely applied for designing concepts of future autonomous vehicles as well as for developing, training, and testing their systems. Let’s look at some of the most useful ones:

CARLA is a popular open-source platform that can help you generate custom maps and simulate traffic scenarios or weather conditions for thorough training of your autonomous driving system.

Cognata is a simulation platform for building ADAS and other AI-powered solutions for autonomous vehicles. It can be used for testing smart vehicle applications by simulating driving behavior in virtual 3D locations.

Apollo Auto is an open autonomous driving platform that offers a perception system for analyzing data from different sensors, a simulator for modeling and testing autonomous vehicle performance, map creation capabilities, and more.

Autoware.Auto is an open-source project providing ROS 2-based solutions for autonomous cars. Currently, the platform handles such tasks as object detection, localization, and mapping.

NVIDIA DRIVE is a set of autonomous vehicle development platforms that includes capabilities for training deep neural networks and a simulation platform for testing and validating autonomous vehicle solutions. These platforms, however, are only accessible to those registered as NVIDIA developers and NVIDIA DRIVE Developer Program for DRIVE AGX participants.

Open-source datasets

AI-based solutions show the best results when they are trained on properly prepared, high-quality datasets.

There are several open-source datasets that you may find useful when working on an AI-based solution for the automotive industry:

PandaSet is a free dataset created by Scale AI and Hesai. The dataset is licensed for both commercial and academic use and can be applied for various autonomous driving challenges. This dataset includes multiple 3D bounding boxes and point cloud segmentation data necessary for recreating realistic, complex urban environments.

nuScenes is another useful open-access dataset that contains high-quality colored images and 400k lidar point clouds relevant to different daytime and weather conditions. The creators of nuScenes also released a Python devkit which makes it easier for AI developers to navigate this complex dataset.

Level 5 is a promising project by Lyft which is going to be acquired by Toyota. This dataset provides you with over 1.3M bounding boxes and 30,000 lidar point clouds collected from Lyft’s autonomous fleet. 

Waymo Open Dataset is a rich dataset with high-resolution sensor data collected by Waymo Driver-operated autonomous vehicles. It also contains labeled data for recognizing vehicles, pedestrians, cyclists, and road signs, and a motion dataset for determining object trajectory.

Berkeley DeepDrive BDD100k is one of the largest datasets for autonomous vehicles, which contains over 100,000 driving experience videos recorded on the roads of California and New York. This dataset can be used for training your models to solve tasks like detecting lanes and objects, tracking multiple objects, tracking segmentations, and more.

Data processing libraries and platforms

Whether you are planning to use AI for designing a new vehicle or enhancing it with driverless capabilities, your AI solution would have to process lots of data collected by different sensors: cameras, GPS, radars, lidars, and so on. Even the ready datasets created for training AI models for autonomous cars are complex and often require additional visualization tools. Below, we list some of the libraries and platforms that you might find helpful when working with data like lidar point clouds and 3D bounding boxes.

deck.gl is a WebGL-powered library created at Uber and maintained by the vis.gl team. This library comes in handy when you need to perform visual exploratory data analysis of massive datasets.

streetscape.gl (also known as AVS) is another library created by the vis.gl team. This library includes tools for visualizing autonomy and robotics data recorded in the XVIZ format. Similar to deck.gl, AVS allows for visualizing point clouds and bounding boxes, and it also supports real-time playback.

Geospatial Data Abstraction Library (GDAL) is a powerful library that comes with a rich set of command line utilities for translating and processing geospatial data. 

NumPy is an efficient library that can be used for data analysis and array computing. Many other popular Python libraries, including Pandas and Matplotlib, are based on NumPy.

Automated Driving Toolbox is a co-simulation framework that provides tools and algorithms for designing, simulating, and testing autonomous driving systems and ADAS. Using this framework, you can recreate different driving scenarios and simulate lidar perception, path planning, sensor fusion, and so on.

Conclusion

The variety of possible applications of machine learning in the automotive industry are impressive. Manufacturers can deploy AI technologies for designing and building new prototypes, improving the efficiency of their supply chains, and enabling predictive maintenance for both factory equipment and vehicles on the road. 

AI is also the power behind driver and passenger assistance services delivering experiences such as driverless transportation, in-car shopping and entertainment, instant insurance claim filing, and so on.

But despite its promising potential, the use of AI in the automotive industry is associated with several challenges. Some of the biggest are associated with algorithm biases, data quality, and understanding how a model came to a certain conclusion.

At Apriorit, we have a team of passionate experts who have already created a number of ambitious AI solutions.