How to Search for Vulnerabilities in Embedded Software

Embedded software is a popular target for hacking, since embedded systems are present in almost all modern devices. Securing embedded software requires expert development skills, as embedded devices are severely limited in the amount of code they can run, and updating security configurations after release is sometimes impossible. That’s why developers have to be extremely attentive when searching for potential vulnerabilities.

In one of our previous articles, we discussed common attacks on embedded systems. This time, one of our experts shares his knowledge on how to conduct swift vulnerability searches for Linux-based embedded software, automate this process, and validate whether security issues indeed threaten your software.

This article will be useful for security testers, QA engineers, and software developers working on embedded software and firmware.

How dangerous are exploits of embedded software vulnerabilities?

Embedded software is code that controls the operation of an embedded system. Such systems are created to perform specific functions in larger mechanisms. They are small, energy-efficient, and capable of working in real time. That’s why such systems are widely used in modern devices: traffic lights, cars, computers, aircraft, refrigerators, pacemakers, etc.

The widespread use of embedded systems makes their software a popular target for hacking. Cybercriminals look for both well-known and zero-day vulnerabilities in embedded software code to gain control over devices or steal the data they collect.

Let’s take a look at several examples of vulnerabilities found in widely used embedded systems and explore possible consequences of their exploits:

• Urgent//11 — Security researchers discovered 11 zero-day vulnerabilities in the VxWorks firmware. This real-time operating system (RTOS) is used in around two billion devices, including medical and manufacturing devices, infrastructure elements, and phones. The vulnerabilities allow hackers to execute code remotely and take control of devices.
• Xiaomi Mijia smart security — A security issue in smart cameras made by Xiaomi allowed their owners (and criminals) to watch video streams from other people’s homes. Since Xiaomi smart cameras were used in security systems, this buggy in-camera firmware made owners’ homes unsecure.
• Car control systems — A team of security researchers conducted an experiment to test the security of car control systems. These systems manage the operation of almost all devices in a modern car. Researchers introduced a series of fake packets into communications with a car’s electrical control units. After that, they were able to activate and deactivate the brakes and interfere with the engine, locks, instrumental panel, lights, and radio. 
• Qualcomm’s Mobile Station Modem — This chip is used in 31% of all smartphones to process voice calls, SMS, and high-definition recording. A group of security researchers discovered that it’s vulnerable to a heap spray attack performed by a malicious application installed on a user’s phone. Hackers can exploit it to listen to a user’s calls or get access to call and SMS history.

These cases clearly show how important vulnerability searching and security testing are. Let’s see how to detect vulnerabilities in embedded firmware and libraries used in embedded software development. We’ll start this process by getting more information on first-day exploits in Linux-based systems.

Gathering reports on first-day vulnerabilities

First-day vulnerabilities are exploits that have already been discovered and reported. Searching for such exploits usually starts with discovering and examining libraries used by the device’s firmware.

With Linux-based firmware, we can simply write down all the libraries and their versions from the firmware’s core. It’s best to identify as many libraries as we can to make sure we don’t miss any vulnerabilities. However, experienced security researchers can look only for specific libraries that they know might contain exploits.

If we are working with monolithic firmware, we have to analyze constants to discover libraries used. For example, we can find some of these libraries by checking debug messages or finding the row with the library version in the code.

Once we have the list of libraries implemented in firmware, we can search for known vulnerabilities in these libraries using data from the following resources:

• National Vulnerability Database
• Japan Vulnerability Notes
• Chinese National Vulnerability Database
• CVE Details
• Common Vulnerabilities and Exposures

When users discover new issues, they can create vulnerability reports and upload them to these databases. Reports can contain data on security issues in any piece of code: firmware, operating systems, drivers, etc. Since anyone can register security issues, vulnerability reports in these databases aren’t always of the best quality. For example, a report may contain a vulnerability description for a single version of a library with no information on other versions.

The same goes for device vulnerabilities, especially for smartphones. With the variety of devices, it’s nearly impossible to test all device versions for a particular vulnerability, so researchers conduct tests on the several models they have access to.

That’s why we have to look for additional sources of data on vulnerabilities, such as bug tracking systems.

Searching for data in bug tracking systems

Sometimes, developers discover vulnerabilities in embedded software but don’t register them in the databases mentioned above because they treat these vulnerabilities as minor bugs. For example, if a developer detects issues like buffer overflow, they can fix them and move on without creating a report. The only way to learn about such vulnerabilities is to check bug tracking systems. 

For example, we can search for Qt platform bugs in the dedicated bug tracking system with a single JQL request:

project = QTBUG AND affectedVersion = 5.3.1 and priority >= "P1: Critical" and text ~"crash"

This request will help us find critical vulnerabilities in Qt 5.3.1 that may lead to the crash of this platform.

Once we’ve finished searching for potential security issues, it’s time to evaluate the level of danger coming from the discovered vulnerabilities. Let’s see how we can determine which bugs require immediate fixing and which can be dealt with later.

Validation of discovered vulnerabilities

Once we have a list of possible software vulnerabilities, our next step is to validate which are an actual threat to our device.

Here are the three reasons why validation is crucial and can’t be skipped:

1. Vulnerability reports may contain mistakes, inaccuracies, or outdated data.
2. Our version of the library may be different from the one a security researcher tested. Even if it has the same version number, the libraries could be compiled from different sources or use different compilation settings.
3. The reported vulnerability can be relevant only for applications with a certain architecture, so the set of commands generated in our application simply may not have this vulnerability.

A reliable way to validate the possible impact of a vulnerability on your software is to check its score in a vulnerability database. Most databases have a system to rank the severity of each vulnerability, such as the Common Vulnerability Scoring System (CVSS) score, that helps software developers determine which security issues they should focus on first. 

For example, vulnerabilities that can allow hackers to execute code remotely or escalate their privileges score 10 out of 10 in CVSS. Bugs with such a high score are quite rare, which is why in most cases we still need to analyze reports before the bug fixing stage.

However, our experience shows that vulnerabilities with a score below 7 usually describe bugs that are hard to reproduce and exploit. Such vulnerabilities don’t allow hackers to control code execution or require them to insert the exploit to an exact memory address. And security testers need to spend hours trying to reproduce such vulnerabilities. That’s why it’s best to focus on more harmful exploits.

This is how you can validate embedded software vulnerabilities in four steps:

1. Filter the list of vulnerability reports with a score of 7 and higher and exclude all reports with lower scores from the list.
2. Rule out reports that contain little to no technical information. If a software vendor forbids publishing technical data about their products, their vulnerability reports won’t contain the names of functions used in the code, links to changes in the source code that can fix the detected issue, ready-to-use exploits, etc. Such reports provide us only with a general description of the vulnerability, with no clues on how to detect and recreate it.
3. Finalize the list of the most likely and dangerous security issues and start validating them in your code. At this stage, you need to use a disassembler to find the exact piece of code that causes each vulnerability and check that it doesn’t contain fixes.
4. Reproduce the environment that makes the vulnerability affect your architecture. If you see that the vulnerability does exist in your code, move to fixing it.

Searching for vulnerabilities and validating that they can threaten your embedded software is a time-consuming process even for an experienced developer. Fortunately, we can significantly speed it up by automating it. Let’s take a look at several tools that can at least partially automate the discovery of code vulnerabilities.

Automating the vulnerability search

Automation tools save hours of work for someone looking for security issues. Usually, they locate a known vulnerability or suspicious code, after which the QA specialist or developer has to analyze it and decide whether this code really creates a vulnerability and needs to be fixed. 

When using automation tools, it’s important to remember that they don’t do all the work for security testers. Also, they might not discover all potential vulnerabilities in software and may return false positives.

The Firmware Analysis and Comparison Tool (FACT) is one of the tools we often use at Apriorit to save time when looking for vulnerabilities in our embedded projects. FACT examines Linux-based firmware for various types of architectures: x86, ARM, MIPS, and PowerPC. It can use an image of the firmware or a tar.gz archive with a rootfs archive inside as an input. The output is a list of versions of all the files inside the firmware and all corresponding vulnerabilities from the National Vulnerability Database.

FACT doesn’t determine file versions with 100% accuracy, and it doesn’t validate discovered vulnerabilities. However, it automates the initial stages of the vulnerability search. For example, FACT needs approximately half an hour to analyze firmware that consists of 3,000 files and takes up 1 GB of disk space. The average developer needs at least several hours to analyze such firmware manually and draft a list of possible vulnerabilities.

Emba is another helpful tool that allows security testers to analyze firmware images for Linux-based embedded devices and RTOS-based systems. It can also analyze kernel configurations. During analysis, emba highlights suspicious code in the firmware that can cause a vulnerability. This tool significantly speeds up vulnerability searches for security testers because they don’t have to analyze the firmware themselves. 

FwAnalyzer is a tool for analyzing file system images. It works with the ext, FAT, SquashFS, and UBIFS file systems. FwAnalyzer compares the file system image to the comparison file created by developers that contains various rules for firmware files and directories. It also helps developers check the contents of files, extract data from them, and detect changes in different versions of files and file trees. As a result of its analysis, FwAnalyzer creates a report that contains all differences it discovered or the data a developer requested.

Conclusion

Collecting as much information on first-day vulnerabilities as possible is one of the first steps to securing embedded software. Although there seems to be a lot of data on such security issues, there’s a risk of spending too much time researching them or missing something important. That’s why developers and security researchers need an algorithm to validate and prioritize discovered vulnerabilities.

At Apriorit, our expert teams of embedded software developers and security testing specialists work hand in hand to ensure the software they create is not only efficient but secure.

Feel free to reach out and leverage our knowledge to improve the security of your embedded software development project!