Man Vs. Machine: Russian Company Compares Human, AI Driving Capabilities

The Moscow-based company, creator of its own automated driving assistance technology known as Cognitive Pilot, has completed a study to determine who – human beings or artificial intelligence, are better able to detect road signs, other vehicles and pedestrians in various traffic situations.

Speaking to Russia’s RIA Novosti, Cognitive Technologies’ autonomous software department head Yuri Minkin said that the tests’ main goal was to determine whether its AI had already outmatched human drivers’ reaction time and accuracy.

Put up against 17 human volunteers, the company’s AI was put to the test in a variety of challenging conditions (dusk, nighttime driving, driving in rain or conditions of blinding sunlight). The volunteers and the AI took turns identifying a series of objects on the road on a single monitor.Taking place between September and November, tests featured the use of a single camera, and were conducted along 27 different urban routes at speeds between 50-60 km/h. Researchers limited the number of objects appearing onscreen simultaneously to three for the human volunteers’ benefit.

The company said that their study confirmed that its AI’s detection capabilities have come very close to matching those of people, even exceeding them in some areas by small margins.

The first stage, studying the quality of detection of objects in good weather and road conditions, showed humans and AI as near-equals, in terms of both accuracy and speed, exceeding 99% accurate detection rate.Meanwhile, the AI proved superior when it came to objects that were not completely visible, hidden behind trees, parked cars or other obstructions. The AI also proved faster in detecting road signs, picking up even obscured signs in just a fraction of a second.

“The testing showed that under more difficult conditions, the human volunteers often noticed the road signs a moment later than the AI. This time gives the [AI] control system an additional advantage for the processing and analysis of information about the situation on the road as a whole,” Minkin said.

The AI was slightly better in terms of speed and accuracy of detecting objects in rainy conditions, (98.3% vs. 97% for humans). In twilight and blinding sunlight, human volunteers proved significantly slower in recognizing road objects, although the overall recognition rate remained 98% for both.

One area where AI lagged behind its human counterparts was in spotting pedestrians in rainy and nighttime conditions (with an accuracy of 98.2% compared to 99.2% for people shown).

“A person is a rather complex object for recognition,” Minkin explained. “Pedestrians do not have a constant form, and can travel embracing, holding hands, carrying an object, etc. And while the AI can match the abilities of a [human driver] in clear conditions, in more difficult conditions, the human’s abilities are still slightly superior.”

According to Minkin, the main takeaway from the study is that artificial intelligence is rapidly approaching the capabilities of human beings in the recognition of objects on the road. Furthermore, the more complex the road conditions, the better the AI performs, relative to its human counterparts.

“It can be expected that with an increase in computing power, and in the quality of sensors and software, the advantage of the AI will become all the more obvious, similarly to how it has become in chess,” he concluded.

Cognitive Technologies president Olga Uskova says that the company has plans to conduct further testing, and plans to use industry experts to come up with a more advanced testing methodology.”But we must understand that this was the first attempt to compare the capabilities of artificial intelligence and people. With this first approximation, we’ve received real results, which can and should be considered, and on whose basis it will be possible to predict the direction in the creation of AI that can drive vehicles,” she noted.