Tesla has had a cabin-facing camera in its cars for years now. Musk had earlier mentioned that they will be using this camera to prevent people from vandalizing cars when the upcoming RoboTaxi network starts. However, they activated this camera for the first time in a software update last year. Using the data from this camera, they were able to determine which drivers weren’t paying attention to the road while using the FSD beta suite. They then removed some drivers from the beta program last month.
There were some doubts regarding the real-time detection system, considering that Musk had said that they won’t be using it till the RoboTaxi network starts up. Many people thought Tesla was manually reviewing footage from the FSD Beta program.
Now, noted Tesla hacker Green has released a video that shows the real-time driver monitoring system using the cabin camera. The AI-based system can detect the driver’s attention on the road. It actively monitors where the driver’s eyes are looking and other distractions such as phone use.
The Real-Time Detection System
Green is a known Tesla hacker. He hacks into Tesla’s systems for hints of upcoming features and releases. He also gets information about the various kinds of data that Tesla collects, and releases most of it through Twitter or YouTube. This is another such video that gives a lot of information about the detection system that Tesla uses.
The video shows a number of different scenarios that Tesla’s system can detect. It also shows the probability of every scenario being true. Green tried to simulate different situations. He first sat on the driver’s seat, tried looking in all directions for understanding the accuracy of the system. He then shifted into the passenger’s seat. The system detects a number of scenarios. This includes “driver’s eyes nominal”, “driver’s eyes down/closed/up”, “view of head truncated”, “driver looking left/right”, “camera dark/blinded”, “driver head down”.
He also tried a few tricks, such as taping a piece of paper with his photograph on the headrest of the driver’s seat.
Then comes the part of using a phone. Interestingly enough, the system does not detect “phone use” when you are holding your phone to your ear. Instead, it detects if the phone is in front of your face. You can see that in the following image.
He also tried to trick the system by sitting in the rear row and putting his head in between the front seats. And as you can see, the system did say that that the driver’s eyes are on the road, but it was only 2.693% sure of that. Thus, the system probably uses other pointers as well, not just the head and eyes of the driver.
And just for fun, perhaps, he also tried taping Elon Musk’s photo on the driver headrest.
Privacy Concerns for Tesla’s Cameras
Recently, there was a report from the Chinese military, saying that it was worried about the Tesla vehicles on its bases. This was due to the cabin-facing camera. Tesla China has responded this week by issuing a statement on the subject. They said that they have activated this camera only in the cars in the FSD beta program. Since the beta program hasn’t been released in China, there is no need for concern.
The following statement is translated from Chinese:
Recently, there has been a discussion on the Internet about “Tesla monitors the owner through the camera in the car”. In order to eliminate unnecessary misunderstandings, the following is specifically stated:
The vehicles used by Tesla users do not infringe on the privacy of users through the in-vehicle camera.
All Tesla user vehicles on the Chinese market have not turned on their in-vehicle cameras, nor are they involved in FSD Beta testing.
Tesla’s privacy protection policy complies with national laws and regulations. Like you, Tesla attaches great importance to the protection of user privacy. We are equipped with a network security system with the world’s leading security level to ensure user privacy protection. Thank you again for your attention to Tesla.
Musk had also commented on this issue a few weeks ago, saying that there is no infringement on the privacy of users. He went on to say that if Tesla was using its cameras to spy on other countries, it would get shut down.
Our Opinion
The real-time driver monitoring system is quite impressive. The system is changing the data every few milliseconds, which shows the speed at which it works. So Tesla is using an AI-based system to monitor drivers and using these results, they decided to remove certain drivers from the beta program.
Of course, there is an argument that they could send warnings to the user when his/her attention to the road isn’t satisfactory according to Tesla. Given the speed at which the data is being processed, they can easily do this. Maybe, since this is just the beta release, they may be still working on that. We can’t comment further till Tesla actually takes some action.
