Though Elon Musk’s takeover of social-media company Twitter is making news, and his rocket company SpaceX is scheduled to test fire its massive fully-reusable rocket Starship/Superheavy into orbit in December, I’m going to revisit a topic that I previously covered some months ago, and the company he was best known for previously, Tesla.
I recently had the opportunity to borrow a friend’s Tesla Model 3 equipped with the Full Self Driving (FSD) beta for a long trip, and I’d estimate I was able to let the car handle the driving about 97% of the time.1 In addition to driving itself, the car also monitors the human who’s supposed to be supervising its performance, to ensure that they’re paying attention, by displaying occasional notifications to “apply slight turning force to the steering wheel.”2 This amounts to either slightly resisting the car’s steering wheel as it rotates for turns by letting it slip past your fingers, or “following along” with the turning, as though guiding a horse. In the beta, the car is also supposed to be able to recognize when you’ve taken your eyes off the road for too long, or are using a mobile device, because to run the beta, you grant it permission to use the car’s interior-facing camera to watch you, to make sure you’re not “abusing” the self-driving feature.3
(The scanning Cylon-eye4 of the Knight Industries Two Thousand, K.I.T.T., from the 1980s T.V. show “Knight Rider,” which featured a talking robotic car.)
To get the coveted FSD beta, you’d have had to get used to the car monitoring your driving without it, and Tesla owners who’re interested in getting access to this early release have done so, driving cautiously to get a good “score” (as though it were a video game), in addition to paying the hefty five-figure price tag in advance to add the FSD option to their car.5 Those who have get some neat features while they’re waiting for FSD, including “autopark” and “summon.”
Summon, which is a bit of a trick to show off to your friends, allows you to designate a spot using the Tesla mobile app that’s near to where the car is parked, and so long as you’re holding down the button in the app, the car will navigate to it with nobody at the wheel. A bit unsettling to see a driverless car coming in your direction across the parking lot, but it does work, and eventually the car will come to a halt close to where you designated.
I found the autopark feature to be much more handy, and only slightly less unsettling, when the car starts backing into the parking space that you’ve designated on the screen, and turning the steering wheel by itself (which is usually a Tesla passenger’s first experience of having the machine do the driving, even before they get FSD). Supposedly it can also parallel park, but I never had occasion to test that. I’ll have a bit more to say about autopark later, in my notes on FSD beta generally.
To enable full-self driving, you’ve got to first input your destination into the car’s navigation system, which allows the car to “navigate on autopilot” even on local streets. Tesla’s cheaper “Enhanced Autopilot” option is only able to “navigate on autopilot” on the highway, but this is the level of functionality that Tesla owners who’ve paid for FSD receive until they are accepted in the FSD beta program, and Enhanced Autopilot also includes “autopark” and “summon.” The base level of Tesla’s driver-assist technology is simply called “Autopilot,” and features traffic-aware cruise control and lane following on the highway. It’s similar to radar-following cruise control on other vehicles, except that the lane-keeping steering makes it feel like driving on a rail, as the car maintains its position precisely between the lane markings.6
I didn’t have time to learn how to edit the suggested routes, and sometimes, as GPS tends to do, it sends you on weird routes that you might not prefer, just because they’re slightly “shorter” or “faster.” Tesla uses Google for its navigation data. The car I tested in was not a subscriber to Tesla’s “Premium Connectivity” package, which includes traffic updates and music services delivered via cellular streaming. Other cars use over-the-air HD digital radio to obtain traffic and weather information for free, so Tesla not using HD radio seems like a step down from my own car, which also has a weather radar screen available.7 HD radio signals aren’t always available outside of metro areas, so I can understand why Tesla doesn’t use it, besides having the income stream of a subscription service they can charge for, in addition to its Supercharger network. Elon Musk has hinted that future Tesla models will include SpaceX’s Starlink satellite receivers as an option, providing internet connectivity even in remote locations.
As the Tesla autopilot technology is camera-based, it slows down when visibility is poor, as a human driver would do, such as in heavy rain or fog. It also does best on roads with clear lane markings. On roads without lane markings, or in parts of parking lots which are wider than a typical road, it seems drawn to either drive in the middle of the road if there’s no approaching traffic (apparently assuming there’s only one lane), or stay towards the right if there is a stretch of unmarked pavement wider than two typical lanes. This behavior caused a rather unexpected result at Shelby Farms, our large urban park here in Memphis. As I was approaching the visitor’s center, where the drive is wide but unmarked and has parking for some cars on the right, the Tesla chose to drive to the right, within a few feet of the parked cars, rather than staying on what I considered to be the “road” portion of the drive. Once the drive narrowed again, it got back in the “lane.”
The revision of FSD beta that I tested also has an issue with speed bumps. It doesn’t seem to notice them most of the time, nor does it seem aware of potholes. While its recognition of other cars, trucks, motorcycles, bicyclists, pedestrians, trash cans, stop signs, stoplights, and traffic cones is very well-developed, FSD beta also seems oblivious to debris in the roadway.8 Driving on Walnut Grove one afternoon, there were some branches placed at the curbside for trash pickup which were hanging partway into the road, and I suppose the car might have noticed them once they got within range of its low-speed ultrasonic sensors, but I took over to steer around them, just in case. As the human "supervisor" of FSD beta, it takes some time to have confidence in its capabilities.
For the things the car is trained to see, it can simultaneously keep track of far more objects in its environment than a human driver can. After all, it has eight exterior cameras for its eyes, and humans only have, as Elon Musk describes it, “two cameras on a slow gimbal.” The on-screen visualization of what the car is aware of is quite impressive. And as Elon Musk has pointed out, to a computer, things in the human world happen extremely slowly. That said, machine vision in a real-time operating system is very hard problem in computer science, and situations in the world are full of ambiguity, something that computers have a much harder time dealing with than humans —so far.
The autopilot’s default behavior is to drive at exactly the posted speed limit, so the car’s speed changes whenever its cameras see a new speed limit sign. You can, in the autopilot settings, establish an “offset” value in miles-per-hour (or in percentage) to cause it to drive faster or slower than the speed limit, according to your comfort level as a driver. You can also adjust the speed on-the-fly, just as you can for ordinary cruise control in other cars, using the right scroll wheel on the hub of the steering wheel. Turning the scroll wheel slowly will change the speed in 1 m.p.h. increments, and rolling it faster changes the speed in 5 m.p.h. increments.
While the FSD beta handles highway on-ramps very well, I’d say it needs some work with off-ramps. I’m assuming this is because the speed limit signs for the ramps are not the standard black-and-white speed limit signs. I’ve noticed that it seems to want to take off-ramps at the speed the cruise control is set for, unless there’s a slower vehicle ahead of it, so I would adjust the speed slower with the scroll wheel when approaching an exit. There are times when you don’t want to wait and see what the car is going to do, and taking an exit ramp too fast is one of those times!
In Enhanced Autopilot and FSD beta, the car uses its awareness of nearby vehicles to determine when it’s safe to change lanes, either to follow the set route, or to pass other cars which are driving more slowly. By default, the car will automatically change lanes to follow the route, but for passing, it will suggest a lane change, and execute it when it’s safe to do so, after you use your turn signal to begin the passing maneuver. Once you’ve passed the other vehicle, autopilot will suggest returning to the outside lane, which is what many state highway laws require as well.9 You can ignore the suggestion, and you can also suggest lane changes yourself, by signaling with the turn signal stalk.
As one might expect, the Tesla with FSD beta is an exceptionally cautious driver. Once when I had changed lanes to pass a semi, and I was already in the left lane, but still behind it, the truck put on its brake lights briefly, and the Tesla hesitated to begin the passing maneuver, as though uncertain of what the semi was going to do. Another example is when I was trying to get it to autopark at a space in front of a restaurant, and there was a group of people waiting on the sidewalk behind the parking space. The Tesla just would not back into the space, and hesitated very noticeably, jerking the steering wheel back and forth, seemingly “indecisive” about the situation. After all, someone could easily step off the curb into the parking space, and there were pedestrians “in the way” of its planned path.10
Autopark seems to be designed with the Supercharger network in mind, so the Tesla always backs into parking spaces, rather than driving forward into them, because the charging port of the car is behind the driver’s side rear taillight. This is a little annoying in a parking lot at times, because if there’s another car following you in the parking lot, they may think you’ve driven past the space, rather than that you’re getting in position to back up. Perhaps driving past the space is necessary for the car to get a good fix on the position of the open parking space, but if not, it’d be nice if you could choose whether to park front-in or back in to the space.
Because the software is still in beta, Tesla stresses the need for human supervision. It can, Tesla cautions, “do the wrong thing at the worst time.” If you happen to be making a very sharp turn, or a turn that is somewhat less than 90 degrees, it seems difficult to keep the “just right” amount of force on the steering wheel to keep FSD beta from disengaging. Occasionally, making left turns from a narrower two lane road onto a road with more lanes, including a left turn lane, it may, like a human driver, not properly judge how wide to take the turn. I had a couple of instances when the car appeared to be headed for the turn lane, rather than the corresponding lane of the wider road, so I had to take over. There have been some challenging driving situations requiring turning left across lanes of traffic with limited visibility where either I didn’t trust the software to handle it, or the car appeared sufficiently “indecisive” that I took over. If the FSD beta is “uncertain” at an intersection, it may take longer than a human might to merge or turn.
The software’s hesitancy occasionally extends to what Tesla owners call “phantom braking,” in which the car applies the brakes briefly to slow down when there’s no apparent reason for it. Phantom braking will sometimes occur when the car can’t immediately determine whether an oncoming vehicle is in its own lane or in a different lane, and so the car slows briefly until it figures it out, and then just as quickly resumes normal speed once it has determined there is no hazard. “Phantom braking” is now a rather rare occurrence in comparison to previous versions of Enhanced Autopilot.
Overall, the FSD beta is at a very advanced stage of development, and Tesla has begun adapting its core machine vision technology to the company’s humanoid robot prototypes, variously code named Optimus, or “Tesla bot.” I occurred to me not long ago that the Tesla car with Full Self Driving isn’t so much a car, but is actually a robot that you can ride around in. More than meets the eye.
The release I tested was 10.69.2.4, but there’s been a more recent update since then.
But not too much, or you’ll disengage autopilot.
I never tested whether this is true, because participation in the FSD beta testing program is conditional on not getting five “strikes” for such inattention. A strike would disable the FSD beta for the remainder of the trip, and accumulating five strikes would reportedly disable the beta at least until the next software release (not to mention getting my friend very upset with me!).
“Knight Rider“ was produced by Glen A. Larson, who also developed Battlestar Galactica, and other science-fiction TV shows of the 70’s and 80’s. My title is, of course a reference to the theme song from the toyetic mecha franchise Transformers, in which cars and other vehicles are revealed to be alien “robots in disguise.”
The current price for FSD on a Tesla is $15,000, which became effective September 5, 2022. My friend paid $10,000 about a year ago.
Current Tesla models only use machine-vision cameras for autopilot features, and lack radar, so it’s not accurate to call autopilot “radar-following cruise control” anymore.
So far as I know, there’s no weather radar app screen in the Tesla, but I don’t know whether there’s one available if you have the Premium Connectivity package.
I assume the software also can recognize large land mammals, like horses, or deer, but I fortunately didn’t have an opportunity to put that to a test. Also fortunately, I didn’t encounter tire shards from blown out semi-truck tires, or big rocks, or any of another several hazards that can occur in interstate travel where I was driving.
Because I’m an attorney, I was curious what the law was in Tennessee regarding autonomous vehicles. Tennessee’s law was enacted in 2015.
Another time, there was a pedestrian slowly jaywalking across the road about 50 yards ahead, and the car detected him and slowed down as we approached, giving the pedestrian time enough to get out of the way.
Good test report. Were you an engineer before law school?
All this reminds me of when I briefly worked remote piloted aircraft (drone) safety as an Air Force contractor. Predator and larger Reaper drones were very pilot-intensive, pilot-in-the-loop. Constant attention to keep it trimmed, but the big problem was (predictably) on landing. Army's Predator-like Grey Eagle drone had an auto-land feature, but Air Force leaders demanded pilots land their own (expletive) airplanes. Meanwhile, the Global Hawk and Navy Triton aircraft have an outstanding autopilot, with the human operator very much pilot-ON-the-loop (not in). The pilot puts in waypoints and commands, and the airplane figures out all the little details like keeping itself in the air. It lands just fine, with close to zero intervention. A car autopilot is actually a far more challenging problem than an airplane autopilot or even autolander. You don't bump into tire debris at 20,000 feet MSL.