Self-driving cars see the road by turning the world around them into data that a computer can understand. Cameras, lidar, radar, GPS, and other sensors work together to detect lanes, vehicles, signs, cyclists, pedestrians, and obstacles. This matters because safe driving requires fast decisions in a changing environment, often with only fractions of a second to react.
The car is not just recording the road, it is building a live model of what is nearby and what might happen next.
The system usually follows a pipeline: sense, perceive, predict, plan, and control. Lidar creates a 3D point cloud, cameras provide color and texture, and radar measures distance and relative speed even in poor visibility. Machine learning models label the scene through tasks like semantic segmentation, then prediction models estimate where each object may move.
Edge cases such as snow, unusual construction zones, glare, and confusing human behavior remain difficult because they are rare, messy, and hard to represent perfectly in training data.
Key Facts
- Lidar measures distance using time of flight: distance = speed of light × time / 2.
- Radar can estimate relative speed using the Doppler effect: faster closing objects shift the reflected signal more.
- Sensor fusion combines camera, lidar, radar, and map data to reduce uncertainty and improve reliability.
- Semantic segmentation assigns a class label to each image pixel, such as road, car, sidewalk, sign, or pedestrian.
- A common driving pipeline is perception → prediction → planning → control.
- Reaction distance can be estimated by d = vt, where v is speed and t is system reaction time.
Vocabulary
- Lidar
- Lidar is a sensor that uses laser pulses to measure distances and build a 3D point cloud of the surroundings.
- Radar
- Radar is a sensor that uses radio waves to detect objects and estimate their distance and relative speed.
- Sensor fusion
- Sensor fusion is the process of combining data from multiple sensors to create a more accurate view of the environment.
- Semantic segmentation
- Semantic segmentation is an AI vision task that labels every pixel in an image with a category such as road, vehicle, or pedestrian.
- Edge case
- An edge case is an unusual or rare situation that can confuse a system because it was not common in its training or testing data.
Common Mistakes to Avoid
- Thinking one sensor is enough, which is wrong because each sensor has weaknesses such as cameras struggling in glare and lidar struggling in heavy snow.
- Confusing detection with prediction, which is wrong because detecting a pedestrian only finds where they are now while prediction estimates where they may move next.
- Assuming more data always means better driving, which is wrong because noisy, delayed, or poorly aligned data can make the world model less accurate.
- Ignoring rare road situations, which is wrong because construction zones, unusual signs, and extreme weather can cause failures even if normal driving works well.
Practice Questions
- 1 A self-driving car is moving at 20 m/s and its perception and planning system takes 0.25 s to react. Using d = vt, how far does the car travel before it begins its response?
- 2 A lidar pulse returns to the sensor after 100 ns. Using distance = speed of light × time / 2 and speed of light = 3.0 × 10^8 m/s, how far away is the object?
- 3 A camera sees lane markings clearly, but radar detects a slow object ahead in thick fog while lidar returns are noisy. Explain why sensor fusion is safer than trusting only the camera in this situation.