Click image to open full size
Autonomous Vehicles: How Self-Driving Systems Work
Cameras, Lidar, Perception, Planning, and Control
Related Tools
Related Labs
Related Worksheets
Related Cheat Sheets
Self-driving vehicles combine sensors, computers, and control systems to move through the world with limited or no human input. They matter because they could reduce crashes, improve mobility, and make transportation more efficient. To do this safely, the vehicle must constantly detect its surroundings, predict what other road users will do, and choose its own actions. Engineering these systems requires careful integration of hardware, software, and real-time decision making.
A self-driving car works by following a pipeline of sensing, perception, localization, planning, and control. Cameras, radar, lidar, GPS, and onboard maps provide data about lanes, cars, pedestrians, and road geometry. Software combines these inputs to estimate the car's position, build a model of the environment, and select a safe path. Control algorithms then convert that path into steering, throttle, and braking commands while continuously updating as conditions change.
Key Facts
- The main autonomy pipeline is sensing -> perception -> localization -> planning -> control.
- Speed is related to distance and time by v = d/t.
- Average acceleration is a = (v_f - v_i)/t.
- Stopping distance depends on reaction and braking, and braking distance grows roughly with v^2.
- Radar measures distance using d = cΔt/2, where c is wave speed and Δt is round-trip time.
- Control systems often minimize error, with tracking error e = desired position - actual position.
Vocabulary
- Perception
- Perception is the process of turning raw sensor data into useful information about lanes, objects, signs, and people.
- Localization
- Localization is the task of estimating the vehicle's exact position and orientation on a road or map.
- Lidar
- Lidar is a sensor that uses laser pulses to measure distances and build a 3D picture of the surroundings.
- Path planning
- Path planning is the process of choosing a safe and efficient route and immediate trajectory for the vehicle to follow.
- Control system
- A control system converts the planned path into steering, acceleration, and braking actions that keep the car on course.
Common Mistakes to Avoid
- Thinking one sensor is enough, which is wrong because cameras, radar, lidar, and maps each have different strengths and weaknesses in weather, lighting, and distance measurement.
- Assuming GPS alone tells the car exactly where it is, which is wrong because GPS can be inaccurate or blocked and must be combined with maps, wheel motion, and other sensors.
- Believing object detection is the same as safe driving, which is wrong because the car must also predict motion, plan around hazards, and control the vehicle smoothly.
- Ignoring system delay, which is wrong because sensing, computation, and braking all take time and can greatly affect stopping distance and safety.
Practice Questions
- 1 A self-driving car travels 90 m in 5.0 s on a straight road. What is its average speed in m/s?
- 2 A vehicle slows from 20 m/s to 8 m/s in 4.0 s. What is its average acceleration?
- 3 Explain why engineers use both cameras and radar on a self-driving car instead of relying on only one of them.