A stereo depth camera lets a robot estimate how far away objects are by using two lenses separated by a known distance. This is similar to human depth perception, where each eye sees a slightly different image. The difference between the two views is called disparity, and it becomes a measurable clue for distance.
Stereo depth matters because robots need reliable 3D information to avoid obstacles, grasp objects, and plan safe paths.
The camera system first finds matching points in the left and right images, then measures how far those points shift horizontally. A large shift means the object is close, while a small shift means the object is farther away. If the focal length, baseline, and disparity are known, depth can be estimated with z = fB / d.
The result is often displayed as a dense depth map, where each pixel stores an estimated distance used for navigation and mapping.
Understanding Robotics: Stereo Depth Camera
Before a stereo camera can measure well, its two cameras must be calibrated. Calibration finds each lens position, its focal length, and the small distortions caused by real glass lenses. Straight edges near the edge of an uncorrected image can appear bent.
Software corrects this distortion, then rectifies the image pair. Rectification rotates and shifts the views so that a point in one image should appear on the same horizontal row in the other. This greatly reduces the search problem.
Instead of searching across the whole second image, the system searches along one line. Accurate calibration is essential because a tiny alignment error can create a false distance estimate.
Finding a true match is harder than it first appears. The software looks for useful visual patterns such as corners, edges, dots, or changes in brightness. A plain white wall has few features, so many locations can look identical.
Repeating patterns cause another problem. The stripes on a fence or rows of windows may be matched to the wrong stripe or window. Low light adds image noise, while motion can blur the images.
Good stereo systems use matching rules that compare small image patches and reject weak matches. They may report a confidence value, meaning how much the system trusts each distance result.
Some parts of a scene cannot be seen by both cameras. An object close to one camera can hide a background region from the other camera. This is called occlusion.
The hidden region has no correct partner in the second view, so its depth is uncertain. Shiny metal, clear plastic, water, and dark surfaces are difficult for another reason. Their appearance can change with viewing angle or lighting.
A stereo camera may produce holes, streaks, or incorrect patches in these areas. Robots often combine stereo data with wheel motion estimates, lidar, or an inertial sensor to make navigation safer when the camera is unsure.
Distance accuracy changes across the scene. A close object creates a larger image shift, so the system can usually measure it more precisely. Far away objects have only a tiny shift.
A difference of one pixel can then change the estimated distance by a large amount. This matters for robots moving quickly toward walls, people, or furniture. A robot may treat uncertain far measurements carefully until it gets closer.
Students can notice the same effect by holding a finger in front of their face and viewing it with one eye at a time. The finger appears to jump more than a distant object.
When learning stereo vision, pay attention to units, image resolution, camera alignment, and confidence. These details decide whether a depth map is useful for a real robot or only looks convincing on a screen.
Key Facts
- Baseline B is the horizontal distance between the left and right camera lenses.
- Disparity d is the horizontal pixel shift between matching features in the left and right images.
- Depth formula for rectified stereo cameras: z = fB / d.
- Larger disparity means smaller depth, so nearby objects show a bigger left to right shift.
- If disparity d = 0 or is very small, the object is extremely far away or difficult to measure accurately.
- Dense depth maps assign a distance value to many pixels, helping robots detect obstacles and plan motion.
Vocabulary
- Stereo vision
- Stereo vision is a depth sensing method that compares two images taken from slightly different viewpoints.
- Baseline
- The baseline is the fixed distance between the optical centers of the two camera lenses.
- Disparity
- Disparity is the difference in horizontal position of the same scene point in the left and right images.
- Focal length
- Focal length is a camera parameter that relates scene geometry to image size and is often measured in pixels for depth calculations.
- Depth map
- A depth map is an image-like grid in which each pixel value represents the estimated distance to a point in the scene.
Common Mistakes to Avoid
- Using dB instead of fB in the depth formula is wrong because depth depends on focal length, baseline, and disparity as z = fB / d.
- Assuming larger disparity means a farther object is wrong because disparity is inversely related to depth.
- Ignoring camera calibration is wrong because inaccurate focal length, lens alignment, or baseline values produce incorrect depth estimates.
- Matching random textureless regions is wrong because stereo matching needs recognizable features or patterns to identify the same point in both images.
Practice Questions
- 1 A stereo camera has focal length f = 800 pixels, baseline B = 0.12 m, and disparity d = 40 pixels. Calculate the depth z.
- 2 A robot sees the same object with disparity d = 24 pixels using a stereo camera with f = 600 pixels and B = 0.10 m. What is the estimated distance to the object?
- 3 Two objects appear in a stereo depth system. Object A has disparity 60 pixels and Object B has disparity 15 pixels. Explain which object is closer and why.