Visual Servoing

Visual servoing uses camera feedback in a closed-loop controller to guide a robot toward a goal defined in image space (Image-Based Visual Servoing, IBVS) or 3D space estimated from images (Position-Based Visual Servoing, PBVS). In IBVS, the controller minimizes the error between detected image features (keypoints, object bounding boxes) and their desired positions in the image plane, without explicitly computing 3D poses. Visual servoing is attractive because it directly compensates for calibration errors and camera-robot misalignment. Modern deep learning variants train neural networks to output servoing velocity commands directly from raw images, enabling robust alignment to novel objects.