Observation Space

The observation space defines all sensor inputs available to the robot policy at each timestep. Common modalities include RGB images from wrist or overhead cameras, depth maps from structured-light or stereo sensors, proprioceptive state (joint positions, velocities, torques), gripper state, end-effector pose, tactile readings, and task-specification inputs such as language embeddings or goal images. The observation space design profoundly affects policy performance and generalization: richer observations carry more information but increase model complexity, training time, and the risk of overfitting to irrelevant visual features.