Replay Buffer

A replay buffer (or experience replay memory) is a dataset of past (state, action, reward, next state, done) transitions collected by an RL agent during environment interaction. At each training step, random mini-batches are sampled from the buffer to train the value function or policy, breaking temporal correlations that would destabilize gradient updates. In offline RL and robot learning, the replay buffer is replaced by a fixed dataset of human demonstrations or previously collected rollouts. Prioritized experience replay weights sampling by temporal-difference error to focus training on informative transitions.