Transformer Architecture

A neural network architecture based on self-attention mechanisms, originally designed for NLP but now dominant across modalities. Transformers process sequences of tokens in parallel, computing attention between all pairs. In robotics, transformers are used for: policy networks (decision transformer, ACT), visual encoders (ViT), and VLA models that unify vision, language, and action.