Specification-Guided Reinforcement Learning

Abstract

This tutorial explores specification-guided reinforcement learning as an alternative to traditional reward-based approaches, where the design of effective reward functions can be tedious, error-prone, and may not capture complex objectives. We introduce formal logical specifications as a more intuitive and precise way to define agent behavior, focusing on the theoretical guarantees and algorithmic aspects of learning from specifications. We examine both fundamental limitations in infinite-horizon settings and practical approaches for finite-horizon specifications.