News

Pre-Publication Notice: "Markov Decision Processes and Reinforcement Learning" (Puterman & Chan)

We're excited to announce the submission of a new book, "Markov Decision Processes and Reinforcement Learning" by Martin Puterman and Tim Chan (to Cambridge University Press for publication).  The book blends foundational theory with modern reinforcement learning insights.
 

Description

This book offers a comprehensive introduction to Markov decision process and reinforcement learning fundamentals using a common mathematical notation and language. Its goal is to provide a solid foundation that enables readers to engage meaningfully with these rapidly evolving fields. Topics covered include finite and infinite horizon models, partially observable models, value function approximation, and simulation-based methods. Rigorous mathematical concepts and algorithmic developments are supported by numerous worked examples.

As an up-to-date successor to Martin L. Puterman’s influential 1994 textbook, this volume assumes familiarity with probability, mathematical notation, and proof techniques.  It is ideally suited for students, researchers, and professionals in operations research, computer science, engineering, and economics.
 
 

📄 Preprints of the Preface, Chapters 1–6, and chapter summaries are now available on GitHub:
🔗 https://lnkd.in/g4vyHsXs