Parameter Sharing Reinforcement Learning for Modeling Multi-Agent Driving Behavior in Roundabout Scenarios

Konstantinidis F, Sackmann M, De Candido O, Hofmann U, Thielecke J, Utschick W (2021)

Publication Language: English

Publication Type: Conference contribution, Conference Contribution

Publication year: 2021

Publisher: Institute of Electrical and Electronics Engineers Inc.

Book Volume: 2021-September

Pages Range: 1974--1981

Conference Proceedings Title: IEEE International Intelligent Transportation Systems Conference (ITSC)

Event location: Hybrid / Indianapolis US

ISBN: 9781728191423

URI: https://ieeexplore.ieee.org/abstract/document/9565031

DOI: 10.1109/ITSC48978.2021.9565031

Abstract

Modeling other drivers' behavior in highly interactive traffic situations, such as roundabouts, is a challenging task. We address this task using a Multi-Agent Reinforcement Learning (MARL) approach that learns a driving policy based on a minimal set of assumptions: drivers want to move forward and avoid collisions while maintaining low accelerations. Each agent's actions depend only on his observation of the local environment; no explicit communication between agents is possible. In order to teach the agents to safely interact with each other, and for example, respect right-of-way rules, we use parameter sharing: During training all vehicles are controlled by the same policy and the aggregated experiences are used to improve the policy. Moreover, parameter sharing enables us to use the efficient Soft Actor Critic (SAC) algorithm for training. The approach is evaluated in a roundabout setting with different traffic densities. Furthermore, the ability of the model to generalize is assessed in an untrained roundabout. In both settings, success rates above 97 \% demonstrate that a safe and transferable policy is learned.

Authors with CRIS profile

Moritz Sackmann Professur für Informationstechnik (Schwerpunkt Ortsbestimmung und Navigation) Jörn Thielecke Professur für Informationstechnik (Schwerpunkt Ortsbestimmung und Navigation)

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) Audi AG DE Germany (DE)

How to cite

APA:

Konstantinidis, F., Sackmann, M., De Candido, O., Hofmann, U., Thielecke, J., & Utschick, W. (2021). Parameter Sharing Reinforcement Learning for Modeling Multi-Agent Driving Behavior in Roundabout Scenarios. In IEEE International Intelligent Transportation Systems Conference (ITSC) (pp. 1974--1981). Hybrid / Indianapolis, US: Institute of Electrical and Electronics Engineers Inc..

MLA:

Konstantinidis, Fabian, et al. "Parameter Sharing Reinforcement Learning for Modeling Multi-Agent Driving Behavior in Roundabout Scenarios." Proceedings of the IEEE International Intelligent Transportation Systems Conference (ITSC), Hybrid / Indianapolis Institute of Electrical and Electronics Engineers Inc., 2021. 1974--1981.

BibTeX: Download