We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AI

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: Offline Reinforcement Learning for Road Traffic Control

Abstract: Traffic signal control is an important problem in urban mobility with a significant potential of economic and environmental impact. While there is a growing interest in Reinforcement Learning (RL) for traffic signal control, the work so far has focussed on learning through simulations which could lead to inaccuracies due to simplifying assumptions. Instead, real experience data on traffic is available and could be exploited at minimal costs. Recent progress in {\em offline} or {\em batch} RL has enabled just that. Model-based offline RL methods, in particular, have been shown to generalize from the experience data much better than others.
We build a model-based learning framework which infers a Markov Decision Process (MDP) from a dataset collected using a cyclic traffic signal control policy that is both commonplace and easy to gather. The MDP is built with pessimistic costs to manage out-of-distribution scenarios using an adaptive shaping of rewards which is shown to provide better regularization compared to the prior related work in addition to being PAC-optimal. Our model is evaluated on a complex signalized roundabout showing that it is possible to build highly performant traffic control policies in a data efficient manner.
Comments: 24 pages
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
ACM classes: I.2.1
Cite as: arXiv:2201.02381 [cs.AI]
  (or arXiv:2201.02381v2 [cs.AI] for this version)

Submission history

From: Mayuresh Kunjir [view email]
[v1] Fri, 7 Jan 2022 09:55:21 GMT (467kb,D)
[v2] Mon, 11 Jul 2022 08:22:00 GMT (2272kb,D)

Link back to: arXiv, form interface, contact.