Title: A Web of Things Architecture for Digital Twin Creation and Model-Based Reinforcement Control

Abstract: Internet of Things (IoT) devices are available in a multitude of scenarios, and provide constant, contextual data which can be leveraged to automatically reconfigure and optimize smart environments. To realize this vision, Artificial Intelligence (AI) and deep learning techniques are usually employed, however they need large quantity of data which is often not feasible in IoT scenarios. Digital Twins (DTs) have recently emerged as an effective way to replicate physical entities in the digital domain, to allow for simulation and testing of models and services. In this paper, we present a novel architecture based on the emerging Web of Things (WoT) standard, which provides a DT of a smart environment and applies Deep Reinforcement Learning (DRL) techniques on real time data. We implement our system in a real deployment, and test it along with a legacy system. Our findings show that the benefits of having a digital twin, specifically for DRL models, allow for faster convergence and finer tuning.