Research Repository

Fast Q-learning for Improved Finite Length Performance of Irregular Repetition Slotted ALOHA

Nisioti, Eleni and Thomos, Nikolaos (2020) 'Fast Q-learning for Improved Finite Length Performance of Irregular Repetition Slotted ALOHA.' IEEE Transactions on Cognitive Communications and Networking, 6 (2). pp. 844-857. ISSN 2332-7731

[img]
Preview
Text
final version.pdf - Accepted Version

Download (699kB) | Preview

Abstract

In this paper, we study the problem of designing adaptive Medium Access Control (MAC) solutions for wireless sensor networks (WSNs) under the Irregular Repetition Slotted ALOHA (IRSA) protocol. In particular, we optimize the degree distribution employed by IRSA for finite frame sizes. Motivated by characteristics of WSNs, such as the restricted computational resources and partial observability, we model the design of IRSA as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). We have theoretically analyzed our solution in terms of optimality of the learned IRSA design and derived guarantees for finding near-optimal policies. These guarantees are generic and can be applied in resource allocation problems that exhibit the waterfall effect, which in our setting manifests itself as a severe degradation in the overall throughput of the network above a particular channel load. Furthermore, we combat the inherent non-stationarity of the learning environment in WSNs by advancing classical Q-learning through the use of virtual experience (VE), a technique that enables the update of multiple state-action pairs per learning iteration and, thus, accelerates convergence. Our simulations confirm the superiority of our learning-based MAC solution compared to traditional IRSA and provide insights into the effect of WSN characteristics on the quality of learned policies.

Item Type: Article
Uncontrolled Keywords: Wireless sensor networks; Convergence; Sensors; Complexity theory; Media Access Protocol; Throughput; Observability; Medium access control; Q-learning; irregular repetition slotted ALOHA; POMDP; independent learning
Divisions: Faculty of Science and Health
Faculty of Science and Health > Computer Science and Electronic Engineering, School of
SWORD Depositor: Elements
Depositing User: Elements
Date Deposited: 09 Dec 2019 10:49
Last Modified: 15 Jan 2022 01:31
URI: http://repository.essex.ac.uk/id/eprint/26157

Actions (login required)

View Item View Item