Zhang, Weilin and Toor, Salman and Al-Naday, Mays (2023) Federated Machine Learning for Resource Allocation in Multi-domain Fog Ecosystems. In: IEEE International Conference on Cloud Networking (Cloudnet) 2023, 2023-11-01 - 2023-11-03, New York Area, USA. (In Press)
Zhang, Weilin and Toor, Salman and Al-Naday, Mays (2023) Federated Machine Learning for Resource Allocation in Multi-domain Fog Ecosystems. In: IEEE International Conference on Cloud Networking (Cloudnet) 2023, 2023-11-01 - 2023-11-03, New York Area, USA. (In Press)
Zhang, Weilin and Toor, Salman and Al-Naday, Mays (2023) Federated Machine Learning for Resource Allocation in Multi-domain Fog Ecosystems. In: IEEE International Conference on Cloud Networking (Cloudnet) 2023, 2023-11-01 - 2023-11-03, New York Area, USA. (In Press)
Abstract
The proliferation of the Internet of Things (IoT) has incentivised extending cloud resources to the edge in what is deemed fog computing. The latter is manifesting as an ecosystem of connected clouds, geo-dispersed and of diverse capacities. In such ecosystem, workload allocation to fog services becomes a non-trivial challenge. Users' demand at the edge is diverse, which does not lend to straightforward resource planning. Conversely, running services at the edge may leverage proximity, but it comes at higher operational cost let alone increasing risk of resource straining. Consequently, there is a need for intelligent yet scalable allocation solutions that counter the adversity of demand, while efficiently distributing load between the edge and farther clouds. Machine learning is increasingly adopted in resource planning. This paper proposes a federated deep reinforcement learning system, based on deep Q-learning network (DQN), for workload distribution in a fog ecosystem. The proposed solution adapts a DQN to optimize local workload allocations, made by single gateways. Federated learning is incorporated to allow multiple gateways in a network to collaboratively build knowledge of users' demand. This is leveraged to establish consensus on the fraction of workload allocated to different fog nodes, using lower data supply and computation resources. System performance is evaluated using realistic demand from Google Cluster Workload Traces 2019. Evaluation results show over 50% reduction in failed allocations when spreading users over larger number of gateways, given fixed number of fog nodes. The results further illustrate the trade-offs between performance and cost under different conditions.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Additional Information: | Published proceedings: _not provided_ |
Uncontrolled Keywords: | Workload Allocation; Federated Learning; Deep Q-network; Fog networks; Federated Average Aggregation |
Divisions: | Faculty of Science and Health Faculty of Science and Health > Computer Science and Electronic Engineering, School of |
SWORD Depositor: | Unnamed user with email elements@essex.ac.uk |
Depositing User: | Unnamed user with email elements@essex.ac.uk |
Date Deposited: | 08 Feb 2024 11:12 |
Last Modified: | 07 Aug 2024 15:10 |
URI: | http://repository.essex.ac.uk/id/eprint/37775 |
Available files
Filename: Federated_Machine_Learning_for_Resource_Allocation_in_Multi-domain_Fog_Ecosystems.pdf