Zhang, Yuchao and Nie, Xiaohui and Jiang, Junchen and Wang, Wendong and Xu, Ke and Zhao, Youjian and Reed, Martin J and Chen, Kai and Wang, Haiyang and Yao, Guang (2021) BDS+: An Inter-Datacenter Data Replication System With Dynamic Bandwidth Separation. IEEE ACM Transactions on Networking, 29 (2). pp. 918-934. DOI https://doi.org/10.1109/tnet.2021.3054924
Zhang, Yuchao and Nie, Xiaohui and Jiang, Junchen and Wang, Wendong and Xu, Ke and Zhao, Youjian and Reed, Martin J and Chen, Kai and Wang, Haiyang and Yao, Guang (2021) BDS+: An Inter-Datacenter Data Replication System With Dynamic Bandwidth Separation. IEEE ACM Transactions on Networking, 29 (2). pp. 918-934. DOI https://doi.org/10.1109/tnet.2021.3054924
Zhang, Yuchao and Nie, Xiaohui and Jiang, Junchen and Wang, Wendong and Xu, Ke and Zhao, Youjian and Reed, Martin J and Chen, Kai and Wang, Haiyang and Yao, Guang (2021) BDS+: An Inter-Datacenter Data Replication System With Dynamic Bandwidth Separation. IEEE ACM Transactions on Networking, 29 (2). pp. 918-934. DOI https://doi.org/10.1109/tnet.2021.3054924
Abstract
Many important cloud services require replicating massive data from one datacenter (DC) to multiple DCs. While the performance of pair-wise inter-DC data transfers has been much improved, prior solutions are insufficient to optimize bulk-data multicast, as they fail to explore the rich inter-DC overlay paths that exist in geo-distributed DCs, as well as the remaining bandwidth reserved for online traffic under fixed bandwidth separation scheme. To take advantage of these opportunities, we present BDS+, a near-optimal network system for large-scale inter-DC data replication. BDS+ is an application-level multicast overlay network with a fully centralized architecture, allowing a central controller to maintain an up-to-date global view of data delivery status of intermediate servers, in order to fully utilize the available overlay paths. Furthermore, in each overlay path, it leverages dynamic bandwidth separation to make use of the remaining available bandwidth reserved for online traffic. By constantly estimating online traffic demand and rescheduling bulk-data transfers accordingly, BDS+ can further speed up the massive data multicast. Through a pilot deployment in one of the largest online service providers and large-scale real-trace simulations, we show that BDS+ can achieve 3-5 x speedup over the provider's existing system and several well-known overlay routing baselines of static bandwidth separation. Moreover, dynamic bandwidth separation can further reduce the completion time of bulk data transfers by 1.2 to 1.3 times.
Item Type: | Article |
---|---|
Uncontrolled Keywords: | Overlay Network; Data Replication; Centralized Control; Dynamic Bandwidth Separation |
Divisions: | Faculty of Science and Health Faculty of Science and Health > Computer Science and Electronic Engineering, School of |
SWORD Depositor: | Unnamed user with email elements@essex.ac.uk |
Depositing User: | Unnamed user with email elements@essex.ac.uk |
Date Deposited: | 24 Feb 2021 15:42 |
Last Modified: | 30 Oct 2024 16:35 |
URI: | http://repository.essex.ac.uk/id/eprint/29935 |
Available files
Filename: main-BDS.pdf