学术报告：Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors

题目：Communication-Efficient Network-Distributed Optimization with Differential-Coded Compressors

主讲人：刘嘉（教授、美国爱荷华州立大学）

日期：2020年1月7日（星期二）

时间：上午10:30 - 11:30

地点：数据科学与ok138cn太阳集团 A201

主持：陈旭 教授

摘要：Network-distributed optimization has attracted significant attention in recent years due to its ever-increasing applications. However, the classic decentralized gradient descent (DGD) algorithm is communication-inefficient for large-scale and high-dimensional network-distributed optimization problems. To address this challenge, many compressed DGD-based algorithms have been proposed. However, most of the existing works have high complexity and assume compressors with bounded noise power.

To overcome these limitations, in this paper, we propose a new differential-coded compressed DGD (DC-DGD) algorithm. The key features of DC-DGD include: i) DC-DGD works with general SNR-constrained compressors, relaxing the bounded noise power assumption; ii) The differential-coded design entails the same convergence rate as the original DGD algorithm; and iii) DC-DGD has the same low-complexity structure as the original DGD due to a self-noise-reduction effect. Moreover, the above features inspire us to develop a hybrid compression scheme that offers a systematic mechanism to minimize the communication cost. Finally, we conduct extensive experiments to verify the efficacy of the proposed DC-DGD and hybrid compressor.

个人介绍：Jia Liu is currently an Assistant Professor in the Dept. of Computer Science and Dept. of Electrical and Computer Engineering (by courtesy) at Iowa State University, where he joined in Aug. 2017. He received his Ph.D. degree from the Bradley Dept. of Electrical and Computer Engineering at Virginia Tech in 2010. He was a Postdoctoral Researcher and subsequently a Research Assistant Professor from Feb. 2010 to Jul. 2017, both in the Dept. of Electrical and Computer Engineering at The Ohio State University. His research areas include theoretical foundations of control and optimization for stochastic networked systems, distributed algorithms design, optimization of cyber-physical systems, data analytics infrastructure, and machine learning. Dr. Liu is a senior member of IEEE and a member of ACM. His work has received numerous awards at top venues, including IEEE INFOCOM'19 Best Paper Award, IEEE INFOCOM'16 Best Paper Award, IEEE INFOCOM'13 Best Paper Runner-up Award, IEEE INFOCOM'11 Best Paper Runner-up Award, and IEEE ICC'08 Best Paper Award. He is a recipient of Bell Labs President Gold Award in 2001. His research has been supported by NSF, AFOSR, AFRL, and ONR.