Asynchronous Federated Learning: convergence and performance in heterogeneous environments

Abstract

Distributed Machine Learning (DML) utilizes several nodes for training machine learning models, with a central node overseeing data distribution and communication. Federated Learning (FL), a Distributed Machine Learning (DML) branch, improves data privacy by retaining data on local devices. The realm of Federated Learning (FL) has primarily functioned synchronously; nevertheless, recent advancements have initiated a new phase of asynchronous FL. This innovative method enables nodes to update the model, independently facilitating exceptional scalability and adaptability. This study thoroughly examines asynchronous federated learning, analyzing its distributed architecture, communication protocols, optimization methods, and the numerous hurdles it faces, such as data heterogeneity, node delays, and convergence problems. The findings illustrate that, despite these challenges, the system attains near-centralized accuracy and exhibits accelerated convergence rates. This serves as a compelling demonstration of the potential of asynchronous federated learning in transforming actual applications.