Label-Aware Aggregation for Improved Federated Learning
Key: AK+23-2
Author: Ahmad Khalil, Aidmar Wainakh, Ephraim Zimmer, Javier Parra-Arnau, Antonio Fernández Anta, Ralf Steinmetz
Date: September 2023
Kind: In proceedings
Abstract: Federated Averaging (FedAvg) is the most common aggregation method used in Federated learning, which performs a weighted averaging of the updates based on the sizes of the individual datasets of each client. A raising discussion in the research community suggests that FedAvg might not be the optimal method since, for instance, it does not fully take into account the variety of the client data distributions. In this paper, we propose a label-aware aggregation method FedLA, that addresses the biased models issue by considering the variety of labels in the weighted averaging. It combines two main properties of the client data, namely data size and label distribution. Through extensive experiments, we demonstrate that FedLA is particularly effective in several heterogeneous data distribution scenarios. Especially when only a small group of the clients is participating in the Federated Learning process. Furthermore, we argue that accurately describing the data distribution is crucial in selecting the appropriate aggregation method. In this regard, we discuss various properties that can be used to describe data distribution and illustrate how these properties can guide the choice of an aggregation method for specific data distributions.
View Full paper (PDF) | Download Full paper (PDF)

The documents distributed by this server have been provided by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, not withstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.