Federated learning
None of these hypotheses are made for federated learning; instead, the datasets are typically heterogeneous and their sizes may span several orders of magnitude.
Moreover, the clients involved in federated learning may be unreliable as they are subject to more failures or drop out since they commonly rely on less powerful communication media (i.e. Wi-Fi) and battery-powered systems (i.e. smartphones and IoT devices) compared to distributed learning where nodes are typically datacenters that have powerful computational capabilities and are connected to one another with fast networks.
This setup prevents single point failures as the model updates are exchanged only between interconnected nodes without the orchestration of the central server.
An increasing number of application domains involve a large set of heterogeneous clients, e.g., mobile phones and IoT devices.
Recently, a new federated learning framework named HeteroFL was developed to address heterogeneous clients equipped with very different computation and communication capabilities.
[3][6] In most cases, the assumption of independent and identically distributed samples across local nodes does not hold for federated learning setups.
To further investigate the effects of non-IID data, the following description considers the main categories presented in the preprint by Peter Kairouz et al. from 2019.
[13] The way the statistical local outputs are pooled and the way the nodes communicate with each other can change from the centralized model explained in the previous section.
This leads to a variety of federated learning approaches: for instance no central orchestrating server, or stochastic communication.
Thus, it requires not only enough local computing power and memory, but also high bandwidth connections to be able to exchange parameters of the machine learning model.
Nevertheless, the devices typically employed in federated learning are communication-constrained, for example IoT devices or smartphones are generally connected to Wi-Fi networks, thus, even if the models are commonly less expensive to be transmitted compared to raw data, federated learning mechanisms may not be suitable in their general form.
Since the local losses are aligned, FedDyn is robust to the different heterogeneity levels and it can safely perform full minimization in each device.
Recently, Vahidian et al.[23] introduced Sub-FedAvg opening a new personalized FL algorithm paradigm by proposing Hybrid Pruning (structured + unstructured pruning) with averaging on the intersection of clients’ drawn subnetworks which simultaneously handles communication efficiency, resource constraints and personalized models accuracies.
[23] Sub-FedAvg is the first work which shows existence of personalized winning tickets for clients in federated learning through experiments.
This requires a matching regularizer constant that must be tuned based on user goals and results in disparate local and global models.
Before that, in a thesis work titled "A Framework for Multi-source Prefetching Through Adaptive Weight",[32] an approach to aggregate predictions from multiple models trained at three location of a request response cycle with was proposed.
In 2017 and 2018, publications have emphasized the development of resource allocation strategies, especially to reduce communication[20] requirements[33] between nodes with gossip algorithms[34] as well as on the characterization of the robustness to differential privacy attacks.
[35] Other research activities focus on the reduction of the bandwidth during training through sparsification and quantization methods,[33] where the machine learning models are sparsified and/or compressed before they are shared with other nodes.
Federated learning typically applies when individual actors need to train models on larger datasets than their own, but cannot afford to share the data in itself with others (e.g., for legal, strategic or economic reasons).
Due to the potential high number of self-driving cars and the need for them to quickly respond to real world situations, traditional cloud approach may generate safety risks.
To solve this problem, the ability to train machine learning models at scale across multiple medical institutions without moving the data is a critical technology.
Recently, a collaboration of 20 different institutions around the world validated the utility of training AI models using federated learning.
[46] A coalition from industry and academia has developed MedPerf,[47] an open source platform that enables validation of medical AI models in real world data.
[48] Robotics includes a wide range of applications of machine learning methods: from perception and decision-making to control.
In the paper,[49] mobile robots learned navigation over diverse environments using the FL-based method, helping generalization.
In the paper,[50] Federated Learning is applied to improve multi-robot navigation under limited communication bandwidth scenarios, which is a current challenge in real-world learning-based robotic tasks.
Federated Learning (FL) is transforming biometric recognition by enabling collaborative model training across distributed data sources while preserving privacy.
By eliminating the need to share sensitive biometric templates like fingerprints, facial images, and iris scans, FL addresses privacy concerns and regulatory constraints, allowing for improved model accuracy and generalizability.
It mitigates challenges of data fragmentation by leveraging scattered datasets, making it particularly effective for diverse biometric applications such as facial and iris recognition.
However, FL faces challenges, including model and data heterogeneity, computational overhead, and vulnerability to security threats like inference attacks.
