Federated learning was introduced by Google in 2016 in a paper titled Communication-Efficient Learning of Deep Networks from Decentralized Data. It’s a new machine learning paradigm that allows us to build machine learning models from private data, without sharing such data to a data center.
The summary of the steps we take to do this is as follows:
A generic model (i.e. neural network) is created at a server. The model will not be trained on the server but on the users’ devices (the majority are mobile devices).
The model is sent to the users’ devices where the training occurs. So the same model (i.e. neural network) is trained parallelly on different devices, according to their private data.
Just the trained model (i.e. parameters or gradients) is shared back to the server.
The server averages the trained parameters from all devices to update the generic model based on the federated averaging algorithm.