The aggregation in FL does weight averaging by assigning the weight to a client as (no of samples owned by a client i / total number of samples). However, the server doesn't know the number of samples held with the client, so we simply tend to up to normally averaging. Few custom aggregation rules who extend from FedAvg also rely on such weights (no of samples owned by a client i / total number of samples). So, how it is taken care of?
Hello, this is indeed a very critical question. But I don't think a simple averaging method can be used. Although it cannot be quantitatively described, qualitatively speaking, users with more data train better models and deserve higher weights.
Second, I think participants should share information such as the amount of local data with the server, as the amount of data is not a privacy concern. If you consider the problem of fake data volume, it makes no sense, because fake data volume is not good for global training. But to expand, the setting of weights is difficult to measure the effect, so we use a weighted average based on the amount of data, but in fact, I think factors such as data distribution should also affect the distribution of weights. There are relatively few studies in this area, probably because it is better to make some breakthroughs in other aspects instead of being entangled in the setting of weights.
At the current stage, federated learning mostly focuses on conceptual and architectural design. For practical use, asking for help from an organization that has actually deployed federated learning services may give you a more accurate answer.
Hope this helps!
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning