Addition of Choco sharing

This merge request adds implementation of ChocoSGD¹ in form of a sharing module decentralizepy.sharing.Choco. As a compression of choice the implementation uses topk sparsification over all elements of a model.

This sharing module adds two new parameters:

• step_size - Step size parameter from the formulation of ChocoSGD in the original paper¹
• alpha - Percentage of model parameters to keep during topk sparsification of a model

---

¹ Koloskova, Anastasia, Sebastian Stich, and Martin Jaggi. "Decentralized stochastic optimization and gossip algorithms with compressed communication." International Conference on Machine Learning. PMLR, 2019.

changed the description

Added via f655967c.

closed