Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • D decentralizepy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 5
    • Issues 5
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 1
    • Merge requests 1
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar

Gitlab upgrade to latest version (15.10.1) is complete. Have a nice day!

  • SaCS
  • decentralizepy
  • Merge requests
  • !1

using uid instead of rank for the femnist data partition

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Jeffrey Wigger requested to merge wigger/decentralizepy:femnist_uid_change into main Feb 17, 2022
  • Overview 0
  • Commits 2
  • Pipelines 0
  • Changes 17

For the femnist dataset the data gets partitioned on every machine based on the rank of the process [0, procs_per_machine]. If the data is not already pre-partitioned then all machines will use the entire dataset, i.e., a sample is used machines times.

This PR changes the Femnist dataset to use the uid for partitioning as it is done in for Celeba.

TODO:
[x] Either change all femnist config files such that n_procs now equals procs_per_machine * machines, or set n_procs in Dataset.py to mapping.n_machines * mapping.procs_per_machine

Update:
Removed the n_procs config option. Both for the Femnist and Celeba datasets the data is now split between the number of global processes.

Edited Feb 17, 2022 by Jeffrey Wigger
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: femnist_uid_change