Skip to content
Snippets Groups Projects
Commit f5f9fd1b authored by aurelien.bellet's avatar aurelien.bellet
Browse files

move metropolis to Sec 2

parent c1840025
No related branches found
No related tags found
No related merge requests found
......@@ -66,19 +66,6 @@ edge with all other cliques, see Figure~\ref{fig:d-cliques-figure} for the
corresponding D-Cliques network in the case of $n=100$ nodes and $c=10$
classes. We will explore sparser inter-clique topologies in Section~\ref{section:interclique-topologies}.
The mixing matrix $W$ required by D-SGD is obtained from standard
Metropolis-Hasting weights~\cite{xiao2004fast} computed from the above
topology, namely:
\begin{equation}
W_{ij} = \begin{cases}
\frac{1}{\max(\text{degree}(i), \text{degree}(j)) + 1} & \text{if}~i \neq
j \text{ and } \{i,j\}\in E,\\
1 - \sum_{j \neq i} W_{ij} & \text{if}~$i = j$, \\
0 & \text{otherwise}.
\end{cases}
\label{eq:metro}
\end{equation}
We construct D-Cliques by initializing cliques at random, using at most $M$
nodes to limit the intra-clique communication costs, then we
swap nodes between pairs of cliques chosen at random such that the swap
......
% !TEX root = main.tex
\section{Problem Statement}
\section{Problem Setting}
\label{section:problem}
......@@ -45,6 +45,19 @@ of Problem
must be doubly
stochastic ($\sum_{j \in N} W_{ij} = 1$ and $\sum_{j \in N} W_{ji} = 1$) and
symmetric, i.e. $W_{ij} = W_{ji}$~\cite{lian2017d-psgd}.
In our experiments, $W$ is given by
standard
Metropolis-Hasting weights~\cite{xiao2004fast} computed from the network
topology $G$, namely:\todo{AB: if we need space we can remove this equation}
\begin{equation}
W_{ij} = \begin{cases}
\frac{1}{\max(\text{degree}(i), \text{degree}(j)) + 1} & \text{if}~i \neq
j \text{ and } \{i,j\}\in E,\\
1 - \sum_{j \neq i} W_{ij} & \text{if } i = j, \\
0 & \text{otherwise}.
\end{cases}
\label{eq:metro}
\end{equation}
\begin{algorithm}[t]
\caption{D-SGD, Node $i$}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment