Talk about pushpin and future work

thesis.tex

@@ -10,7 +10,7 @@
 %   http://www.latex-project.org/lppl.txt
 %
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-\documentclass[a4paper,11pt,oneside]{report}
+\documentclass[a4paper,11pt,oneside]{article}
 % Options: MScThesis, BScThesis, MScProject, BScProject
 \usepackage[BScProject,lablogo]{EPFLreport}
 \usepackage{xspace}
@@ -44,38 +44,39 @@
 % \maketoc
 
 
-\section{Another application: Dat Protocol}
+\section{Another application of Smallworld, the Dat Protocol}
 
-% TODO: add ref + why is it different from SSB
-% both built on append only logs, but Dat more active, and mature libraries
-% TODO: high level motivation
 
-The Dat Protocol [ref] is a peer-to-peer protocol that allows sharing files in a distributed network.
 
-It differs from BitTorrent in that contents can be updated by their author.
+The Dat Protocol\footnote{\url{https://www.datprotocol.com/}} is a peer-to-peer protocol that allows sharing files in a distributed network. It is based on append-only logs SSB but its community is more active, and it provides more mature libraries.
 
+As a distributed file sharing protocol, it differs from BitTorrent in that contents can be updated by their author.
 
-\subsection{Installation}
+This report aims at showing that Dat can be used with a Smallworld setup, to continuously replicate files among peers, similarly to a Dropbox instance. We will explain how to configure Dat to continously replicate files, and we will cover the case where we replicate files between A and B with the Raspberry Pi acting as an intermediary in case only one of the clients is online at a given time.
+% \subsection{Installation}
 
-The installation of Dat is quite straightforward. The dat-cli command utility allows to simply interact with Dat files. The commands in Figure \ref{cmd:dat_install} are used to install dat-cli.
-% TODO: what we've achieved in the report: brings a service that is similar to something like Dropbox, with different
-% TODO: try replicating remotely
-% TODO: explain what's dat-cli, why we use it
-% TODO: Remove dat-cli
+% The installation of Dat is quite straightforward. The dat-cli command utility allows to simply interact with Dat files. The commands in Figure \ref{cmd:dat_install} are used to install dat-cli.
+% % TODO: try replicating remotely
 
 
-It must be installed on both the emitter and the receiver devices.
+% It must be installed on both the emitter and the receiver devices.
 
-\begin{figure}
-  \begin{lstlisting}[label=cmd:dat_install, caption=Installing dat-cli]
-    wget -qO- https://raw.githubusercontent.com/datproject/dat/master/download.sh | bash
+% \begin{figure}
+%   \begin{lstlisting}[label=cmd:dat_install, caption=Installing dat-cli]
+%     wget -qO- https://raw.githubusercontent.com/datproject/dat/master/download.sh | bash
     
-    # or
-    npm install -g dat-cli
-  \end{lstlisting}
-\end{figure}
+%     # or
+%     npm install -g dat-cli
+%   \end{lstlisting}
+% \end{figure}
 
-\subsection{Using dat}
+\subsection{Real time synchronization with dat-store}
+
+While Dat provides a command tool called \emph{dat-cli}, it is not able to run in background, and cannot be remotely controlled and is thus not very practical for this application. \\
+A more suited tool is \emph{dat-store}. It provides a service that can be run in background, can be remotely controlled, and does all the work for you.
+
+Dat-store introduces the concept of provider. Providers are instances of dat-store running either locally or remotely. The default one is your local instance, but you can also configure other providers, like the Raspberry Pi. \\
+This notion is particularly useful for remote control of other providers. \\
 
 % It is possible to create a shared folder using \emph{dat share} in the folder to share. This command will give a dat url starting with \emph{dat://} that can be used to retrieve the files on other clients.
 
@@ -87,14 +88,14 @@ It must be installed on both the emitter and the receiver devices.
 %   \item \emph{dat pull}, tries to retrieve a newer version of the files
 % \end{itemize}
 
-\subsubsection{Real time synchronization with dat-store}
+% \subsubsection{Real time synchronization with dat-store}
 % TODO: what's dat-store: reverse the order of the following
-A tool to do just that is \emph{dat-store}. It provides a service that can be run in background, can be remotely controlled, and does all the work for you.
+% A tool to do just that is . 
 
-On the opposite, Dat does not directly provide a daemon to run in background and automatically synchronize folders for you. \\
+% On the opposite, Dat does not directly provide a daemon to run in background and automatically synchronize folders for you. \\
 
 
-It should be installed on both the client and the Raspberry Pi using 
+Dat-store should be installed on both the client and the Raspberry Pi using 
 \begin{lstlisting}
   npm install -g dat-store
 \end{lstlisting}
@@ -161,20 +162,15 @@ sudo systemctl status dat-store
   \end{lstlisting}
 \end{figure}
 
-% TODO: explain during the introduction of dat-store
-Dat-store introduces the concept of provider. Providers are instances of dat-store running either locally or remotely. The default one is your local instance, but you can also configure other providers, like the Raspberry Pi. \\
-This notion is particularly useful for remote control of other providers. \\
-
-%TODO: add reference
-Dat-store provides interesting commands that we will use in our demo: [reference]
+Dat-store provides interesting commands that we will use in our demo\footnote{See the documentation at \url{https://github.com/datproject/dat-store}.}
 \begin{itemize}
   \item \emph{dat-store add <url|path> [provider]}: Adds a folder or a dat url to the dat-store of the specified provider.
   \item \emph{dat-store set-provider <url> [provider]}: Sets the url of the specified provider.
   \item \emph{dat-store list [provider]}: Retrieves the list of available Dats in the specified provider.
   \item \emph{dat-store clone <path> <url> [provider]}: Clones \emph{<url>} into a local folder.
 \end{itemize}
-% TODO: Remove the chapter numbers
-% TODO: move to introduction,, this report will describe how to use Dat as if it was a dropbox like service and we will cover the case where we replicate between A and B where the rasp is an intermediary.
+
+\subsection{Demonstration with 2 clients, and a Raspberry Pi}
 
 We can then imagine the case where we have two clients A and B and one Raspberry Pi used as a permanent store to synchronize a folder from A to B.
 
@@ -198,6 +194,19 @@ Then, client B executes:
   dat-store clone ./mydat <url obtained from the previous command>
 \end{lstlisting}
 
+\section{Beyond}
+
+We also experimented with Pushpin\footnote{\url{https://github.com/automerge/pushpin}}, which is a collaborative coarkboard app. It is based on hypermerge\footnote{\url{https://github.com/automerge/hypermerge}}, a library providing a JSON-like structure that can be edited concurrently, without worrying about conflicts. And hypermerge is itself based on the Dat protocol.
+
+However, while using it we suffered from a few instabilities, among them Electron crashes from time to time, and the synchronization stops until the window is reloaded. \\
+On top of that, we were unable to install Pushpin on the Raspberry Pi to have a replicating node as it requires compiling Node 14 and the Raspberry Pi ran out of memory while doing the compilation.
+
+\section{Possible improvements and future work}
+
+A common need we noticed between SSB and Dat is to provide a way to determine when two devices ended their synchronization. SSB provides a Node.js client\footnote{\url{https://github.com/ssbc/ssb-client}} that can be used to determine when new posts are received. A proof of concept is showcased at \url{https://github.com/GuilhemN/ssb-copy-follows/tree/POSTS}. This script could be adapted to make a LED blink for instance to notify the user that the devices have synchronized.
+
+Another possible improvement for SSB is the refactoring of the AutoFollow with \emph{ssb-client} to copy the follows of its owner. Similarly a proof of concept is available at \url{https://github.com/GuilhemN/ssb-copy-follows/blob/master/index.js}. It could be integrated to a web page exposed at \verb|http://raspberrypi.local/|.
+
 
 \cleardoublepage
 \phantomsection
@@ -206,6 +215,7 @@ Then, client B executes:
 
 \end{document}
 
+
 % TODO: talk about pushpin + reference and what we've observed: that it didn't sync well + npm issue on rasp
 
 % TODO: need for knowing when the replication is done, state + motivate the problem, sketch the solutions
\ No newline at end of file