Skip to content
Snippets Groups Projects
thesis.tex 8.11 KiB
Newer Older
Guilhem Niot's avatar
Guilhem Niot committed
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% EPFL report package, main thesis file
% Goal: provide formatting for theses and project reports
% Author: Mathias Payer <mathias.payer@epfl.ch>
%
% This work may be distributed and/or modified under the
% conditions of the LaTeX Project Public License, either version 1.3
% of this license or (at your option) any later version.
% The latest version of this license is in
%   http://www.latex-project.org/lppl.txt
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[a4paper,11pt,oneside]{report}
% Options: MScThesis, BScThesis, MScProject, BScProject
\usepackage[BScProject,lablogo]{EPFLreport}
\usepackage{xspace}
\usepackage{listings}
\usepackage{caption}
\usepackage{subcaption}

\renewcommand{\lstlistingname}{Configuration}
\lstset{
  basicstyle=\footnotesize\ttfamily,
  columns=flexible,
  literate={-}{-}1,
  breaklines=true,
}
\lstset{captionpos=b}
\AtBeginDocument{\def\chapterautorefname{Chapter}}
\AtBeginDocument{\def\sectionautorefname{Section}}%
\AtBeginDocument{\def\subsectionautorefname{Subsection}}%
%

Guilhem Niot's avatar
Guilhem Niot committed
\title{Smallworld: Adding Dat support}
Guilhem Niot's avatar
Guilhem Niot committed
\author{Paulette Vazquez and Guilhem Niot}
\supervisor{Dr. Erick Lavoie}
\adviser{Professor Anne-Marie Kermarrec}

\newcommand{\sysname}{FooSystem\xspace}

\begin{document}
\maketitle

Guilhem Niot's avatar
Guilhem Niot committed
% \maketoc
Guilhem Niot's avatar
Guilhem Niot committed
\section{Another application: Dat Protocol}
Guilhem Niot's avatar
Guilhem Niot committed
% TODO: add ref + why is it different from SSB
% both built on append only logs, but Dat more active, and mature libraries
% TODO: high level motivation

The Dat Protocol [ref] is a peer-to-peer protocol that allows sharing files in a distributed network.
Guilhem Niot's avatar
Guilhem Niot committed

It differs from BitTorrent in that contents can be updated by their author.


Guilhem Niot's avatar
Guilhem Niot committed
\subsection{Installation}

The installation of Dat is quite straightforward. The dat-cli command utility allows to simply interact with Dat files. The commands in Figure \ref{cmd:dat_install} are used to install dat-cli.
% TODO: what we've achieved in the report: brings a service that is similar to something like Dropbox, with different
% TODO: try replicating remotely
% TODO: explain what's dat-cli, why we use it
% TODO: Remove dat-cli
Guilhem Niot's avatar
Guilhem Niot committed


It must be installed on both the emitter and the receiver devices.

\begin{figure}
  \begin{lstlisting}[label=cmd:dat_install, caption=Installing dat-cli]
    wget -qO- https://raw.githubusercontent.com/datproject/dat/master/download.sh | bash
    
    # or
Guilhem Niot's avatar
Guilhem Niot committed
    npm install -g dat-cli
Guilhem Niot's avatar
Guilhem Niot committed
  \end{lstlisting}
\end{figure}

Guilhem Niot's avatar
Guilhem Niot committed
\subsection{Using dat}
Guilhem Niot's avatar
Guilhem Niot committed
% It is possible to create a shared folder using \emph{dat share} in the folder to share. This command will give a dat url starting with \emph{dat://} that can be used to retrieve the files on other clients.
Guilhem Niot's avatar
Guilhem Niot committed
% A set of useful commands:
% \begin{itemize}
%   \item \emph{dat share}, to share a folder (starts a daemon that waits for incoming connexions)
%   \item \emph{dat sync}, tries to retrieve newer version of the files, and share the files for incoming connexions
%   \item \emph{dat clone <url> <folder>}, retrieves files from a dat url
%   \item \emph{dat pull}, tries to retrieve a newer version of the files
% \end{itemize}
Guilhem Niot's avatar
Guilhem Niot committed
\subsubsection{Real time synchronization with dat-store}
% TODO: what's dat-store: reverse the order of the following
A tool to do just that is \emph{dat-store}. It provides a service that can be run in background, can be remotely controlled, and does all the work for you.
Guilhem Niot's avatar
Guilhem Niot committed
On the opposite, Dat does not directly provide a daemon to run in background and automatically synchronize folders for you. \\
Guilhem Niot's avatar
Guilhem Niot committed


It should be installed on both the client and the Raspberry Pi using 
\begin{lstlisting}
  npm install -g dat-store
\end{lstlisting}

Guilhem Niot's avatar
Guilhem Niot committed
\textbf{Note:} In case you have permission errors, you may need to fix the permissions of your NPM installations. See the \href{https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally#manually-change-npms-default-directory}{manual guide}. Using sudo instead won't fix the problem.  \\
Guilhem Niot's avatar
Guilhem Niot committed
You should then configure a systemd service to run the store in background. You have to configure both your device and the Raspberry Pi. For the user device you can follow Configuration \ref{cmd:dat_store_systemd_user}. For the Raspberry Pi you can follow Configuration \ref{cmd:dat_store_systemd_rasp}.
Guilhem Niot's avatar
Guilhem Niot committed
\begin{figure}
  \begin{lstlisting}[label=cmd:dat_store_systemd_user, caption=Configure dat-store systemd service on the user device]
    
# This will create the service file.
sudo cat << EOF | sudo tee /etc/systemd/system/dat-store.service > /dev/null
[Unit]
Description=Dat storage provider, keeps dats alive in the background.

[Service]
Type=simple
# Check that dat-store is present at this location
# If it's not, replace the path with its location
ExecStart=$(which dat-store) run-service
Restart=always

[Install]
WantedBy=multi-user.target
EOF

sudo chmod 644 /etc/systemd/system/dat-store.service

sudo systemctl daemon-reload
sudo systemctl enable dat-store
sudo systemctl start dat-store

sudo systemctl status dat-store
  \end{lstlisting}
\end{figure}

\begin{figure}
  \begin{lstlisting}[label=cmd:dat_store_systemd_rasp, caption=Configure dat-store systemd service on the raspberry pi]
    
# This will create the service file.
sudo cat << EOF | sudo tee /etc/systemd/system/dat-store.service > /dev/null
[Unit]
Description=Dat storage provider exposed to the internet (for the raspberry pi).

[Service]
Type=simple
# Check that dat-store is present at this location
# If it's not, replace the path with its location
ExecStart=$(which dat-store) run-service --expose-to-internet
Restart=always

[Install]
WantedBy=multi-user.target
EOF

sudo chmod 644 /etc/systemd/system/dat-store.service

sudo systemctl daemon-reload
sudo systemctl enable dat-store
sudo systemctl start dat-store

sudo systemctl status dat-store
  \end{lstlisting}
\end{figure}

Guilhem Niot's avatar
Guilhem Niot committed
% TODO: explain during the introduction of dat-store
Dat-store introduces the concept of provider. Providers are instances of dat-store running either locally or remotely. The default one is your local instance, but you can also configure other providers, like the Raspberry Pi. \\
Guilhem Niot's avatar
Guilhem Niot committed
This notion is particularly useful for remote control of other providers. \\

Guilhem Niot's avatar
Guilhem Niot committed
%TODO: add reference
Dat-store provides interesting commands that we will use in our demo: [reference]
Guilhem Niot's avatar
Guilhem Niot committed
\begin{itemize}
  \item \emph{dat-store add <url|path> [provider]}: Adds a folder or a dat url to the dat-store of the specified provider.
  \item \emph{dat-store set-provider <url> [provider]}: Sets the url of the specified provider.
  \item \emph{dat-store list [provider]}: Retrieves the list of available Dats in the specified provider.
  \item \emph{dat-store clone <path> <url> [provider]}: Clones \emph{<url>} into a local folder.
\end{itemize}
Guilhem Niot's avatar
Guilhem Niot committed
% TODO: Remove the chapter numbers
% TODO: move to introduction,, this report will describe how to use Dat as if it was a dropbox like service and we will cover the case where we replicate between A and B where the rasp is an intermediary.
Guilhem Niot's avatar
Guilhem Niot committed

We can then imagine the case where we have two clients A and B and one Raspberry Pi used as a permanent store to synchronize a folder from A to B.

First, A executes the following commands:
\begin{lstlisting}[label=cmd:dat_store_example_A, caption=Commands executed by client A]

  dat-store set-provider http://raspberrypi.local:3472 raspberry
Guilhem Niot's avatar
Guilhem Niot committed
  mkdir mydat
  dat-store add ./mydat
Guilhem Niot's avatar
Guilhem Niot committed
  dat-store list
  dat-store add <url from previous command corresponding to ./mydat> raspberry
  # You can check that the content is actually replicated by accessing
  # http://raspberrypi.local/gateway/<token from hyper url corresponding to ./mydat>/
Guilhem Niot's avatar
Guilhem Niot committed
\end{lstlisting}

Then, client B executes:
\begin{lstlisting}[label=cmd:dat_store_example_A, caption=Commands executed by client B]
  dat-store set-provider http://raspberrypi.local:3472 raspberry
Guilhem Niot's avatar
Guilhem Niot committed
  dat-store list raspberry
  dat-store clone ./mydat <url obtained from the previous command>
\end{lstlisting}


\cleardoublepage
\phantomsection
\addcontentsline{toc}{chapter}{Bibliography}
\printbibliography

\end{document}
Guilhem Niot's avatar
Guilhem Niot committed

% TODO: talk about pushpin + reference and what we've observed: that it didn't sync well + npm issue on rasp

% TODO: need for knowing when the replication is done, state + motivate the problem, sketch the solutions