Start from clean GIT
This commit is contained in:
commit
e841f14a95
2112 changed files with 6638085 additions and 0 deletions
327
thesis/chapters/setup.tex
Normal file
327
thesis/chapters/setup.tex
Normal file
|
@ -0,0 +1,327 @@
|
|||
\chapter{Measurement Setup}
|
||||
\label{chap:setup}
|
||||
|
||||
This chapter describes the setup used for configuring, and controlling the
|
||||
experiments inside the lab. Furthermore,it is described how measurement data are
|
||||
collected, processed and stored. The general software architecture is shown in
|
||||
\autoref{fig:components}. The components are distributed across several
|
||||
different platforms.
|
||||
|
||||
Measurement data for multiple parameters were collected from the network. The
|
||||
variables and their data sources are listed in \autoref{tab:params}. The packet
|
||||
captures can be substituted with outputs on the serial output. For telling the
|
||||
energy consumption of an implementation, a \ac{PM} of one sample per second is
|
||||
sufficient.
|
||||
|
||||
\begin{table}[h]
|
||||
\centering
|
||||
\caption{Measured variables at each node}
|
||||
\begin{tabular}{ll}
|
||||
\toprule
|
||||
Variable & Source \\
|
||||
\midrule
|
||||
Network latency & PCAP, serial output \\
|
||||
Delivery ratio & PCAP, serial output \\
|
||||
Control overhead & PCAP, serial output \\
|
||||
Number of DIO, DAO & PCAP, serial output \\
|
||||
DODAG state & serial output \\
|
||||
Routing table & serial output \\
|
||||
Node reset times & HN event log \\
|
||||
RSSI & HN radio receiver \\
|
||||
Energy consumption & HN power monitor \\
|
||||
\end{tabular}
|
||||
\label{tab:params}
|
||||
\end{table}
|
||||
|
||||
When evaluating such a large set of variables from multiple different sources,
|
||||
the amount of generated data and if enough processing power and storage space is
|
||||
available has to carefully considered. \autoref{tab:amountdata} shows the
|
||||
estimated amounts of data generated for each variable. For 50 runs this merely
|
||||
amounts to 500 MByte of raw data, which can be further compressed using text
|
||||
compression, since they, especially in the case of the serial log, contain many
|
||||
repeating strings.
|
||||
|
||||
\begin{table}
|
||||
\centering
|
||||
\caption{Estimated amount of data generated by an experiment with 44 nodes,
|
||||
lasting 600 seconds}
|
||||
\begin{tabular}{lrr}
|
||||
\toprule
|
||||
Source & Single (bytes) & Total (KBytes)\\
|
||||
\midrule
|
||||
Serial output & 300 & 7920 \\
|
||||
Event log & 40 & 1056 \\
|
||||
Consumption log & 46 & 1214 \\
|
||||
RSSI log & 46 & 1214 \\
|
||||
\midrule
|
||||
Total & & 11404 \\
|
||||
\end{tabular}
|
||||
\label{tab:amountdata}
|
||||
\end{table}
|
||||
|
||||
\section{Testlab Nodes}
|
||||
|
||||
As previously mentioned in \autoref{sec:architecture}, each node inside the
|
||||
testlab is consists of the \ac{ON} that runs the firmware supplied by the user
|
||||
and the \ac{HN} that controls and monitors the \ac{ON}. In this section, the
|
||||
software running on the node and how data is collected from it is further
|
||||
described.
|
||||
|
||||
\subsection{Open Node}
|
||||
|
||||
Each \ac{ON} inside the test lab runs a version of the \emph{Contiki} operating
|
||||
system. The node at the root of the \ac{DAG} is programmed to act as a sink for
|
||||
the \ac{UDP} traffic that is periodically emitted the other nodes that act as
|
||||
sources. Each such packet contains a sequence number. The reception of these
|
||||
packets at the sink and the emission from the source are logged to the serial
|
||||
output of the node node. This makes it possible to detect whether a packet has
|
||||
been received when comparing the log of the sink and that of the respective
|
||||
sender.
|
||||
|
||||
Further entries inside the log on the serial output from each node include the
|
||||
number of emitted \ac{DIO} and \ac{DAO} messages, changes of the routing table and changes
|
||||
of the preferred parent.
|
||||
|
||||
Different versions of the firmware have been produced for the \ac{DAG} root and
|
||||
all other nodes in the network. The firmware of the root creates a new
|
||||
\ac{DAG} with the node itself at the root. All other nodes will attempt to join
|
||||
this \ac{DAG}.
|
||||
|
||||
\subsection{Porting the Firmware}
|
||||
|
||||
The firmware supplied to the \ac{ON} comes in different versions which each have
|
||||
different features enabled. The default configuration of \emph{Contiki} 2.7 with
|
||||
added device support for the \emph{M3} node is provided by \fitlab. The hardened
|
||||
implementation has been developed based on an older version of \emph{Contiki}, that
|
||||
also did not include device support for the \emph{M3} node. In preparation for
|
||||
the evaluation, the hardened implementation has been ported to this newer version
|
||||
of \emph{Contiki} and some changes and additions had to be made to take into
|
||||
account changed software interfaces and device specific differences.
|
||||
|
||||
Some build options specific to the \emph{Z1} where required for the hardened
|
||||
implementation to function. These options have been added as necessary.
|
||||
|
||||
A missing consistency check has been discovered when restoring the routing state
|
||||
from flash memory and has then been added that erases the state from the flash
|
||||
if an inconsistency is discovered.
|
||||
|
||||
When testing the hardened implementation, it came to attention that a restart of
|
||||
the \ac{DAG} root will result in it becoming unable to reconfigure as the
|
||||
\ac{DAG} root, leading to the network being unusable until the state is purged
|
||||
from the flash memory of the root node. This is caused by the root node
|
||||
restoring the previously recorded \ac{DIO} messages from its flash and replaying
|
||||
them onto the \ac{RPL} module before it configures itself as the root of a
|
||||
\ac{DAG}. This leads to it joining an existing \ac{DAG} discovered from the
|
||||
\ac{DIO} messages. This \ac{DAG} is the same \ac{DAG} as was the node previously
|
||||
the root of. The node joins this \ac{DAG} and selects one of its nodes as a
|
||||
preferred parent, setting its own rank to a rank larger than its parent. It is
|
||||
likely that the new parent node previously had selected the former root node as
|
||||
a parent. In this case, the new parent node of the former root node discovers
|
||||
that its own parent has a larger rank than In this case, the new parent node of
|
||||
the former root node discovers that its own parent has a larger rank than
|
||||
itself. This triggers the former root node to be dropped as the parent at this
|
||||
node. Thus the tree looses its root and becomes unusable but also unable to
|
||||
recover, since the root node believes to be already part of a \ac{DAG} and will
|
||||
not create a new \ac{DAG} with itself as the root node.
|
||||
|
||||
\subsection{Host Node}
|
||||
|
||||
The \ac{HN} collects the log output from the \ac{ON} and offers it to
|
||||
connecting clients of a \ac{TCP} network socket. The \ac{HN} also can record
|
||||
\acp{PCAP} in the direct vicinity of the \ac{ON} and forwards them in a similar
|
||||
manner to the serial output. Alternatively, the \ac{HN} can record the local
|
||||
\ac{RSSI}. For each event, including start, stop and reset of the attached
|
||||
\ac{ON}, a corresponding entry is appended to an event log. The \ac{HN} can also
|
||||
record accurate values for the power consumption, voltage and current from the
|
||||
power management unit of the \ac{ON}.
|
||||
|
||||
For the purpose of measuring the energy consumption of the node in
|
||||
reaction to a transient node failure, a high \ac{AM} can be selected since
|
||||
\ac{RPL} takes some time to react to the change in the network, so the state
|
||||
will persist for a multiple second. This way it is possible to reduce
|
||||
the noise component and acquire more accurate data.
|
||||
|
||||
The \ac{HN} stores both the consumption data and the \ac{RSSI} in the
|
||||
\ac{OML}. These log files are stored locally and periodically collected and
|
||||
stored by the site server.
|
||||
|
||||
\section{Site Server}
|
||||
|
||||
At each site, a shared server is provided running a multi-user system
|
||||
(\emph{Linux}). The server periodically queries the events, \ac{RSSI} and
|
||||
consumption of each \ac{HN} and for each node in the form of log files, stores
|
||||
them inside the home directory of the user who owns the experiment. The forwarded
|
||||
serial output of the nodes and the output of the network sniffer are not stored
|
||||
directly on the shared server because a large amount of data might be generated
|
||||
this way. Instead, scripts are provided to aggregate the serial output and
|
||||
stream of network packages into two aggregate streams of messages. These
|
||||
streams can then be forwarded and stored for later analysis on the local
|
||||
computer.
|
||||
|
||||
\section{Local Computer}
|
||||
|
||||
The local computer submits the experiment to the \ac{API}, controls the execution
|
||||
of the experiment and collects the results from the site server. The data is
|
||||
stored, processed and later displayed to the user.
|
||||
|
||||
\subsection{Orchestration Scripts}
|
||||
|
||||
For the purpose of configuring the experiment through the \ac{API}, an
|
||||
extendable set of scripts has been created. The \texttt{run} script submits the
|
||||
configuration of the experiment to the \ac{API}, monitors the state of the
|
||||
experiment, sets up forwardings for the aggregated serial output and packet
|
||||
captures and loads and calls into hook functions that can be specified per
|
||||
experiment. The control flow of the script is shown in \autoref{fig:orchest}.
|
||||
|
||||
\begin{figure}[h]
|
||||
\centering
|
||||
\begin{tikzpicture}
|
||||
\tikzumlset{fill class=tubsLightBlue20, fill object=tubsLightBlue20, fill component=tubsLightBlue40}
|
||||
\begin{umlseqdiag}
|
||||
\umlobject[class=script]{run}
|
||||
\begin{umlcallself}[op=pre]{run}
|
||||
\end{umlcallself}
|
||||
\begin{umlcallself}[op=wait running]{run}
|
||||
\end{umlcallself}
|
||||
\umlobject[class=hook]{during}
|
||||
\begin{umlcall}[op=subshell,type=asynchron]{run}{during}
|
||||
\umlobject[class=Subshell]{track}
|
||||
\begin{umlcall}[op=subshell,type=asynchron]{run}{track}
|
||||
\begin{umlcallself}[op=serial aggregator,type=asynchron]{track}
|
||||
\begin{umlcallself}[op=sniffer aggregator,type=asynchron]{track}
|
||||
\end{umlcallself}
|
||||
\end{umlcallself}
|
||||
\end{umlcall}
|
||||
\end{umlcall}
|
||||
\umlobject[class=hook]{post}
|
||||
\begin{umlcallself}[op=wait terminated]{run}
|
||||
\end{umlcallself}
|
||||
\begin{umlcall}{run}{post}
|
||||
\end{umlcall}
|
||||
\begin{umlcallself}[op=save results]{run}
|
||||
\end{umlcallself}
|
||||
\end{umlseqdiag}
|
||||
\end{tikzpicture}
|
||||
\caption{A Sequence diagram of the orchestration scripts}
|
||||
\label{fig:orchest}
|
||||
\end{figure}
|
||||
|
||||
First, the script calls the \texttt{pre} hook function from a script file
|
||||
associated with the experiment, this function can be used for setting up the
|
||||
environment at the local computer with settings specific to the experiment. Then
|
||||
the script sends send the configuration of the experiment together with the
|
||||
associated firmware files to the \ac{REST}-\ac{API} and waits for the experiment
|
||||
to be started on the test-lab and stores the ID announced by the \ac{API}.
|
||||
|
||||
After the \ac{API} has marked the experiment as \texttt{Running}, the script
|
||||
proceeds by calling into the \texttt{during} function of the experiment. This
|
||||
function can be used for manipulating the experiment during its executing. In
|
||||
the case of the experiments run in the evaluation, this function takes care of
|
||||
resetting a node at a random time during a phase with resets and flashing the different versions of the firmware at the
|
||||
required time.
|
||||
|
||||
After the \texttt{during} function completes, the orchestration script copies
|
||||
any further results from the side server to the local computer.
|
||||
|
||||
\subsection{Aggregation and Storage}
|
||||
|
||||
For the storage and analysis of the measurement data, the local computer connects
|
||||
to the shared server. The aggregated serial output and \acp{PCAP} are
|
||||
transported to the local computer on one \ac{SSH} connections and each stored
|
||||
into a separate files. The local computer also collects the log files containing
|
||||
the events, energy consumption and \ac{RSSI} from the shared server and stores them for
|
||||
later analysis.
|
||||
|
||||
After all data of an experiment has been collected, the contents of the
|
||||
different files is pre-processed according to the criteria specified for the
|
||||
evaluation and this data is then stored into a
|
||||
\emph{SQLite3} database for later analysis. The main reason for pre-processing
|
||||
the data is to be able to perform the analysis within adequate computation time.
|
||||
|
||||
The pre-processed data is then fetched from the database using \ac{SQL}. Numeric
|
||||
processing of the data is done using \emph{SciPy}
|
||||
\footnote{\url{https://www.scipy.org/}}, a software library for scientific
|
||||
computation that is mostly written in the programming language \emph{Python}
|
||||
but binds to functions written in lower level programming languages that contain
|
||||
optimized code for certain tasks. The results of the analysis are then displayed
|
||||
using \emph{Matplotlib}, a \emph{Python} library for displaying structured
|
||||
data-sets.
|
||||
|
||||
|
||||
\begin{landscape}
|
||||
|
||||
\begin{figure}
|
||||
\centering
|
||||
|
||||
\begin{tikzpicture}
|
||||
\tikzumlset{fill component=tubsLightBlue20, fill port=tubsLightBlue40}
|
||||
|
||||
\begin{umlcomponent}[x=-0.5,y=15,fill=white]{Testlab Node}
|
||||
\end{umlcomponent}
|
||||
\begin{umlcomponent}[x=3.5,y=15,fill=white]{Testlab Node}
|
||||
\end{umlcomponent}
|
||||
\begin{umlcomponent}[x=7.5,y=15,fill=white]{Testlab Node}
|
||||
\end{umlcomponent}
|
||||
|
||||
\begin{umlcomponent}[x=-1,y=14,fill=white]{Testlab Node}
|
||||
\end{umlcomponent}
|
||||
\begin{umlcomponent}[x=3,y=14,fill=white]{Testlab Node}
|
||||
\end{umlcomponent}
|
||||
\begin{umlcomponent}[x=7,y=14,fill=white]{Testlab Node}
|
||||
\end{umlcomponent}
|
||||
|
||||
\begin{umlcomponent}[y=1]{Testlab Node}
|
||||
\begin{umlcomponent}{Open Node}
|
||||
\umlbasiccomponent{UDP Sink / Source}
|
||||
\umlbasiccomponent[y=3]{Logger}
|
||||
\umlbasiccomponent[y=6]{Powertrace}
|
||||
\end{umlcomponent}
|
||||
|
||||
\begin{umlcomponent}[x=6]{Host Node}
|
||||
\umlbasiccomponent{Consumption}
|
||||
\umlbasiccomponent[y=2.5]{RSSI}
|
||||
\umlbasiccomponent[y=5]{Event Log}
|
||||
\umlbasiccomponent[y=7.5]{Sniffer}
|
||||
\umlbasiccomponent[y=10]{Forwarded Serial}
|
||||
\end{umlcomponent}
|
||||
\end{umlcomponent}
|
||||
|
||||
\begin{umlcomponent}[y=1,x=14.5]{Shared Server}
|
||||
\umlbasiccomponent[y=0]{API}
|
||||
\umlbasiccomponent[y=2.5]{Log files}
|
||||
\umlbasiccomponent[y=5]{Sniffer Aggregator}
|
||||
\umlbasiccomponent[y=7.5]{Serial Aggregator}
|
||||
\end{umlcomponent}
|
||||
|
||||
\begin{umlcomponent}[y=1,x=20,fill=tubsLightGreen20]{Local Computer}
|
||||
\umlbasiccomponent[y=12]{Orchestration}
|
||||
\umlbasiccomponent[y=8]{Database}
|
||||
\umlbasiccomponent[y=4]{Analysis}
|
||||
\begin{umlcomponent}[y=0]{Presentation}
|
||||
\umlactor[scale=0.5]{User}
|
||||
\end{umlcomponent}
|
||||
|
||||
\umlassemblyconnector[interface=Parser]{Database-north-port}{Orchestration-south-port}
|
||||
\umlassemblyconnector[interface=SQL]{Analysis-north-port}{Database-south-port}
|
||||
\umlassemblyconnector[interface=Python]{Presentation-north-port}{Analysis-south-port}
|
||||
|
||||
\end{umlcomponent}
|
||||
|
||||
\umlbasiccomponent[x=14.5,y=14,fill=tubsLightBlue20]{API Server}
|
||||
\umlassemblyconnector[interface=REST,with port]{Orchestration-west-port}{API Server}
|
||||
\umlassemblyconnector[interface=SSH,with port]{API Server-south-port}{Shared Server-north-port}
|
||||
|
||||
\umlassemblyconnector[interface=SSH,with port]{Orchestration}{Shared Server}
|
||||
\umlassemblyconnector[interface=ON Con,with port]{Host Node}{Open Node}
|
||||
%\umlHVHassemblyconnector[interface=Configuration,with port]{API}{Host Node}
|
||||
\umlHVHassemblyconnector[with port,arm2=+2cm]{Log files}{Consumption}
|
||||
\umlassemblyconnector[interface=Log Collection,with port]{Log files}{RSSI}
|
||||
\umlHVHassemblyconnector[with port,arm2=+2cm]{Log files}{Event Log}
|
||||
\umlassemblyconnector[interface=TCP,with port]{Serial Aggregator}{Forwarded Serial}
|
||||
\umlassemblyconnector[interface=TCP,with port]{Sniffer Aggregator}{Sniffer}
|
||||
\end{tikzpicture}
|
||||
\caption{Software components used in the evaluation}
|
||||
\label{fig:components}
|
||||
\end{figure}
|
||||
|
||||
\end{landscape}
|
Loading…
Add table
Add a link
Reference in a new issue