327 lines
No EOL
15 KiB
TeX
327 lines
No EOL
15 KiB
TeX
\chapter{Measurement Setup}
|
|
\label{chap:setup}
|
|
|
|
This chapter describes the setup used for configuring, and controlling the
|
|
experiments inside the lab. Furthermore,it is described how measurement data are
|
|
collected, processed and stored. The general software architecture is shown in
|
|
\autoref{fig:components}. The components are distributed across several
|
|
different platforms.
|
|
|
|
Measurement data for multiple parameters were collected from the network. The
|
|
variables and their data sources are listed in \autoref{tab:params}. The packet
|
|
captures can be substituted with outputs on the serial output. For telling the
|
|
energy consumption of an implementation, a \ac{PM} of one sample per second is
|
|
sufficient.
|
|
|
|
\begin{table}[h]
|
|
\centering
|
|
\caption{Measured variables at each node}
|
|
\begin{tabular}{ll}
|
|
\toprule
|
|
Variable & Source \\
|
|
\midrule
|
|
Network latency & PCAP, serial output \\
|
|
Delivery ratio & PCAP, serial output \\
|
|
Control overhead & PCAP, serial output \\
|
|
Number of DIO, DAO & PCAP, serial output \\
|
|
DODAG state & serial output \\
|
|
Routing table & serial output \\
|
|
Node reset times & HN event log \\
|
|
RSSI & HN radio receiver \\
|
|
Energy consumption & HN power monitor \\
|
|
\end{tabular}
|
|
\label{tab:params}
|
|
\end{table}
|
|
|
|
When evaluating such a large set of variables from multiple different sources,
|
|
the amount of generated data and if enough processing power and storage space is
|
|
available has to carefully considered. \autoref{tab:amountdata} shows the
|
|
estimated amounts of data generated for each variable. For 50 runs this merely
|
|
amounts to 500 MByte of raw data, which can be further compressed using text
|
|
compression, since they, especially in the case of the serial log, contain many
|
|
repeating strings.
|
|
|
|
\begin{table}
|
|
\centering
|
|
\caption{Estimated amount of data generated by an experiment with 44 nodes,
|
|
lasting 600 seconds}
|
|
\begin{tabular}{lrr}
|
|
\toprule
|
|
Source & Single (bytes) & Total (KBytes)\\
|
|
\midrule
|
|
Serial output & 300 & 7920 \\
|
|
Event log & 40 & 1056 \\
|
|
Consumption log & 46 & 1214 \\
|
|
RSSI log & 46 & 1214 \\
|
|
\midrule
|
|
Total & & 11404 \\
|
|
\end{tabular}
|
|
\label{tab:amountdata}
|
|
\end{table}
|
|
|
|
\section{Testlab Nodes}
|
|
|
|
As previously mentioned in \autoref{sec:architecture}, each node inside the
|
|
testlab is consists of the \ac{ON} that runs the firmware supplied by the user
|
|
and the \ac{HN} that controls and monitors the \ac{ON}. In this section, the
|
|
software running on the node and how data is collected from it is further
|
|
described.
|
|
|
|
\subsection{Open Node}
|
|
|
|
Each \ac{ON} inside the test lab runs a version of the \emph{Contiki} operating
|
|
system. The node at the root of the \ac{DAG} is programmed to act as a sink for
|
|
the \ac{UDP} traffic that is periodically emitted the other nodes that act as
|
|
sources. Each such packet contains a sequence number. The reception of these
|
|
packets at the sink and the emission from the source are logged to the serial
|
|
output of the node node. This makes it possible to detect whether a packet has
|
|
been received when comparing the log of the sink and that of the respective
|
|
sender.
|
|
|
|
Further entries inside the log on the serial output from each node include the
|
|
number of emitted \ac{DIO} and \ac{DAO} messages, changes of the routing table and changes
|
|
of the preferred parent.
|
|
|
|
Different versions of the firmware have been produced for the \ac{DAG} root and
|
|
all other nodes in the network. The firmware of the root creates a new
|
|
\ac{DAG} with the node itself at the root. All other nodes will attempt to join
|
|
this \ac{DAG}.
|
|
|
|
\subsection{Porting the Firmware}
|
|
|
|
The firmware supplied to the \ac{ON} comes in different versions which each have
|
|
different features enabled. The default configuration of \emph{Contiki} 2.7 with
|
|
added device support for the \emph{M3} node is provided by \fitlab. The hardened
|
|
implementation has been developed based on an older version of \emph{Contiki}, that
|
|
also did not include device support for the \emph{M3} node. In preparation for
|
|
the evaluation, the hardened implementation has been ported to this newer version
|
|
of \emph{Contiki} and some changes and additions had to be made to take into
|
|
account changed software interfaces and device specific differences.
|
|
|
|
Some build options specific to the \emph{Z1} where required for the hardened
|
|
implementation to function. These options have been added as necessary.
|
|
|
|
A missing consistency check has been discovered when restoring the routing state
|
|
from flash memory and has then been added that erases the state from the flash
|
|
if an inconsistency is discovered.
|
|
|
|
When testing the hardened implementation, it came to attention that a restart of
|
|
the \ac{DAG} root will result in it becoming unable to reconfigure as the
|
|
\ac{DAG} root, leading to the network being unusable until the state is purged
|
|
from the flash memory of the root node. This is caused by the root node
|
|
restoring the previously recorded \ac{DIO} messages from its flash and replaying
|
|
them onto the \ac{RPL} module before it configures itself as the root of a
|
|
\ac{DAG}. This leads to it joining an existing \ac{DAG} discovered from the
|
|
\ac{DIO} messages. This \ac{DAG} is the same \ac{DAG} as was the node previously
|
|
the root of. The node joins this \ac{DAG} and selects one of its nodes as a
|
|
preferred parent, setting its own rank to a rank larger than its parent. It is
|
|
likely that the new parent node previously had selected the former root node as
|
|
a parent. In this case, the new parent node of the former root node discovers
|
|
that its own parent has a larger rank than In this case, the new parent node of
|
|
the former root node discovers that its own parent has a larger rank than
|
|
itself. This triggers the former root node to be dropped as the parent at this
|
|
node. Thus the tree looses its root and becomes unusable but also unable to
|
|
recover, since the root node believes to be already part of a \ac{DAG} and will
|
|
not create a new \ac{DAG} with itself as the root node.
|
|
|
|
\subsection{Host Node}
|
|
|
|
The \ac{HN} collects the log output from the \ac{ON} and offers it to
|
|
connecting clients of a \ac{TCP} network socket. The \ac{HN} also can record
|
|
\acp{PCAP} in the direct vicinity of the \ac{ON} and forwards them in a similar
|
|
manner to the serial output. Alternatively, the \ac{HN} can record the local
|
|
\ac{RSSI}. For each event, including start, stop and reset of the attached
|
|
\ac{ON}, a corresponding entry is appended to an event log. The \ac{HN} can also
|
|
record accurate values for the power consumption, voltage and current from the
|
|
power management unit of the \ac{ON}.
|
|
|
|
For the purpose of measuring the energy consumption of the node in
|
|
reaction to a transient node failure, a high \ac{AM} can be selected since
|
|
\ac{RPL} takes some time to react to the change in the network, so the state
|
|
will persist for a multiple second. This way it is possible to reduce
|
|
the noise component and acquire more accurate data.
|
|
|
|
The \ac{HN} stores both the consumption data and the \ac{RSSI} in the
|
|
\ac{OML}. These log files are stored locally and periodically collected and
|
|
stored by the site server.
|
|
|
|
\section{Site Server}
|
|
|
|
At each site, a shared server is provided running a multi-user system
|
|
(\emph{Linux}). The server periodically queries the events, \ac{RSSI} and
|
|
consumption of each \ac{HN} and for each node in the form of log files, stores
|
|
them inside the home directory of the user who owns the experiment. The forwarded
|
|
serial output of the nodes and the output of the network sniffer are not stored
|
|
directly on the shared server because a large amount of data might be generated
|
|
this way. Instead, scripts are provided to aggregate the serial output and
|
|
stream of network packages into two aggregate streams of messages. These
|
|
streams can then be forwarded and stored for later analysis on the local
|
|
computer.
|
|
|
|
\section{Local Computer}
|
|
|
|
The local computer submits the experiment to the \ac{API}, controls the execution
|
|
of the experiment and collects the results from the site server. The data is
|
|
stored, processed and later displayed to the user.
|
|
|
|
\subsection{Orchestration Scripts}
|
|
|
|
For the purpose of configuring the experiment through the \ac{API}, an
|
|
extendable set of scripts has been created. The \texttt{run} script submits the
|
|
configuration of the experiment to the \ac{API}, monitors the state of the
|
|
experiment, sets up forwardings for the aggregated serial output and packet
|
|
captures and loads and calls into hook functions that can be specified per
|
|
experiment. The control flow of the script is shown in \autoref{fig:orchest}.
|
|
|
|
\begin{figure}[h]
|
|
\centering
|
|
\begin{tikzpicture}
|
|
\tikzumlset{fill class=tubsLightBlue20, fill object=tubsLightBlue20, fill component=tubsLightBlue40}
|
|
\begin{umlseqdiag}
|
|
\umlobject[class=script]{run}
|
|
\begin{umlcallself}[op=pre]{run}
|
|
\end{umlcallself}
|
|
\begin{umlcallself}[op=wait running]{run}
|
|
\end{umlcallself}
|
|
\umlobject[class=hook]{during}
|
|
\begin{umlcall}[op=subshell,type=asynchron]{run}{during}
|
|
\umlobject[class=Subshell]{track}
|
|
\begin{umlcall}[op=subshell,type=asynchron]{run}{track}
|
|
\begin{umlcallself}[op=serial aggregator,type=asynchron]{track}
|
|
\begin{umlcallself}[op=sniffer aggregator,type=asynchron]{track}
|
|
\end{umlcallself}
|
|
\end{umlcallself}
|
|
\end{umlcall}
|
|
\end{umlcall}
|
|
\umlobject[class=hook]{post}
|
|
\begin{umlcallself}[op=wait terminated]{run}
|
|
\end{umlcallself}
|
|
\begin{umlcall}{run}{post}
|
|
\end{umlcall}
|
|
\begin{umlcallself}[op=save results]{run}
|
|
\end{umlcallself}
|
|
\end{umlseqdiag}
|
|
\end{tikzpicture}
|
|
\caption{A Sequence diagram of the orchestration scripts}
|
|
\label{fig:orchest}
|
|
\end{figure}
|
|
|
|
First, the script calls the \texttt{pre} hook function from a script file
|
|
associated with the experiment, this function can be used for setting up the
|
|
environment at the local computer with settings specific to the experiment. Then
|
|
the script sends send the configuration of the experiment together with the
|
|
associated firmware files to the \ac{REST}-\ac{API} and waits for the experiment
|
|
to be started on the test-lab and stores the ID announced by the \ac{API}.
|
|
|
|
After the \ac{API} has marked the experiment as \texttt{Running}, the script
|
|
proceeds by calling into the \texttt{during} function of the experiment. This
|
|
function can be used for manipulating the experiment during its executing. In
|
|
the case of the experiments run in the evaluation, this function takes care of
|
|
resetting a node at a random time during a phase with resets and flashing the different versions of the firmware at the
|
|
required time.
|
|
|
|
After the \texttt{during} function completes, the orchestration script copies
|
|
any further results from the side server to the local computer.
|
|
|
|
\subsection{Aggregation and Storage}
|
|
|
|
For the storage and analysis of the measurement data, the local computer connects
|
|
to the shared server. The aggregated serial output and \acp{PCAP} are
|
|
transported to the local computer on one \ac{SSH} connections and each stored
|
|
into a separate files. The local computer also collects the log files containing
|
|
the events, energy consumption and \ac{RSSI} from the shared server and stores them for
|
|
later analysis.
|
|
|
|
After all data of an experiment has been collected, the contents of the
|
|
different files is pre-processed according to the criteria specified for the
|
|
evaluation and this data is then stored into a
|
|
\emph{SQLite3} database for later analysis. The main reason for pre-processing
|
|
the data is to be able to perform the analysis within adequate computation time.
|
|
|
|
The pre-processed data is then fetched from the database using \ac{SQL}. Numeric
|
|
processing of the data is done using \emph{SciPy}
|
|
\footnote{\url{https://www.scipy.org/}}, a software library for scientific
|
|
computation that is mostly written in the programming language \emph{Python}
|
|
but binds to functions written in lower level programming languages that contain
|
|
optimized code for certain tasks. The results of the analysis are then displayed
|
|
using \emph{Matplotlib}, a \emph{Python} library for displaying structured
|
|
data-sets.
|
|
|
|
|
|
\begin{landscape}
|
|
|
|
\begin{figure}
|
|
\centering
|
|
|
|
\begin{tikzpicture}
|
|
\tikzumlset{fill component=tubsLightBlue20, fill port=tubsLightBlue40}
|
|
|
|
\begin{umlcomponent}[x=-0.5,y=15,fill=white]{Testlab Node}
|
|
\end{umlcomponent}
|
|
\begin{umlcomponent}[x=3.5,y=15,fill=white]{Testlab Node}
|
|
\end{umlcomponent}
|
|
\begin{umlcomponent}[x=7.5,y=15,fill=white]{Testlab Node}
|
|
\end{umlcomponent}
|
|
|
|
\begin{umlcomponent}[x=-1,y=14,fill=white]{Testlab Node}
|
|
\end{umlcomponent}
|
|
\begin{umlcomponent}[x=3,y=14,fill=white]{Testlab Node}
|
|
\end{umlcomponent}
|
|
\begin{umlcomponent}[x=7,y=14,fill=white]{Testlab Node}
|
|
\end{umlcomponent}
|
|
|
|
\begin{umlcomponent}[y=1]{Testlab Node}
|
|
\begin{umlcomponent}{Open Node}
|
|
\umlbasiccomponent{UDP Sink / Source}
|
|
\umlbasiccomponent[y=3]{Logger}
|
|
\umlbasiccomponent[y=6]{Powertrace}
|
|
\end{umlcomponent}
|
|
|
|
\begin{umlcomponent}[x=6]{Host Node}
|
|
\umlbasiccomponent{Consumption}
|
|
\umlbasiccomponent[y=2.5]{RSSI}
|
|
\umlbasiccomponent[y=5]{Event Log}
|
|
\umlbasiccomponent[y=7.5]{Sniffer}
|
|
\umlbasiccomponent[y=10]{Forwarded Serial}
|
|
\end{umlcomponent}
|
|
\end{umlcomponent}
|
|
|
|
\begin{umlcomponent}[y=1,x=14.5]{Shared Server}
|
|
\umlbasiccomponent[y=0]{API}
|
|
\umlbasiccomponent[y=2.5]{Log files}
|
|
\umlbasiccomponent[y=5]{Sniffer Aggregator}
|
|
\umlbasiccomponent[y=7.5]{Serial Aggregator}
|
|
\end{umlcomponent}
|
|
|
|
\begin{umlcomponent}[y=1,x=20,fill=tubsLightGreen20]{Local Computer}
|
|
\umlbasiccomponent[y=12]{Orchestration}
|
|
\umlbasiccomponent[y=8]{Database}
|
|
\umlbasiccomponent[y=4]{Analysis}
|
|
\begin{umlcomponent}[y=0]{Presentation}
|
|
\umlactor[scale=0.5]{User}
|
|
\end{umlcomponent}
|
|
|
|
\umlassemblyconnector[interface=Parser]{Database-north-port}{Orchestration-south-port}
|
|
\umlassemblyconnector[interface=SQL]{Analysis-north-port}{Database-south-port}
|
|
\umlassemblyconnector[interface=Python]{Presentation-north-port}{Analysis-south-port}
|
|
|
|
\end{umlcomponent}
|
|
|
|
\umlbasiccomponent[x=14.5,y=14,fill=tubsLightBlue20]{API Server}
|
|
\umlassemblyconnector[interface=REST,with port]{Orchestration-west-port}{API Server}
|
|
\umlassemblyconnector[interface=SSH,with port]{API Server-south-port}{Shared Server-north-port}
|
|
|
|
\umlassemblyconnector[interface=SSH,with port]{Orchestration}{Shared Server}
|
|
\umlassemblyconnector[interface=ON Con,with port]{Host Node}{Open Node}
|
|
%\umlHVHassemblyconnector[interface=Configuration,with port]{API}{Host Node}
|
|
\umlHVHassemblyconnector[with port,arm2=+2cm]{Log files}{Consumption}
|
|
\umlassemblyconnector[interface=Log Collection,with port]{Log files}{RSSI}
|
|
\umlHVHassemblyconnector[with port,arm2=+2cm]{Log files}{Event Log}
|
|
\umlassemblyconnector[interface=TCP,with port]{Serial Aggregator}{Forwarded Serial}
|
|
\umlassemblyconnector[interface=TCP,with port]{Sniffer Aggregator}{Sniffer}
|
|
\end{tikzpicture}
|
|
\caption{Software components used in the evaluation}
|
|
\label{fig:components}
|
|
\end{figure}
|
|
|
|
\end{landscape} |