MT/content/introduction.tex

\chapter{Introduction}
\label{chapter:Introduction}

Embedded applications are increasingly required to provide real-time
performance \cite{hopkins2006debug}.  This means that the correct behavior of a
system is not only dependent on the logical results of a computation, but also
on the physical instant in which these are produced \cite{kopetz2011real}.  For
hard real-time applications violation of a deadline will result in damage of
the system or its environment \cite{tokuda1990real}.

Due to the pervasive nature of embedded systems and their use for critical
applications, e.g., medical devices or advanced driver assistance systems,
measures to ensure the correctness of time dependent functionality must be
taken \cite{konrad2005real}.  Therefore, debugging and validation are a
fundamental part of the development process of such applications
\cite{dixon2013advantages}.

Different techniques to debug embedded systems exist \cite{schneider2004ten}.
The simplest one is a classical \lstinline{printf} statement in C (or the
equivalent in another language).  More sophisticated debug technologies require
on-chip debug logic in the embedded processor.  On-chip debug generally
supports two different types of functionality: run-control debug and real-time
trace \cite{dixon2013advantages}.

The former allows it to stop and examine the state of a system at points of
interest, so called breakpoints.  This approach is intrusive, or in other words
changes the runtime behavior of an application.  This is not acceptable for
time critical applications, e.g., engine control units that require continuous
execution of the processor in order to control feedback loops and to maintain
mechanical stability \cite{dixon2013advantages}.

Real-time trace recording or tracing however, allows it to analyze and debug
a system without stopping the execution.  It works by recording processor
events such as function calls and data accesses.  The captured events can be
used to reconstruct and analyze the runtime behavior of an application.

Since timing is an integral part in the development of safe and secure
real-time applications, timing dependencies should be included in the software
interface specifications \cite{lutz1993analyzing}.  One way to specify these
dependencies are timing requirements, e.g., the maximum response time for a
certain task \cite{deubzer2011robust}.  Via tracing system engineers are
capable of validating those requirements on target.

\glsdesc{ta} (\gls{ta}) provides the \gls{ta} Tool Suite, a collection of tools
for the system design, simulation, automated optimization, and target
verification of embedded real-time multi-core and many-core systems
\cite{tatoolsuite}.  These features work on the basis of system models.
Consequently, requirements are defined for system entities such as tasks,
runnables, signals, and semaphores.

On the contrary, trace recording produces events on software level.  This means
a trace contains information about function entries and exits, and data read
and write accesses. As a consequence, the specified system requirements cannot
be evaluated.

However, by mapping software events to the corresponding system events it is
possible to transform a software to a system level trace.  \glsdesc{btf}
(\gls{btf}) is a trace format on system level and is used in this thesis
because of its native support for multi-core environments.  To the best of my
knowledge, the possibility of a software to system mapping has only been shown
for a small subset of all entities specified by \gls{btf}.

In this thesis the feasibility of mapping all event actions contained in the
\gls{btf} standard is discussed, evaluated and validated.  Furthermore,
different real-time trace techniques are discussed with respect to their
versatility for the timing analysis of embedded multi-core real-time
applications.

\section{Motivation}
\label{section:motivation}

Transformation of software events to system events is required for the timing
analysis of embedded real-time systems as discussed in the previous section.
Moreover, system traces can also be used for different other use cases which
are covered in the following.

\subsubsection{Simulation Validation}

A simulation can be executed for a timing model by the TA Simulator.  The
resulting simulated trace can be evaluated to validate the compliance of an
application with the specified requirements.

A simulated and a hardware based system trace will never be equal by
definition because a model is an abstraction of reality.  Nevertheless,
simulation supports engineers in validating system behavior in early design
stages.  It can abstract complex problems and analyze non-deterministic system
behavior \cite{sifakis2003building}.

However, a simulation is still a software which is vulnerable to bugs and can
potentially produce wrong results.  A deviation to reality due to the
abstraction cannot be classified as a wrong result, on the other hand an
implementation error can be.

Via tracing it is possible to validate the correctness of simulated traces.
This is especially useful if a new simulation feature is implemented.  In this
case a system trace recorded from hardware can provide valuable insights in the
actual behavior.

\subsubsection{OS Overhead Measurement}

Another aspect that is relevant for the development of embedded applications is
the overhead caused by the operating system (\gls{os})
\cite{zeng2011mechanisms}.  Overheads are execution periods where the processor
is not used by the actual application but by the \gls{os}  for example,
context switches and inter-core communication mechanisms.

Especially for applications with a high processor utilization the additional
overhead caused by the \gls{os} plays a critical role.  Fulfillment of timing
requirements may be feasible or not depending on the overhead
\cite{maxmaster}.  In order to take this into consideration a good
understanding of the execution times required by \gls{os} routines is
necessary.  System traces recorded on hardware allow it to determine the exact
execution times for these overheads easily.

\subsubsection{Model Reconstruction}

The initial creation of a timing model for an existing application is a tedious
process if it must be done manually.  Model reconstruction can simplify this
task by creating a timing model automatically.  It works by analyzing a
system trace recorded from hardware.  By detecting common timing patterns in
the trace a model of the application can be created
\cite{sailer2014reconstruction}.


\section{Related Work}

The two main topics discussed in this thesis are tracing and hardware to system
mapping.  While the former has been an important topic in the literature over
the last three decades, the necessity for the latter has only become important
in recent years.

\subsubsection{Tracing}

Ferrari \cite{ferrari1978computer} gives an comprehensive overview of major
computer performance evaluation techniques and their application to various
types of performance problems.  In his book \emph{Computer Systems Performance
Evaluation} he distinguishes between three trace measurement techniques:
software, hybrid, and hardware based trace measurement.  It is important to
understand that these techniques do not directly relate to the trace
abstraction levels discussed in the previous sections.  The concepts described
in his book which was released in 1978 are still relevant today, the
implementation is outdated.

Mink et al.\ \cite{mink1989performance} discuss hardware based performance
measurement in more detail.  They argue that hardware tracing is the only
sufficient trace technique for recording resource utilization information
because of the high signal speeds involved and the fact that not all signals
are visible to software measurement techniques.  Resource utilization is
concerned with detailed information about the operation of the hardware such as
cache hit ratios and access delays.  Moreover, they mention that software based
tracing is intrusive and thus changes the runtime characteristics of an
application.

Kraft et al.\ \cite{kraft2010trace} discuss trace measurement in the context of
five industrial projects.  They argue that hardware trace solutions require
large, expensive equipment mainly intended for lab use.  Additionally, they
claim that software based trace solutions can also remain active in
applications post-release.  Based on this arguments they use a software based
trace measurement approach in their paper.  They introduce a software
instrumentation approach with a very low overhead according to their
measurement results.


\subsubsection{Hardware to System Mapping}

Lauterbach \cite{lauterbach2015third} provides a possibility to export task and
runnable system events for traces recorded via hardware tracing.  However,
their approach is limited to a subset of the existing task and runnable events.
For example, runnable preempt and resume, and task wait events are not covered
by the Lauterbach export even though this information is relevant for the
real-time analysis.  Lauterbach uses the information from the \glsdesc{orti}
(\gls{orti}) files and relies solely on function trace events for the export.

Kraft et al.\ \cite{kraft2010trace} also discuss how task events on system
level can be recorded.  They argue that it is difficult to detect which entity
blocks a task because the scheduling status of the \gls{os} only provides
information about the entity type blocking the task not the entity itself.
They suggest code instrumentation as a pragmatic solution to work around this
problem, admitting that this approach is problematic because the
instrumentation points have to be maintained by the developer.


\section{Interrogation}
\label{section:interrogation}

Timing analysis of embedded system requires a trace, i.e., a sequence of
events, with sufficient duration and timestamp accuracy.  The minimum trace
duration is dependent on the application and requirements that should be
validated.  Fundamentally, the longer the trace duration the more information
for the real-time analysis of the application are acquired.  However, more data
requires longer processing times.  Therefore, a trace duration of at least one
second is demanded in this thesis to provide a tradeoff between processing time
and sufficient length for the real-life use-cases discussed in
\autoref{section:motivation}.

Timestamp accuracy is important for the real-time analysis because if the
resolution is too low no meaningful analysis may be feasible.  For example, if
events can only be recorded in the range of milliseconds, the analysis of
requirements in the microseconds range is not feasible.

Kraft et al. \cite{kraft2010trace} also state that a timestamp accuracy in the
milliseconds range is too coarse-grained for embedded systems timing analysis.
Especially for validation of simulation tools and model reconstruction cycle
accurate timestamps would provide enormous benefits.  From these requirements
the first hypothesis that should be evaluated in this thesis can be derived.

\begin{hyp}
\label{hyp:1}
There exists a trace technique that allows recording of cycle accurate traces
for embedded multi-core real-time system with a duration of at least one
second.
\end{hyp}

Trace techniques output a trace on software level, i.e., a sequence of software
events.  These events provide information about the code segments executed by
an application and the memory regions accessed.  This information allows deep
insights into the runtime behavior of an embedded system, but is not sufficient
for its real-time analysis.

Traces on system level or in other words, sequences of system events are
required for the real-time analysis of embedded multi-core applications.  In
the context of this thesis system events are defined as all events that are
contained in the \gls{btf} specification and not explicitly excluded in
\autoref{subsection:btf_entity_types}.  With an understanding of the underlying
\gls{os} mechanisms it may be possible to map software to system events.

\gls{osek} and \gls{autosar} are common standards for the development of
applications in the automotive industry.  These standards are discussed in more
detail later.  \gls{osek} compliant operating systems feature a so-called
\gls{orti} file.

The aim of \gls{orti} is to make \gls{os} internal data visible to external
tools \cite{osekortia}.  This means it is possible via \gls{orti} to relate
software level entities to their respective interpretation on system level.  It
must be examined if a mapping for all \gls{btf} entities is feasible.

\begin{hyp}
\label{hyp:2}
A complete mapping from software to system entities is feasible based on the
information included in the \gls{orti} file for an \gls{osek} compliant
\glsdesc{os}.
\end{hyp}

If Hypothesis~\ref{hyp:2} does not hold other ways to achieve a complete
software to system mapping must be found.  An \gls{os} must keep track of the
states of all relevant system objects internally.  Otherwise, it would not be
possible to execute appropriate actions if required.  For example, if one task
activates another one the \gls{os} must determine whether the corresponding
task is allowed to be activated or if the maximum number of activations has
already been exceeded.

By analyzing the internal data structures of an \gls{os} it may be possible to
construct a mapping from software to system entities.  Considering the previous
example, there might be an \gls{os} data structure that keeps track of the
remaining activations for each task entity.  If the field for a task is
incremented, an entity of the corresponding task terminates. If it is
decremented a new task instance is activated.

\begin{hyp}
\label{hyp:3}
A complete mapping from software to system entities is feasible for an
\gls{osek} compliant \glsdesc{os}.
\end{hyp}


\section{Outline}

In oder to transform a trace recorded from hardware to a trace on system level
an understanding of the underlying operating system mechanisms is required.  An
\gls{os} standard commonly used in the automotive industry is \gls{osekos}.  It
is discussed in \autoref{section:osekvdxos}.

The real-time behavior of an embedded multi-core application can be represented
by a system trace.  Based on a system trace an application can be examined and
specified timing requirements can be validated.  \gls{btf} is a system level
trace format and used in this thesis.  It is discussed in
\autoref{chapter:btf}.

There exist different techniques to record traces of embedded applications.  In
\autoref{section:trace_measurement} an overview of these techniques is
provided.  It is then argued why hardware tracing is the only technique
sufficient for the validation of embedded real-time applications.  Accordingly,
hardware tracing is then discussed in more detail.

On the basis of the information in \autoref{chapter:fundamentals} the mapping
between software entities and system entities is described in
\autoref{chapter:mapping}.  Mapping is done for all \gls{btf} entities that are
relevant for the analysis of embedded multi-core applications as discussed in
\autoref{subsection:btf_entity_types}.

In \autoref{chapter:validation} the mapping is validated.  For that reason
criteria to compare \gls{btf} traces are established in
\autoref{subsection:validation_techniques}.  Based on these criteria simulated
traces and traces recorded hardware are compared and evaluated.  This is done
in two steps.  Firstly, test applications are created manually to cover all
possible \gls{btf} actions in \autoref{subsection:systematic_tests}.  Secondly,
applications are created randomly to avoid selection bias in the creation of
test cases in \autoref{subsection:randomized_tests}.

Finally, the results of this thesis are discussed in
\autoref{chapter:conclusion} and possible topics for future work are outlined
in \autoref{chapter:future_work}.