diff --git a/project_task_sheets/phase_03/project_phase03_tasks.zip b/project_task_sheets/phase_03/project_phase03_tasks.zip index ebb9463..70a4f5b 100644 Binary files a/project_task_sheets/phase_03/project_phase03_tasks.zip and b/project_task_sheets/phase_03/project_phase03_tasks.zip differ diff --git a/project_task_sheets/phase_05/project_phase05_tasks/Makefile b/project_task_sheets/phase_05/project_phase05_tasks/Makefile index ef3a241..f678320 100644 --- a/project_task_sheets/phase_05/project_phase05_tasks/Makefile +++ b/project_task_sheets/phase_05/project_phase05_tasks/Makefile @@ -11,7 +11,7 @@ latexmkflags = all : $(target) -dev : latexmkflags = -pvc +dev : latexmkflags = -pvc -interaction=nonstopmode dev : all $(target) : $(package) diff --git a/project_task_sheets/phase_05/project_phase05_tasks/Solution_Phase05_MichaelChen.tex b/project_task_sheets/phase_05/project_phase05_tasks/Solution_Phase05_MichaelChen.tex index fc3b04b..81f5916 100644 --- a/project_task_sheets/phase_05/project_phase05_tasks/Solution_Phase05_MichaelChen.tex +++ b/project_task_sheets/phase_05/project_phase05_tasks/Solution_Phase05_MichaelChen.tex @@ -5,6 +5,15 @@ \usepackage{listings} \usepackage{enumitem} \usepackage{subcaption} +\usepackage[acronym]{glossaries} + +\newacronym{bnf}{BNF}{Backus-Naur form} +\newacronym{fsm}{FSM}{finite state machine} +\newacronym{tsc}{TSC}{terminal symbol coverage} +\newacronym{pdc}{PDC}{production coverage} +\newacronym{dc}{DC}{derivation coverage} +\newacronym{moc}{MOC}{mutation operator coverage} +\newacronym{mpc}{MPC}{mutation production coverage} \usepackage{pgf} \usepackage{tikz} @@ -63,8 +72,8 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\def\name{[Add name here]} -\def\group{[Add group here]} +\def\name{Michael Chen} +\def\group{Group 01 (fastjson)} \begin{document} \projectinfo{5}{Software Testing - Syntax Coverage\small}{\today}{\name}{\group} @@ -79,32 +88,57 @@ \begin{enumerate} \item \textit{Production Rule}, \textit{Generator}, and \textit{Terminal} \begin{answer} - [TODO: Add answer here] + A Grammar is a set of \textit{words} or, as called here, \textit{strings}, that it accepts. Grammars can be described in different ways such as set notation, regular expressions, or in \gls{bnf}. \Gls{bnf} is set of production rules, also called rewrite rules, that map a set of symbols to a single symbol which is the name of the production rule. Symbols can be either non-terminals which represent non-trivial sytactic constructs such as loops or declarations in programming, while terminal symbols represent tokens, such as integer or string literals in programming languages. A generator is a \gls{fsm} that, given a grammar in \gls{bnf}, can generate a word of the grammar by recursively applying productions on the starting symbol until the syntax tree's leafs are only terminal symbols. \end{answer} \item \textit{Mutant}, \textit{Ground String}, and \textit{Mutation Operator} \begin{answer} - [TODO: Add answer here] + A ground string is a string that is accepted by a grammar. In other terms it is a string where we already know that a proof exists, that shows the string is in the grammar (ground truth). We can now change this string a little bit by applying specific rules, called mutation operators, that generate new strings that may or may not be accepted by the grammar, called mutants. \end{answer} - \item \textit{RIP Model} - \begin{answer} - [TODO: Add answer here] - \end{answer} + \item \textit{RIP Model} + \begin{answer} + The RIP model declares three requirements that are fulfilled when a failure occurrs. If all three of these are true, only then a failure can be detected. Firstly, the location of the fault in the code must be reached (\textbf{Reachability}), then, the program's internal state must be corrupted, i.e. an error occurred (\textbf{Infection}) and, finally, the error must have propagated to the program's output where the failure can then be observed (\textbf{Propagation}). + \end{answer} \end{enumerate} \item Name and describe \textit{2 syntax-based coverage criteria}. Does one of these two criteria subsume the other? Explain why, or provide a counterexample. \begin{answer} - [TODO: Add answer here] + Two criteria on mutation operators are \gls{moc} and \gls{mpc}. \Gls{moc} introduces test requirements for every mutation operator, where the test is applied to the mutant created by the operator. This is trivially subsumed by the \gls{mpc}, which in addition to that, requires that for every production in the grammar, which the operator is applicable to, a test requirement is added. \end{answer} \item What does it mean to \textit{"kill a mutant"}? Explain the concept with an own code example. \begin{answer} - [TODO: Add answer here] + A test (strongly) kills a mutant if all three conditions of the RIP model are satisfied for it's test execution. Consider the following mutated \texttt{max} program: + \begin{lstlisting}[language=C] + int max(int a, int b) { + // return a > b ? a : b; // original + return a < b ? a : b; // mutant + } + \end{lstlisting} + + A test that kills this mutant could be the following: + \begin{lstlisting}[language=C] + void kill_max_mutation() { + assert(max(1, 3) == 3); + } + \end{lstlisting} + + \begin{enumerate} + \item This test satisfies the reachability requirement trivially. + \item The program is infected because the changed operator causes the program to select the wrong branch of the ternary expression. + \item And finally the error propagated to the function output, in this case also trivially because the incorrect value is directly returned. + \end{enumerate} + Because the RIP model is satisfied the mutation is successfully killed by the test. \end{answer} \item Name and describe the \textit{4 different classified types} of mutants in the context of mutant killing. \begin{answer} - [TODO: Add answer here] + \begin{itemize} + \item \textbf{Dead mutants} have been successfully killed by a test case. + \item \textbf{Trivial mutants} are killed by any any test case in that the variation is so severe, that the program will almost certainly result in a failure. + \item \textbf{Stillborn mutants} cannot be killed by any test case because they are syntactically illegal, i.e. cannot be executed. + \item \textbf{Equivalent mutants} can also not be killed, because the mutation does not cause the program to behave differently. + \end{itemize} \end{answer} \end{enumerate}