Ticket #2099: sage-doctesting.tex

File sage-doctesting.tex, 3.2 KB (added by ncalexan, 12 years ago)
Line 
1\documentclass{article}
2
3% Copied from devel/doc/commontex/macros.tex
4\usepackage{xspace}
5\newcommand{\Sage}{{\sf Sage}\xspace}
6\newcommand{\Latex}{\LaTeX\xspace} 
7
8\begin{document}
9
10\title{The \Sage testing architecture}
11\maketitle
12
13\section{Introduction}
14
15The testing philosophy of \Sage is that
16\begin{itemize}
17\item every function should be doctested for correctness; and
18\item every doctest should be reproducible.
19\end{itemize}
20
21Reproducible means that a user should be able to copy the text of any doctest
22in the system, paste it into a fresh \Sage command shell (or the notebook) and
23receive the expected output, verbatim.
24
25\section{At runtime}
26
27\texttt{sage-sage} is the main entry point to the testing infrastructure.
28The relevant flags are \texttt{-t, -tnew, -testall}.
29
30The file \texttt{sage-test} (command line \texttt{sage -t}) is the testing
31``master process''.  The file \texttt{sage-doctest} is the testing ``slave
32process''.
33
34The master \texttt{sage-test} collects files from the command line for testing
35and discards files that should not be tested, such as images and internally
36generated doctest files.  For each file to be tested, the master spawns a
37slave \texttt{sage-doctest} that processes a single file.
38
39The slave \texttt{sage-doctest}:
40\begin{itemize}
41\item Parses the source code of the indicated file for doctests;
42\item Writes a \texttt{.doctest} Python file containing code that executes the
43  doctests and checks the results;
44\item Spawns a \Sage instance that imports the \Sage library and executes the
45  \texttt{.doctest} file in the newly created clean environment.
46\end{itemize}
47
48\section{Files and code organization}
49
50The main files are \texttt{sage-sage}, \texttt{sage-test}, and
51\texttt{sage-doctest}.
52
53\subsection{Other files that are relevant}
54
55The file \texttt{sage-maketest} (command line \texttt{sage -testall}) is a
56helper that tests the \Latex documentation and every file in the \Sage
57library.
58
59The file \texttt{sage-test-new} (command line \texttt{sage -tnew}) is a helper
60that queries \texttt{hg} for files that have been added or modified but not
61yet committed to the current repository and tests only those changes.
62
63\subsection{Files that need to be considered when updating}
64
65The file \texttt{sage-fixdoctests} changes the expected text of each doctest
66to the resulting text, thereby making every test (that does not raise or
67expect an exception) pass by definition.  If the testing architecture changes,
68this needs to be updated or declared obsolete and removed.
69
70\subsection{Files that are obsolete}
71
72The following doctest related files appear to be obsolete:
73\texttt{sage-doctest\_old\_ver, sage-doctest\_py, sage-doctest\_pyx,
74  sage-doctest\_sage}. They are in the \texttt{local/bin} directory and
75\texttt{hg} repository, but there appears to be no reference to them in
76\texttt{sage-test} or \texttt{sage-doctest}.
77
78The file \texttt{sage-doctest\_tex} is \emph{almost} obsolete.  It appears to be used
79to test \texttt{.sage} files by default and every file if the \texttt{-sage}
80flag is set.  The relevant code (from \texttt{sage-test}) is
81
82\begin{verbatim}
83if use_sage_only or ext == '.sage':
84    return test(F, 'doctest_tex ' + opts)
85\end{verbatim}
86
87The Boolean \texttt{use\_sage\_only} is set by the \texttt{flag}.
88
89\end{document}