mirror of
https://github.com/python/cpython
synced 2024-10-08 20:20:47 +00:00
Library documentation for the 'test' package. Still needs to be checked by Fred.
This commit is contained in:
parent
aed0a4a138
commit
066f3928b2
|
@ -118,6 +118,7 @@ and how to embed it in other applications.
|
|||
\input{libpydoc}
|
||||
\input{libdoctest}
|
||||
\input{libunittest}
|
||||
\input{libtest}
|
||||
\input{libmath}
|
||||
\input{libcmath}
|
||||
\input{librandom}
|
||||
|
|
275
Doc/lib/libtest.tex
Normal file
275
Doc/lib/libtest.tex
Normal file
|
@ -0,0 +1,275 @@
|
|||
\section{\module{test} ---
|
||||
Regression tests package for Python}
|
||||
|
||||
\declaremodule{standard}{test}
|
||||
|
||||
\sectionauthor{Brett Cannon}{brett@python.org}
|
||||
|
||||
|
||||
\modulesynopsis{Regression tests package containing the testing suite for
|
||||
Python.}
|
||||
|
||||
|
||||
The \module{test} package contains all regression tests for Python as well as
|
||||
the modules \module{test_support} and \module{regrtest.py}.
|
||||
\module{test_support} is used to enhance your tests while \module{regrtest.py}
|
||||
drives the testing suite.
|
||||
|
||||
Each module in the \module{test} package whose name starts with
|
||||
\code{'test_'} is a testing suite for a specific module or feature.
|
||||
All new tests should be written using the \module{unittest} module; using
|
||||
\module{unittest} is not required but makes the tests more flexible and
|
||||
maintenance of the tests easier.
|
||||
Some older tests are written to use \module{doctest} and a ``traditional''
|
||||
testing style; these styles of tests will not be covered.
|
||||
|
||||
\begin{seealso}
|
||||
\seemodule{unittest}{Writing PyUnit regression tests.}
|
||||
\seemodule{doctest}{Tests embedded in documentation strings.}
|
||||
\end{seealso}
|
||||
|
||||
|
||||
\subsection{test_support \label{test_support-docs}}
|
||||
|
||||
The \module{test_support} module contains functions for assisting with writing
|
||||
regression tests.
|
||||
|
||||
|
||||
The \module{test_support} module defines the following exceptions:
|
||||
|
||||
\begin{excdesc}{TestFailed}
|
||||
Exception to be raised when a test fails.
|
||||
\end{excdesc}
|
||||
|
||||
\begin{excdesc}{TestSkipped}
|
||||
Subclass of \exception{TestFailed}.
|
||||
Raised when a test is skipped.
|
||||
This occurs when a needed resource (such as a network connection) is not
|
||||
available at the time of testing.
|
||||
\end{excdesc}
|
||||
|
||||
\begin{excdesc}{ResourceDenied}
|
||||
Subclass of \exception{TestSkipped}.
|
||||
Raised when a resource (such as a network connection) is not available.
|
||||
Raised by the \function{requires} function.
|
||||
\end{excdesc}
|
||||
|
||||
|
||||
The \module{test_support} module defines the following constants:
|
||||
|
||||
\begin{datadesc}{verbose}
|
||||
\constant{True} when verbose output is enabled.
|
||||
Should be checked when more detailed information is desired about a running
|
||||
test.
|
||||
\var{verbose} is set by \module{regrtest.py}.
|
||||
\end{datadesc}
|
||||
|
||||
\begin{datadesc}{have_unicode}
|
||||
\constant{True} when Unicode support is available.
|
||||
\end{datadesc}
|
||||
|
||||
\begin{datadesc}{is_jython}
|
||||
\constant{True} if the running interpreter is Jython.
|
||||
\end{datadesc}
|
||||
|
||||
\begin{datadesc}{TESTFN}
|
||||
Set to the path that a temporary file may be created at.
|
||||
Any temporary that is created should be closed and unlinked (removed).
|
||||
\end{datadesc}
|
||||
|
||||
|
||||
The \module{test_support} module defines the following functions:
|
||||
|
||||
\begin{funcdesc}{forget}{module_name}
|
||||
Removes the module named \var{module_name} from \module{sys.modules} and deletes
|
||||
any byte-compiled files of the module.
|
||||
\end{funcdesc}
|
||||
|
||||
\begin{funcdesc}{is_resource_enabled}{resource}
|
||||
Returns \constant{True} if \var{resource} is enabled and available.
|
||||
The list of available resources is only set when \module{regrtest.py} is
|
||||
executing the tests.
|
||||
\end{funcdest}
|
||||
|
||||
\begin{funcdesc}{requires}{resource\optional{, msg}}
|
||||
Raises \exception{ResourceDenied} if \var{resource} is not available.
|
||||
\var{msg} is the argument to \exception{ResourceDenied} if it is raised.
|
||||
Always returns true if called by a function whose \var{__name__} is
|
||||
\code{"__main__"}.
|
||||
Used when tests are executed by \module{regrtest.py}.
|
||||
\end{funcdesc}
|
||||
|
||||
\begin{funcdesc}{findfile}{filename}
|
||||
Return the path to the file named \var{filename}.
|
||||
If no match is found \var{filename} is returned.
|
||||
This does not equal a failure since it could be the path to the file.
|
||||
\end{funcdesc}
|
||||
|
||||
\begin{funcdesc}{run_unittest}{*classes}
|
||||
Execute \class{unittest.TestCase} subclasses passed to the function.
|
||||
The function scans the classes for methods starting with the name
|
||||
\code{"test_"} and executes the tests individually.
|
||||
This is the preferred way to execute tests.
|
||||
\end{datadesc}
|
||||
|
||||
\begin{funcdesc}{run_suite}{suite\optional{, testclass=None}}
|
||||
Execute the \class{unittest.TestSuite} instance, \var{suite}.
|
||||
The optional argument \var{testclass} accepts one of the test classes in the
|
||||
suite so as to print out more detailed information on where the testing suite
|
||||
originated from.
|
||||
\end{funcdesc}
|
||||
|
||||
|
||||
|
||||
\subsection{Writing Unit Tests for the \module{test} package \label{writing-tests}}
|
||||
|
||||
It is preferred that tests for the \module{test} package use the
|
||||
\module{unittest} module and follow a few guidelines.
|
||||
One is to have the name of all the test methods start with \code{"test_"} as
|
||||
well as the module's name.
|
||||
This is needed so that the methods are recognized by the test driver as
|
||||
test methods.
|
||||
Also, no documentation string for the method should be included.
|
||||
A comment (such as
|
||||
\var{# Tests function returns only True or False}) should be used to provide
|
||||
documentation for test methods.
|
||||
This is done because documentation strings get printed out if they exist and
|
||||
thus what test is being run is not stated.
|
||||
|
||||
A basic boilerplate is often used:
|
||||
|
||||
\begin{verbatim}
|
||||
import unittest
|
||||
from test import test_support
|
||||
|
||||
class MyTestCase1(unittest.TestCase):
|
||||
|
||||
# Only use setUp() and tearDown() if necessary
|
||||
|
||||
def setUp(self):
|
||||
... code to execute in preparation for tests ...
|
||||
|
||||
def tearDown(self):
|
||||
... code to execute to clean up after tests ...
|
||||
|
||||
def test_feature_one(self):
|
||||
# Test feature one.
|
||||
... testing code ...
|
||||
|
||||
def test_feature_two(self):
|
||||
# Test feature two.
|
||||
... testing code ...
|
||||
|
||||
... more test methods ...
|
||||
|
||||
class MyTestCase2(unittest.TestCase):
|
||||
... same structure as MyTestCase1 ...
|
||||
|
||||
... more test classes ...
|
||||
|
||||
def test_main():
|
||||
test_support.run_unittest(MyTestCase1,
|
||||
MyTestCase2,
|
||||
... list other tests ...
|
||||
)
|
||||
|
||||
if __name__ == '__main__':
|
||||
test_main()
|
||||
\end{verbatim}
|
||||
|
||||
This boilerplate code allows the testing suite to be run by \module{regrtest.py}
|
||||
as well as on its own as a script.
|
||||
|
||||
The goal for regression testing is to try to break code.
|
||||
This leads to a few guidelines to be followed:
|
||||
|
||||
\begin{itemize}
|
||||
\item The testing suite should exercise all classes, functions, and
|
||||
constants.
|
||||
This includes not just the external API that is to be presented to the
|
||||
outside world but also "private" code.
|
||||
\item Whitebox testing (examining the code being tested when the tests are
|
||||
being written) is preferred.
|
||||
Blackbox testing (testing only the published user interface) is not
|
||||
complete enough to make sure all boundary and edge cases are tested.
|
||||
\item Make sure all possible values are tested including invalid ones.
|
||||
This makes sure that not only all valid values are acceptable but also
|
||||
that improper values are handled correctly.
|
||||
\item Exhaust as many code paths as possible.
|
||||
Test where branching occurs and thus tailor input to make sure as many
|
||||
different paths through the code are taken.
|
||||
\item Add an explicit test for any bugs discovered for the tested code.
|
||||
This will make sure that the error does not crop up again if the code is
|
||||
changed in the future.
|
||||
\item Make sure to clean up after your tests (such as close and remove all
|
||||
temporary files).
|
||||
\item Import as few modules as possible and do it as soon as possible.
|
||||
This minimizes external dependencies of tests and also minimizes possible
|
||||
anomalous behavior from side-effects of importing a module.
|
||||
\item Try to maximize code reuse.
|
||||
On occasion tests will vary by something as small as what type of input
|
||||
they take.
|
||||
Minimize code duplication by subclassing a basic test class with a class
|
||||
that specifies the input:
|
||||
\begin{verbatim}
|
||||
class TestFuncAcceptsSequences(unittest.TestCase):
|
||||
|
||||
func = mySuperWhammyFunction
|
||||
|
||||
def test_func(self):
|
||||
self.func(self.arg)
|
||||
|
||||
class AcceptLists(TestFuncAcceptsSequences):
|
||||
arg = [1,2,3]
|
||||
|
||||
class AcceptStrings(TestFuncAcceptsSequences):
|
||||
arg = 'abc'
|
||||
|
||||
class AcceptTuples(TestFuncAcceptsSequences):
|
||||
arg = (1,2,3)
|
||||
\end{verbatim}
|
||||
\end{itemize}
|
||||
|
||||
\begin{seealso}
|
||||
\seetitle{Test Driven Development}{A book by Kent Beck on writing tests before
|
||||
code}
|
||||
\end{seealso}
|
||||
|
||||
|
||||
|
||||
\subsection{Running tests Using \module{regrtest.py} \label{regrtest}}
|
||||
|
||||
\module{regrtest.py} is the script used to drive Python's regression test
|
||||
suite.
|
||||
Running the script by itself automatically starts running all
|
||||
regression tests in the \module{test} package.
|
||||
It does this by finding all modules in the package whose name starts with
|
||||
\code{test_}, importing them, and executing the function \function{test_main}
|
||||
if present.
|
||||
The names of tests to execute may also be passed to the script.
|
||||
Specifying a single regression test (\code{python regrtest.py test_spam.py})
|
||||
will minimize output and only print whether the test passed or failed and thus
|
||||
minimize output.
|
||||
|
||||
Running \module{regrtest.py} directly allows what resources are
|
||||
available for tests to use to be set.
|
||||
You do this by using the \code{-u} command-line option.
|
||||
Run \code{python regrtest.py -uall} to turn on all resources;
|
||||
specifying \code{all} as an option for \code{-u} enables all possible
|
||||
resources.
|
||||
If all but one resource is desired (a more common case), a
|
||||
comma-separated list of resources that are not desired may be listed after
|
||||
\code{all}.
|
||||
The command \code{python regrtest.py -uall,-audio,-largefile} will run
|
||||
\module{regrtest.py} with all resources except the audio and largefile
|
||||
resources.
|
||||
For a list of all resources and more command-line options, run
|
||||
\code{python regrtest.py -h}.
|
||||
|
||||
Some other ways to execute the regression tests depend on what platform the
|
||||
tests are being executed on.
|
||||
On \UNIX{}, you can run \code{make test} at the top-level directory
|
||||
where Python was built.
|
||||
On Windows, executing \code{rt.bat} from your PCBuild directory will run all
|
||||
regression tests.
|
||||
|
Loading…
Reference in a new issue