mirror of
https://github.com/python/cpython
synced 2024-10-06 17:05:01 +00:00
360e4b8fb1
svn+ssh://pythondev@svn.python.org/python/branches/p3yk ................ r55326 | guido.van.rossum | 2007-05-14 15:07:35 -0700 (Mon, 14 May 2007) | 2 lines Don't use err.message, use err.args[0]. ................ r55327 | guido.van.rossum | 2007-05-14 15:11:37 -0700 (Mon, 14 May 2007) | 259 lines Merged revisions 54988-55226,55228-55323 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r54995 | neal.norwitz | 2007-04-26 23:45:32 -0700 (Thu, 26 Apr 2007) | 3 lines This gets the test working on Solaris. It seems a little hokey to me, but the test passed on Linux and Solaris, hopefully other platforms too. ........ r55002 | georg.brandl | 2007-04-27 12:20:00 -0700 (Fri, 27 Apr 2007) | 2 lines Version fix (bug #1708710) ........ r55021 | neal.norwitz | 2007-04-29 16:53:24 -0700 (Sun, 29 Apr 2007) | 1 line There really are some tests that are problematic. ........ r55024 | kristjan.jonsson | 2007-04-30 08:17:46 -0700 (Mon, 30 Apr 2007) | 1 line Complete revamp of PCBuild8 directory. Use subdirectories for each project under the main pcbuild solution. Now make extensive use of property sheets to simplify project configuration. x64 build fully supported, and the process for building PGO version (Profiler Guided Optimization) simplified. All projects are now present, except _ssl, which needs to be reimplemented. Also, some of the projects that require external libraries need extra work to fully compile on x64. ........ r55025 | thomas.heller | 2007-04-30 08:44:17 -0700 (Mon, 30 Apr 2007) | 4 lines Make sure to call PyErr_NoMemory() in several places where PyMem_Malloc() could potentially fail. Will backport to the release25-maint branch. ........ r55027 | thomas.heller | 2007-04-30 09:04:57 -0700 (Mon, 30 Apr 2007) | 8 lines When accessing the .value attribute of a c_wchar_p instance, and the instance does not point to a valid wchar_t zero-terminated string, raise a ValueError. c_char_p does this already. The ValueError message now contains the correct pointer address. Will backport to release25-maint. ........ r55036 | georg.brandl | 2007-04-30 23:04:11 -0700 (Mon, 30 Apr 2007) | 2 lines Bug #1710295: exceptions are now new-style classes. ........ r55038 | georg.brandl | 2007-04-30 23:08:15 -0700 (Mon, 30 Apr 2007) | 2 lines Patch #1710352: add missing public functions to locale.__all__. ........ r55041 | vinay.sajip | 2007-05-01 03:20:03 -0700 (Tue, 01 May 2007) | 1 line Added new optional credentials argument to SMTPHandler.__init__, and smtp.login() is now called in SMTPHandler.emit() if credentials are specified. ........ r55042 | vinay.sajip | 2007-05-01 03:21:45 -0700 (Tue, 01 May 2007) | 1 line Added documentation for new optional credentials argument to SMTPHandler.__init__(). ........ r55070 | neal.norwitz | 2007-05-01 21:47:55 -0700 (Tue, 01 May 2007) | 3 lines Stop using PyMem_FREE while the GIL is not held. For details see: http://mail.python.org/pipermail/python-dev/2007-May/072896.html ........ r55080 | armin.rigo | 2007-05-02 12:23:31 -0700 (Wed, 02 May 2007) | 6 lines Fix for #1303614 and #1174712: - __dict__ descriptor abuse for subclasses of built-in types - subclassing from both ModuleType and another built-in types Thanks zseil for the patch. ........ r55083 | georg.brandl | 2007-05-02 13:02:29 -0700 (Wed, 02 May 2007) | 3 lines Actually raise an exception before calling ast_error_finish. Triggers an assertion otherwise. ........ r55087 | neal.norwitz | 2007-05-02 23:47:18 -0700 (Wed, 02 May 2007) | 1 line Handle a couple of uncaught errors. This should be backported ........ r55090 | neal.norwitz | 2007-05-03 00:20:57 -0700 (Thu, 03 May 2007) | 4 lines Remove dead code. This code couldn't be reached because earlier in the function there is another check for z != Py_None. ........ r55092 | thomas.heller | 2007-05-03 05:02:08 -0700 (Thu, 03 May 2007) | 1 line Fix building _ctypes.pyd for x64 / Windows. ........ r55093 | thomas.heller | 2007-05-03 05:05:20 -0700 (Thu, 03 May 2007) | 1 line Don't truncate pointers to integers (on win64 platform). ........ r55094 | walter.doerwald | 2007-05-03 08:13:55 -0700 (Thu, 03 May 2007) | 3 lines Clarify the behaviour of PyUnicode_DecodeUTF16(): A BOM is only skipped in native order mode, and only if it's the first two bytes. ........ r55101 | kristjan.jonsson | 2007-05-03 13:04:53 -0700 (Thu, 03 May 2007) | 2 lines Fix pcbuild8 after recent overhaul: Added the version resource to python26.dll. Adjust stacksize to 2Mb and made large address aware for 32 bits, and set stacksize to 3Mb for 64 bits. Todo: Set .dll optimized load addresses, and side-by-side packaging of the python26.dll. ........ r55102 | kristjan.jonsson | 2007-05-03 13:09:56 -0700 (Thu, 03 May 2007) | 1 line Fix those parts in the testsuite that assumed that sys.maxint would cause overflow on x64. Now the testsuite is well behaved on that platform. ........ r55103 | kristjan.jonsson | 2007-05-03 13:27:03 -0700 (Thu, 03 May 2007) | 11 lines Fix problems in x64 build that were discovered by the testsuite: - Reenable modules on x64 that had been disabled aeons ago for Itanium. - Cleared up confusion about compilers for 64 bit windows. There is only Itanium and x64. Added macros MS_WINI64 and MS_WINX64 for those rare cases where it matters, such as the disabling of modules above. - Set target platform (_WIN32_WINNT and WINVER) to 0x0501 (XP) for x64, and 0x0400 (NT 4.0) otherwise, which are the targeted minimum platforms. - Fixed thread_nt.h. The emulated InterlockedCompareExchange function didn?\194?\180t work on x64, probaby due to the lack of a "volatile" specifier. Anyway, win95 is no longer a target platform. - Itertools module used wrong constant to check for overflow in count() - PyInt_AsSsize_t couldn't deal with attribute error when accessing the __long__ member. - PyLong_FromSsize_t() incorrectly specified that the operand were unsigned. With these changes, the x64 passes the testsuite, for those modules present. ........ r55107 | kristjan.jonsson | 2007-05-03 17:25:08 -0700 (Thu, 03 May 2007) | 1 line Revert compiler comment to AMD64 for x64/AMD64 builds. ........ r55115 | thomas.heller | 2007-05-04 00:14:39 -0700 (Fri, 04 May 2007) | 4 lines Fix some ctypes test crashes, when running with a debug Python version on win64 by using proper argtypes and restype function attributes. ........ r55117 | thomas.heller | 2007-05-04 01:20:41 -0700 (Fri, 04 May 2007) | 4 lines On 64-bit Windows, ffi_arg must be 8 bytes long. This fixes the remaining crashes in the ctypes tests, when functions return float or double types. ........ r55120 | kristjan.jonsson | 2007-05-04 08:48:15 -0700 (Fri, 04 May 2007) | 1 line Update the pcbuild8 solution. Straightened out the _ctypes project by using a .vsproj file and a masm64.rules file to avoid redundancy ........ r55121 | kristjan.jonsson | 2007-05-04 10:28:06 -0700 (Fri, 04 May 2007) | 1 line Minor fix of PCBuild8/_ctypes vcproj, moving include dir into the .vsprops file. ........ r55129 | thomas.heller | 2007-05-04 12:54:22 -0700 (Fri, 04 May 2007) | 3 lines Do not truncate 64-bit pointers to 32-bit integers. Fixes SF #1703286, will backport to release25-maint. ........ r55131 | thomas.heller | 2007-05-04 12:56:32 -0700 (Fri, 04 May 2007) | 1 line Oops, these tests do not run on Windows CE. ........ r55140 | brett.cannon | 2007-05-04 18:34:02 -0700 (Fri, 04 May 2007) | 2 lines Deprecate BaseException.message as per PEP 352. ........ r55154 | georg.brandl | 2007-05-05 11:55:37 -0700 (Sat, 05 May 2007) | 2 lines Bug #1713535: typo in logging example. ........ r55158 | vinay.sajip | 2007-05-06 10:53:37 -0700 (Sun, 06 May 2007) | 1 line Updates of recent changes to logging. ........ r55165 | neal.norwitz | 2007-05-07 00:02:26 -0700 (Mon, 07 May 2007) | 1 line Verify changes to the trunk go to the normal checkins list ........ r55169 | kristjan.jonsson | 2007-05-07 09:46:54 -0700 (Mon, 07 May 2007) | 1 line As per Armin Rigo's suggestion, remove special handing from intobject.c to deal with the peculiarities of classobject's implementation of the number protocol. The nb_long method of classobject now falls back to nb_int if there is no __long__ attribute present. ........ r55197 | collin.winter | 2007-05-08 21:14:36 -0700 (Tue, 08 May 2007) | 9 lines Fix a bug in test.test_support.open_urlresource(). If the call to requires() doesn't precede the filesystem check, we get the following situation: 1. ./python Lib/test/regrtest.py test_foo # test needs urlfetch, not enabled, so skipped 2. ./python Lib/test/regrtest.py -u urlfetch test_foo # test runs 3. ./python Lib/test/regrtest.py test_foo # test runs (!) By moving the call to requires() *before* the filesystem check, the fact that fetched files are cached on the local disk becomes an implementation detail, rather than a semantics-changing point of note. ........ r55198 | neal.norwitz | 2007-05-08 23:43:15 -0700 (Tue, 08 May 2007) | 1 line Add markup for True/False. Will backport ........ r55205 | walter.doerwald | 2007-05-09 11:10:47 -0700 (Wed, 09 May 2007) | 4 lines Backport checkin: Fix a segfault when b"" was passed to b2a_qp() -- it was using strchr() instead of memchr(). ........ r55241 | neal.norwitz | 2007-05-10 22:55:15 -0700 (Thu, 10 May 2007) | 6 lines Don't ever report a failure when the sum of the reference count differences are zero. This should help reduce the false positives. The message about references leaking is maintained to provide as much info as possible rather than simply suppressing the message at the source. ........ r55242 | neal.norwitz | 2007-05-10 23:23:01 -0700 (Thu, 10 May 2007) | 1 line Fix typo in docstring (the module is popen2, not 3). ........ r55244 | neal.norwitz | 2007-05-10 23:56:52 -0700 (Thu, 10 May 2007) | 1 line Remove trailing whitespace in docstring ........ r55245 | neal.norwitz | 2007-05-10 23:57:33 -0700 (Thu, 10 May 2007) | 1 line Deprecate os.popen* and popen2 module in favor of the subprocess module. ........ r55247 | neal.norwitz | 2007-05-11 00:13:30 -0700 (Fri, 11 May 2007) | 1 line Deprecate os.popen* and popen2 module in favor of the subprocess module. (forgot the doc) ........ r55253 | georg.brandl | 2007-05-11 02:41:37 -0700 (Fri, 11 May 2007) | 3 lines Remove an XXX that is unnecessary. ........ r55258 | georg.brandl | 2007-05-11 04:04:26 -0700 (Fri, 11 May 2007) | 2 lines Patch #1714700: clarify os.linesep vs. tfiles opened in text mode. (backport) ........ r55259 | georg.brandl | 2007-05-11 04:43:56 -0700 (Fri, 11 May 2007) | 2 lines Update DDJ link. ........ r55273 | raymond.hettinger | 2007-05-11 10:59:59 -0700 (Fri, 11 May 2007) | 1 line Better tests for posixpath.commonprefix ........ r55287 | georg.brandl | 2007-05-12 14:06:41 -0700 (Sat, 12 May 2007) | 2 lines Bug #1046945: document SWIG options of distutils. ........ r55290 | georg.brandl | 2007-05-13 01:04:07 -0700 (Sun, 13 May 2007) | 2 lines Add bz2 to content encodings. ........ r55297 | neal.norwitz | 2007-05-13 13:45:05 -0700 (Sun, 13 May 2007) | 3 lines Remove Amoeba doc which was removed in version 1.0! according to Misc/HISTORY. Hopefully Guido won't shed a tear. :-) ........ r55298 | neal.norwitz | 2007-05-13 13:54:19 -0700 (Sun, 13 May 2007) | 1 line Remove references to stdwin which was removed long ago. ........ r55299 | neal.norwitz | 2007-05-13 14:13:42 -0700 (Sun, 13 May 2007) | 3 lines Remove support for freebsd[23] which haven't been released since 2000 or earlier. http://www.freebsd.org/releases/index.html ........ r55320 | raymond.hettinger | 2007-05-14 13:52:31 -0700 (Mon, 14 May 2007) | 1 line Small speedup. ........ ................ |
||
---|---|---|
.. | ||
crashers | ||
decimaltestdata | ||
leakers | ||
output | ||
185test.db | ||
__init__.py | ||
audiotest.au | ||
autotest.py | ||
bad_coding.py | ||
bad_coding2.py | ||
badsyntax_future3.py | ||
badsyntax_future4.py | ||
badsyntax_future5.py | ||
badsyntax_future6.py | ||
badsyntax_future7.py | ||
badsyntax_future8.py | ||
badsyntax_future9.py | ||
badsyntax_nocaret.py | ||
cfgparser.1 | ||
check_soundcard.vbs | ||
cjkencodings_test.py | ||
doctest_aliases.py | ||
double_const.py | ||
empty.vbs | ||
exception_hierarchy.txt | ||
fork_wait.py | ||
greyrgb.uue | ||
infinite_reload.py | ||
inspect_fodder.py | ||
inspect_fodder2.py | ||
list_tests.py | ||
mapping_tests.py | ||
outstanding_bugs.py | ||
pickletester.py | ||
pyclbr_input.py | ||
pydocfodder.py | ||
pystone.py | ||
re_tests.py | ||
README | ||
regex_tests.py | ||
regrtest.py | ||
reperf.py | ||
sample_doctest.py | ||
seq_tests.py | ||
sgml_input.html | ||
sortperf.py | ||
ssl_cert.pem | ||
ssl_key.pem | ||
string_tests.py | ||
test.xml | ||
test.xml.out | ||
test___all__.py | ||
test___future__.py | ||
test__locale.py | ||
test_aepack.py | ||
test_anydbm.py | ||
test_applesingle.py | ||
test_array.py | ||
test_ast.py | ||
test_asynchat.py | ||
test_atexit.py | ||
test_audioop.py | ||
test_augassign.py | ||
test_base64.py | ||
test_bigaddrspace.py | ||
test_bigmem.py | ||
test_binascii.py | ||
test_binhex.py | ||
test_binop.py | ||
test_bisect.py | ||
test_bool.py | ||
test_bsddb.py | ||
test_bsddb3.py | ||
test_buffer.py | ||
test_bufio.py | ||
test_builtin.py | ||
test_bytes.py | ||
test_bz2.py | ||
test_calendar.py | ||
test_call.py | ||
test_capi.py | ||
test_cfgparser.py | ||
test_cgi.py | ||
test_charmapcodec.py | ||
test_class.py | ||
test_cmath.py | ||
test_cmd_line.py | ||
test_code.py | ||
test_codeccallbacks.py | ||
test_codecencodings_cn.py | ||
test_codecencodings_hk.py | ||
test_codecencodings_jp.py | ||
test_codecencodings_kr.py | ||
test_codecencodings_tw.py | ||
test_codecmaps_cn.py | ||
test_codecmaps_hk.py | ||
test_codecmaps_jp.py | ||
test_codecmaps_kr.py | ||
test_codecmaps_tw.py | ||
test_codecs.py | ||
test_codeop.py | ||
test_coding.py | ||
test_collections.py | ||
test_colorsys.py | ||
test_commands.py | ||
test_compare.py | ||
test_compile.py | ||
test_complex.py | ||
test_complex_args.py | ||
test_contains.py | ||
test_contextlib.py | ||
test_cookie.py | ||
test_cookielib.py | ||
test_copy.py | ||
test_copy_reg.py | ||
test_cpickle.py | ||
test_cProfile.py | ||
test_crypt.py | ||
test_csv.py | ||
test_ctypes.py | ||
test_curses.py | ||
test_datetime.py | ||
test_dbm.py | ||
test_decimal.py | ||
test_decorators.py | ||
test_defaultdict.py | ||
test_deque.py | ||
test_descr.py | ||
test_descrtut.py | ||
test_dict.py | ||
test_dictviews.py | ||
test_difflib.py | ||
test_difflib_expect.html | ||
test_dircache.py | ||
test_dis.py | ||
test_distutils.py | ||
test_dl.py | ||
test_doctest.py | ||
test_doctest.txt | ||
test_doctest2.py | ||
test_doctest2.txt | ||
test_doctest3.txt | ||
test_doctest4.txt | ||
test_dumbdbm.py | ||
test_dummy_thread.py | ||
test_dummy_threading.py | ||
test_email.py | ||
test_email_codecs.py | ||
test_email_renamed.py | ||
test_enumerate.py | ||
test_eof.py | ||
test_errno.py | ||
test_exception_variations.py | ||
test_exceptions.py | ||
test_extcall.py | ||
test_fcntl.py | ||
test_file.py | ||
test_filecmp.py | ||
test_fileinput.py | ||
test_fileio.py | ||
test_float.py | ||
test_fnmatch.py | ||
test_fork1.py | ||
test_format.py | ||
test_fpformat.py | ||
test_frozen.py | ||
test_ftplib.py | ||
test_funcattrs.py | ||
test_functools.py | ||
test_future.py | ||
test_future1.py | ||
test_future2.py | ||
test_future3.py | ||
test_gc.py | ||
test_gdbm.py | ||
test_generators.py | ||
test_genericpath.py | ||
test_genexps.py | ||
test_getargs.py | ||
test_getargs2.py | ||
test_getopt.py | ||
test_gettext.py | ||
test_glob.py | ||
test_global.py | ||
test_grammar.py | ||
test_grp.py | ||
test_gzip.py | ||
test_hash.py | ||
test_hashlib.py | ||
test_heapq.py | ||
test_hexoct.py | ||
test_hmac.py | ||
test_hotshot.py | ||
test_htmllib.py | ||
test_htmlparser.py | ||
test_httplib.py | ||
test_imageop.py | ||
test_imaplib.py | ||
test_imp.py | ||
test_import.py | ||
test_importhooks.py | ||
test_index.py | ||
test_inspect.py | ||
test_io.py | ||
test_ioctl.py | ||
test_isinstance.py | ||
test_iter.py | ||
test_iterlen.py | ||
test_itertools.py | ||
test_keywordonlyarg.py | ||
test_largefile.py | ||
test_linuxaudiodev.py | ||
test_list.py | ||
test_listcomps.py | ||
test_locale.py | ||
test_logging.py | ||
test_long.py | ||
test_long_future.py | ||
test_longexp.py | ||
test_macfs.py | ||
test_macostools.py | ||
test_macpath.py | ||
test_mailbox.py | ||
test_marshal.py | ||
test_math.py | ||
test_metaclass.py | ||
test_mhlib.py | ||
test_mimetools.py | ||
test_mimetypes.py | ||
test_MimeWriter.py | ||
test_minidom.py | ||
test_mmap.py | ||
test_module.py | ||
test_modulefinder.py | ||
test_multibytecodec.py | ||
test_multibytecodec_support.py | ||
test_multifile.py | ||
test_mutants.py | ||
test_netrc.py | ||
test_new.py | ||
test_nis.py | ||
test_normalization.py | ||
test_ntpath.py | ||
test_old_mailbox.py | ||
test_opcodes.py | ||
test_openpty.py | ||
test_operator.py | ||
test_optparse.py | ||
test_os.py | ||
test_ossaudiodev.py | ||
test_parser.py | ||
test_peepholer.py | ||
test_pep247.py | ||
test_pep263.py | ||
test_pep277.py | ||
test_pep292.py | ||
test_pep352.py | ||
test_pickle.py | ||
test_pickletools.py | ||
test_pkg.py | ||
test_pkgimport.py | ||
test_platform.py | ||
test_plistlib.py | ||
test_poll.py | ||
test_popen.py | ||
test_popen2.py | ||
test_poplib.py | ||
test_posix.py | ||
test_posixpath.py | ||
test_pow.py | ||
test_pprint.py | ||
test_profile.py | ||
test_profilehooks.py | ||
test_pty.py | ||
test_pwd.py | ||
test_pyclbr.py | ||
test_pyexpat.py | ||
test_queue.py | ||
test_quopri.py | ||
test_random.py | ||
test_re.py | ||
test_repr.py | ||
test_resource.py | ||
test_rfc822.py | ||
test_rgbimg.py | ||
test_richcmp.py | ||
test_robotparser.py | ||
test_runpy.py | ||
test_sax.py | ||
test_scope.py | ||
test_scriptpackages.py | ||
test_select.py | ||
test_set.py | ||
test_setcomps.py | ||
test_sgmllib.py | ||
test_shelve.py | ||
test_shlex.py | ||
test_shutil.py | ||
test_signal.py | ||
test_site.py | ||
test_slice.py | ||
test_smtplib.py | ||
test_socket.py | ||
test_socket_ssl.py | ||
test_socketserver.py | ||
test_sort.py | ||
test_sqlite.py | ||
test_startfile.py | ||
test_str.py | ||
test_strftime.py | ||
test_string.py | ||
test_StringIO.py | ||
test_stringprep.py | ||
test_strop.py | ||
test_strptime.py | ||
test_struct.py | ||
test_structmembers.py | ||
test_structseq.py | ||
test_subprocess.py | ||
test_sunaudiodev.py | ||
test_sundry.py | ||
test_support.py | ||
test_symtable.py | ||
test_syntax.py | ||
test_sys.py | ||
test_tarfile.py | ||
test_tcl.py | ||
test_telnetlib.py | ||
test_tempfile.py | ||
test_textwrap.py | ||
test_thread.py | ||
test_threaded_import.py | ||
test_threadedtempfile.py | ||
test_threading.py | ||
test_threading_local.py | ||
test_threadsignals.py | ||
test_time.py | ||
test_timeout.py | ||
test_tokenize.py | ||
test_trace.py | ||
test_traceback.py | ||
test_tuple.py | ||
test_types.py | ||
test_ucn.py | ||
test_unary.py | ||
test_unicode.py | ||
test_unicode_file.py | ||
test_unicodedata.py | ||
test_unittest.py | ||
test_univnewlines.py | ||
test_unpack.py | ||
test_unpack_ex.py | ||
test_urllib.py | ||
test_urllib2.py | ||
test_urllib2net.py | ||
test_urllibnet.py | ||
test_urlparse.py | ||
test_userdict.py | ||
test_userlist.py | ||
test_userstring.py | ||
test_uu.py | ||
test_uuid.py | ||
test_wait3.py | ||
test_wait4.py | ||
test_warnings.py | ||
test_wave.py | ||
test_weakref.py | ||
test_whichdb.py | ||
test_winreg.py | ||
test_winsound.py | ||
test_with.py | ||
test_wsgiref.py | ||
test_xdrlib.py | ||
test_xml_etree.py | ||
test_xml_etree_c.py | ||
test_xmlrpc.py | ||
test_xpickle.py | ||
test_xrange.py | ||
test_zipfile.py | ||
test_zipfile64.py | ||
test_zipimport.py | ||
test_zlib.py | ||
testall.py | ||
testcodec.py | ||
testimg.uue | ||
testimgr.uue | ||
testrgb.uue | ||
testtar.tar | ||
tf_inherit_check.py | ||
threaded_import_hangers.py | ||
time_hashlib.py | ||
tokenize_tests.txt | ||
warning_tests.py | ||
xmltests.py |
+++++++++++++++++++++++++++++++ Writing Python Regression Tests +++++++++++++++++++++++++++++++ :Author: Skip Montanaro :Contact: skip@mojam.com Introduction ============ If you add a new module to Python or modify the functionality of an existing module, you should write one or more test cases to exercise that new functionality. There are different ways to do this within the regression testing facility provided with Python; any particular test should use only one of these options. Each option requires writing a test module using the conventions of the selected option: - unittest_ based tests - doctest_ based tests - "traditional" Python test modules Regardless of the mechanics of the testing approach you choose, you will be writing unit tests (isolated tests of functions and objects defined by the module) using white box techniques. Unlike black box testing, where you only have the external interfaces to guide your test case writing, in white box testing you can see the code being tested and tailor your test cases to exercise it more completely. In particular, you will be able to refer to the C and Python code in the CVS repository when writing your regression test cases. .. _unittest: http://www.python.org/doc/current/lib/module-unittest.html .. _doctest: http://www.python.org/doc/current/lib/module-doctest.html unittest-based tests ------------------ The unittest_ framework is based on the ideas of unit testing as espoused by Kent Beck and the `Extreme Programming`_ (XP) movement. The specific interface provided by the framework is tightly based on the JUnit_ Java implementation of Beck's original SmallTalk test framework. Please see the documentation of the unittest_ module for detailed information on the interface and general guidelines on writing unittest-based tests. The test_support helper module provides a function for use by unittest-based tests in the Python regression testing framework, ``run_unittest()``. This is the primary way of running tests in the standard library. You can pass it any number of the following: - classes derived from or instances of ``unittest.TestCase`` or ``unittest.TestSuite``. These will be handed off to unittest for converting into a proper TestSuite instance. - a string; this must be a key in sys.modules. The module associated with that string will be scanned by ``unittest.TestLoader.loadTestsFromModule``. This is usually seen as ``test_support.run_unittest(__name__)`` in a test module's ``test_main()`` function. This has the advantage of picking up new tests automatically, without you having to add each new test case manually. All test methods in the Python regression framework have names that start with "``test_``" and use lower-case names with words separated with underscores. Test methods should *not* have docstrings! The unittest module prints the docstring if there is one, but otherwise prints the function name and the full class name. When there's a problem with a test, the latter information makes it easier to find the source for the test than the docstring. All unittest-based tests in the Python test suite use boilerplate that looks like this (with minor variations):: import unittest from test import test_support class MyTestCase1(unittest.TestCase): # Define setUp and tearDown only if needed def setUp(self): unittest.TestCase.setUp(self) ... additional initialization... def tearDown(self): ... additional finalization... unittest.TestCase.tearDown(self) def test_feature_one(self): # Testing feature one ...unit test for feature one... def test_feature_two(self): # Testing feature two ...unit test for feature two... ...etc... class MyTestCase2(unittest.TestCase): ...same structure as MyTestCase1... ...etc... def test_main(): test_support.run_unittest(__name__) if __name__ == "__main__": test_main() This has the advantage that it allows the unittest module to be used as a script to run individual tests as well as working well with the regrtest framework. .. _Extreme Programming: http://www.extremeprogramming.org/ .. _JUnit: http://www.junit.org/ doctest based tests ------------------- Tests written to use doctest_ are actually part of the docstrings for the module being tested. Each test is written as a display of an interactive session, including the Python prompts, statements that would be typed by the user, and the output of those statements (including tracebacks, although only the exception msg needs to be retained then). The module in the test package is simply a wrapper that causes doctest to run over the tests in the module. The test for the difflib module provides a convenient example:: import difflib from test import test_support test_support.run_doctest(difflib) If the test is successful, nothing is written to stdout (so you should not create a corresponding output/test_difflib file), but running regrtest with -v will give a detailed report, the same as if passing -v to doctest. A second argument can be passed to run_doctest to tell doctest to search ``sys.argv`` for -v instead of using test_support's idea of verbosity. This is useful for writing doctest-based tests that aren't simply running a doctest'ed Lib module, but contain the doctests themselves. Then at times you may want to run such a test directly as a doctest, independent of the regrtest framework. The tail end of test_descrtut.py is a good example:: def test_main(verbose=None): from test import test_support, test_descrtut test_support.run_doctest(test_descrtut, verbose) if __name__ == "__main__": test_main(1) If run via regrtest, ``test_main()`` is called (by regrtest) without specifying verbose, and then test_support's idea of verbosity is used. But when run directly, ``test_main(1)`` is called, and then doctest's idea of verbosity is used. See the documentation for the doctest module for information on writing tests using the doctest framework. "traditional" Python test modules --------------------------------- The mechanics of how the "traditional" test system operates are fairly straightforward. When a test case is run, the output is compared with the expected output that is stored in .../Lib/test/output. If the test runs to completion and the actual and expected outputs match, the test succeeds, if not, it fails. If an ``ImportError`` or ``test_support.TestSkipped`` error is raised, the test is not run. Executing Test Cases ==================== If you are writing test cases for module spam, you need to create a file in .../Lib/test named test_spam.py. In addition, if the tests are expected to write to stdout during a successful run, you also need to create an expected output file in .../Lib/test/output named test_spam ("..." represents the top-level directory in the Python source tree, the directory containing the configure script). If needed, generate the initial version of the test output file by executing:: ./python Lib/test/regrtest.py -g test_spam.py from the top-level directory. Any time you modify test_spam.py you need to generate a new expected output file. Don't forget to desk check the generated output to make sure it's really what you expected to find! All in all it's usually better not to have an expected-out file (note that doctest- and unittest-based tests do not). To run a single test after modifying a module, simply run regrtest.py without the -g flag:: ./python Lib/test/regrtest.py test_spam.py While debugging a regression test, you can of course execute it independently of the regression testing framework and see what it prints:: ./python Lib/test/test_spam.py To run the entire test suite: - [UNIX, + other platforms where "make" works] Make the "test" target at the top level:: make test - [WINDOWS] Run rt.bat from your PCBuild directory. Read the comments at the top of rt.bat for the use of special -d, -O and -q options processed by rt.bat. - [OTHER] You can simply execute the two runs of regrtest (optimized and non-optimized) directly:: ./python Lib/test/regrtest.py ./python -O Lib/test/regrtest.py But note that this way picks up whatever .pyc and .pyo files happen to be around. The makefile and rt.bat ways run the tests twice, the first time removing all .pyc and .pyo files from the subtree rooted at Lib/. Test cases generate output based upon values computed by the test code. When executed, regrtest.py compares the actual output generated by executing the test case with the expected output and reports success or failure. It stands to reason that if the actual and expected outputs are to match, they must not contain any machine dependencies. This means your test cases should not print out absolute machine addresses (e.g. the return value of the id() builtin function) or floating point numbers with large numbers of significant digits (unless you understand what you are doing!). Test Case Writing Tips ====================== Writing good test cases is a skilled task and is too complex to discuss in detail in this short document. Many books have been written on the subject. I'll show my age by suggesting that Glenford Myers' `"The Art of Software Testing"`_, published in 1979, is still the best introduction to the subject available. It is short (177 pages), easy to read, and discusses the major elements of software testing, though its publication predates the object-oriented software revolution, so doesn't cover that subject at all. Unfortunately, it is very expensive (about $100 new). If you can borrow it or find it used (around $20), I strongly urge you to pick up a copy. The most important goal when writing test cases is to break things. A test case that doesn't uncover a bug is much less valuable than one that does. In designing test cases you should pay attention to the following: * Your test cases should exercise all the functions and objects defined in the module, not just the ones meant to be called by users of your module. This may require you to write test code that uses the module in ways you don't expect (explicitly calling internal functions, for example - see test_atexit.py). * You should consider any boundary values that may tickle exceptional conditions (e.g. if you were writing regression tests for division, you might well want to generate tests with numerators and denominators at the limits of floating point and integer numbers on the machine performing the tests as well as a denominator of zero). * You should exercise as many paths through the code as possible. This may not always be possible, but is a goal to strive for. In particular, when considering if statements (or their equivalent), you want to create test cases that exercise both the true and false branches. For loops, you should create test cases that exercise the loop zero, one and multiple times. * You should test with obviously invalid input. If you know that a function requires an integer input, try calling it with other types of objects to see how it responds. * You should test with obviously out-of-range input. If the domain of a function is only defined for positive integers, try calling it with a negative integer. * If you are going to fix a bug that wasn't uncovered by an existing test, try to write a test case that exposes the bug (preferably before fixing it). * If you need to create a temporary file, you can use the filename in ``test_support.TESTFN`` to do so. It is important to remove the file when done; other tests should be able to use the name without cleaning up after your test. .. _"The Art of Software Testing": http://www.amazon.com/exec/obidos/ISBN=0471043281 Regression Test Writing Rules ============================= Each test case is different. There is no "standard" form for a Python regression test case, though there are some general rules (note that these mostly apply only to the "classic" tests; unittest_- and doctest_- based tests should follow the conventions natural to those frameworks):: * If your test case detects a failure, raise ``TestFailed`` (found in ``test.test_support``). * Import everything you'll need as early as possible. * If you'll be importing objects from a module that is at least partially platform-dependent, only import those objects you need for the current test case to avoid spurious ``ImportError`` exceptions that prevent the test from running to completion. * Print all your test case results using the ``print`` statement. For non-fatal errors, print an error message (or omit a successful completion print) to indicate the failure, but proceed instead of raising ``TestFailed``. * Use ``assert`` sparingly, if at all. It's usually better to just print what you got, and rely on regrtest's got-vs-expected comparison to catch deviations from what you expect. ``assert`` statements aren't executed at all when regrtest is run in -O mode; and, because they cause the test to stop immediately, can lead to a long & tedious test-fix, test-fix, test-fix, ... cycle when things are badly broken (and note that "badly broken" often includes running the test suite for the first time on new platforms or under new implementations of the language). Miscellaneous ============= There is a test_support module in the test package you can import for your test case. Import this module using either:: import test.test_support or:: from test import test_support test_support provides the following useful objects: * ``TestFailed`` - raise this exception when your regression test detects a failure. * ``TestSkipped`` - raise this if the test could not be run because the platform doesn't offer all the required facilities (like large file support), even if all the required modules are available. * ``ResourceDenied`` - this is raised when a test requires a resource that is not available. Primarily used by 'requires'. * ``verbose`` - you can use this variable to control print output. Many modules use it. Search for "verbose" in the test_*.py files to see lots of examples. * ``forget(module_name)`` - attempts to cause Python to "forget" that it loaded a module and erase any PYC files. * ``is_resource_enabled(resource)`` - Returns a boolean based on whether the resource is enabled or not. * ``requires(resource [, msg])`` - if the required resource is not available the ResourceDenied exception is raised. * ``verify(condition, reason='test failed')``. Use this instead of:: assert condition[, reason] ``verify()`` has two advantages over ``assert``: it works even in -O mode, and it raises ``TestFailed`` on failure instead of ``AssertionError``. * ``have_unicode`` - true if Unicode is available, false otherwise. * ``is_jython`` - true if the interpreter is Jython, false otherwise. * ``TESTFN`` - a string that should always be used as the filename when you need to create a temp file. Also use ``try``/``finally`` to ensure that your temp files are deleted before your test completes. Note that you cannot unlink an open file on all operating systems, so also be sure to close temp files before trying to unlink them. * ``sortdict(dict)`` - acts like ``repr(dict.items())``, but sorts the items first. This is important when printing a dict value, because the order of items produced by ``dict.items()`` is not defined by the language. * ``findfile(file)`` - you can call this function to locate a file somewhere along sys.path or in the Lib/test tree - see test_linuxaudiodev.py for an example of its use. * ``fcmp(x,y)`` - you can call this function to compare two floating point numbers when you expect them to only be approximately equal withing a fuzz factor (``test_support.FUZZ``, which defaults to 1e-6). * ``check_syntax_error(testcase, statement)`` - make sure that the statement is *not* correct Python syntax. Python and C statement coverage results are currently available at http://www.musi-cal.com/~skip/python/Python/dist/src/ As of this writing (July, 2000) these results are being generated nightly. You can refer to the summaries and the test coverage output files to see where coverage is adequate or lacking and write test cases to beef up the coverage. Some Non-Obvious regrtest Features ================================== * Automagic test detection: When you create a new test file test_spam.py, you do not need to modify regrtest (or anything else) to advertise its existence. regrtest searches for and runs all modules in the test directory with names of the form test_xxx.py. * Miranda output: If, when running test_spam.py, regrtest does not find an expected-output file test/output/test_spam, regrtest pretends that it did find one, containing the single line test_spam This allows new tests that don't expect to print anything to stdout to not bother creating expected-output files. * Two-stage testing: To run test_spam.py, regrtest imports test_spam as a module. Most tests run to completion as a side-effect of getting imported. After importing test_spam, regrtest also executes ``test_spam.test_main()``, if test_spam has a ``test_main`` attribute. This is rarely required with the "traditional" Python tests, and you shouldn't create a module global with name test_main unless you're specifically exploiting this gimmick. This usage does prove useful with unittest-based tests as well, however; defining a ``test_main()`` which is run by regrtest and a script-stub in the test module ("``if __name__ == '__main__': test_main()``") allows the test to be used like any other Python test and also work with the unittest.py-as-a-script approach, allowing a developer to run specific tests from the command line.