mirror of
https://github.com/python/cpython
synced 2024-09-16 00:27:20 +00:00
c5bace2bf7
Add basic fuzz tests for a few common builtin functions. This is an easy place to start, and these functions are probably safe. We'll want to add more fuzz tests later. Lets bootstrap using these. While the fuzz tests are included in CPython and compiled / tested on a very basic level inside CPython itself, the actual fuzzing happens as part of oss-fuzz (https://github.com/google/oss-fuzz). The reason to include the tests in CPython is to make sure that they're maintained as part of the CPython project, especially when (as some eventually will) they use internal implementation details in the test. (This will be necessary sometimes because e.g. the fuzz test should never enter Python's interpreter loop, whereas some APIs only expose themselves publicly as Python functions.) This particular set of changes is part of testing Python's builtins, tracked internally at Google by b/37562550. The _xxtestfuzz module that this change adds need not be shipped with binary distributions of Python. |
||
|---|---|---|
| .. | ||
| _xxtestfuzz.c | ||
| fuzz_tests.txt | ||
| fuzzer.c | ||
| README.rst | ||
Fuzz Tests for CPython
======================
These fuzz tests are designed to be included in Google's `oss-fuzz`_ project.
oss-fuzz works against a library exposing a function of the form
``int LLVMFuzzerTestOneInput(const uint8_t* data, size_t length)``. We provide
that library (``fuzzer.c``), and include a ``_fuzz`` module for testing with
some toy values -- no fuzzing occurs in Python's test suite.
oss-fuzz will regularly pull from CPython, discover all the tests in
``fuzz_tests.txt``, and run them -- so adding a new test here means it will
automatically be run in oss-fuzz, while also being smoke-tested as part of
CPython's test suite.
Adding a new fuzz test
----------------------
Add the test name on a new line in ``fuzz_tests.txt``.
In ``fuzzer.c``, add a function to be run::
int $test_name (const char* data, size_t size) {
...
return 0;
}
And invoke it from ``LLVMFuzzerTestOneInput``::
#if _Py_FUZZ_YES(fuzz_builtin_float)
rv |= _run_fuzz(data, size, fuzz_builtin_float);
#endif
``LLVMFuzzerTestOneInput`` will run in oss-fuzz, with each test in
``fuzz_tests.txt`` run separately.
What makes a good fuzz test
---------------------------
Libraries written in C that might handle untrusted data are worthwhile. The
more complex the logic (e.g. parsing), the more likely this is to be a useful
fuzz test. See the existing examples for reference, and refer to the
`oss-fuzz`_ docs.
.. _oss-fuzz: https://github.com/google/oss-fuzz