tokei/fuzz/README.md
Michael Macnair 66967a1b9e
Fuzzing (#726)
* Add fuzzing support

See fuzz/README.md for details. Example bug: #725

* improve fuzz docs

* fuzzing: make relevant config part of input

Most of the config is to do with what files to parse, however
one setting - treat_doc_strings_as_comments - does impact parse_from_slice

* fuzzing: improve docs + config clarity

* fuzz/README.md: install instructions + another todo item
2021-05-09 20:05:14 +02:00

1.5 KiB

Fuzzing Tokei

Tokei can be fuzzed using libFuzzer, via cargo-fuzz.

First install cargo-fuzz: cargo install cargo-fuzz.

To launch a fuzzing job: cargo +nightly fuzz run <target> - it will run until you kill it with ctrl-c.

To use multiple cores: cargo +nightly fuzz run <target> --jobs=6

To speed things up (at the expensive of missing bugs that only manifest in larger files): cargo +nightly fuzz run <target> -- -max_len=200

Available fuzz targets:

  • parse_from_slice_panic - checks that all of the LanguageType instances' parse_from_slice function doesn't panic.
  • parse_from_slice_total - checks that the language stats pass a basic test of reporting no more total lines than there are new lines in the file. At the time of writing there are low-hanging bugs here.

With the two parse_from_slice fuzz targets, it makes sense to share a common corpus directory as they have identical input formats, e.g.: cargo +nightly fuzz run parse_from_slice_{panic,total} fuzz/corpus/common

Potential improvements:

  • Build the fuzz harnesses in CI, so they don't rot.
  • Do some coverage analysis to check if we're missing any code we would benefit from fuzzing (once it's integrated into cargo-fuzz)
  • Tighten the parse_from_slice_total fuzz target to check the total lines exactly matches the number of lines in the file. Only once any bugs found with the current fuzzer are fixed.
  • Check in a minimized corpus, and run regression over it in CI.