linux/tools/testing/selftests
Masami Hiramatsu 915de2adb5 ftracetest: Add POSIX.3 standard and XFAIL result codes
Add XFAIL and POSIX 1003.3 standard codes (UNRESOLVED/
UNTESTED/UNSUPPORTED) as result codes. These are used for the
results that test case is expected to fail or unsupported
feature (by config).

To return these result code, this introduces exit_unresolved,
exit_untested, exit_unsupported and exit_xfail functions,
which use real-time signals to notify the result code to
ftracetest.

This also set "errexit" option for the testcases, so that
the tests don't need to exit explicitly.

Note that if the test returns UNRESOLVED/UNSUPPORTED/FAIL,
its test log including executed commands is shown on console
and main logfile as below.

  ------
  # ./ftracetest samples/
  === Ftrace unit tests ===
  [1] failure-case example        [FAIL]
  execute: /home/fedora/ksrc/linux-3/tools/testing/selftests/ftrace/samples/fail.tc
  + . /home/fedora/ksrc/linux-3/tools/testing/selftests/ftrace/samples/fail.tc
  ++ cat non-exist-file
  cat: non-exist-file: No such file or directory
  [2] pass-case example   [PASS]
  [3] unresolved-case example     [UNRESOLVED]
  execute: /home/fedora/ksrc/linux-3/tools/testing/selftests/ftrace/samples/unresolved.tc
  + . /home/fedora/ksrc/linux-3/tools/testing/selftests/ftrace/samples/unresolved.tc
  ++ trap exit_unresolved INT
  ++ kill -INT 29324
  +++ exit_unresolved
  +++ kill -s 38 29265
  +++ exit 0
  [4] unsupported-case example    [UNSUPPORTED]
  execute: /home/fedora/ksrc/linux-3/tools/testing/selftests/ftrace/samples/unsupported.tc
  + . /home/fedora/ksrc/linux-3/tools/testing/selftests/ftrace/samples/unsupported.tc
  ++ exit_unsupported
  ++ kill -s 40 29265
  ++ exit 0
  [5] untested-case example       [UNTESTED]
  [6] xfail-case example  [XFAIL]

  # of passed:  1
  # of failed:  1
  # of unresolved:  1
  # of untested:  1
  # of unsupported:  1
  # of xfailed:  1
  # of undefined(test bug):  0
  ------

Link: http://lkml.kernel.org/p/20140929120211.30203.99510.stgit@kbuild-f20.novalocal

Acked-by: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-10-03 16:44:02 -04:00
..
breakpoints breakpoint selftests: print failure status instead of cause make error 2012-12-17 17:15:27 -08:00
cpu-hotplug tools: selftests - create a separate hotplug target for full range test 2014-07-11 18:13:06 -07:00
efivarfs efivars: efivarfs_valid_name() should handle pstore syntax 2013-03-06 14:46:04 +00:00
firmware test: add firmware_class loader test 2014-07-17 18:44:19 -07:00
ftrace ftracetest: Add POSIX.3 standard and XFAIL result codes 2014-10-03 16:44:02 -04:00
ipc tools: selftests: fix build issue with make kselftests target 2014-08-29 16:28:17 -07:00
kcmp tools: selftests: fix build issue with make kselftests target 2014-08-29 16:28:17 -07:00
memfd tools: selftests: fix build issue with make kselftests target 2014-08-29 16:28:17 -07:00
memory-hotplug tools: selftests - create a separate hotplug target for full range test 2014-07-11 18:13:06 -07:00
mount mnt: Add tests for unprivileged remount cases that have found to be faulty 2014-07-31 17:13:15 -07:00
mqueue tools: Fix mqueue Makefile compile linking order 2014-07-11 18:11:18 -07:00
net net: filter: BPF testsuite 2014-05-12 00:23:55 -04:00
powerpc selftests/powerpc: Add test of per-event excludes 2014-07-28 14:11:33 +10:00
ptrace tools/testing/selftests/ptrace/peeksiginfo.c: add PAGE_SIZE definition 2014-08-08 15:57:25 -07:00
rcutorture Merge branches 'doc.2014.07.08a', 'fixes.2014.07.09a', 'maintainers.2014.07.08b', 'nocbs.2014.07.07a' and 'torture.2014.07.07a' into HEAD 2014-07-09 09:16:54 -07:00
sysctl tools/testing/selftests/sysctl: validate sysctl_writes_strict 2014-06-06 16:08:13 -07:00
timers tools/testing/selftests: fix uninitialized variable 2013-10-16 21:35:53 -07:00
user test: check copy_to/from_user boundary validation 2014-01-23 16:36:57 -08:00
vm selftests: add .gitignore for vm 2013-07-03 16:08:07 -07:00
Makefile ftracetest: Initial commit for ftracetest 2014-09-23 09:31:05 -04:00
README.txt tools: selftests - create a separate hotplug target for full range test 2014-07-11 18:13:06 -07:00

Linux Kernel Selftests

The kernel contains a set of "self tests" under the tools/testing/selftests/
directory. These are intended to be small unit tests to exercise individual
code paths in the kernel.

On some systems, hot-plug tests could hang forever waiting for cpu and
memory to be ready to be offlined. A special hot-plug target is created
to run full range of hot-plug tests. In default mode, hot-plug tests run
in safe mode with a limited scope. In limited mode, cpu-hotplug test is
run on a single cpu as opposed to all hotplug capable cpus, and memory
hotplug test is run on 2% of hotplug capable memory instead of 10%.

Running the selftests (hotplug tests are run in limited mode)
=============================================================

To build the tests:

  $ make -C tools/testing/selftests


To run the tests:

  $ make -C tools/testing/selftests run_tests

- note that some tests will require root privileges.

To run only tests targeted for a single subsystem: (including
hotplug targets in limited mode)

  $  make -C tools/testing/selftests TARGETS=cpu-hotplug run_tests

See the top-level tools/testing/selftests/Makefile for the list of all possible
targets.

Running the full range hotplug selftests
========================================

To build the tests:

  $ make -C tools/testing/selftests hotplug

To run the tests:

  $ make -C tools/testing/selftests run_hotplug

- note that some tests will require root privileges.

Contributing new tests
======================

In general, the rules for for selftests are

 * Do as much as you can if you're not root;

 * Don't take too long;

 * Don't break the build on any architecture, and

 * Don't cause the top-level "make run_tests" to fail if your feature is
   unconfigured.