mirror of
https://github.com/git/git
synced 2024-08-27 11:39:22 +00:00
![Johannes Schindelin](/assets/img/avatar_default.png)
It does not make much sense to run the (expensive) valgrind tests and not look at the output. To prevent output from scrolling out of reach, the parameter --tee is implied, too. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
276 lines
9.1 KiB
Plaintext
276 lines
9.1 KiB
Plaintext
Core GIT Tests
|
|
==============
|
|
|
|
This directory holds many test scripts for core GIT tools. The
|
|
first part of this short document describes how to run the tests
|
|
and read their output.
|
|
|
|
When fixing the tools or adding enhancements, you are strongly
|
|
encouraged to add tests in this directory to cover what you are
|
|
trying to fix or enhance. The later part of this short document
|
|
describes how your test scripts should be organized.
|
|
|
|
|
|
Running Tests
|
|
-------------
|
|
|
|
The easiest way to run tests is to say "make". This runs all
|
|
the tests.
|
|
|
|
*** t0000-basic.sh ***
|
|
* ok 1: .git/objects should be empty after git-init in an empty repo.
|
|
* ok 2: .git/objects should have 256 subdirectories.
|
|
* ok 3: git-update-index without --add should fail adding.
|
|
...
|
|
* ok 23: no diff after checkout and git-update-index --refresh.
|
|
* passed all 23 test(s)
|
|
*** t0100-environment-names.sh ***
|
|
* ok 1: using old names should issue warnings.
|
|
* ok 2: using old names but having new names should not issue warnings.
|
|
...
|
|
|
|
Or you can run each test individually from command line, like
|
|
this:
|
|
|
|
$ sh ./t3001-ls-files-killed.sh
|
|
* ok 1: git-update-index --add to add various paths.
|
|
* ok 2: git-ls-files -k to show killed files.
|
|
* ok 3: validate git-ls-files -k output.
|
|
* passed all 3 test(s)
|
|
|
|
You can pass --verbose (or -v), --debug (or -d), and --immediate
|
|
(or -i) command line argument to the test, or by setting GIT_TEST_OPTS
|
|
appropriately before running "make".
|
|
|
|
--verbose::
|
|
This makes the test more verbose. Specifically, the
|
|
command being run and their output if any are also
|
|
output.
|
|
|
|
--debug::
|
|
This may help the person who is developing a new test.
|
|
It causes the command defined with test_debug to run.
|
|
|
|
--immediate::
|
|
This causes the test to immediately exit upon the first
|
|
failed test.
|
|
|
|
--long-tests::
|
|
This causes additional long-running tests to be run (where
|
|
available), for more exhaustive testing.
|
|
|
|
--valgrind::
|
|
Execute all Git binaries with valgrind and exit with status
|
|
126 on errors (just like regular tests, this will only stop
|
|
the test script when running under -i). Valgrind errors
|
|
go to stderr, so you might want to pass the -v option, too.
|
|
|
|
Since it makes no sense to run the tests with --valgrind and
|
|
not see any output, this option implies --verbose. For
|
|
convenience, it also implies --tee.
|
|
|
|
--tee::
|
|
In addition to printing the test output to the terminal,
|
|
write it to files named 't/test-results/$TEST_NAME.out'.
|
|
As the names depend on the tests' file names, it is safe to
|
|
run the tests with this option in parallel.
|
|
|
|
Skipping Tests
|
|
--------------
|
|
|
|
In some environments, certain tests have no way of succeeding
|
|
due to platform limitation, such as lack of 'unzip' program, or
|
|
filesystem that do not allow arbitrary sequence of non-NUL bytes
|
|
as pathnames.
|
|
|
|
You should be able to say something like
|
|
|
|
$ GIT_SKIP_TESTS=t9200.8 sh ./t9200-git-cvsexport-commit.sh
|
|
|
|
and even:
|
|
|
|
$ GIT_SKIP_TESTS='t[0-4]??? t91?? t9200.8' make
|
|
|
|
to omit such tests. The value of the environment variable is a
|
|
SP separated list of patterns that tells which tests to skip,
|
|
and either can match the "t[0-9]{4}" part to skip the whole
|
|
test, or t[0-9]{4} followed by ".$number" to say which
|
|
particular test to skip.
|
|
|
|
Note that some tests in the existing test suite rely on previous
|
|
test item, so you cannot arbitrarily disable one and expect the
|
|
remainder of test to check what the test originally was intended
|
|
to check.
|
|
|
|
|
|
Naming Tests
|
|
------------
|
|
|
|
The test files are named as:
|
|
|
|
tNNNN-commandname-details.sh
|
|
|
|
where N is a decimal digit.
|
|
|
|
First digit tells the family:
|
|
|
|
0 - the absolute basics and global stuff
|
|
1 - the basic commands concerning database
|
|
2 - the basic commands concerning the working tree
|
|
3 - the other basic commands (e.g. ls-files)
|
|
4 - the diff commands
|
|
5 - the pull and exporting commands
|
|
6 - the revision tree commands (even e.g. merge-base)
|
|
7 - the porcelainish commands concerning the working tree
|
|
8 - the porcelainish commands concerning forensics
|
|
9 - the git tools
|
|
|
|
Second digit tells the particular command we are testing.
|
|
|
|
Third digit (optionally) tells the particular switch or group of switches
|
|
we are testing.
|
|
|
|
If you create files under t/ directory (i.e. here) that is not
|
|
the top-level test script, never name the file to match the above
|
|
pattern. The Makefile here considers all such files as the
|
|
top-level test script and tries to run all of them. A care is
|
|
especially needed if you are creating a common test library
|
|
file, similar to test-lib.sh, because such a library file may
|
|
not be suitable for standalone execution.
|
|
|
|
|
|
Writing Tests
|
|
-------------
|
|
|
|
The test script is written as a shell script. It should start
|
|
with the standard "#!/bin/sh" with copyright notices, and an
|
|
assignment to variable 'test_description', like this:
|
|
|
|
#!/bin/sh
|
|
#
|
|
# Copyright (c) 2005 Junio C Hamano
|
|
#
|
|
|
|
test_description='xxx test (option --frotz)
|
|
|
|
This test registers the following structure in the cache
|
|
and tries to run git-ls-files with option --frotz.'
|
|
|
|
|
|
Source 'test-lib.sh'
|
|
--------------------
|
|
|
|
After assigning test_description, the test script should source
|
|
test-lib.sh like this:
|
|
|
|
. ./test-lib.sh
|
|
|
|
This test harness library does the following things:
|
|
|
|
- If the script is invoked with command line argument --help
|
|
(or -h), it shows the test_description and exits.
|
|
|
|
- Creates an empty test directory with an empty .git/objects
|
|
database and chdir(2) into it. This directory is 't/trash directory'
|
|
if you must know, but I do not think you care.
|
|
|
|
- Defines standard test helper functions for your scripts to
|
|
use. These functions are designed to make all scripts behave
|
|
consistently when command line arguments --verbose (or -v),
|
|
--debug (or -d), and --immediate (or -i) is given.
|
|
|
|
|
|
End with test_done
|
|
------------------
|
|
|
|
Your script will be a sequence of tests, using helper functions
|
|
from the test harness library. At the end of the script, call
|
|
'test_done'.
|
|
|
|
|
|
Test harness library
|
|
--------------------
|
|
|
|
There are a handful helper functions defined in the test harness
|
|
library for your script to use.
|
|
|
|
- test_expect_success <message> <script>
|
|
|
|
This takes two strings as parameter, and evaluates the
|
|
<script>. If it yields success, test is considered
|
|
successful. <message> should state what it is testing.
|
|
|
|
Example:
|
|
|
|
test_expect_success \
|
|
'git-write-tree should be able to write an empty tree.' \
|
|
'tree=$(git-write-tree)'
|
|
|
|
- test_expect_failure <message> <script>
|
|
|
|
This is NOT the opposite of test_expect_success, but is used
|
|
to mark a test that demonstrates a known breakage. Unlike
|
|
the usual test_expect_success tests, which say "ok" on
|
|
success and "FAIL" on failure, this will say "FIXED" on
|
|
success and "still broken" on failure. Failures from these
|
|
tests won't cause -i (immediate) to stop.
|
|
|
|
- test_debug <script>
|
|
|
|
This takes a single argument, <script>, and evaluates it only
|
|
when the test script is started with --debug command line
|
|
argument. This is primarily meant for use during the
|
|
development of a new test script.
|
|
|
|
- test_done
|
|
|
|
Your test script must have test_done at the end. Its purpose
|
|
is to summarize successes and failures in the test script and
|
|
exit with an appropriate error code.
|
|
|
|
- test_tick
|
|
|
|
Make commit and tag names consistent by setting the author and
|
|
committer times to defined stated. Subsequent calls will
|
|
advance the times by a fixed amount.
|
|
|
|
- test_commit <message> [<filename> [<contents>]]
|
|
|
|
Creates a commit with the given message, committing the given
|
|
file with the given contents (default for both is to reuse the
|
|
message string), and adds a tag (again reusing the message
|
|
string as name). Calls test_tick to make the SHA-1s
|
|
reproducible.
|
|
|
|
- test_merge <message> <commit-or-tag>
|
|
|
|
Merges the given rev using the given message. Like test_commit,
|
|
creates a tag and calls test_tick before committing.
|
|
|
|
Tips for Writing Tests
|
|
----------------------
|
|
|
|
As with any programming projects, existing programs are the best
|
|
source of the information. However, do _not_ emulate
|
|
t0000-basic.sh when writing your tests. The test is special in
|
|
that it tries to validate the very core of GIT. For example, it
|
|
knows that there will be 256 subdirectories under .git/objects/,
|
|
and it knows that the object ID of an empty tree is a certain
|
|
40-byte string. This is deliberately done so in t0000-basic.sh
|
|
because the things the very basic core test tries to achieve is
|
|
to serve as a basis for people who are changing the GIT internal
|
|
drastically. For these people, after making certain changes,
|
|
not seeing failures from the basic test _is_ a failure. And
|
|
such drastic changes to the core GIT that even changes these
|
|
otherwise supposedly stable object IDs should be accompanied by
|
|
an update to t0000-basic.sh.
|
|
|
|
However, other tests that simply rely on basic parts of the core
|
|
GIT working properly should not have that level of intimate
|
|
knowledge of the core GIT internals. If all the test scripts
|
|
hardcoded the object IDs like t0000-basic.sh does, that defeats
|
|
the purpose of t0000-basic.sh, which is to isolate that level of
|
|
validation in one place. Your test also ends up needing
|
|
updating when such a change to the internal happens, so do _not_
|
|
do it and leave the low level of validation to t0000-basic.sh.
|