Commit graph

61 commits

Author SHA1 Message Date
Jeff King a6ca9dfa5c t/Makefile: stop setting GIT_CONFIG
Once upon a time, the setting of GIT_CONFIG in the
environment could affect how tests ran. Commit 9c3796f (Fix
setting config variables with an alternative GIT_CONFIG,
2006-06-20) unconditionally set GIT_CONFIG in the Makefile
when running tests to give us a known starting point.

This is insufficient for running the tests outside of the
Makefile, however, and 8565d2d (Make tests independent of
global config files, 2007-02-15) later set GIT_CONFIG
directly in test-lib.sh. At that point the Makefile setting
was redundant, but we never removed it. Let's do so now.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-21 14:10:30 -07:00
Junio C Hamano 31d176d083 Merge branch 'jk/test-output'
When TEST_OUTPUT_DIRECTORY setting is used, it was handled somewhat
inconsistently between the test framework and t/Makefile, and logic
to summarize the results looked at a wrong place.

* jk/test-output:
  t/Makefile: don't define TEST_RESULTS_DIRECTORY recursively
  test output: respect $TEST_OUTPUT_DIRECTORY
  t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY
2013-05-29 14:29:11 -07:00
John Keeping b3e0c4ed07 t/Makefile: don't define TEST_RESULTS_DIRECTORY recursively
Commit 54bb901 (t/Makefile: fix result handling with
TEST_OUTPUT_DIRECTORY - 2013-04-26) incorrectly defined
TEST_RESULTS_DIRECTORY relative to itself, when it should be relative to
TEST_OUTPUT_DIRECTORY.  Fix this.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-06 08:20:28 -07:00
John Keeping 6a776acabf t/Makefile: remove smoke test targets
Commit d24fbca (Remove Git's support for smoke testing - 2011-12-23)
removed the smoke test support from the test suite but it was
re-added by commit 342e9ef (Introduce a performance testing
framework - 2012-02-17).  This appears to be the result of a
mis-rebase, since re-adding the smoke testing infrastructure does
not relate to the subject of that commit.

The current 'smoke' target is broken since the 'harness' script it
uses no longer exists, so just reapply this section of commit d24fbca
and remove all of the smoke testing section in the makefile.

Signed-off-by: John Keeping <john@keeping.me.uk>
Acked-by: Thomas Rast <trast@inf.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-28 14:04:41 -07:00
John Keeping 54bb9015c8 t/Makefile: fix result handling with TEST_OUTPUT_DIRECTORY
When TEST_OUTPUT_DIRECTORY is set, the test results will be generated in
"$TEST_OUTPUT_DIRECTORY/test-results", which may not be the same as
"test-results" in t/Makefile.  This causes the aggregate-results target
to fail as it finds no count files.

Fix this by introducing TEST_RESULTS_DIRECTORY and using it wherever
test-results is referenced.

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-04-27 13:41:31 -07:00
Junio C Hamano 4249d850cf Merge branch 'tb/test-shell-lint'
Check for common mistakes in the test scripts, based on simple
pattern-matching.

* tb/test-shell-lint:
  test: Add check-non-portable-shell.pl
2013-01-10 13:47:04 -08:00
Jeff King 81127d74c4 tests: turn on test-lint by default
The test Makefile knows about a few "lint" checks for common
errors. However, they are not enabled as part of "make test"
by default, which means that many people do not bother
running them. Since they are both quick to run and accurate
(i.e., no false positives), there should be no harm in
turning them on and helping submitters catch errors earlier.

We could just set:

  TEST_LINT = test-lint

to enable all tests. But that would be unnecessarily
annoying later on if we add slower or less accurate tests
that should not be part of the default. Instead, we name the
tests individually.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-03 08:03:46 -08:00
Torsten Bögershausen c7ce70ace9 test: Add check-non-portable-shell.pl
Add the perl script "check-non-portable-shell.pl" to detect
non-portable shell syntax.

"echo -n" is an example of a shell command working on Linux, but not
on Mac OS X.

These shell commands are checked and reported as error:

 - "echo -n" (printf should be used)
 - "sed -i" (GNUism; use a temp file instead)
 - "declare" (bashism, often used with arrays)
 - "which" (unreliable exit status and output; use type instead)
 - "test a == b" (bashism for "test a = b")

"make test-lint-shell-syntax" can be used to run only the check.

Helped-By: Jeff King <peff@peff.net>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-02 16:06:42 -08:00
Stefano Lattarini dd6fc7ca91 Makefile: whitespace style fixes in macro definitions
Consistently use a single space before and after the "=" (or ":=", "+=",
etc.) in assignments to make macros.  Granted, this was not a big deal,
but I did find the needless inconsistency quite distracting.

Signed-off-by: Stefano Lattarini <stefano.lattarini@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-10 08:35:56 -08:00
Michael Haggerty 60f26f6348 t/Makefile: retain cache t/.prove across prove runs
prove(1) can write a summary of its test results and timings into a
cache file, t/.prove, then use this information during later runs for
various purposes.  But deleting t/.prove after every test run defeats
this purpose.  So do not delete t/.prove as part of "make
DEFAILT_TEST_TARGET=prove test".  (Continue to delete the file on
"make clean".)

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-20 14:19:18 -07:00
Thomas Rast 342e9ef2d9 Introduce a performance testing framework
This introduces a performance testing framework under t/perf/.  It
tries to be as close to the test-lib.sh infrastructure as possible,
and thus should be easy to get used to for git developers.

The following points were considered for the implementation:

1. You usually want to compare arbitrary revisions/build trees against
   each other.  They may not have the performance test under
   consideration, or even the perf-lib.sh infrastructure.

   To cope with this, the 'run' script lets you specify arbitrary
   build dirs and revisions.  It even automatically builds the revisions
   if it doesn't have them at hand yet.

2. Usually you would not want to run all tests.  It would take too
   long anyway.  The 'run' script lets you specify which tests to run;
   or you can also do it manually.  There is a Makefile for
   discoverability and 'make clean', but it is not meant for
   real-world use.

3. Creating test repos from scratch in every test is extremely
   time-consuming, and shipping or downloading such large/weird repos
   is out of the question.

   We leave this decision to the user.  Two different sizes of test
   repos can be configured, and the scripts just copy one or more of
   those (using hardlinks for the object store).  By default it tries
   to use the build tree's git.git repository.

   This is fairly fast and versatile.  Using a copy instead of a clone
   preserves many properties that the user may want to test for, such
   as lots of loose objects, unpacked refs, etc.

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-17 08:21:22 -08:00
Junio C Hamano 6f5e880c68 Sync with maint
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-13 11:48:00 -08:00
Ævar Arnfjörð Bjarmason d24fbca7a5 Remove Git's support for smoke testing
I'm no longer running the Git smoke testing service at
smoke.git.nix.is due to Smolder being a fragile piece of software not
having time to follow through on making it easy for third parties to
run and submit their own smoke tests.

So remove the support in Git for sending smoke tests to
smoke.git.nix.is, it's still easy to modify the test suite to submit
smokes somewhere else.

This reverts the following commits:

    Revert "t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target" -- e38efac87d
    Revert "t/README: Document the Smoke testing" -- d15e9ebc5c
    Revert "t/Makefile: Create test-results dir for smoke target" -- 617344d77b
    Revert "tests: Infrastructure for Git smoke testing" -- b6b84d1b74

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-13 02:29:07 -08:00
Kirill Smelkov aae5239be2 t/Makefile: Use $(sort ...) explicitly where needed
Starting from GNU Make 3.82 $(wildcard ...) no longer sorts the result
(from NEWS):

    * WARNING: Backward-incompatibility!
      Wildcards were not documented as returning sorted values, but the results
      have been sorted up until this release..  If your makefiles require sorted
      results from wildcard expansions, use the $(sort ...)  function to request
      it explicitly.

    2a59dc32aa

I usually watch test progress visually, and if tests are sorted, even
with make -j4 they go more or less incrementally by their t number. On
the other side, without sorting, tests are executed in seemingly random
order even for -j1. Let's please maintain sane tests order for perceived
prettyness.

Another note is that in GNU Make sort also works as uniq, so after sort
being removed, we might expect e.g. $(wildcard *.sh a.*) to produce
duplicates for e.g. "a.sh". From this point of view, adding sort could
be seen as hardening t/Makefile from accidentally introduced dups.

It turned out that prevous releases of GNU Make did not perform full
sort in $(wildcard), only sorting results for each pattern, that's why
explicit sort-as-uniq is relevant even for older makes.

Signed-off-by: Kirill Smelkov <kirr@navytux.spb.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-22 14:55:16 -08:00
Jeff King 7ef4d6b928 t/Makefile: pass test opts to valgrind target properly
The valgrind target just reinvokes make with GIT_TEST_OPTS
set to "--valgrind". However, it does this using an
environment variable, which means GIT_TEST_OPTS in your
config.mak would override it, and "make valgrind" would
simply run the test suite without valgrind on.

Instead, we should pass GIT_TEST_OPTS on the command-line,
overriding what's in config.mak, and take care to append to
whatever the user has there already.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-17 11:40:39 -07:00
Jeff King d38732c28c tests: add some script lint checks
There are some common but minor errors we tend to make in
writing test scripts:

  1. Scripts are left non-executable. This is not usually
     noticed immediately because "make test" does not need
     the bit, but it is a matter of git policy to make them
     executable (and is a slight convenience when running
     individual scripts).

  2. Two scripts are allocated the same number. Usually this
     happens on separate branches, and the problem only
     comes about during a merge. But since there is no
     textual conflict, the merger would have to be very
     observant to notice.

     This is also a minor error, but can make GIT_SKIP_TESTS
     ambiguous.

This patch introduces a "test-lint" target which checks
both. It is not invoked by default. You can invoke it as
"make test-lint", or you can make it a prerequisite of
running the tests by specifying "TEST_LINT = test-lint" in
your config.mak or on the command line.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-12-13 11:37:44 -08:00
Junio C Hamano 4739c19d7f Merge branch 'mg/make-prove'
* mg/make-prove:
  test: allow running the tests under "prove"
2010-11-17 15:00:46 -08:00
Michael J Gruber 28d836c815 test: allow running the tests under "prove"
You can run "make DEFAULT_TEST_TARGET=prove test" to run the test under
"prove" (or $(PROVE) if set).  The output is a bit easier to read when
running many tests in parallel.

Signed-off-by: Michael J Gruber <git@drmicha.warpmail.net>
Liked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Liked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-10-26 11:25:48 -07:00
Jakub Narebski 958a846721 gitweb/Makefile: Add 'test' and 'test-installed' targets
The 'test-installed' target in gitweb/Makefile tests installed gitweb,
using the same destination directory that 'install' target uses.

The 'test' target is just a convenience wrapper invoking 'gitweb-test'
target of t/Makefile.

Signed-off-by: Jakub Narebski <jnareb@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-09-30 11:50:26 -07:00
Junio C Hamano 9b777a165e cvs tests: do not touch test CVS repositories shipped with source
Some tests in t96xx series (cvsimport) want to write into the control area
(CVSROOT) of their test CVS repositories, but this does not work well when
the source area is made read-only (test trash directories are moved via
--root=else/where option).

Copy the supplied test CVS repository to a scratch place at the beginning
of these tests.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:43:24 -07:00
Ævar Arnfjörð Bjarmason 617344d77b t/Makefile: Create test-results dir for smoke target
Change the smoke target to create a test-results directory. This was
done implicitly by the test-lib before my "test-lib: Don't write
test-results when HARNESS_ACTIVE" patch, but after that smoking from
the pu branch hasn't worked.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:43:23 -07:00
Ævar Arnfjörð Bjarmason e38efac87d t/README: Add SMOKE_{COMMENT,TAGS}= to smoke_report target
The smoke server supports a free form text field with comments about a
report, and a comma delimited list of tags. Change the smoke_report
target to expose this functionality. Now smokers can send more data
that explains and categorizes the reports they're submitting.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:14 -07:00
Ævar Arnfjörð Bjarmason 4678a5cd71 t/Makefile: Can't include GIT-BUILD-OPTIONS, it's a .sh
Change the smoke testing portion of t/Makefile not to include
GIT-BUILD-OPTIONS. It's a shellscript, not a Makefile snippet, so it
had the nasty side-effect of sneaking e.g. SHELL_PATH = '/bin/sh'
(with quotes) everywhere.

Just add our own PERL_PATH variable as a workaround. The t/Makefile
already has e.g. an equivalent SHELL_PATH and TAR option which
duplicate the definitions in GIT-BUILD-OPTIONS.

Reported-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:14 -07:00
Ævar Arnfjörð Bjarmason b6b84d1b74 tests: Infrastructure for Git smoke testing
Add the capability to send smoke reports from the Git test suite.

Currently we only notice bugs in the test suite when it's run
manually. Bugs in Git that only occur on obscure platforms or setups
that the core developers aren't using can thus go unnoticed.

This series aims to change that. With it, anyone that's interested in
avoiding bitrot in Git can volunteer to run a smoke tester. A smoke
tester periodically compiles the latest version of Git, runs the test
suite, and submits a report to a central server indicating how the
test run went.

A smoke tester might run something like this in cron:

    #!/bin/sh
    cd ~/g/git
    git fetch
    for branch in maint master next pu; do
        git checkout origin/$i &&
        make clean all &&
        cd t &&
        make smoke_report
    done

The smoker might want to compile git with non-default flags, include
bisecting functionality or run the tests under valgrind. Doing that is
outside the scope of this patch, this just adds a report submission
mechanism. But including a canonical smoke runner is something we'll
want to include eventually.

What this does now is add smoke and smoke_report targets to t/Makefile
(this example only uses a few tests for demonstration):

    $ make clean smoke
    rm -f -r 'trash directory'.* test-results
    rm -f t????/cvsroot/CVSROOT/?*
    rm -f -r valgrind/bin
    rm -f .prove
    perl ./harness --git-version="1.7.2.1.173.gc9b40" \
                    --no-verbose \
                    --archive="test-results/git-smoke.tar.gz" \
                    t0000-basic.sh t0001-init.sh t0002-gitfile.sh t0003-attributes.sh t0004-unwritable.sh t0005-signals.sh t0006-date.sh
    t0000-basic.sh ....... ok
    t0001-init.sh ........ ok
    t0002-gitfile.sh ..... ok
    t0003-attributes.sh .. ok
    t0004-unwritable.sh .. ok
    t0005-signals.sh ..... ok
    t0006-date.sh ........ ok
    All tests successful.

    Test Summary Report
    -------------------
    t0000-basic.sh     (Wstat: 0 Tests: 46 Failed: 0)
      TODO passed:   5
    Files=7, Tests=134,  3 wallclock secs ( 0.06 usr  0.05 sys +  0.23 cusr  1.33 csys =  1.67 CPU)
    Result: PASS

    TAP Archive created at /home/avar/g/git/t/test-results/git-smoke.tar.gz

The smoke target uses TAP::Harness::Archive to aggregate the test
results into a tarball. The tarball contains two things, the output of
every test file that was run, and a metadata file:

Tarball contents:

    $ tar xzvf git-smoke.tar.gz
    t0004-unwritable.sh
    t0001-init.sh
    t0002-gitfile.sh
    t0005-signals.sh
    t0000-basic.sh
    t0003-attributes.sh
    t0006-date.sh
    meta.yml

A test report:

    $ cat t0005-signals.sh
    ok 1 - sigchain works
    # passed all 1 test(s)
    1..1

A metadata file:

    ---
    extra_properties:
    file_attributes:
      -
        description: t0000-basic.sh
        end_time: 1280437324.61398
        start_time: 1280437324.22186
      -
        description: t0001-init.sh
        end_time: 1280437325.12346
        start_time: 1280437324.62393
      -
        description: t0002-gitfile.sh
        end_time: 1280437325.29428
        start_time: 1280437325.13646
      -
        description: t0003-attributes.sh
        end_time: 1280437325.59678
        start_time: 1280437325.30565
      -
        description: t0004-unwritable.sh
        end_time: 1280437325.77376
        start_time: 1280437325.61003
      -
        description: t0005-signals.sh
        end_time: 1280437325.85426
        start_time: 1280437325.78727
      -
        description: t0006-date.sh
        end_time: 1280437326.2362
        start_time: 1280437325.86768
    file_order:
      - t0000-basic.sh
      - t0001-init.sh
      - t0002-gitfile.sh
      - t0003-attributes.sh
      - t0004-unwritable.sh
      - t0005-signals.sh
      - t0006-date.sh
    start_time: 1280437324
    stop_time: 1280437326

The "extra_properties" hash is where we'll stick Git-specific info,
like whether Git was compiled with gettext or the fallback regex
engine, and what branch we're compiling. Currently no metadata like
this is included.

The entire tarball is then submitted to a central smokebox at
smoke.git.nix.is. This is done with curl(1) via the "smoke_report"
target:

    $ make smoke_report
    curl \
                    -H "Expect: " \
                    -F project=Git \
                    -F architecture=x86_64 \
                    -F platform=Linux \
                    -F revision="1.7.2.1.173.gc9b40" \
                    -F report_file=@test-results/git-smoke.tar.gz \
                    http://smoke.git.nix.is/app/projects/process_add_report/1 \
            | grep -v ^Redirecting
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100  117k  100    63  100  117k      3   6430  0:00:21  0:00:18  0:00:03     0
    Reported #8 added.

Reports are then made available on the smokebox via a web interface:

    http://smoke.git.nix.is/app/projects/smoke_reports/1

The smoke reports are also mirrored to a Git repository hosted on
GitHub:

    http://github.com/gitsmoke/smoke-reports

The Smolder SQLite database that contains metadata about the reports
is also made available:

    http://github.com/gitsmoke/smoke-database

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-08-18 12:42:14 -07:00
Ævar Arnfjörð Bjarmason 9a3028b987 tests: Ignore the Test::Harness .prove file
We document how to run prove with the --state option in t/README. This
produces a .prove YAML file in the current directory. Change the t/
gitignore to ignore it, and clean it up on `make clean'.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-25 23:14:48 -07:00
Thomas Rast 7ea3ddf818 Only run aggregate-results over actual counts
The current make target 'aggregate-results' scanned all files matching
test-results/t*-*.  Normally these are only the test counts (and the
exit values, which are ignored), but with --tee the suite also dumps
all output.  Furthermore, with --verbose t1450 contains several lines
starting with "broken link from ..." which matches the criteria used
by aggregate-results.sh.

Rename the counts output files to *.counts, and only scan those.

Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Acked-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-07-16 13:25:01 -07:00
Junio C Hamano 8d676d85f7 Merge branch 'gv/portable'
* gv/portable:
  test-lib: use DIFF definition from GIT-BUILD-OPTIONS
  build: propagate $DIFF to scripts
  Makefile: Tru64 portability fix
  Makefile: HP-UX 10.20 portability fixes
  Makefile: HPUX11 portability fixes
  Makefile: SunOS 5.6 portability fix
  inline declaration does not work on AIX
  Allow disabling "inline"
  Some platforms lack socklen_t type
  Make NO_{INET_NTOP,INET_PTON} configured independently
  Makefile: some platforms do not have hstrerror anywhere
  git-compat-util.h: some platforms with mmap() lack MAP_FAILED definition
  test_cmp: do not use "diff -u" on platforms that lack one
  fixup: do not unconditionally disable "diff -u"
  tests: use "test_cmp", not "diff", when verifying the result
  Do not use "diff" found on PATH while building and installing
  enums: omit trailing comma for portability
  Makefile: -lpthread may still be necessary when libc has only pthread stubs
  Rewrite dynamic structure initializations to runtime assignment
  Makefile: pass CPPFLAGS through to fllow customization

Conflicts:
	Makefile
	wt-status.h
2010-06-21 06:02:44 -07:00
Brandon Casey 6508eedf67 t/aggregate-results: accomodate systems with small max argument list length
IRIX 6.5 has a default maximum argument list length of 20480.  The file
glob that is passed to aggregate-results currently exceeds this length, and
so the script cannot run successfully.  Work around this issue by passing
the file names in via the standard input rather than the argument list.

Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-06-02 09:36:49 -07:00
Gary V. Vaughan b2b0026eed test_cmp: do not use "diff -u" on platforms that lack one
By default the testsuite calls 'diff -u' whenever a file comparison is
called for.  Unfortunately that throws a "diff: unknown option '-u'"
error for most non-GNU diffs.

This patch sets GIT_TEST_CMP to 'cmp' on all the architectures where
that happens.  The previous version of this patch forgot to export
GIT_TEST_CMP from t/Makefile, which is why 'make test' continued to
fail most tests on most architectures - test-lib.sh was falling back
on its default of `diff -u' for GIT_TEST_CMP.  This version of this
patch shows a vast improvement in testsuite results where either GNU
diff is in the path at configure time, or where Makefile knows that
GIT_TEST_CMP=cmp is required.

Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-31 17:36:27 -07:00
Junio C Hamano 7b3bdbb335 fixup: do not unconditionally disable "diff -u"
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-31 17:35:20 -07:00
Gary V. Vaughan 4fdf71be1c tests: use "test_cmp", not "diff", when verifying the result
In tests, call test_cmp rather than raw diff where possible (i.e. if
the output does not go to a pipe), to allow the use of, say, 'cmp'
when the default 'diff -u' is not compatible with a vendor diff.

When that is not possible, use $DIFF, as set in GIT-BUILD-OPTIONS.

Signed-off-by: Gary V. Vaughan <gary@thewrittenword.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-05-31 16:59:27 -07:00
Junio C Hamano 34b383e7cd t/Makefile: remove test artifacts upon "make clean"
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2010-03-13 12:41:20 -08:00
Jeff King c14417c4f5 t/Makefile: include config.mak
This is useful if you want to specify GIT_TEST_OPTS that you
always use.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-08-09 10:51:47 -07:00
Junio C Hamano b0085a7167 Merge branch 'maint'
* maint:
  test-lib: Clean up comments and Makefile.
  diff --no-index: Do not generate patch output if other output is requested
2009-03-28 01:16:46 -07:00
Emil Sit 81db4abf84 test-lib: Clean up comments and Makefile.
Bring documentation in test-lib and clean target
in Makefile in-line with abc5d372.

Signed-off-by: Emil Sit <sit@emilsit.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-03-28 00:11:27 -07:00
Johannes Schindelin 3da9365234 Tests: let --valgrind imply --verbose and --tee
It does not make much sense to run the (expensive) valgrind tests and
not look at the output.

To prevent output from scrolling out of reach, the parameter --tee is
implied, too.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-03 22:01:23 -08:00
Johannes Schindelin 7f6fdea119 t/Makefile: provide a 'valgrind' target
It is easy to forget running valgrinded tests without -v, and it is
also easy to forget to redirect the output to "tee" (lest the output
scroll out of the terminal's buffer).  Running "make valgrind" will
take care of all that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2009-02-03 22:01:14 -08:00
Johannes Schindelin abc5d372ec Enable parallel tests
On multiprocessor machines, or with I/O heavy tests (that leave the
CPU waiting a lot), it makes sense to parallelize the tests.

However, care has to be taken that the different jobs use different
trash directories.

This commit does so, by creating the trash directories with a suffix
that is unique with regard to the test, as it is the test's base name.

Further, the trash directory is removed in the test itself if
everything went fine, so that the trash directories do not
pile up only to be removed at the very end.

If a test failed, the trash directory is not removed.  Chances are
that the exact error message is lost in the clutter, but you can still
see what test failed from the name of the trash directory, and repeat
the test (without -j).

If all was good, you will see the aggregated results.

Suggestions to simplify this commit came from Junio and René.

There still is an issue with tests that want to run a server process and
listen to a fixed port (http and svn) --- they cannot run in parallel but
this patch does not address this issue.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 22:24:28 -07:00
Johannes Schindelin e3df89a4b1 tests: Clarify dependencies between tests, 'aggregate-results' and 'clean'
The Makefile targets 'aggregate-results' and 'clean' pretended to be
independent.  This is not true, of course, since aggregate-results
needs the results _before_ they are removed.

Likewise, the tests should have been run already when the results are
to be aggregated.

However, as it is legitimate to run only a few tests, and then aggregate
just those results, so another target is introduced, that depends on all
tests, then aggregates the results, and only then removes the results.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-08-08 18:52:28 -07:00
SungHyun Nam dd33927ccd t/Makefile: use specified shell when running aggregation script
Signed-off-by: SungHyun Nam <goweol@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-07-16 08:59:14 -07:00
Sverre Rabbelier f8d5ffc2c7 Hook up the result aggregation in the test makefile.
This patch makes 'make' output the aggregated results at the end of each build.
The 'git-test-result' file is removed both before and after each build.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08 15:08:19 -07:00
Sverre Rabbelier 2d84e9fb6d Modify test-lib.sh to output stats to t/test-results/*
This change is needed order to aggregate data on the test run later on.

Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-06-08 15:07:27 -07:00
Brandon Casey 9231e3a953 t/Makefile: "trash" directory was renamed recently
Signed-off-by: Brandon Casey <drafnel@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2008-05-18 21:39:58 -07:00
Emil Medve 4cb08df553 Use $(RM) in Makefiles instead of 'rm -f'
Signed-off-by: Emil Medve <Emilian.Medve@Freescale.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-07-14 23:31:01 -07:00
Junio C Hamano a6080a0a44 War on whitespace
This uses "git-apply --whitespace=strip" to fix whitespace errors that have
crept in to our source files over time.  There are a few files that need
to have trailing whitespaces (most notably, test vectors).  The results
still passes the test, and build result in Documentation/ area is unchanged.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2007-06-07 00:04:01 -07:00
Eric Wong ed92f17026 git-svn: remove non-delta fetch code paths
We have less code to worry about now.  As a bonus, --revision
can be used to reliably skip parts of history whenever fetch is
run, not just the first time.  I'm not sure why anybody would
want to skip history in the middle, however...

For people (nearly everyone at the moment) without the
do_switch() function in their Perl SVN library, the entire tree
must be refetched if --follow-parent is used and a parent is
found.  Future versions of SVN will have a working do_switch()
function accessible via Perl.

Accessing repositories on the local machine (especially file://
ones) is also slightly slower as a result; but I suspect most
git-svn users will be using it to access remote repositories.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-12-28 01:39:09 -08:00
Eric Wong b9c8518722 git-svn: remove support for the svn command-line client
Using the command-line client was great for prototyping and
getting something working quickly.  Eventually I found time
to study the library documentation and add support for using
the libraries which are much faster and more flexible when
it comes to supporting new features.

Note that we require version 1.1 of the SVN libraries, whereas
we supported the command-line svn client down to version 1.0.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-12-16 05:17:17 -08:00
Eric Wong e15161198a git-svn: update tests for recent changes
* Enable test for delta transfers in full-svn-test.

* Run tests against the root of the repository so we won't have
  to revisit 308906fa6e and
  efe4631def again.
  The graft-branches test still runs as before.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-27 22:23:06 -08:00
Junio C Hamano 7cdbff14d4 remove merge-recursive-old
This frees the Porcelain-ish that comes with the core Python-free.

Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-11-21 20:55:39 -08:00
Eric Wong 60d02ccc18 git-svn: migrate out of contrib
Allow NO_SVN_TESTS to be defined to skip git-svn tests.  These
tests are time-consuming due to SVN being slow, and even more so
if SVN Perl libraries are not available.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <junkio@cox.net>
2006-07-06 17:02:47 -07:00