git/t/perf
Junio C Hamano 0f9e62e084 Merge branch 'jk/pack-bitmap'
Borrow the bitmap index into packfiles from JGit to speed up
enumeration of objects involved in a commit range without having to
fully traverse the history.

* jk/pack-bitmap: (26 commits)
  ewah: unconditionally ntohll ewah data
  ewah: support platforms that require aligned reads
  read-cache: use get_be32 instead of hand-rolled ntoh_l
  block-sha1: factor out get_be and put_be wrappers
  do not discard revindex when re-preparing packfiles
  pack-bitmap: implement optional name_hash cache
  t/perf: add tests for pack bitmaps
  t: add basic bitmap functionality tests
  count-objects: recognize .bitmap in garbage-checking
  repack: consider bitmaps when performing repacks
  repack: handle optional files created by pack-objects
  repack: turn exts array into array-of-struct
  repack: stop using magic number for ARRAY_SIZE(exts)
  pack-objects: implement bitmap writing
  rev-list: add bitmap mode to speed up object lists
  pack-objects: use bitmaps when packing objects
  pack-objects: split add_object_entry
  pack-bitmap: add support for bitmap indexes
  documentation: add documentation for the bitmap format
  ewah: compressed bitmap implementation
  ...
2014-02-27 14:01:48 -08:00
..
.gitignore t/perf: add "trash directory" to .gitignore 2012-09-17 14:27:55 -07:00
aggregate.perl
Makefile
min_time.perl
p0000-perf-lib-sanity.sh perf: export some important test-lib variables 2012-03-08 12:07:50 -08:00
p0001-rev-list.sh t/perf: time rev-list with UNINTERESTING commits 2014-01-21 14:46:17 -08:00
p0002-read-cache.sh read-cache: add simple performance test 2013-06-09 17:03:00 -07:00
p4000-diff-algorithms.sh p4000: use -3000 when promising -3000 2012-03-09 02:07:23 -08:00
p4001-diff-no-index.sh diff: don't read index when --no-index is given 2013-12-12 12:23:02 -08:00
p4211-line-log.sh Implement line-history search (git log -L) 2013-03-28 10:29:22 -07:00
p5302-pack-index.sh index-pack: support multithreaded delta resolving 2012-05-07 15:48:15 -07:00
p5310-pack-bitmaps.sh pack-bitmap: implement optional name_hash cache 2013-12-30 12:19:23 -08:00
p7810-grep.sh Add a performance test for git-grep 2012-02-17 08:21:34 -08:00
perf-lib.sh test: replace shebangs with descriptions in shell libraries 2013-11-26 14:23:52 -08:00
README Documentation: Update 'linux-2.6.git' -> 'linux.git' 2013-06-22 23:36:48 -07:00
run

Git performance tests
=====================

This directory holds performance testing scripts for git tools.  The
first part of this document describes the various ways in which you
can run them.

When fixing the tools or adding enhancements, you are strongly
encouraged to add tests in this directory to cover what you are
trying to fix or enhance.  The later part of this short document
describes how your test scripts should be organized.


Running Tests
-------------

The easiest way to run tests is to say "make".  This runs all
the tests on the current git repository.

    === Running 2 tests in this tree ===
    [...]
    Test                                     this tree
    ---------------------------------------------------------
    0001.1: rev-list --all                   0.54(0.51+0.02)
    0001.2: rev-list --all --objects         6.14(5.99+0.11)
    7810.1: grep worktree, cheap regex       0.16(0.16+0.35)
    7810.2: grep worktree, expensive regex   7.90(29.75+0.37)
    7810.3: grep --cached, cheap regex       3.07(3.02+0.25)
    7810.4: grep --cached, expensive regex   9.39(30.57+0.24)

You can compare multiple repositories and even git revisions with the
'run' script:

    $ ./run . origin/next /path/to/git-tree p0001-rev-list.sh

where . stands for the current git tree.  The full invocation is

    ./run [<revision|directory>...] [--] [<test-script>...]

A '.' argument is implied if you do not pass any other
revisions/directories.

You can also manually test this or another git build tree, and then
call the aggregation script to summarize the results:

    $ ./p0001-rev-list.sh
    [...]
    $ GIT_BUILD_DIR=/path/to/other/git ./p0001-rev-list.sh
    [...]
    $ ./aggregate.perl . /path/to/other/git ./p0001-rev-list.sh

aggregate.perl has the same invocation as 'run', it just does not run
anything beforehand.

You can set the following variables (also in your config.mak):

    GIT_PERF_REPEAT_COUNT
	Number of times a test should be repeated for best-of-N
	measurements.  Defaults to 3.

    GIT_PERF_MAKE_OPTS
	Options to use when automatically building a git tree for
	performance testing.  E.g., -j6 would be useful.

    GIT_PERF_REPO
    GIT_PERF_LARGE_REPO
	Repositories to copy for the performance tests.  The normal
	repo should be at least git.git size.  The large repo should
	probably be about linux.git size for optimal results.
	Both default to the git.git you are running from.

You can also pass the options taken by ordinary git tests; the most
useful one is:

--root=<directory>::
	Create "trash" directories used to store all temporary data during
	testing under <directory>, instead of the t/ directory.
	Using this option with a RAM-based filesystem (such as tmpfs)
	can massively speed up the test suite.


Naming Tests
------------

The performance test files are named as:

	pNNNN-commandname-details.sh

where N is a decimal digit.  The same conventions for choosing NNNN as
for normal tests apply.


Writing Tests
-------------

The perf script starts much like a normal test script, except it
sources perf-lib.sh:

	#!/bin/sh
	#
	# Copyright (c) 2005 Junio C Hamano
	#

	test_description='xxx performance test'
	. ./perf-lib.sh

After that you will want to use some of the following:

	test_perf_default_repo  # sets up a "normal" repository
	test_perf_large_repo    # sets up a "large" repository

	test_perf_default_repo sub  # ditto, in a subdir "sub"

        test_checkout_worktree  # if you need the worktree too

At least one of the first two is required!

You can use test_expect_success as usual.  For actual performance
tests, use

	test_perf 'descriptive string' '
		command1 &&
		command2
	'

test_perf spawns a subshell, for lack of better options.  This means
that

* you _must_ export all variables that you need in the subshell

* you _must_ flag all variables that you want to persist from the
  subshell with 'test_export':

	test_perf 'descriptive string' '
		foo=$(git rev-parse HEAD) &&
		test_export foo
	'

  The so-exported variables are automatically marked for export in the
  shell executing the perf test.  For your convenience, test_export is
  the same as export in the main shell.

  This feature relies on a bit of magic using 'set' and 'source'.
  While we have tried to make sure that it can cope with embedded
  whitespace and other special characters, it will not work with
  multi-line data.