It is much easier to diff the output against a previous
one when the fields are sorted.
Helped-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes it easier to use the aggregate script
on the command line when one wants to get the
"environment" fields set in the codespeed output.
Previously setting GIT_REPO_NAME was needed
for this purpose.
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes it easier to use the aggregate script
on the command line, to get results from
subsections.
Previously setting GIT_PERF_SUBSECTION was needed
for this purpose.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Codespeed (https://github.com/tobami/codespeed/) is an open source
project that can be used to track how some software performs over
time. It stores performance test results in a database and can show
nice graphs and charts on a web interface.
As it can be interesting to use Codespeed to see how Git performance
evolves over time and releases, let's implement a Codespeed output
in "perf/aggregate.perl".
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As we want to implement another kind of output than
the current output for the perf test results, let's
refactor the existing code that outputs the results
in its own print_default_results() function.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The way we check ENV{GIT_PERF_SUBSECTION} could trigger
comparison between undef and "" that may be flagged by
use of strict & warnings. Let's fix that.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When tests are run for a subsection defined in a config file, it is
better if the results for the current subsection are not overwritting
the results of a previous subsection.
So let's store the results for a subsection in a subdirectory of
"test-results/" with the subsection name.
The aggregate.perl, when it is run for a subsection, should then
aggregate the results found in "test-results/$GIT_PERF_SUBSECTION/".
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Change the test descriptions from being treated as binary blobs by
perl to being treated as UTF-8. This ensures that e.g. a test
description like "æ" is counted as 1 character, not 2.
I have WIP performance tests for non-ASCII grep patterns on another
topic that are affected by this.
Now instead of:
$ ./run p0000-perf-lib-sanity.sh
[...]
0000.4: export a weird var 0.00(0.00+0.00)
0000.5: éḿíẗ ńöń-ÁŚĆÍÍ ćḧáŕáćẗéŕś 0.00(0.00+0.00)
0000.7: important variables available in subshells 0.00(0.00+0.00)
[...]
We emit:
[...]
0000.4: export a weird var 0.00(0.00+0.00)
0000.5: éḿíẗ ńöń-ÁŚĆÍÍ ćḧáŕáćẗéŕś 0.00(0.00+0.00)
0000.7: important variables available in subshells 0.00(0.00+0.00)
[...]
Fixes code originally added in 342e9ef2d9 ("Introduce a performance
testing framework", 2012-02-17).
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
aggregate.perl did not work when Git.pm is not installed to a directory
contained in the default Perl library path list or PERLLIB.
This commit prepends the Perl library path of the current Git source
tree to enable this.
Note that this commit adds a hard-coded relative path
use lib '../../perl/blib/lib';
instead of the flexible environment-based variant
use lib (split(/:/, $ENV{GITPERLLIB}));
which is used in tests written in Perl.
The hard-coded variant is used because the whole performance test
framework does it that way (and GITPERLLIB is not set there).
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This introduces a performance testing framework under t/perf/. It
tries to be as close to the test-lib.sh infrastructure as possible,
and thus should be easy to get used to for git developers.
The following points were considered for the implementation:
1. You usually want to compare arbitrary revisions/build trees against
each other. They may not have the performance test under
consideration, or even the perf-lib.sh infrastructure.
To cope with this, the 'run' script lets you specify arbitrary
build dirs and revisions. It even automatically builds the revisions
if it doesn't have them at hand yet.
2. Usually you would not want to run all tests. It would take too
long anyway. The 'run' script lets you specify which tests to run;
or you can also do it manually. There is a Makefile for
discoverability and 'make clean', but it is not meant for
real-world use.
3. Creating test repos from scratch in every test is extremely
time-consuming, and shipping or downloading such large/weird repos
is out of the question.
We leave this decision to the user. Two different sizes of test
repos can be configured, and the scripts just copy one or more of
those (using hardlinks for the object store). By default it tries
to use the build tree's git.git repository.
This is fairly fast and versatile. Using a copy instead of a clone
preserves many properties that the user may want to test for, such
as lots of loose objects, unpacked refs, etc.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
