test_cmp() is primarily meant to compare text files (and display the
difference for debug purposes).
Raw "cmp" is better suited to compare binary files (tar, zip, etc.).
On MinGW, test_cmp is a shell function mingw_test_cmp that tries to
read both files into environment, stripping CR characters (introduced
in commit 4d715ac0).
This function usually speeds things up, as fork is extremly slow on
Windows. But no wonder that this function is extremely slow and
sometimes even crashes when comparing large tar or zip files.
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit ee27ca4, we started restricting remote git-archive
invocations to only accessing reachable commits. This
matches what upload-pack allows, but does restrict some
useful cases (e.g., HEAD:foo). We loosened this in 0f544ee,
which allows `foo:bar` as long as `foo` is a ref tip.
However, that still doesn't allow many useful things, like:
1. Commits accessible from a ref, like `foo^:bar`, which
are reachable
2. Arbitrary sha1s, even if they are reachable.
We can do a full object-reachability check for these cases,
but it can be quite expensive if the client has sent us the
sha1 of a tree; we have to visit every sub-tree of every
commit in the worst case.
Let's instead give site admins an escape hatch, in case they
prefer the more liberal behavior. For many sites, the full
object database is public anyway (e.g., if you allow dumb
walker access), or the site admin may simply decide the
security/convenience tradeoff is not worth it.
This patch adds a new config option to disable the
restrictions added in ee27ca4. It defaults to off, meaning
there is no change in behavior by default.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In t5000, we test the built-in ".tar.gz" config for
git-archive. To make our tests portable, we check that we
have a way to both gzip and gunzip, and we respected
environment variables to point to alternate commands for
doing these operations.
However, the $GZIP variable did not actually do anything, as
changing it would not affect the baked-in value in
archive-tar.c. Moreover, setting the variable $GZIP
influences gzip itself. From the gzip man page:
The environment variable GZIP can hold a set of default
options for gzip. These options are interpreted first and
can be overwritten by explicit command line parameters.
We could rename this variable, and use it to set up custom
config (or even have a Makefile knob to affect the built
binary), but it is not worth the trouble; nobody has ever
reported a problem with the baked-in default, and they can
always change it via config if they need to. Let's just drop
the variable and use "gzip" in the test (keeping the
prerequisite, of course).
While we're at it, we can drop the GUNZIP variable and
prerequisite; it uses "gzip -d", so if we have GZIP, we
will have both.
We can also use test_lazy_prereq for the gzip prerequisite,
which is simpler and behaves more consistently with the rest
of git (e.g., by making output available when the test is
run with "-v").
Noticed-by: Christian Hesse <mail@eworm.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git tar-tree" has been a thin wrapper around "git archive" since commit
fd88d9c (Remove upload-tar and make git-tar-tree a thin wrapper to
git-archive, 2006-09-24), which also made it print a message indicating
that git-tar-tree is deprecated.
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a file with a long name to the test archive in order to check
entries with pax extended headers. Also add a check for tar versions
that doen't understand this format. Those versions should extract the
headers as a regular files. Add code to check_tar() to interpret the
path header if present, so that our tests work even with those tar
versions.
It's important to use the fallback code only if needed to still be
able to detect git archive errorously creating pax headers as regular
file entries (with a suitable tar version, of course).
The archive used to check for pax header support in tar was generated
using GNU tar 1.26 and its option --format=pax.
Tested successfully on NetBSD 6.1, which has a tar version lacking pax
header support.
Signed-off-by: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Just compare the archives created by git tar-tree with the ones created
using git archive with the equivalent options, whose contents are
checked already, instead of extracting them again.
Signed-off-by: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Perform the full range of checks against all archived files instead of
looking only at the file type of a few of them. Also add a test of a
git archive with a prefix ending in with a slash, i.e. adding a full
directory level.
Signed-off-by: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create a helper function that extracts a tar archive and checks its
contents, modelled after check_zip in t5003.
Signed-off-by: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Create the directories b and c just before they are needed instead of
up front. For t5003 it turns out we don't need them at all. For t5000
it makes the coming modifications easier.
Signed-off-by: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of creating extra archives for testing substitutions, set the
attribute export-subst and overwrite the marked file with the expected
(expanded) content right between committing and archiving. Thus
placeholder expansion based on the committed content is performed with
each archive creation and the comparison with the contents of directory
a yields the correct result. We can then remove the special tests for
export-subst.
Signed-off-by: René Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is fewer lines of code, but more importantly, fixes a
bogus pointer offset. We are looking for "tar." in the
section, but later assume that the dot we found is at offset
9, not 3. This is a holdover from an earlier iteration of
767cf45 which called the section "tarfilter".
As a result, we could erroneously reject some filters with
dots in their name, as well as read uninitialized memory.
Reported by (and test by) René Scharfe.
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This change makes the code smaller and we can put it at the top of
the script, its rightful place as setup code.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
InfoZIP's unzip takes default parameters from the environment variable
UNZIP. Unset it in the test library and use GIT_UNZIP for specifying
alternate versions of the unzip command instead.
t0024 wasn't even using variable for the actual extraction. t5000
was, but when setting it to InfoZIP's unzip it would try to extract
from itself (because it treats the contents of $UNZIP as parameters),
which failed of course.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Factor out a function for checking the contents of ZIP archives. It
extracts their contents and compares them to the original files. This
removes some duplicate code. Tests that just create archives can lose
their UNZIP prerequisite.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After an entry has been streamed out, its CRC and sizes are written as
part of a data descriptor.
For simplicity, we make the buffer for the compressed chunks twice as
big as for the uncompressed ones, to be sure the result fit in even
if deflate makes them bigger.
t5000 verifies output. t1050 makes sure the command always respects
core.bigfilethreshold
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Write a data descriptor containing the CRC of the entry and its sizes
after streaming it out. For simplicity, do that only if we're storing
files (option -0) for now.
t5000 verifies output. t1050 makes sure the command always respects
core.bigfilethreshold
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
t5000 verifies output while t1050 makes sure the command always
respects core.bigfilethreshold
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Usually git is careful not to allow clients to fetch
arbitrary objects from the database; for example, objects
received via upload-pack must be reachable from a ref.
Upload-archive breaks this by feeding the client's tree-ish
directly to get_sha1, which will accept arbitrary hex sha1s,
reflogs, etc.
This is not a problem if all of your objects are publicly
reachable anyway (or at least public to anybody who can run
upload-archive). Or if you are making the repo available by
dumb protocols like http or rsync (in which case the client
can read your whole object db directly).
But for sites which allow access only through smart
protocols, clients may be able to fetch trees from commits
that exist in the server's object database but are not
referenced (e.g., because history was rewound).
This patch tightens upload-archive's lookup to use dwim_ref
rather than get_sha1. This means a remote client can only
fetch the tip of a named ref, not an arbitrary sha1 or
reflog entry.
This also restricts some legitimate requests, too:
1. Reachable non-tip commits, like:
git archive --remote=$url v1.0~5
2. Sub-trees of reachable commits, like:
git archive --remote=$url v1.7.7:Documentation
Local requests continue to use get_sha1, and are not
restricted at all.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The POSIX-function fork is not supported on Windows. Use our
start_command API instead, respawning ourselves in a special
"writer" mode to follow the alternate code path.
Remove the NOT_MINGW-prereq for t5000, as git-archive --remote
now works.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The POSIX-function fork is not supported on Windows. Use our
start_command API instead.
As this is the last call-site that depends on the fork-stub in
compat/mingw.h, remove that as well.
Add an undocumented flag to git-archive that tells it that the
action originated from a remote, so features can be disabled.
Thanks to Jeff King for work on this part.
Remove the NOT_MINGW-prereq for t5000, as git-archive --remote
now works.
Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
These depend on a working git-upload-archive, which is broken on Windows,
because it depends on fork().
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Some tar filters may be very expensive to run, so sites do
not want to expose them via upload-archive. This patch lets
users configure tar.<filter>.remote to turn them off.
By default, gzip filters are left on, as they are about as
expensive as creating zip archives.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This works exactly as if the user had configured it via:
[tar "tgz"]
command = gzip -cn
[tar "tar.gz"]
command = gzip -cn
but since it is so common, it's convenient to have it
builtin without the user needing to do anything.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It's common to pipe the tar output produce by "git archive"
through gzip or some other compressor. Locally, this can
easily be done by using a shell pipe. When requesting a
remote archive, though, it cannot be done through the
upload-archive interface.
This patch allows configurable tar filters, so that one
could define a "tar.gz" format that automatically pipes tar
output through gzip.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code to guess an output archive's format consumed any --format
options and built a new one. Jonathan noticed that it does so in an
unsafe way, risking to overflow the static buffer fmt_opt.
Change the code to keep the existing --format options intact and to only
add a new one if a format could be guessed based on the output file name.
The new option is added as the first one, allowing the existing ones to
overrule it, i.e. explicit --format options given on the command line win
over format guesses, as before.
To simplify the code further, format_from_name() is changed to return the
full --format option, thus no potentially dangerous sprintf() calls are
needed any more.
Reported-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With --prefix=string that does not end with a slash, the top-level entries
are written out with the specified prefix as expected, but no paths in the
directories are added.
Fix this by adding the prefix in write_archive_entry() instead of letting
get_pathspec() and read_tree_recursive() pair; they are designed to only
handle prefixes that are path components.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add a small test case for git archive --remote (and thus
git-upload-archive), which so far went untested.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We are interested in using archive mostly from a bare repository, so it
should not add .gitattributes to the work tree.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Many tests depend on that symbolic links work. This introduces a check
that sets the prerequisite tag SYMLINKS if the file system supports
symbolic links. Since so many tests have to check for this prerequisite,
we do the check in test-lib.sh, so that we don't need to repeat the test
in many scripts.
To check for 'ln -s' failures, you can use a FAT partition on Linux:
$ mkdosfs -C git-on-fat 1000000
$ sudo mount -o loop,uid=j6t,gid=users,shortname=winnt git-on-fat /mnt
Clone git to /mnt and
$ GIT_SKIP_TESTS='t0001.1[34] t0010 t1301 t403[34] t4129.[47] t5701.7
t7701.3 t9100 t9101.26 t9119 t9124.[67] t9200.10 t9600.6' \
make test
(These additionally skipped tests depend on POSIX permissions that FAT on
Linux does not provide.)
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Some tests report that some tests will be skipped. They used
'test_expect_success' with a trivially successful test. Nowadays we have
the helper function 'say' for this purpose.
In on case, 'say_color skip' is replaced by 'say' because the former is
not intended as a public API.
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Several old tests were written before test_cmp was introduced, convert
these to test_cmp.
If were are at it, fix the order of the arguments where necessary to
make expected come first, so the command shows how the test result
deviates from the correct output.
Signed-off-by: Miklos Vajna <vmiklos@frugalware.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When archiving a repository there is no way to specify a file as output.
This patch adds a new option "--output" that redirects the output to a
file instead of stdout.
Signed-off-by: Carlos Manuel Duclos Vergara <carlos.duclos@nokia.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The test was broken on admittedly broken combination of Windows, Cygwin,
and ActiveState Perl.
Signed-off-by: Alex Riesen <ariesen@harmanbecker.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This moves the call to git_config to a place where it doesn't break the
logic for using git archive in a bare repository but retains the fix to
make git archive respect core.autocrlf.
Tests are by René Scharfe.
Signed-off-by: Charles Bailey <charles@hashpling.org>
Tested-by: Deskin Miller <deskinm@umich.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Makefile records paths to a few programs in GIT-BUILD-OPTIONS file. These
paths need to be quoted twice: once to protect specials from the shell
that runs the generated GIT-BUILD-OPTIONS file, and again to protect them
(and the first level of quoting itself) from the shell that runs the
"echo" inside the Makefile.
You can test this by trying:
$ ln -s /bin/tar "$HOME/Tes' program/tar"
$ make TAR="$HOME/Tes' program/tar" test
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Paths marked with this attribute are not output to git-archive
output.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The output of 'tar tv' varies from system to system. In
particular, the t5000 was expecting to parse the date from
something like:
-rw-rw-r-- root/root 0 2008-05-13 04:27 file
but FreeBSD's tar produces this:
-rw-rw-r-- 0 root root 0 May 13 04:27 file
Instead of relying on tar's output, let's just extract the
file using tar and stat the result using perl.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ulrik Sverdrup noticed that git-archive doesn't correctly apply the attribute
export-subst when the option --prefix is given, too.
When it checked if a file has the attribute turned on, git-archive would try
to look up the full path -- including the prefix -- in .gitattributes. That's
wrong, as the prefix doesn't need to have any relation to any existing
directories, tracked or not.
This patch makes git-archive ignore the prefix when looking up if value of the
attribute export-subst for a file.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Before the strbuf conversion, result was a char pointer. The if
statement checked for it being not NULL, which meant that no
"$Format:...$" string had been found and no replacement had to be
made. format_subst() returned NULL in that case -- the caller
then simply kept the original file content, as it was unaffected
by the expansion.
The length of the string being 0 is not the same as the string
being NULL (expansion to an empty string vs. no expansion at all),
so checking result.len != 0 is not a full replacement for the old
NULL check.
However, I doubt the subtle optimization explained above resulted
in a notable speed-up anyway. Simplify the code and add the tail
of the file to the expanded string unconditionally.
[jc: added a test to expose the breakage this fixes]
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As suggested by Junio and Johannes, change the name of the former
attribute specfile to export-subst to indicate its function rather
than purpose and to make clear that it is not applied to working tree
files.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As suggested by Johannes, --pretty=format: placeholders in specfiles
need to be wrapped in $Format:...$ now. This syntax change restricts
the expansion of placeholders and makes it easier to use with files
that contain non-placeholder percent signs.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
