mirror of
https://github.com/git/git
synced 2024-08-28 12:09:19 +00:00
![Steffen Prohaska](/assets/img/avatar_default.png)
Checking out 2GB or more through an external filter (see test) fails
on Mac OS X 10.8.4 (12E55) for a 64-bit executable with:
error: read from external filter cat failed
error: cannot feed the input to external filter cat
error: cat died of signal 13
error: external filter cat failed 141
error: external filter cat failed
The reason is that read() immediately returns with EINVAL when asked
to read more than 2GB. According to POSIX [1], if the value of
nbyte passed to read() is greater than SSIZE_MAX, the result is
implementation-defined. The write function has the same restriction
[2]. Since OS X still supports running 32-bit executables, the
32-bit limit (SSIZE_MAX = INT_MAX = 2GB - 1) seems to be also
imposed on 64-bit executables under certain conditions. For write,
the problem has been addressed earlier [6c642a].
Address the problem for read() and write() differently, by limiting
size of IO chunks unconditionally on all platforms in xread() and
xwrite(). Large chunks only cause problems, like causing latencies
when killing the process, even if OS X was not buggy. Doing IO in
reasonably sized smaller chunks should have no negative impact on
performance.
The compat wrapper clipped_write() introduced earlier [6c642a] is
not needed anymore. It will be reverted in a separate commit. The
new test catches read and write problems.
Note that 'git add' exits with 0 even if it prints filtering errors
to stderr. The test, therefore, checks stderr. 'git add' should
probably be changed (sometime in another commit) to exit with
nonzero if filtering fails. The test could then be changed to use
test_must_fail.
Thanks to the following people for suggestions and testing:
Johannes Sixt <j6t@kdbg.org>
John Keeping <john@keeping.me.uk>
Jonathan Nieder <jrnieder@gmail.com>
Kyle J. McKay <mackyle@gmail.com>
Linus Torvalds <torvalds@linux-foundation.org>
Torsten Bögershausen <tboegi@web.de>
[1] http://pubs.opengroup.org/onlinepubs/009695399/functions/read.html
[2] http://pubs.opengroup.org/onlinepubs/009695399/functions/write.html
[6c642a] commit 6c642a8786
compate/clipped-write.c: large write(2) fails on Mac OS X/XNU
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
208 lines
5.5 KiB
Bash
Executable file
208 lines
5.5 KiB
Bash
Executable file
#!/bin/sh
|
|
|
|
test_description='blob conversion via gitattributes'
|
|
|
|
. ./test-lib.sh
|
|
|
|
cat <<EOF >rot13.sh
|
|
#!$SHELL_PATH
|
|
tr \
|
|
'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ' \
|
|
'nopqrstuvwxyzabcdefghijklmNOPQRSTUVWXYZABCDEFGHIJKLM'
|
|
EOF
|
|
chmod +x rot13.sh
|
|
|
|
test_expect_success setup '
|
|
git config filter.rot13.smudge ./rot13.sh &&
|
|
git config filter.rot13.clean ./rot13.sh &&
|
|
|
|
{
|
|
echo "*.t filter=rot13"
|
|
echo "*.i ident"
|
|
} >.gitattributes &&
|
|
|
|
{
|
|
echo a b c d e f g h i j k l m
|
|
echo n o p q r s t u v w x y z
|
|
echo '\''$Id$'\''
|
|
} >test &&
|
|
cat test >test.t &&
|
|
cat test >test.o &&
|
|
cat test >test.i &&
|
|
git add test test.t test.i &&
|
|
rm -f test test.t test.i &&
|
|
git checkout -- test test.t test.i
|
|
'
|
|
|
|
script='s/^\$Id: \([0-9a-f]*\) \$/\1/p'
|
|
|
|
test_expect_success check '
|
|
|
|
cmp test.o test &&
|
|
cmp test.o test.t &&
|
|
|
|
# ident should be stripped in the repository
|
|
git diff --raw --exit-code :test :test.i &&
|
|
id=$(git rev-parse --verify :test) &&
|
|
embedded=$(sed -ne "$script" test.i) &&
|
|
test "z$id" = "z$embedded" &&
|
|
|
|
git cat-file blob :test.t > test.r &&
|
|
|
|
./rot13.sh < test.o > test.t &&
|
|
cmp test.r test.t
|
|
'
|
|
|
|
# If an expanded ident ever gets into the repository, we want to make sure that
|
|
# it is collapsed before being expanded again on checkout
|
|
test_expect_success expanded_in_repo '
|
|
{
|
|
echo "File with expanded keywords"
|
|
echo "\$Id\$"
|
|
echo "\$Id:\$"
|
|
echo "\$Id: 0000000000000000000000000000000000000000 \$"
|
|
echo "\$Id: NoSpaceAtEnd\$"
|
|
echo "\$Id:NoSpaceAtFront \$"
|
|
echo "\$Id:NoSpaceAtEitherEnd\$"
|
|
echo "\$Id: NoTerminatingSymbol"
|
|
echo "\$Id: Foreign Commit With Spaces \$"
|
|
} >expanded-keywords.0 &&
|
|
|
|
{
|
|
cat expanded-keywords.0 &&
|
|
printf "\$Id: NoTerminatingSymbolAtEOF"
|
|
} >expanded-keywords &&
|
|
cat expanded-keywords >expanded-keywords-crlf &&
|
|
git add expanded-keywords expanded-keywords-crlf &&
|
|
git commit -m "File with keywords expanded" &&
|
|
id=$(git rev-parse --verify :expanded-keywords) &&
|
|
|
|
{
|
|
echo "File with expanded keywords"
|
|
echo "\$Id: $id \$"
|
|
echo "\$Id: $id \$"
|
|
echo "\$Id: $id \$"
|
|
echo "\$Id: $id \$"
|
|
echo "\$Id: $id \$"
|
|
echo "\$Id: $id \$"
|
|
echo "\$Id: NoTerminatingSymbol"
|
|
echo "\$Id: Foreign Commit With Spaces \$"
|
|
} >expected-output.0 &&
|
|
{
|
|
cat expected-output.0 &&
|
|
printf "\$Id: NoTerminatingSymbolAtEOF"
|
|
} >expected-output &&
|
|
{
|
|
append_cr <expected-output.0 &&
|
|
printf "\$Id: NoTerminatingSymbolAtEOF"
|
|
} >expected-output-crlf &&
|
|
{
|
|
echo "expanded-keywords ident"
|
|
echo "expanded-keywords-crlf ident text eol=crlf"
|
|
} >>.gitattributes &&
|
|
|
|
rm -f expanded-keywords expanded-keywords-crlf &&
|
|
|
|
git checkout -- expanded-keywords &&
|
|
test_cmp expanded-keywords expected-output &&
|
|
|
|
git checkout -- expanded-keywords-crlf &&
|
|
test_cmp expanded-keywords-crlf expected-output-crlf
|
|
'
|
|
|
|
# The use of %f in a filter definition is expanded to the path to
|
|
# the filename being smudged or cleaned. It must be shell escaped.
|
|
# First, set up some interesting file names and pet them in
|
|
# .gitattributes.
|
|
test_expect_success 'filter shell-escaped filenames' '
|
|
cat >argc.sh <<-EOF &&
|
|
#!$SHELL_PATH
|
|
cat >/dev/null
|
|
echo argc: \$# "\$@"
|
|
EOF
|
|
normal=name-no-magic &&
|
|
special="name with '\''sq'\'' and \$x" &&
|
|
echo some test text >"$normal" &&
|
|
echo some test text >"$special" &&
|
|
git add "$normal" "$special" &&
|
|
git commit -q -m "add files" &&
|
|
echo "name* filter=argc" >.gitattributes &&
|
|
|
|
# delete the files and check them out again, using a smudge filter
|
|
# that will count the args and echo the command-line back to us
|
|
git config filter.argc.smudge "sh ./argc.sh %f" &&
|
|
rm "$normal" "$special" &&
|
|
git checkout -- "$normal" "$special" &&
|
|
|
|
# make sure argc.sh counted the right number of args
|
|
echo "argc: 1 $normal" >expect &&
|
|
test_cmp expect "$normal" &&
|
|
echo "argc: 1 $special" >expect &&
|
|
test_cmp expect "$special" &&
|
|
|
|
# do the same thing, but with more args in the filter expression
|
|
git config filter.argc.smudge "sh ./argc.sh %f --my-extra-arg" &&
|
|
rm "$normal" "$special" &&
|
|
git checkout -- "$normal" "$special" &&
|
|
|
|
# make sure argc.sh counted the right number of args
|
|
echo "argc: 2 $normal --my-extra-arg" >expect &&
|
|
test_cmp expect "$normal" &&
|
|
echo "argc: 2 $special --my-extra-arg" >expect &&
|
|
test_cmp expect "$special" &&
|
|
:
|
|
'
|
|
|
|
test_expect_success 'required filter success' '
|
|
git config filter.required.smudge cat &&
|
|
git config filter.required.clean cat &&
|
|
git config filter.required.required true &&
|
|
|
|
echo "*.r filter=required" >.gitattributes &&
|
|
|
|
echo test >test.r &&
|
|
git add test.r &&
|
|
rm -f test.r &&
|
|
git checkout -- test.r
|
|
'
|
|
|
|
test_expect_success 'required filter smudge failure' '
|
|
git config filter.failsmudge.smudge false &&
|
|
git config filter.failsmudge.clean cat &&
|
|
git config filter.failsmudge.required true &&
|
|
|
|
echo "*.fs filter=failsmudge" >.gitattributes &&
|
|
|
|
echo test >test.fs &&
|
|
git add test.fs &&
|
|
rm -f test.fs &&
|
|
test_must_fail git checkout -- test.fs
|
|
'
|
|
|
|
test_expect_success 'required filter clean failure' '
|
|
git config filter.failclean.smudge cat &&
|
|
git config filter.failclean.clean false &&
|
|
git config filter.failclean.required true &&
|
|
|
|
echo "*.fc filter=failclean" >.gitattributes &&
|
|
|
|
echo test >test.fc &&
|
|
test_must_fail git add test.fc
|
|
'
|
|
|
|
test -n "$GIT_TEST_LONG" && test_set_prereq EXPENSIVE
|
|
|
|
test_expect_success EXPENSIVE 'filter large file' '
|
|
git config filter.largefile.smudge cat &&
|
|
git config filter.largefile.clean cat &&
|
|
for i in $(test_seq 1 2048); do printf "%1048576d" 1; done >2GB &&
|
|
echo "2GB filter=largefile" >.gitattributes &&
|
|
git add 2GB 2>err &&
|
|
! test -s err &&
|
|
rm -f 2GB &&
|
|
git checkout -- 2GB 2>err &&
|
|
! test -s err
|
|
'
|
|
|
|
test_done
|