git/t/t6135-pathspec-with-attrs.sh

424 lines
9.7 KiB
Bash
Raw Normal View History

#!/bin/sh
test_description='test labels in pathspecs'
. ./test-lib.sh
test_expect_success 'setup a tree' '
cat <<-\EOF >expect &&
fileA
fileAB
fileAC
fileB
fileBC
fileC
fileNoLabel
fileSetLabel
fileUnsetLabel
fileValue
fileWrongLabel
sub/fileA
sub/fileAB
sub/fileAC
sub/fileB
sub/fileBC
sub/fileC
sub/fileNoLabel
sub/fileSetLabel
sub/fileUnsetLabel
sub/fileValue
sub/fileWrongLabel
EOF
mkdir sub &&
while read path
do
echo content >$path &&
git add $path || return 1
done <expect &&
git commit -m "initial commit" &&
git ls-files >actual &&
test_cmp expect actual
'
test_expect_success 'pathspec with no attr' '
test_must_fail git ls-files ":(attr:)"
'
test_expect_success 'pathspec with labels and non existent .gitattributes' '
git ls-files ":(attr:label)" >actual &&
test_must_be_empty actual
'
test_expect_success 'pathspec with labels and non existent .gitattributes (2)' '
test_must_fail git grep content HEAD -- ":(attr:label)"
'
test_expect_success 'setup .gitattributes' '
cat <<-\EOF >.gitattributes &&
fileA labelA
fileB labelB
fileC labelC
fileAB labelA labelB
fileAC labelA labelC
fileBC labelB labelC
fileUnsetLabel -label
fileSetLabel label
fileValue label=foo
fileWrongLabel label☺
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
newFileA* labelA
newFileB* labelB
EOF
dir: match "attr" pathspec magic with correct paths The match_pathspec_item() function takes "prefix" value, allowing a caller to chop off the common leading prefix of pathspec pattern strings from the path and only use the remainder of the path to match the pathspec patterns (after chopping the same leading prefix of them, of course). This "common leading prefix" optimization has two main features: * discard the entries in the in-core index that are outside of the common leading prefix; if you are doing "ls-files one/a one/b", we know all matches must be from "one/", so first the code discards all entries outside the "one/" directory from the in-core index. This allows us to work on a smaller dataset. * allow skipping the comparison of the leading bytes when matching pathspec with path. When "ls-files" finds the path "one/a/1" in the in-core index given "one/a" and "one/b" as the pathspec, knowing that common leading prefix "one/" was found lets the pathspec matchinery not to bother comparing "one/" part, and allows it to feed "a/1" down, as long as the pathspec element "one/a" gets corresponding adjustment to "a". When the "attr" pathspec magic is in effect, however, the current code breaks down. The attributes, other than the ones that are built-in and the ones that come from the $GIT_DIR/info/attributes file and the top-level .gitattributes file, are lazily read from the filesystem on-demand, as we encounter each path and ask if it matches the pathspec. For example, if you say "git ls-files "(attr:label)sub/" in a repository with a file "sub/file" that is given the 'label' attribute in "sub/.gitattributes": * The common prefix optimization finds that "sub/" is the common prefix and prunes the in-core index so that it has only entries inside that directory. This is desirable. * The code then walks the in-core index, finds "sub/file", and eventually asks do_match_pathspec() if it matches the given pathspec. * do_match_pathspec() calls match_pathspec_item() _after_ stripping the common prefix "sub/" from the path, giving it "file", plus the length of the common prefix (4-bytes), so that the pathspec element "(attr:label)sub/" can be treated as if it were "(attr:label)". The last one is what breaks the match in the current code, as the pathspec subsystem ends up asking the attribute subsystem to find the attribute attached to the path "file". We need to ask about the attributes on "sub/file" when calling match_pathspec_attrs(); this can be done by looking at "prefix" bytes before the beginning of "name", which is the same trick already used by another piece of the code in the same match_pathspec_item() function. Unfortunately this was not discovered so far because the code works with slightly different arguments, e.g. $ git ls-files "(attr:label)sub" $ git ls-files "(attr:label)sub/" "no/such/dir/" would have reported "sub/file" as a path with the 'label' attribute just fine, because neither would trigger the common prefix optimization. Reported-by: Matthew Hughes <mhughes@uw.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-08 21:35:33 +00:00
echo fileSetLabel label1 >sub/.gitattributes &&
git add .gitattributes sub/.gitattributes &&
git commit -m "add attributes"
'
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
test_expect_success 'setup .gitignore' '
cat <<-\EOF >.gitignore &&
actual
expect
pathspec_file
EOF
git add .gitignore &&
git commit -m "add gitignore"
'
test_expect_success 'check specific set attr' '
cat <<-\EOF >expect &&
fileSetLabel
sub/fileSetLabel
EOF
git ls-files ":(attr:label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check set attr with pathspec pattern' '
echo sub/fileSetLabel >expect &&
git ls-files ":(attr:label)sub" >actual &&
test_cmp expect actual &&
git ls-files ":(attr:label)sub/" >actual &&
test_cmp expect actual
'
test_expect_success 'check specific set attr in tree-ish' '
cat <<-\EOF >expect &&
HEAD:fileSetLabel
HEAD:sub/fileSetLabel
EOF
git grep -l content HEAD ":(attr:label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check specific set attr with pathspec pattern in tree-ish' '
echo HEAD:sub/fileSetLabel >expect &&
git grep -l content HEAD ":(attr:label)sub" >actual &&
test_cmp expect actual &&
git grep -l content HEAD ":(attr:label)sub/" >actual &&
test_cmp expect actual
'
test_expect_success 'check specific unset attr' '
cat <<-\EOF >expect &&
fileUnsetLabel
sub/fileUnsetLabel
EOF
git ls-files ":(attr:-label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check specific unset attr (2)' '
cat <<-\EOF >expect &&
HEAD:fileUnsetLabel
HEAD:sub/fileUnsetLabel
EOF
git grep -l content HEAD ":(attr:-label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check specific value attr' '
cat <<-\EOF >expect &&
fileValue
sub/fileValue
EOF
git ls-files ":(attr:label=foo)" >actual &&
test_cmp expect actual &&
git ls-files ":(attr:label=bar)" >actual &&
test_must_be_empty actual
'
test_expect_success 'check specific value attr (2)' '
cat <<-\EOF >expect &&
HEAD:fileValue
HEAD:sub/fileValue
EOF
git grep -l content HEAD ":(attr:label=foo)" >actual &&
test_cmp expect actual &&
test_must_fail git grep -l content HEAD ":(attr:label=bar)"
'
test_expect_success 'check unspecified attr' '
cat <<-\EOF >expect &&
.gitattributes
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
.gitignore
fileA
fileAB
fileAC
fileB
fileBC
fileC
fileNoLabel
fileWrongLabel
dir: match "attr" pathspec magic with correct paths The match_pathspec_item() function takes "prefix" value, allowing a caller to chop off the common leading prefix of pathspec pattern strings from the path and only use the remainder of the path to match the pathspec patterns (after chopping the same leading prefix of them, of course). This "common leading prefix" optimization has two main features: * discard the entries in the in-core index that are outside of the common leading prefix; if you are doing "ls-files one/a one/b", we know all matches must be from "one/", so first the code discards all entries outside the "one/" directory from the in-core index. This allows us to work on a smaller dataset. * allow skipping the comparison of the leading bytes when matching pathspec with path. When "ls-files" finds the path "one/a/1" in the in-core index given "one/a" and "one/b" as the pathspec, knowing that common leading prefix "one/" was found lets the pathspec matchinery not to bother comparing "one/" part, and allows it to feed "a/1" down, as long as the pathspec element "one/a" gets corresponding adjustment to "a". When the "attr" pathspec magic is in effect, however, the current code breaks down. The attributes, other than the ones that are built-in and the ones that come from the $GIT_DIR/info/attributes file and the top-level .gitattributes file, are lazily read from the filesystem on-demand, as we encounter each path and ask if it matches the pathspec. For example, if you say "git ls-files "(attr:label)sub/" in a repository with a file "sub/file" that is given the 'label' attribute in "sub/.gitattributes": * The common prefix optimization finds that "sub/" is the common prefix and prunes the in-core index so that it has only entries inside that directory. This is desirable. * The code then walks the in-core index, finds "sub/file", and eventually asks do_match_pathspec() if it matches the given pathspec. * do_match_pathspec() calls match_pathspec_item() _after_ stripping the common prefix "sub/" from the path, giving it "file", plus the length of the common prefix (4-bytes), so that the pathspec element "(attr:label)sub/" can be treated as if it were "(attr:label)". The last one is what breaks the match in the current code, as the pathspec subsystem ends up asking the attribute subsystem to find the attribute attached to the path "file". We need to ask about the attributes on "sub/file" when calling match_pathspec_attrs(); this can be done by looking at "prefix" bytes before the beginning of "name", which is the same trick already used by another piece of the code in the same match_pathspec_item() function. Unfortunately this was not discovered so far because the code works with slightly different arguments, e.g. $ git ls-files "(attr:label)sub" $ git ls-files "(attr:label)sub/" "no/such/dir/" would have reported "sub/file" as a path with the 'label' attribute just fine, because neither would trigger the common prefix optimization. Reported-by: Matthew Hughes <mhughes@uw.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-08 21:35:33 +00:00
sub/.gitattributes
sub/fileA
sub/fileAB
sub/fileAC
sub/fileB
sub/fileBC
sub/fileC
sub/fileNoLabel
sub/fileWrongLabel
EOF
git ls-files ":(attr:!label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check unspecified attr (2)' '
cat <<-\EOF >expect &&
HEAD:.gitattributes
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
HEAD:.gitignore
HEAD:fileA
HEAD:fileAB
HEAD:fileAC
HEAD:fileB
HEAD:fileBC
HEAD:fileC
HEAD:fileNoLabel
HEAD:fileWrongLabel
dir: match "attr" pathspec magic with correct paths The match_pathspec_item() function takes "prefix" value, allowing a caller to chop off the common leading prefix of pathspec pattern strings from the path and only use the remainder of the path to match the pathspec patterns (after chopping the same leading prefix of them, of course). This "common leading prefix" optimization has two main features: * discard the entries in the in-core index that are outside of the common leading prefix; if you are doing "ls-files one/a one/b", we know all matches must be from "one/", so first the code discards all entries outside the "one/" directory from the in-core index. This allows us to work on a smaller dataset. * allow skipping the comparison of the leading bytes when matching pathspec with path. When "ls-files" finds the path "one/a/1" in the in-core index given "one/a" and "one/b" as the pathspec, knowing that common leading prefix "one/" was found lets the pathspec matchinery not to bother comparing "one/" part, and allows it to feed "a/1" down, as long as the pathspec element "one/a" gets corresponding adjustment to "a". When the "attr" pathspec magic is in effect, however, the current code breaks down. The attributes, other than the ones that are built-in and the ones that come from the $GIT_DIR/info/attributes file and the top-level .gitattributes file, are lazily read from the filesystem on-demand, as we encounter each path and ask if it matches the pathspec. For example, if you say "git ls-files "(attr:label)sub/" in a repository with a file "sub/file" that is given the 'label' attribute in "sub/.gitattributes": * The common prefix optimization finds that "sub/" is the common prefix and prunes the in-core index so that it has only entries inside that directory. This is desirable. * The code then walks the in-core index, finds "sub/file", and eventually asks do_match_pathspec() if it matches the given pathspec. * do_match_pathspec() calls match_pathspec_item() _after_ stripping the common prefix "sub/" from the path, giving it "file", plus the length of the common prefix (4-bytes), so that the pathspec element "(attr:label)sub/" can be treated as if it were "(attr:label)". The last one is what breaks the match in the current code, as the pathspec subsystem ends up asking the attribute subsystem to find the attribute attached to the path "file". We need to ask about the attributes on "sub/file" when calling match_pathspec_attrs(); this can be done by looking at "prefix" bytes before the beginning of "name", which is the same trick already used by another piece of the code in the same match_pathspec_item() function. Unfortunately this was not discovered so far because the code works with slightly different arguments, e.g. $ git ls-files "(attr:label)sub" $ git ls-files "(attr:label)sub/" "no/such/dir/" would have reported "sub/file" as a path with the 'label' attribute just fine, because neither would trigger the common prefix optimization. Reported-by: Matthew Hughes <mhughes@uw.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-08 21:35:33 +00:00
HEAD:sub/.gitattributes
HEAD:sub/fileA
HEAD:sub/fileAB
HEAD:sub/fileAC
HEAD:sub/fileB
HEAD:sub/fileBC
HEAD:sub/fileC
HEAD:sub/fileNoLabel
HEAD:sub/fileWrongLabel
EOF
git grep -l ^ HEAD ":(attr:!label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check multiple unspecified attr' '
cat <<-\EOF >expect &&
.gitattributes
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
.gitignore
fileC
fileNoLabel
fileWrongLabel
dir: match "attr" pathspec magic with correct paths The match_pathspec_item() function takes "prefix" value, allowing a caller to chop off the common leading prefix of pathspec pattern strings from the path and only use the remainder of the path to match the pathspec patterns (after chopping the same leading prefix of them, of course). This "common leading prefix" optimization has two main features: * discard the entries in the in-core index that are outside of the common leading prefix; if you are doing "ls-files one/a one/b", we know all matches must be from "one/", so first the code discards all entries outside the "one/" directory from the in-core index. This allows us to work on a smaller dataset. * allow skipping the comparison of the leading bytes when matching pathspec with path. When "ls-files" finds the path "one/a/1" in the in-core index given "one/a" and "one/b" as the pathspec, knowing that common leading prefix "one/" was found lets the pathspec matchinery not to bother comparing "one/" part, and allows it to feed "a/1" down, as long as the pathspec element "one/a" gets corresponding adjustment to "a". When the "attr" pathspec magic is in effect, however, the current code breaks down. The attributes, other than the ones that are built-in and the ones that come from the $GIT_DIR/info/attributes file and the top-level .gitattributes file, are lazily read from the filesystem on-demand, as we encounter each path and ask if it matches the pathspec. For example, if you say "git ls-files "(attr:label)sub/" in a repository with a file "sub/file" that is given the 'label' attribute in "sub/.gitattributes": * The common prefix optimization finds that "sub/" is the common prefix and prunes the in-core index so that it has only entries inside that directory. This is desirable. * The code then walks the in-core index, finds "sub/file", and eventually asks do_match_pathspec() if it matches the given pathspec. * do_match_pathspec() calls match_pathspec_item() _after_ stripping the common prefix "sub/" from the path, giving it "file", plus the length of the common prefix (4-bytes), so that the pathspec element "(attr:label)sub/" can be treated as if it were "(attr:label)". The last one is what breaks the match in the current code, as the pathspec subsystem ends up asking the attribute subsystem to find the attribute attached to the path "file". We need to ask about the attributes on "sub/file" when calling match_pathspec_attrs(); this can be done by looking at "prefix" bytes before the beginning of "name", which is the same trick already used by another piece of the code in the same match_pathspec_item() function. Unfortunately this was not discovered so far because the code works with slightly different arguments, e.g. $ git ls-files "(attr:label)sub" $ git ls-files "(attr:label)sub/" "no/such/dir/" would have reported "sub/file" as a path with the 'label' attribute just fine, because neither would trigger the common prefix optimization. Reported-by: Matthew Hughes <mhughes@uw.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-08 21:35:33 +00:00
sub/.gitattributes
sub/fileC
sub/fileNoLabel
sub/fileWrongLabel
EOF
git ls-files ":(attr:!labelB !labelA !label)" >actual &&
test_cmp expect actual
'
test_expect_success 'check label with more labels but excluded path' '
cat <<-\EOF >expect &&
fileAB
fileB
fileBC
EOF
git ls-files ":(attr:labelB)" ":(exclude)sub/" >actual &&
test_cmp expect actual
'
test_expect_success 'check label excluding other labels' '
cat <<-\EOF >expect &&
fileAB
fileB
fileBC
sub/fileAB
sub/fileB
EOF
git ls-files ":(attr:labelB)" ":(exclude,attr:labelC)sub/" >actual &&
test_cmp expect actual
'
test_expect_success 'fail on multiple attr specifiers in one pathspec item' '
test_must_fail git ls-files . ":(attr:labelB,attr:labelC)" 2>actual &&
test_grep "Only one" actual
'
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
test_expect_success 'fail if attr magic is used in places not implemented' '
# The main purpose of this test is to check that we actually fail
# when you attempt to use attr magic in commands that do not implement
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
# attr magic. This test does not advocate check-ignore to stay that way.
# When you teach the command to grok the pathspec, you need to find
# another command to replace it for the test.
test_must_fail git check-ignore ":(attr:labelB)" 2>actual &&
test_grep "magic not supported" actual
'
attr: enable attr pathspec magic for git-add and git-stash Allow users to limit or exclude files based on file attributes during git-add and git-stash. For example, the chromium project would like to use $ git add . ':(exclude,attr:submodule)' as submodules are managed by an external tool, forbidding end users to record changes with "git add". Allowing "git add" to often records changes that users do not want in their commits. This commit does not change any attr magic implementation. It is only adding attr as an allowed pathspec in git-add and git-stash, which was previously blocked by GUARD_PATHSPEC and a pathspec mask in parse_pathspec()). However, we fix a bug in prefix_magic() where attr values were unintentionally removed. This was triggerable when parse_pathspec() is called with PATHSPEC_PREFIX_ORIGIN as a flag, which was the case for git-stash (Bug originally filed here [*]) Furthermore, while other commands hit this code path it did not result in unexpected behavior because this bug only impacts the pathspec->items->original field which is NOT used to filter paths. However, git-stash does use pathspec->items->original when building args used to call other git commands. (See add_pathspecs() usage and implementation in stash.c) It is possible that when the attr pathspec feature was first added in b0db704652 (pathspec: allow querying for attributes, 2017-03-13), "PATHSPEC_ATTR" was just unintentionally left out of a few GUARD_PATHSPEC() invocations. Later, to get a more user-friendly error message when attr was used with git-add, PATHSPEC_ATTR was added as a mask to git-add's invocation of parse_pathspec() 84d938b732 (add: do not accept pathspec magic 'attr', 2018-09-18). However, this user-friendly error message was never added for git-stash. [Reference] * https://lore.kernel.org/git/CAMmZTi-0QKtj7Q=sbC5qhipGsQxJFOY-Qkk1jfkRYwfF5FcUVg@mail.gmail.com/) Signed-off-by: Joanna Wang <jojwang@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-11-03 16:34:48 +00:00
test_expect_success 'check that attr magic works for git stash push' '
cat <<-\EOF >expect &&
A sub/newFileA-foo
EOF
>sub/newFileA-foo &&
>sub/newFileB-foo &&
git stash push --include-untracked -- ":(exclude,attr:labelB)" &&
git stash show --include-untracked --name-status >actual &&
test_cmp expect actual
'
test_expect_success 'check that attr magic works for git add --all' '
cat <<-\EOF >expect &&
sub/newFileA-foo
EOF
>sub/newFileA-foo &&
>sub/newFileB-foo &&
git add --all ":(exclude,attr:labelB)" &&
git diff --name-only --cached >actual &&
git restore -W -S . &&
test_cmp expect actual
'
test_expect_success 'check that attr magic works for git add -u' '
cat <<-\EOF >expect &&
sub/fileA
EOF
>sub/newFileA-foo &&
>sub/newFileB-foo &&
>sub/fileA &&
>sub/fileB &&
git add -u ":(exclude,attr:labelB)" &&
git diff --name-only --cached >actual &&
git restore -S -W . && rm sub/new* &&
test_cmp expect actual
'
test_expect_success 'check that attr magic works for git add <path>' '
cat <<-\EOF >expect &&
fileA
fileB
sub/fileA
EOF
>fileA &&
>fileB &&
>sub/fileA &&
>sub/fileB &&
git add ":(exclude,attr:labelB)sub/*" &&
git diff --name-only --cached >actual &&
git restore -S -W . &&
test_cmp expect actual
'
test_expect_success 'check that attr magic works for git -add .' '
cat <<-\EOF >expect &&
sub/fileA
EOF
>fileA &&
>fileB &&
>sub/fileA &&
>sub/fileB &&
cd sub &&
git add . ":(exclude,attr:labelB)" &&
cd .. &&
git diff --name-only --cached >actual &&
git restore -S -W . &&
test_cmp expect actual
'
test_expect_success 'check that attr magic works for git add --pathspec-from-file' '
cat <<-\EOF >pathspec_file &&
:(exclude,attr:labelB)
EOF
cat <<-\EOF >expect &&
sub/newFileA-foo
EOF
>sub/newFileA-foo &&
>sub/newFileB-foo &&
git add --all --pathspec-from-file=pathspec_file &&
git diff --name-only --cached >actual &&
test_cmp expect actual
'
test_expect_success 'abort on giving invalid label on the command line' '
test_must_fail git ls-files . ":(attr:☺)"
'
test_expect_success 'abort on asking for wrong magic' '
test_must_fail git ls-files . ":(attr:-label=foo)" &&
test_must_fail git ls-files . ":(attr:!label=foo)"
'
test_expect_success 'check attribute list' '
cat <<-EOF >>.gitattributes &&
* whitespace=indent,trail,space
EOF
git ls-files ":(attr:whitespace=indent\,trail\,space)" >actual &&
git ls-files >expect &&
test_cmp expect actual
'
test_expect_success 'backslash cannot be the last character' '
test_must_fail git ls-files ":(attr:label=foo\\ labelA=bar)" 2>actual &&
test_grep "not allowed as last character in attr value" actual
'
test_expect_success 'backslash cannot be used as a value' '
test_must_fail git ls-files ":(attr:label=f\\\oo)" 2>actual &&
test_grep "for value matching" actual
'
dir: match "attr" pathspec magic with correct paths The match_pathspec_item() function takes "prefix" value, allowing a caller to chop off the common leading prefix of pathspec pattern strings from the path and only use the remainder of the path to match the pathspec patterns (after chopping the same leading prefix of them, of course). This "common leading prefix" optimization has two main features: * discard the entries in the in-core index that are outside of the common leading prefix; if you are doing "ls-files one/a one/b", we know all matches must be from "one/", so first the code discards all entries outside the "one/" directory from the in-core index. This allows us to work on a smaller dataset. * allow skipping the comparison of the leading bytes when matching pathspec with path. When "ls-files" finds the path "one/a/1" in the in-core index given "one/a" and "one/b" as the pathspec, knowing that common leading prefix "one/" was found lets the pathspec matchinery not to bother comparing "one/" part, and allows it to feed "a/1" down, as long as the pathspec element "one/a" gets corresponding adjustment to "a". When the "attr" pathspec magic is in effect, however, the current code breaks down. The attributes, other than the ones that are built-in and the ones that come from the $GIT_DIR/info/attributes file and the top-level .gitattributes file, are lazily read from the filesystem on-demand, as we encounter each path and ask if it matches the pathspec. For example, if you say "git ls-files "(attr:label)sub/" in a repository with a file "sub/file" that is given the 'label' attribute in "sub/.gitattributes": * The common prefix optimization finds that "sub/" is the common prefix and prunes the in-core index so that it has only entries inside that directory. This is desirable. * The code then walks the in-core index, finds "sub/file", and eventually asks do_match_pathspec() if it matches the given pathspec. * do_match_pathspec() calls match_pathspec_item() _after_ stripping the common prefix "sub/" from the path, giving it "file", plus the length of the common prefix (4-bytes), so that the pathspec element "(attr:label)sub/" can be treated as if it were "(attr:label)". The last one is what breaks the match in the current code, as the pathspec subsystem ends up asking the attribute subsystem to find the attribute attached to the path "file". We need to ask about the attributes on "sub/file" when calling match_pathspec_attrs(); this can be done by looking at "prefix" bytes before the beginning of "name", which is the same trick already used by another piece of the code in the same match_pathspec_item() function. Unfortunately this was not discovered so far because the code works with slightly different arguments, e.g. $ git ls-files "(attr:label)sub" $ git ls-files "(attr:label)sub/" "no/such/dir/" would have reported "sub/file" as a path with the 'label' attribute just fine, because neither would trigger the common prefix optimization. Reported-by: Matthew Hughes <mhughes@uw.co.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-07-08 21:35:33 +00:00
test_expect_success 'reading from .gitattributes in a subdirectory (1)' '
git ls-files ":(attr:label1)" >actual &&
test_write_lines "sub/fileSetLabel" >expect &&
test_cmp expect actual
'
test_expect_success 'reading from .gitattributes in a subdirectory (2)' '
git ls-files ":(attr:label1)sub" >actual &&
test_write_lines "sub/fileSetLabel" >expect &&
test_cmp expect actual
'
test_expect_success 'reading from .gitattributes in a subdirectory (3)' '
git ls-files ":(attr:label1)sub/" >actual &&
test_write_lines "sub/fileSetLabel" >expect &&
test_cmp expect actual
'
test_expect_success POSIXPERM 'pathspec with builtin_objectmode attr can be used' '
>mode_exec_file_1 &&
git status -s ":(attr:builtin_objectmode=100644)mode_exec_*" >actual &&
echo ?? mode_exec_file_1 >expect &&
test_cmp expect actual &&
git add mode_exec_file_1 &&
chmod +x mode_exec_file_1 &&
git status -s ":(attr:builtin_objectmode=100755)mode_exec_*" >actual &&
echo AM mode_exec_file_1 >expect &&
test_cmp expect actual
'
test_expect_success POSIXPERM 'builtin_objectmode attr can be excluded' '
>mode_1_regular &&
>mode_1_exec &&
chmod +x mode_1_exec &&
git status -s ":(exclude,attr:builtin_objectmode=100644)" "mode_1_*" >actual &&
echo ?? mode_1_exec >expect &&
test_cmp expect actual &&
git status -s ":(exclude,attr:builtin_objectmode=100755)" "mode_1_*" >actual &&
echo ?? mode_1_regular >expect &&
test_cmp expect actual
'
test_done