git/t/t7411-submodule-config.sh

259 lines
7.3 KiB
Bash
Raw Normal View History

submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
#!/bin/sh
#
# Copyright (c) 2014 Heiko Voigt
#
test_description='Test submodules config cache infrastructure
This test verifies that parsing .gitmodules configurations directly
from the database and from the worktree works.
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
'
TEST_NO_CREATE_REPO=1
. ./test-lib.sh
test_expect_success 'submodule config cache setup' '
mkdir submodule &&
(cd submodule &&
git init &&
echo a >a &&
git add . &&
git commit -ma
) &&
mkdir super &&
(cd super &&
git init &&
git submodule add ../submodule &&
git submodule add ../submodule a &&
git commit -m "add as submodule and as a" &&
git mv a b &&
git commit -m "move a to b"
)
'
test_expect_success 'configuration parsing with error' '
test_when_finished "rm -rf repo" &&
test_create_repo repo &&
cat >repo/.gitmodules <<-\EOF &&
[submodule "s"]
path
ignore
EOF
(
cd repo &&
test_must_fail test-tool submodule-config "" s 2>actual &&
test_i18ngrep "bad config" actual
)
'
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
cat >super/expect <<EOF
Submodule name: 'a' for path 'a'
Submodule name: 'a' for path 'b'
Submodule name: 'submodule' for path 'submodule'
Submodule name: 'submodule' for path 'submodule'
EOF
test_expect_success 'test parsing and lookup of submodule config by path' '
(cd super &&
test-tool submodule-config \
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
HEAD^ a \
HEAD b \
HEAD^ submodule \
HEAD submodule \
>actual &&
test_cmp expect actual
)
'
test_expect_success 'test parsing and lookup of submodule config by name' '
(cd super &&
test-tool submodule-config --name \
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
HEAD^ a \
HEAD a \
HEAD^ submodule \
HEAD submodule \
>actual &&
test_cmp expect actual
)
'
cat >super/expect_error <<EOF
Submodule name: 'a' for path 'b'
Submodule name: 'submodule' for path 'submodule'
EOF
test_expect_success 'error in history of one submodule config lets continue, stderr message contains blob ref' '
ORIG=$(git -C super rev-parse HEAD) &&
test_when_finished "git -C super reset --hard $ORIG" &&
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
(cd super &&
cp .gitmodules .gitmodules.bak &&
echo " value = \"" >>.gitmodules &&
git add .gitmodules &&
mv .gitmodules.bak .gitmodules &&
git commit -m "add error" &&
sha1=$(git rev-parse HEAD) &&
test-tool submodule-config \
HEAD b \
HEAD submodule \
>actual \
2>actual_stderr &&
test_cmp expect_error actual &&
test_i18ngrep "submodule-blob $sha1:.gitmodules" actual_stderr >/dev/null
)
'
test_expect_success 'using different treeishs works' '
(
cd super &&
git tag new_tag &&
tree=$(git rev-parse HEAD^{tree}) &&
commit=$(git rev-parse HEAD^{commit}) &&
test-tool submodule-config $commit b >expect &&
test-tool submodule-config $tree b >actual.1 &&
test-tool submodule-config new_tag b >actual.2 &&
test_cmp expect actual.1 &&
test_cmp expect actual.2
)
'
test_expect_success 'error in history in fetchrecursesubmodule lets continue' '
ORIG=$(git -C super rev-parse HEAD) &&
test_when_finished "git -C super reset --hard $ORIG" &&
(cd super &&
git config -f .gitmodules \
submodule.submodule.fetchrecursesubmodules blabla &&
git add .gitmodules &&
git config --unset -f .gitmodules \
submodule.submodule.fetchrecursesubmodules &&
git commit -m "add error in fetchrecursesubmodules" &&
test-tool submodule-config \
HEAD b \
HEAD submodule \
>actual &&
test_cmp expect_error actual
)
'
submodule: support reading .gitmodules when it's not in the working tree When the .gitmodules file is not available in the working tree, try using the content from the index and from the current branch. This covers the case when the file is part of the repository but for some reason it is not checked out, for example because of a sparse checkout. This makes it possible to use at least the 'git submodule' commands which *read* the gitmodules configuration file without fully populating the working tree. Writing to .gitmodules will still require that the file is checked out, so check for that before calling config_set_in_gitmodules_file_gently. Add a similar check also in git-submodule.sh::cmd_add() to anticipate the eventual failure of the "git submodule add" command when .gitmodules is not safely writeable; this prevents the command from leaving the repository in a spurious state (e.g. the submodule repository was cloned but .gitmodules was not updated because config_set_in_gitmodules_file_gently failed). Moreover, since config_from_gitmodules() now accesses the global object store, it is necessary to protect all code paths which call the function against concurrent access to the global object store. Currently this only happens in builtin/grep.c::grep_submodules(), so call grep_read_lock() before invoking code involving config_from_gitmodules(). Finally, add t7418-submodule-sparse-gitmodules.sh to verify that reading from .gitmodules succeeds and that writing to it fails when the file is not checked out. NOTE: there is one rare case where this new feature does not work properly yet: nested submodules without .gitmodules in their working tree. This has been documented with a warning and a test_expect_failure item in t7814, and in this case the current behavior is not altered: no config is read. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-25 16:18:12 +00:00
test_expect_success 'reading submodules config from the working tree with "submodule--helper config"' '
(cd super &&
echo "../submodule" >expect &&
git submodule--helper config submodule.submodule.url >actual &&
test_cmp expect actual
)
'
test_expect_success 'unsetting submodules config from the working tree with "submodule--helper config --unset"' '
(cd super &&
git submodule--helper config --unset submodule.submodule.url &&
git submodule--helper config submodule.submodule.url >actual &&
test_must_be_empty actual
)
'
test_expect_success 'writing submodules config with "submodule--helper config"' '
(cd super &&
echo "new_url" >expect &&
git submodule--helper config submodule.submodule.url "new_url" &&
git submodule--helper config submodule.submodule.url >actual &&
test_cmp expect actual
)
'
test_expect_success 'overwriting unstaged submodules config with "submodule--helper config"' '
test_when_finished "git -C super checkout .gitmodules" &&
(cd super &&
echo "newer_url" >expect &&
git submodule--helper config submodule.submodule.url "newer_url" &&
git submodule--helper config submodule.submodule.url >actual &&
test_cmp expect actual
)
'
test_expect_success 'writeable .gitmodules when it is in the working tree' '
git -C super submodule--helper config --check-writeable
'
test_expect_success 'writeable .gitmodules when it is nowhere in the repository' '
ORIG=$(git -C super rev-parse HEAD) &&
test_when_finished "git -C super reset --hard $ORIG" &&
(cd super &&
git rm .gitmodules &&
git commit -m "remove .gitmodules from the current branch" &&
git submodule--helper config --check-writeable
)
'
test_expect_success 'non-writeable .gitmodules when it is in the index but not in the working tree' '
test_when_finished "git -C super checkout .gitmodules" &&
(cd super &&
rm -f .gitmodules &&
test_must_fail git submodule--helper config --check-writeable
)
'
test_expect_success 'non-writeable .gitmodules when it is in the current branch but not in the index' '
ORIG=$(git -C super rev-parse HEAD) &&
test_when_finished "git -C super reset --hard $ORIG" &&
(cd super &&
git rm .gitmodules &&
test_must_fail git submodule--helper config --check-writeable
)
'
submodule: support reading .gitmodules when it's not in the working tree When the .gitmodules file is not available in the working tree, try using the content from the index and from the current branch. This covers the case when the file is part of the repository but for some reason it is not checked out, for example because of a sparse checkout. This makes it possible to use at least the 'git submodule' commands which *read* the gitmodules configuration file without fully populating the working tree. Writing to .gitmodules will still require that the file is checked out, so check for that before calling config_set_in_gitmodules_file_gently. Add a similar check also in git-submodule.sh::cmd_add() to anticipate the eventual failure of the "git submodule add" command when .gitmodules is not safely writeable; this prevents the command from leaving the repository in a spurious state (e.g. the submodule repository was cloned but .gitmodules was not updated because config_set_in_gitmodules_file_gently failed). Moreover, since config_from_gitmodules() now accesses the global object store, it is necessary to protect all code paths which call the function against concurrent access to the global object store. Currently this only happens in builtin/grep.c::grep_submodules(), so call grep_read_lock() before invoking code involving config_from_gitmodules(). Finally, add t7418-submodule-sparse-gitmodules.sh to verify that reading from .gitmodules succeeds and that writing to it fails when the file is not checked out. NOTE: there is one rare case where this new feature does not work properly yet: nested submodules without .gitmodules in their working tree. This has been documented with a warning and a test_expect_failure item in t7814, and in this case the current behavior is not altered: no config is read. Signed-off-by: Antonio Ospite <ao2@ao2.it> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-25 16:18:12 +00:00
test_expect_success 'reading submodules config from the index when .gitmodules is not in the working tree' '
ORIG=$(git -C super rev-parse HEAD) &&
test_when_finished "git -C super reset --hard $ORIG" &&
(cd super &&
git submodule--helper config submodule.submodule.url "staged_url" &&
git add .gitmodules &&
rm -f .gitmodules &&
echo "staged_url" >expect &&
git submodule--helper config submodule.submodule.url >actual &&
test_cmp expect actual
)
'
test_expect_success 'reading submodules config from the current branch when .gitmodules is not in the index' '
ORIG=$(git -C super rev-parse HEAD) &&
test_when_finished "git -C super reset --hard $ORIG" &&
(cd super &&
git rm .gitmodules &&
echo "../submodule" >expect &&
git submodule--helper config submodule.submodule.url >actual &&
test_cmp expect actual
)
'
test_expect_success 'reading nested submodules config' '
(cd super &&
git init submodule/nested_submodule &&
echo "a" >submodule/nested_submodule/a &&
git -C submodule/nested_submodule add a &&
git -C submodule/nested_submodule commit -m "add a" &&
git -C submodule submodule add ./nested_submodule &&
git -C submodule add nested_submodule &&
git -C submodule commit -m "added nested_submodule" &&
git add submodule &&
git commit -m "updated submodule" &&
echo "./nested_submodule" >expect &&
test-tool submodule-nested-repo-config \
submodule submodule.nested_submodule.url >actual &&
test_cmp expect actual
)
'
get_oid: handle NULL repo->index When get_oid() and its helpers see an index name like ":.gitmodules", they try to load the index on demand, like: if (repo->index->cache) repo_read_index(repo); However, that misses the case when "repo->index" itself is NULL; we'll segfault in the conditional. This never happens with the_repository; there we always point its index field to &the_index. But a submodule repository may have a NULL index field until somebody calls repo_read_index(). This bug is triggered by t7411, but it was hard to notice because it's in an expect_failure block. That test was added by 2b1257e463 (t/helper: add test-submodule-nested-repo-config, 2018-10-25). Back then we had no easy way to access the .gitmodules blob of a submodule repo, so we expected (and got) an error message to that effect. Later, d9b8b8f896 (submodule-config.c: use repo_get_oid for reading .gitmodules, 2019-04-16) started looking in the correct repo, which is when we started triggering the segfault. With this fix, the test starts passing (once we clean it up as its comment instructs). Note that as far as I know, this bug could not be triggered outside of the test suite. It requires resolving an index name in a submodule, and all of the code paths (aside from test-tool) which do that either load the index themselves, or always pass the_repository. Ultimately it comes from 3a7a698e93 (sha1-name.c: remove implicit dependency on the_index, 2019-01-12), which replaced a check of "the_index.cache" with "repo->index->cache". So even if there is another way to trigger it, it wouldn't affect any versions before then. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-14 13:54:55 +00:00
test_expect_success 'reading nested submodules config when .gitmodules is not in the working tree' '
test_when_finished "git -C super/submodule checkout .gitmodules" &&
(cd super &&
echo "./nested_submodule" >expect &&
rm submodule/.gitmodules &&
test-tool submodule-nested-repo-config \
submodule submodule.nested_submodule.url >actual 2>warning &&
get_oid: handle NULL repo->index When get_oid() and its helpers see an index name like ":.gitmodules", they try to load the index on demand, like: if (repo->index->cache) repo_read_index(repo); However, that misses the case when "repo->index" itself is NULL; we'll segfault in the conditional. This never happens with the_repository; there we always point its index field to &the_index. But a submodule repository may have a NULL index field until somebody calls repo_read_index(). This bug is triggered by t7411, but it was hard to notice because it's in an expect_failure block. That test was added by 2b1257e463 (t/helper: add test-submodule-nested-repo-config, 2018-10-25). Back then we had no easy way to access the .gitmodules blob of a submodule repo, so we expected (and got) an error message to that effect. Later, d9b8b8f896 (submodule-config.c: use repo_get_oid for reading .gitmodules, 2019-04-16) started looking in the correct repo, which is when we started triggering the segfault. With this fix, the test starts passing (once we clean it up as its comment instructs). Note that as far as I know, this bug could not be triggered outside of the test suite. It requires resolving an index name in a submodule, and all of the code paths (aside from test-tool) which do that either load the index themselves, or always pass the_repository. Ultimately it comes from 3a7a698e93 (sha1-name.c: remove implicit dependency on the_index, 2019-01-12), which replaced a check of "the_index.cache" with "repo->index->cache". So even if there is another way to trigger it, it wouldn't affect any versions before then. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-05-14 13:54:55 +00:00
test_must_be_empty warning &&
test_cmp expect actual
)
'
submodule: implement a config API for lookup of .gitmodules values In a superproject some commands need to interact with submodules. They need to query values from the .gitmodules file either from the worktree of from certain revisions. At the moment this is quite hard since a caller would need to read the .gitmodules file from the history and then parse the values. We want to provide an API for this so we have one place to get values from .gitmodules from any revision (including the worktree). The API is realized as a cache which allows us to lazily read .gitmodules configurations by commit into a runtime cache which can then be used to easily lookup values from it. Currently only the values for path or name are stored but it can be extended for any value needed. It is expected that .gitmodules files do not change often between commits. Thats why we lookup the .gitmodules sha1 from a commit and then either lookup an already parsed configuration or parse and cache an unknown one for each sha1. The cache is lazily build on demand for each requested commit. This cache can be used for all purposes which need knowledge about submodule configurations. Example use cases are: * Recursive submodule checkout needs to lookup a submodule name from its path when a submodule first appears. This needs be done before this configuration exists in the worktree. * The implementation of submodule support for 'git archive' needs to lookup the submodule name to generate the archive when given a revision that is not checked out. * 'git fetch' when given the --recurse-submodules=on-demand option (or configuration) needs to lookup submodule names by path from the database rather than reading from the worktree. For new submodule it needs to lookup the name from its path to allow cloning new submodules into the .git folder so they can be checked out without any network interaction when the user does a checkout of that revision. Signed-off-by: Heiko Voigt <hvoigt@hvoigt.net> Signed-off-by: Stefan Beller <sbeller@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-18 00:21:57 +00:00
test_done