Commit graph

475 commits

Author SHA1 Message Date
Antonio Ospite
2502ffc0cf submodule--helper: add a new 'config' subcommand
Add a new 'config' subcommand to 'submodule--helper', this extra level
of indirection makes it possible to add some flexibility to how the
submodules configuration is handled.

Signed-off-by: Antonio Ospite <ao2@ao2.it>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-10-09 12:40:21 +09:00
Junio C Hamano
f84b9b09d4 Sync with 2.19.1
* maint:
  Git 2.19.1
  Git 2.18.1
  Git 2.17.2
  fsck: detect submodule paths starting with dash
  fsck: detect submodule urls starting with dash
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:53:39 -07:00
Junio C Hamano
1958ad504b Sync with 2.18.1
* maint-2.18:
  Git 2.18.1
  Git 2.17.2
  fsck: detect submodule paths starting with dash
  fsck: detect submodule urls starting with dash
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:50:45 -07:00
Junio C Hamano
44f87dac99 Sync with 2.17.2
* maint-2.17:
  Git 2.17.2
  fsck: detect submodule paths starting with dash
  fsck: detect submodule urls starting with dash
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:45:01 -07:00
Junio C Hamano
e43aab778c Sync with 2.16.5
* maint-2.16:
  Git 2.16.5
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:41:02 -07:00
Junio C Hamano
424aac653a Sync with 2.15.3
* maint-2.15:
  Git 2.15.3
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:35:43 -07:00
Junio C Hamano
902df9f5c4 Sync with Git 2.14.4
* maint-2.14:
  Git 2.14.5
  submodule-config: ban submodule paths that start with a dash
  submodule-config: ban submodule urls that start with dash
  submodule--helper: use "--" to signal end of clone options
2018-09-27 11:20:22 -07:00
Jeff King
98afac7a7c submodule--helper: use "--" to signal end of clone options
When we clone a submodule, we call "git clone $url $path".
But there's nothing to say that those components can't begin
with a dash themselves, confusing git-clone into thinking
they're options. Let's pass "--" to make it clear what we
expect.

There's no test here, because it's actually quite hard to
make these names work, even with "git clone" parsing them
correctly. And we're going to restrict these cases even
further in future commits. So we'll leave off testing until
then; this is just the minimal fix to prevent us from doing
something stupid with a badly formed entry.

Reported-by: joernchen <joernchen@phenoelit.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-27 09:34:55 -07:00
Nguyễn Thái Ngọc Duy
2abf350385 revision.c: remove implicit dependency on the_index
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-21 09:51:19 -07:00
Junio C Hamano
4d6d6ef1fc Merge branch 'sb/submodule-update-in-c'
"git submodule update" is getting rewritten piece-by-piece into C.

* sb/submodule-update-in-c:
  submodule--helper: introduce new update-module-mode helper
  submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree
  builtin/submodule--helper: factor out method to update a single submodule
  builtin/submodule--helper: store update_clone information in a struct
  builtin/submodule--helper: factor out submodule updating
  git-submodule.sh: rename unused variables
  git-submodule.sh: align error reporting for update mode to use path
2018-09-17 13:53:51 -07:00
Jonathan Nieder
f178c13fda Revert "Merge branch 'sb/submodule-core-worktree'"
This reverts commit 7e25437d35, reversing
changes made to 00624d608c.

v2.19.0-rc0~165^2~1 (submodule: ensure core.worktree is set after
update, 2018-06-18) assumes an "absorbed" submodule layout, where the
submodule's Git directory is in the superproject's .git/modules/
directory and .git in the submodule worktree is a .git file pointing
there.  In particular, it uses $GIT_DIR/modules/$name to find the
submodule to find out whether it already has core.worktree set, and it
uses connect_work_tree_and_git_dir if not, resulting in

	fatal: could not open sub/.git for writing

The context behind that patch: v2.19.0-rc0~165^2~2 (submodule: unset
core.worktree if no working tree is present, 2018-06-12) unsets
core.worktree when running commands like "git checkout
--recurse-submodules" to switch to a branch without the submodule.  If
a user then uses "git checkout --no-recurse-submodules" to switch back
to a branch with the submodule and runs "git submodule update", this
patch is needed to ensure that commands using the submodule directly
are aware of the path to the worktree.

It is late in the release cycle, so revert the whole 3-patch series.
We can try again later for 2.20.

Reported-by: Allan Sandfeld Jensen <allan.jensen@qt.io>
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-09-07 19:05:20 -07:00
Jean-Noël Avila
27c929edd6 i18n: fix mistakes in translated strings
Fix typos and convert a question which does not expect to be replied
to a simple advice.

Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-23 14:29:12 -07:00
Junio C Hamano
85c54ecc5f Merge branch 'sb/submodule-cleanup'
A few preliminary minor clean-ups in the area around submodules.

* sb/submodule-cleanup:
  builtin/submodule--helper: remove stray new line
  t7410: update to new style
2018-08-20 12:41:33 -07:00
Stefan Beller
e6b09b184d builtin/submodule--helper: remove stray new line
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-16 10:42:56 -07:00
Stefan Beller
ee69b2a90c submodule--helper: introduce new update-module-mode helper
This chews off a bit of the shell part of the update command in
git-submodule.sh. When writing the C code, keep in mind that the
submodule--helper part will go away eventually and we want to have
a C function that is able to determine the submodule update strategy,
it as a nicety, make determine_submodule_update_strategy accessible
for arbitrary repositories.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-14 14:01:04 -07:00
Stefan Beller
74d4731da1 submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree
e98317508c (submodule: ensure core.worktree is set after update,
2018-06-18) was overly aggressive in calling connect_work_tree_and_git_dir
as that ensures both the 'core.worktree' configuration is set as well as
setting up correct gitlink file pointing at the git directory.

We do not need to check for the gitlink in this part of the cmd_update
in git-submodule.sh, as the initial call to update-clone will have ensured
that. So we can reduce the work to only (check and potentially) set the
'core.worktree' setting.

While at it move the check from shell to C as that proves to be useful in
a follow up patch, as we do not need the 'name' in shell now.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-14 14:01:04 -07:00
Nguyễn Thái Ngọc Duy
6d2df284e7 dir.c: remove an implicit dependency on the_index in pathspec code
Make the match_patchspec API and friends take an index_state instead
of assuming the_index in dir.c. All external call sites are converted
blindly to keep the patch simple and retain current behavior.
Individual call sites may receive further updates to use the right
index instead of the_index.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-13 14:14:42 -07:00
Stefan Beller
c94d9dc286 builtin/submodule--helper: factor out method to update a single submodule
In a later patch we'll find this method handy.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-03 15:37:12 -07:00
Stefan Beller
f1d15713fa builtin/submodule--helper: store update_clone information in a struct
The information that is printed for update_submodules in
'submodule--helper update-clone' and consumed by 'git submodule update'
is stored as a string per submodule. This made sense at the time of
48308681b0 (git submodule update: have a dedicated helper for cloning,
2016-02-29), but as we want to migrate the rest of the submodule update
into C, we're better off having access to the raw information in a helper
struct.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-03 15:37:12 -07:00
Stefan Beller
90efe595c5 builtin/submodule--helper: factor out submodule updating
Separate the command line parsing from the actual execution of the command
within the repository. For now there is not a lot of execution as
most of it is still in git-submodule.sh.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-03 15:37:12 -07:00
Stefan Beller
9eca701f69 git-submodule.sh: rename unused variables
The 'mode' variable is not used in cmd_update for its original purpose,
rename it to 'dummy' as it only serves the purpose to abort quickly
documenting this knowledge.

The variable 'stage' is also not used any more in cmd_update, so remove it.

This went unnoticed as first each function used the commonly used
submodule listing, which was converted in 74703a1e4d (submodule: rewrite
`module_list` shell function in C, 2015-09-02). When cmd_update was
using its own function starting in 48308681b0 (git submodule update:
have a dedicated helper for cloning, 2016-02-29), its removal was missed.

A later patch in this series also touches the communication between
the submodule helper and git-submodule.sh, but let's have this as
a preparatory patch, as it eases the next patch, which stores the
raw data instead of the line printed for this communication.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-08-03 15:37:12 -07:00
Junio C Hamano
b9632c9d95 Merge branch 'ao/config-from-gitmodules'
Tighten the API to make it harder to misuse in-tree .gitmodules
file, even though it shares the same syntax with configuration
files, to read random configuration items from it.

* ao/config-from-gitmodules:
  submodule-config: reuse config_from_gitmodules in repo_read_gitmodules
  submodule-config: pass repository as argument to config_from_gitmodules
  submodule-config: make 'config_from_gitmodules' private
  submodule-config: add helper to get 'update-clone' config from .gitmodules
  submodule-config: add helper function to get 'fetch' config from .gitmodules
  config: move config_from_gitmodules to submodule-config.c
2018-07-18 12:20:31 -07:00
Junio C Hamano
7e25437d35 Merge branch 'sb/submodule-core-worktree'
"git submodule" did not correctly adjust core.worktree setting that
indicates whether/where a submodule repository has its associated
working tree across various state transitions, which has been
corrected.

* sb/submodule-core-worktree:
  submodule deinit: unset core.worktree
  submodule: ensure core.worktree is set after update
  submodule: unset core.worktree if no working tree is present
2018-07-18 12:20:28 -07:00
Antonio Ospite
057449978e submodule-config: add helper to get 'update-clone' config from .gitmodules
Add a helper function to make it clearer that retrieving 'update-clone'
configuration from the .gitmodules file is a special case supported
solely for backward compatibility purposes.

This change removes one direct use of 'config_from_gitmodules' for
options not strictly related to submodules: "submodule.fetchjobs" does
not describe a property of a submodule, but a behavior of other commands
when dealing with submodules, so it does not really belong to the
.gitmodules file.

This is in the effort to communicate better that .gitmodules is not to
be used as a mechanism to store arbitrary configuration in the
repository that any command can retrieve.

Signed-off-by: Antonio Ospite <ao2@ao2.it>
Acked-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-26 12:56:12 -07:00
Junio C Hamano
02f27d0dd0 Merge branch 'sb/plug-misc-leaks'
Misc leak plugging.

* sb/plug-misc-leaks:
  sequencer.c: plug mem leak in git_sequencer_config
  sequencer.c: plug leaks in do_pick_commit
  submodule--helper: plug mem leak in print_default_remote
  refs/packed-backend.c: close fd of empty file
2018-06-25 13:22:41 -07:00
Junio C Hamano
ea27893a65 Merge branch 'pc/submodule-helper-foreach'
The bulk of "git submodule foreach" has been rewritten in C.

* pc/submodule-helper-foreach:
  submodule: port submodule subcommand 'foreach' from shell to C
  submodule foreach: document variable '$displaypath'
  submodule foreach: document '$sm_path' instead of '$path'
  submodule foreach: correct '$path' in nested submodules from a subdirectory
2018-06-25 13:22:35 -07:00
Stefan Beller
984cd77ddb submodule deinit: unset core.worktree
When a submodule is deinit'd, the working tree is gone, so the setting of
core.worktree is bogus. Unset it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19 09:28:13 -07:00
Stefan Beller
e98317508c submodule: ensure core.worktree is set after update
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-19 09:28:13 -07:00
Stefan Beller
e6be8e2f9f submodule--helper: plug mem leak in print_default_remote
Signed-off-by: Stefan Beller <sbeller@google.com>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[jc: no need for remote to be const char *]
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-06-01 14:52:47 +09:00
Junio C Hamano
e12cbeaa62 Merge branch 'bw/ref-prefix-for-configured-refspec'
"git fetch $there $refspec" that talks over protocol v2 can take
advantage of server-side ref filtering; the code has been extended
so that this mechanism triggers also when fetching with configured
refspec.

* bw/ref-prefix-for-configured-refspec: (38 commits)
  fetch: generate ref-prefixes when using a configured refspec
  refspec: consolidate ref-prefix generation logic
  submodule: convert push_unpushed_submodules to take a struct refspec
  remote: convert check_push_refs to take a struct refspec
  remote: convert match_push_refs to take a struct refspec
  http-push: store refspecs in a struct refspec
  transport: remove transport_verify_remote_names
  send-pack: store refspecs in a struct refspec
  transport: convert transport_push to take a struct refspec
  push: convert to use struct refspec
  push: check for errors earlier
  remote: convert match_explicit_refs to take a struct refspec
  remote: convert get_ref_match to take a struct refspec
  remote: convert query_refspecs to take a struct refspec
  remote: convert apply_refspecs to take a struct refspec
  remote: convert get_stale_heads to take a struct refspec
  fetch: convert prune_refs to take a struct refspec
  fetch: convert get_ref_map to take a struct refspec
  fetch: convert do_fetch to take a struct refspec
  refspec: remove the deprecated functions
  ...
2018-05-30 21:51:26 +09:00
Junio C Hamano
2a98a8794e Merge branch 'cf/submodule-progress-dissociate'
"git submodule update" and "git submodule add" supported the
"--reference" option to borrow objects from a neighbouring local
repository like "git clone" does, but lacked the more recent
invention "--dissociate".  Also "git submodule add" has been taught
to take the "--progress" option.

* cf/submodule-progress-dissociate:
  submodule: add --dissociate option to add/update commands
  submodule: add --progress option to add command
  submodule: clean up substitutions in script
2018-05-30 14:04:09 +09:00
Junio C Hamano
7913f53b56 Sync with Git 2.17.1
* maint: (25 commits)
  Git 2.17.1
  Git 2.16.4
  Git 2.15.2
  Git 2.14.4
  Git 2.13.7
  fsck: complain when .gitmodules is a symlink
  index-pack: check .gitmodules files with --strict
  unpack-objects: call fsck_finish() after fscking objects
  fsck: call fsck_finish() after fscking objects
  fsck: check .gitmodules content
  fsck: handle promisor objects in .gitmodules check
  fsck: detect gitmodules files
  fsck: actually fsck blob data
  fsck: simplify ".git" check
  index-pack: make fsck error message more specific
  verify_path: disallow symlinks in .gitmodules
  update-index: stat updated files earlier
  verify_dotfile: mention case-insensitivity in comment
  verify_path: drop clever fallthrough
  skip_prefix: add case-insensitive variant
  ...
2018-05-29 17:10:05 +09:00
Junio C Hamano
68f95b26e4 Sync with Git 2.16.4
* maint-2.16:
  Git 2.16.4
  Git 2.15.2
  Git 2.14.4
  Git 2.13.7
  verify_path: disallow symlinks in .gitmodules
  update-index: stat updated files earlier
  verify_dotfile: mention case-insensitivity in comment
  verify_path: drop clever fallthrough
  skip_prefix: add case-insensitive variant
  is_{hfs,ntfs}_dotgitmodules: add tests
  is_ntfs_dotgit: match other .git files
  is_hfs_dotgit: match other .git files
  is_ntfs_dotgit: use a size_t for traversing string
  submodule-config: verify submodule names as paths
2018-05-22 14:25:26 +09:00
Junio C Hamano
023020401d Sync with Git 2.15.2
* maint-2.15:
  Git 2.15.2
  Git 2.14.4
  Git 2.13.7
  verify_path: disallow symlinks in .gitmodules
  update-index: stat updated files earlier
  verify_dotfile: mention case-insensitivity in comment
  verify_path: drop clever fallthrough
  skip_prefix: add case-insensitive variant
  is_{hfs,ntfs}_dotgitmodules: add tests
  is_ntfs_dotgit: match other .git files
  is_hfs_dotgit: match other .git files
  is_ntfs_dotgit: use a size_t for traversing string
  submodule-config: verify submodule names as paths
2018-05-22 14:18:06 +09:00
Junio C Hamano
9e0f06d55d Sync with Git 2.14.4
* maint-2.14:
  Git 2.14.4
  Git 2.13.7
  verify_path: disallow symlinks in .gitmodules
  update-index: stat updated files earlier
  verify_dotfile: mention case-insensitivity in comment
  verify_path: drop clever fallthrough
  skip_prefix: add case-insensitive variant
  is_{hfs,ntfs}_dotgitmodules: add tests
  is_ntfs_dotgit: match other .git files
  is_hfs_dotgit: match other .git files
  is_ntfs_dotgit: use a size_t for traversing string
  submodule-config: verify submodule names as paths
2018-05-22 14:15:14 +09:00
Junio C Hamano
7b01c71b64 Sync with Git 2.13.7
* maint-2.13:
  Git 2.13.7
  verify_path: disallow symlinks in .gitmodules
  update-index: stat updated files earlier
  verify_dotfile: mention case-insensitivity in comment
  verify_path: drop clever fallthrough
  skip_prefix: add case-insensitive variant
  is_{hfs,ntfs}_dotgitmodules: add tests
  is_ntfs_dotgit: match other .git files
  is_hfs_dotgit: match other .git files
  is_ntfs_dotgit: use a size_t for traversing string
  submodule-config: verify submodule names as paths
2018-05-22 14:10:49 +09:00
Jeff King
0383bbb901 submodule-config: verify submodule names as paths
Submodule "names" come from the untrusted .gitmodules file,
but we blindly append them to $GIT_DIR/modules to create our
on-disk repo paths. This means you can do bad things by
putting "../" into the name (among other things).

Let's sanity-check these names to avoid building a path that
can be exploited. There are two main decisions:

  1. What should the allowed syntax be?

     It's tempting to reuse verify_path(), since submodule
     names typically come from in-repo paths. But there are
     two reasons not to:

       a. It's technically more strict than what we need, as
          we really care only about breaking out of the
          $GIT_DIR/modules/ hierarchy.  E.g., having a
          submodule named "foo/.git" isn't actually
          dangerous, and it's possible that somebody has
          manually given such a funny name.

       b. Since we'll eventually use this checking logic in
          fsck to prevent downstream repositories, it should
          be consistent across platforms. Because
          verify_path() relies on is_dir_sep(), it wouldn't
          block "foo\..\bar" on a non-Windows machine.

  2. Where should we enforce it? These days most of the
     .gitmodules reads go through submodule-config.c, so
     I've put it there in the reading step. That should
     cover all of the C code.

     We also construct the name for "git submodule add"
     inside the git-submodule.sh script. This is probably
     not a big deal for security since the name is coming
     from the user anyway, but it would be polite to remind
     them if the name they pick is invalid (and we need to
     expose the name-checker to the shell anyway for our
     test scripts).

     This patch issues a warning when reading .gitmodules
     and just ignores the related config entry completely.
     This will generally end up producing a sensible error,
     as it works the same as a .gitmodules file which is
     missing a submodule entry (so "submodule update" will
     barf, but "git clone --recurse-submodules" will print
     an error but not abort the clone.

     There is one minor oddity, which is that we print the
     warning once per malformed config key (since that's how
     the config subsystem gives us the entries). So in the
     new test, for example, the user would see three
     warnings. That's OK, since the intent is that this case
     should never come up outside of malicious repositories
     (and then it might even benefit the user to see the
     message multiple times).

Credit for finding this vulnerability and the proof of
concept from which the test script was adapted goes to
Etienne Stalmans.

Signed-off-by: Jeff King <peff@peff.net>
2018-05-21 23:50:11 -04:00
Casey Fitzpatrick
a0ef29341a submodule: add --dissociate option to add/update commands
Add --dissociate option to add and update commands, both clone helper commands
that already have the --reference option --dissociate pairs with.

Signed-off-by: Casey Fitzpatrick <kcghost@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-22 12:22:42 +09:00
Brandon Williams
9c8361b289 submodule--helper: convert push_check to use struct refspec
Convert 'push_check()' to use 'struct refspec'.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-18 06:19:42 +09:00
Brandon Williams
0ad4a5ff50 refspec: rename struct refspec to struct refspec_item
In preparation for introducing an abstraction around a collection of
refspecs (much like how a 'struct pathspec' is a collection of 'struct
pathspec_item's) rename the existing 'struct refspec' to 'struct
refspec_item'.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-18 06:19:41 +09:00
Brandon Williams
ec0cb49655 refspec: move refspec parsing logic into its own file
In preparation for performing a refactor on refspec related code, move
the refspec parsing logic into its own file.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-18 06:19:41 +09:00
Prathamesh Chavan
fc1b9243cd submodule: port submodule subcommand 'foreach' from shell to C
This aims to make git-submodule foreach a builtin. 'foreach' is ported to
the submodule--helper, and submodule--helper is called from
git-submodule.sh.

Helped-by: Brandon Williams <bmwill@google.com>
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-05-11 10:11:54 +09:00
Junio C Hamano
0cd58d8ba9 Merge branch 'nd/submodule-status-fix'
"git submodule status" did not check the symbolic revision name it
computed for the submodule HEAD is not the NULL, and threw it at
printf routines, which has been corrected.

* nd/submodule-status-fix:
  submodule--helper: don't print null in 'submodule status'
2018-05-08 15:59:31 +09:00
Junio C Hamano
0c7ecb7c31 Merge branch 'sb/submodule-move-nested'
Moving a submodule that itself has submodule in it with "git mv"
forgot to make necessary adjustment to the nested sub-submodules;
now the codepath learned to recurse into the submodules.

* sb/submodule-move-nested:
  submodule: fixup nested submodules after moving the submodule
  submodule-config: remove submodule_from_cache
  submodule-config: add repository argument to submodule_from_{name, path}
  submodule-config: allow submodule_free to handle arbitrary repositories
  grep: remove "repo" arg from non-supporting funcs
  submodule.h: drop declaration of connect_work_tree_and_git_dir
2018-05-08 15:59:17 +09:00
Nguyễn Thái Ngọc Duy
0b5e2ea7cf submodule--helper: don't print null in 'submodule status'
The function compute_rev_name() can return NULL sometimes (e.g. right
after 'submodule init'). The current code makes 'submodule status'
print this:

 19d97bf5af05312267c2e874ee6bcf584d9e9681 sha1collisiondetection ((null))

This ugly 'null' adds no value to the user using this command. More
importantly printf() on some platform can't handle NULL as a string
and will crash instead of printing '(null)'.

Check for this and skip printing this part (the alternative is
printing '(n/a)' or something but I think that is just noise).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-04-19 06:24:23 +09:00
Junio C Hamano
103251a318 Merge branch 'rs/status-with-removed-submodule'
"git submodule status" misbehaved on a submodule that has been
removed from the working tree.

* rs/status-with-removed-submodule:
  submodule: check for NULL return of get_submodule_ref_store()
2018-04-11 13:09:56 +09:00
Junio C Hamano
cf0b1793ea Merge branch 'sb/object-store'
Refactoring the internal global data structure to make it possible
to open multiple repositories, work with and then close them.

Rerolled by Duy on top of a separate preliminary clean-up topic.
The resulting structure of the topics looked very sensible.

* sb/object-store: (27 commits)
  sha1_file: allow sha1_loose_object_info to handle arbitrary repositories
  sha1_file: allow map_sha1_file to handle arbitrary repositories
  sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
  sha1_file: allow open_sha1_file to handle arbitrary repositories
  sha1_file: allow stat_sha1_file to handle arbitrary repositories
  sha1_file: allow sha1_file_name to handle arbitrary repositories
  sha1_file: add repository argument to sha1_loose_object_info
  sha1_file: add repository argument to map_sha1_file
  sha1_file: add repository argument to map_sha1_file_1
  sha1_file: add repository argument to open_sha1_file
  sha1_file: add repository argument to stat_sha1_file
  sha1_file: add repository argument to sha1_file_name
  sha1_file: allow prepare_alt_odb to handle arbitrary repositories
  sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
  sha1_file: add repository argument to prepare_alt_odb
  sha1_file: add repository argument to link_alt_odb_entries
  sha1_file: add repository argument to read_info_alternates
  sha1_file: add repository argument to link_alt_odb_entry
  sha1_file: add raw_object_store argument to alt_odb_usable
  pack: move approximate object count to object store
  ...
2018-04-11 13:09:55 +09:00
Stefan Beller
da62f786d2 submodule: fixup nested submodules after moving the submodule
connect_work_tree_and_git_dir is used to connect a submodule worktree with
its git directory and vice versa after events that require a reconnection
such as moving around the working tree. As submodules can have nested
submodules themselves, we'd also want to fix the nested submodules when
asked to. Add an option to recurse into the nested submodules and connect
them as well.

As submodules are identified by their name (which determines their git
directory in relation to their superproject's git directory) internally
and by their path in the working tree of the superproject, we need to
make sure that the mapping of name <-> path is kept intact. We can do
that in the git-mv command by writing out the gitmodules file first
and then forcing a reload of the submodule config machinery.

Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-29 09:44:51 -07:00
Stefan Beller
3b8fb393bc submodule-config: add repository argument to submodule_from_{name, path}
This enables submodule_from_{name, path} to handle arbitrary repositories.
All callers just pass in the_repository, a later patch will pass in other
repos.

While at it remove the extern key word from the declarations.

Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-29 09:44:50 -07:00
René Scharfe
74b6bda32f submodule: check for NULL return of get_submodule_ref_store()
If we can't find a ref store for a submodule then assume the latter
is not initialized (or was removed).  Print a status line accordingly
instead of causing a segmentation fault by passing NULL as the first
parameter of refs_head_ref().

Reported-by: Jeremy Feusi <jeremy@feusi.co>
Reviewed-by: Stefan Beller <sbeller@google.com>
Initial-Test-By: Stefan Beller <sbeller@google.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-28 15:27:02 -07:00
Junio C Hamano
2081fa73b4 Merge branch 'pc/submodule-helper'
Hotfix.

* pc/submodule-helper:
  submodule deinit: handle non existing pathspecs gracefully
2018-03-28 11:04:25 -07:00
Stefan Beller
9748e39d0c submodule deinit: handle non existing pathspecs gracefully
This fixes a regression introduced in 2e612731b5 (submodule: port
submodule subcommand 'deinit' from shell to C, 2018-01-15), when
handling pathspecs that do not exist gracefully. This restores the
historic behavior of reporting the pathspec as unknown and returning
instead of reporting a bug.

Reported-by: Peter Oberndorfer <kumbayo84@arcor.de>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-27 22:07:13 -07:00
Stefan Beller
0d4a132144 object-store: migrate alternates struct and functions from cache.h
Migrate the struct alternate_object_database and all its related
functions to the object store as these functions are easier found in
that header. The migration is just a verbatim copy, no need to
include the object store header at any C file, because cache.h includes
repository.h which in turn includes the object-store.h

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-03-23 11:06:01 -07:00
Junio C Hamano
7fb6aefd2a Merge branch 'nd/parseopt-completion'
Teach parse-options API an option to help the completion script,
and make use of the mechanism in command line completion.

* nd/parseopt-completion: (45 commits)
  completion: more subcommands in _git_notes()
  completion: complete --{reuse,reedit}-message= for all notes subcmds
  completion: simplify _git_notes
  completion: don't set PARSE_OPT_NOCOMPLETE on --rerere-autoupdate
  completion: use __gitcomp_builtin in _git_worktree
  completion: use __gitcomp_builtin in _git_tag
  completion: use __gitcomp_builtin in _git_status
  completion: use __gitcomp_builtin in _git_show_branch
  completion: use __gitcomp_builtin in _git_rm
  completion: use __gitcomp_builtin in _git_revert
  completion: use __gitcomp_builtin in _git_reset
  completion: use __gitcomp_builtin in _git_replace
  remote: force completing --mirror= instead of --mirror
  completion: use __gitcomp_builtin in _git_remote
  completion: use __gitcomp_builtin in _git_push
  completion: use __gitcomp_builtin in _git_pull
  completion: use __gitcomp_builtin in _git_notes
  completion: use __gitcomp_builtin in _git_name_rev
  completion: use __gitcomp_builtin in _git_mv
  completion: use __gitcomp_builtin in _git_merge_base
  ...
2018-03-14 12:01:07 -07:00
Prathamesh Chavan
2e612731b5 submodule: port submodule subcommand 'deinit' from shell to C
The same mechanism is used even for porting this submodule
subcommand, as used in the ported subcommands till now.
The function cmd_deinit in split up after porting into four
functions: module_deinit(), for_each_listed_submodule(),
deinit_submodule() and deinit_submodule_cb().

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-16 11:29:50 -08:00
Prathamesh Chavan
13424764db submodule: port submodule subcommand 'sync' from shell to C
Port the submodule subcommand 'sync' from shell to C using the same
mechanism as that used for porting submodule subcommand 'status'.
Hence, here the function cmd_sync() is ported from shell to C.
This is done by introducing four functions: module_sync(),
sync_submodule(), sync_submodule_cb() and print_default_remote().

The function print_default_remote() is introduced for getting
the default remote as stdout.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-01-16 11:29:48 -08:00
Junio C Hamano
41a05ee3a6 Merge branch 'pc/submodule-helper'
A message fix.

* pc/submodule-helper:
  submodule--helper.c: i18n: add a missing space in message
2017-12-13 13:28:56 -08:00
Jean-Noel Avila
ed5bdd5bab submodule--helper.c: i18n: add a missing space in message
The message spans over 2 lines but the C concatenation does not add
the needed space between the two lines.

Signed-off-by: Jean-Noel Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-11-26 12:52:17 +09:00
Junio C Hamano
e7e456f500 Merge branch 'bc/object-id'
Conversion from uchar[20] to struct object_id continues.

* bc/object-id: (25 commits)
  refs/files-backend: convert static functions to object_id
  refs: convert read_raw_ref backends to struct object_id
  refs: convert peel_object to struct object_id
  refs: convert resolve_ref_unsafe to struct object_id
  worktree: convert struct worktree to object_id
  refs: convert resolve_gitlink_ref to struct object_id
  Convert remaining callers of resolve_gitlink_ref to object_id
  sha1_file: convert index_path and index_fd to struct object_id
  refs: convert reflog_expire parameter to struct object_id
  refs: convert read_ref_at to struct object_id
  refs: convert peel_ref to struct object_id
  builtin/pack-objects: convert to struct object_id
  pack-bitmap: convert traverse_bitmap_commit_list to object_id
  refs: convert dwim_log to struct object_id
  builtin/reflog: convert remaining unsigned char uses to object_id
  refs: convert dwim_ref and expand_ref to struct object_id
  refs: convert read_ref and read_ref_full to object_id
  refs: convert resolve_refdup and refs_resolve_refdup to struct object_id
  Convert check_connected to use struct object_id
  refs: update ref transactions to use struct object_id
  ...
2017-11-06 14:24:27 +09:00
Junio C Hamano
a1bf46ed9d Merge branch 'pc/submodule-helper'
GSoC.

* pc/submodule-helper:
  submodule: port submodule subcommand 'status' from shell to C
  submodule--helper: introduce for_each_listed_submodule()
  submodule--helper: introduce get_submodule_displaypath()
2017-11-06 14:24:23 +09:00
brian m. carlson
0f2dc722dd refs: convert resolve_refdup and refs_resolve_refdup to struct object_id
All of the callers already pass the hash member of struct object_id, so
update them to pass a pointer to the struct directly,

This transformation was done with an update to declaration and
definition and the following semantic patch:

@@
expression E1, E2, E3, E4;
@@
- resolve_refdup(E1, E2, E3.hash, E4)
+ resolve_refdup(E1, E2, &E3, E4)

@@
expression E1, E2, E3, E4;
@@
- resolve_refdup(E1, E2, E3->hash, E4)
+ resolve_refdup(E1, E2, E3, E4)

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-16 11:05:50 +09:00
Prathamesh Chavan
a9f8a37584 submodule: port submodule subcommand 'status' from shell to C
This aims to make git-submodule 'status' a built-in. Hence, the function
cmd_status() is ported from shell to C. This is done by introducing
four functions: module_status(), submodule_status_cb(),
submodule_status() and print_status().

The function module_status() acts as the front-end of the subcommand.
It parses subcommand's options and then calls the function
module_list_compute() for computing the list of submodules. Then
this functions calls for_each_listed_submodule() looping through the
list obtained.

Then for_each_listed_submodule() calls submodule_status_cb() for each of
the submodule in its list. The function submodule_status_cb() calls
submodule_status() after passing appropriate arguments to the funciton.
Function submodule_status() is responsible for generating the status
each submodule it is called for, and then calls print_status().

Finally, the function print_status() handles the printing of submodule's
status.

Function set_name_rev() is also ported from git-submodule to the
submodule--helper builtin function compute_rev_name(), which now
generates the value of the revision name as required.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-07 17:52:35 +09:00
Prathamesh Chavan
9f580a6260 submodule--helper: introduce for_each_listed_submodule()
Introduce function for_each_listed_submodule() and replace a loop
in module_init() with a call to it.

The new function will also be used in other parts of the
system in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-07 17:52:35 +09:00
Prathamesh Chavan
74a10642aa submodule--helper: introduce get_submodule_displaypath()
Introduce function get_submodule_displaypath() to replace the code
occurring in submodule_init() for generating displaypath of the
submodule with a call to it.

This new function will also be used in other parts of the system
in later patches.

Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Prathamesh Chavan <pc44800@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-10-02 09:35:35 +09:00
Junio C Hamano
73ecdc606e Merge branch 'rs/resolve-ref-optional-result'
Code clean-up.

* rs/resolve-ref-optional-result:
  refs: pass NULL to resolve_ref_unsafe() if hash is not needed
  refs: pass NULL to refs_resolve_ref_unsafe() if hash is not needed
  refs: make sha1 output parameter of refs_resolve_ref_unsafe() optional
2017-09-28 14:47:56 +09:00
René Scharfe
744c040b19 refs: pass NULL to resolve_ref_unsafe() if hash is not needed
This allows us to get rid of some write-only variables, among them seven
SHA1 buffers.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-24 10:18:21 +09:00
Jeff King
1cf01a34ea consistently use "fallthrough" comments in switches
Gcc 7 adds -Wimplicit-fallthrough, which can warn when a
switch case falls through to the next case. The general idea
is that the compiler can't tell if this was intentional or
not, so you should annotate any intentional fall-throughs as
such, leaving it to complain about any unannotated ones.

There's a GNU __attribute__ which can be used for
annotation, but of course we'd have to #ifdef it away on
non-gcc compilers. Gcc will also recognize
specially-formatted comments, which matches our current
practice. Let's extend that practice to all of the
unannotated sites (which I did look over and verify that
they were behaving as intended).

Ideally in each case we'd actually give some reasons in the
comment about why we're falling through, or what we're
falling through to. And gcc does support that with
-Wimplicit-fallthrough=2, which relaxes the comment pattern
matching to anything that contains "fallthrough" (or a
variety of spelling variants). However, this isn't the
default for -Wimplicit-fallthrough, nor for -Wextra. In the
name of simplicity, it's probably better for us to support
the default level, which requires "fallthrough" to be the
only thing in the comment (modulo some window dressing like
"else" and some punctuation; see the gcc manual for the
complete set of patterns).

This patch suppresses all warnings due to
-Wimplicit-fallthrough. We might eventually want to add that
to the DEVELOPER Makefile knob, but we should probably wait
until gcc 7 is more widely adopted (since earlier versions
will complain about the unknown warning type).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-09-22 12:49:57 +09:00
Junio C Hamano
eba2a68f25 Merge branch 'bw/push-options-recursively-to-submodules' into maint
"git push --recurse-submodules $there HEAD:$target" was not
propagated down to the submodules, but now it is.

* bw/push-options-recursively-to-submodules:
  submodule--helper: teach push-check to handle HEAD
2017-09-10 17:02:49 +09:00
Junio C Hamano
614ea03a71 Merge branch 'bw/submodule-config-cleanup'
Code clean-up to avoid mixing values read from the .gitmodules file
and values read from the .git/config file.

* bw/submodule-config-cleanup:
  submodule: remove gitmodules_config
  unpack-trees: improve loading of .gitmodules
  submodule-config: lazy-load a repository's .gitmodules file
  submodule-config: move submodule-config functions to submodule-config.c
  submodule-config: remove support for overlaying repository config
  diff: stop allowing diff to have submodules configured in .git/config
  submodule: remove submodule_config callback routine
  unpack-trees: don't respect submodule.update
  submodule: don't rely on overlayed config when setting diffopts
  fetch: don't overlay config with submodule-config
  submodule--helper: don't overlay config in update-clone
  submodule--helper: don't overlay config in remote_submodule_branch
  add, reset: ensure submodules can be added or reset
  submodule: don't use submodule_from_name
  t7411: check configuration parsing errors
2017-08-26 22:55:08 -07:00
Junio C Hamano
0869277033 Merge branch 'js/run-process-parallel-api-fix' into maint
API fix.

* js/run-process-parallel-api-fix:
  run_processes_parallel: change confusing task_cb convention
2017-08-23 14:33:49 -07:00
Junio C Hamano
5aa0b6c506 Merge branch 'bw/grep-recurse-submodules'
"git grep --recurse-submodules" has been reworked to give a more
consistent output across submodule boundary (and do its thing
without having to fork a separate process).

* bw/grep-recurse-submodules:
  grep: recurse in-process using 'struct repository'
  submodule: merge repo_read_gitmodules and gitmodules_config
  submodule: check for unmerged .gitmodules outside of config parsing
  submodule: check for unstaged .gitmodules outside of config parsing
  submodule: remove fetch.recursesubmodules from submodule-config parsing
  submodule: remove submodule.fetchjobs from submodule-config parsing
  config: add config_from_gitmodules
  cache.h: add GITMODULES_FILE macro
  repository: have the_repository use the_index
  repo_read_index: don't discard the index
2017-08-22 10:29:01 -07:00
Junio C Hamano
a49794d108 Merge branch 'bw/push-options-recursively-to-submodules'
"git push --recurse-submodules $there HEAD:$target" was not
propagated down to the submodules, but now it is.

* bw/push-options-recursively-to-submodules:
  submodule--helper: teach push-check to handle HEAD
2017-08-22 10:29:00 -07:00
Junio C Hamano
4a636e7682 Merge branch 'js/run-process-parallel-api-fix'
API fix.

* js/run-process-parallel-api-fix:
  run_processes_parallel: change confusing task_cb convention
2017-08-11 13:27:02 -07:00
Brandon Williams
557a5998df submodule: remove gitmodules_config
Now that the submodule-config subsystem can lazily read the gitmodules
file we no longer need to explicitly pre-read the gitmodules by calling
'gitmodules_config()' so let's remove it.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03 13:11:02 -07:00
Brandon Williams
2cc67fe54a submodule: remove submodule_config callback routine
Remove the last remaining caller of 'submodule_config()' as well as the
function itself.

With 'submodule_config()' being removed the submodule-config API can be
a little simpler as callers don't need to worry about whether or not
they need to overlay the repository's config on top of the
submodule-config.  This also makes it more difficult to accidentally
add non-submodule specific configuration to the .gitmodules file.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03 13:11:01 -07:00
Brandon Williams
ec6141a0f2 submodule--helper: don't overlay config in update-clone
Don't rely on overlaying the repository's config on top of the
submodule-config, instead query the repository's config directly for the
url and the update strategy configuration.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03 13:11:01 -07:00
Brandon Williams
177257ccc7 submodule--helper: don't overlay config in remote_submodule_branch
Don't rely on overlaying the repository's config on top of the
submodule-config, instead query the repository's config directly for the
branch field.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-03 13:11:01 -07:00
Junio C Hamano
a46ddc992b Merge branch 'bc/object-id' into bw/submodule-config-cleanup
* bc/object-id:
  sha1_name: convert uses of 40 to GIT_SHA1_HEXSZ
  sha1_name: convert GET_SHA1* flags to GET_OID*
  sha1_name: convert get_sha1* to get_oid*
  Convert remaining callers of get_sha1 to get_oid.
  builtin/unpack-file: convert to struct object_id
  bisect: convert bisect_checkout to struct object_id
  builtin/update_ref: convert to struct object_id
  sequencer: convert to struct object_id
  remote: convert struct push_cas to struct object_id
  submodule: convert submodule config lookup to use object_id
  builtin/merge-tree: convert remaining caller of get_sha1 to object_id
  builtin/fsck: convert remaining caller of get_sha1 to object_id
  tag: convert gpg_verify_tag to use struct object_id
  commit: convert lookup_commit_graft to struct object_id
2017-08-02 14:34:28 -07:00
Brandon Williams
f20e7c1ea2 submodule: remove submodule.fetchjobs from submodule-config parsing
The '.gitmodules' file should only contain information pertinent to
configuring individual submodules (name to path mapping, URL where to
obtain the submodule, etc.) while other configuration like the number of
jobs to use when fetching submodules should be a part of the
repository's config.

Remove the 'submodule.fetchjobs' configuration option from the general
submodule-config parsing and instead rely on using the
'config_from_gitmodules()' in order to maintain backwards compatibility
with this config being placed in the '.gitmodules' file.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-08-02 14:26:46 -07:00
Johannes Schindelin
c1e860f1dc run_processes_parallel: change confusing task_cb convention
By declaring the task_cb parameter of type `void **`, the signature of
the get_next_task method suggests that the "task-specific cookie" can be
defined in that method, and the signatures of the start_failure and of
the task_finished methods declare that parameter of type `void *`,
suggesting that those methods are mere users of said cookie.

That convention makes a total lot of sense, because the tasks are pretty
much dead when one of the latter two methods is called: there would be
little use to reset that cookie at that point because nobody would be
able to see the change afterwards.

However, this is not what the code actually does. For all three methods,
it passes the *address* of pp->children[i].data.

As reasoned above, this behavior makes no sense. So let's change the
implementation to adhere to the convention suggested by the signatures.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-21 11:58:46 -07:00
Brandon Williams
c7be7201a7 submodule--helper: teach push-check to handle HEAD
In 06bf4ad1d (push: propagate remote and refspec with
--recurse-submodules) push was taught how to propagate a refspec down to
submodules when the '--recurse-submodules' flag is given.  The only refspecs
that are allowed to be propagated are ones which name a ref which exists
in both the superproject and the submodule, with the caveat that 'HEAD'
was disallowed.

This patch teaches push-check (the submodule helper which determines if
a refspec can be propagated to a submodule) to permit propagating 'HEAD'
if and only if the superproject and the submodule both have the same
named branch checked out and the submodule is not in a detached head
state.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-20 14:58:26 -07:00
brian m. carlson
cd73de4714 submodule: convert submodule config lookup to use object_id
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-17 13:54:37 -07:00
Junio C Hamano
127c13aa41 Merge branch 'jc/utf8-fprintf' into maint
Code cleanup.

* jc/utf8-fprintf:
  submodule--helper: do not call utf8_fprintf() unnecessarily
2017-07-10 13:59:09 -07:00
Junio C Hamano
62458ea333 Merge branch 'jc/utf8-fprintf'
Code cleanup.

* jc/utf8-fprintf:
  submodule--helper: do not call utf8_fprintf() unnecessarily
2017-07-06 18:14:44 -07:00
Junio C Hamano
dc4b4a61ba submodule--helper: do not call utf8_fprintf() unnecessarily
The helper function utf8_fprintf(fp, ...) has exactly the same
effect to the output stream fp as fprintf(fp, ...) does, and the
only difference is that its return value counts in display columns
consumed (assuming that the payload is encoded in UTF-8), as opposed
to number of bytes.

There is no reason to call it unless the caller cares about its
return value.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-28 14:27:10 -07:00
Brandon Williams
627d9342fe submodule: convert is_submodule_initialized to work on a repository
Convert 'is_submodule_initialized()' to take a repository object and
while we're at it, lets rename the function to 'is_submodule_active()'
and remove the NEEDSWORK comment.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-23 18:24:34 -07:00
Junio C Hamano
25bf951381 Merge branches 'bw/ls-files-sans-the-index' and 'bw/config-h' into bw/repo-object
* bw/ls-files-sans-the-index:
  ls-files: factor out tag calculation
  ls-files: factor out debug info into a function
  ls-files: convert show_files to take an index
  ls-files: convert show_ce_entry to take an index
  ls-files: convert prune_cache to take an index
  ls-files: convert ce_excluded to take an index
  ls-files: convert show_ru_info to take an index
  ls-files: convert show_other_files to take an index
  ls-files: convert show_killed_files to take an index
  ls-files: convert write_eolinfo to take an index
  ls-files: convert overlay_tree_on_cache to take an index
  tree: convert read_tree to take an index parameter
  convert: convert renormalize_buffer to take an index
  convert: convert convert_to_git to take an index
  convert: convert convert_to_git_filter_fd to take an index
  convert: convert crlf_to_git to take an index
  convert: convert get_cached_convert_stats_ascii to take an index

* bw/config-h:
  config: don't implicitly use gitdir or commondir
  config: respect commondir
  setup: teach discover_git_directory to respect the commondir
  config: don't include config.h by default
  config: remove git_config_iter
  config: create config.h
  alias: use the early config machinery to expand aliases
  t7006: demonstrate a problem with aliases in subdirectories
  t1308: relax the test verifying that empty alias values are disallowed
  help: use early config when autocorrecting aliases
  config: report correct line number upon error
  discover_git_directory(): avoid setting invalid git_dir
2017-06-23 18:24:00 -07:00
Junio C Hamano
50ad8561de Merge branch 'jk/consistent-h'
"git $cmd -h" for builtin commands calls the implementation of the
command (i.e. cmd_$cmd() function) without doing any repository
set-up, and the commands that expect RUN_SETUP is done by the Git
potty needs to be prepared to show the help text without barfing.

* jk/consistent-h:
  t0012: test "-h" with builtins
  git: add hidden --list-builtins option
  version: convert to parse-options
  diff- and log- family: handle "git cmd -h" early
  submodule--helper: show usage for "-h"
  remote-{ext,fd}: print usage message on invalid arguments
  upload-archive: handle "-h" option early
  credential: handle invalid arguments earlier
2017-06-19 12:38:45 -07:00
Brandon Williams
b2141fc1d2 config: don't include config.h by default
Stop including config.h by default in cache.h.  Instead only include
config.h in those files which require use of the config system.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-06-15 12:56:22 -07:00
Jeff King
f0994fa85d submodule--helper: show usage for "-h"
Normal users shouldn't ever call submodule--helper, but it
doesn't hurt to give them a normal usage message if they try
"-h".

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-30 14:45:04 +09:00
Brandon Williams
2249d4dbc1 pathspec: remove PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP
Since (ae8d08242 pathspec: pass directory indicator to
match_pathspec_item()) the path matching logic has been able to cope
with submodules without needing to strip off a trailing slash if a path
refers to a submodule.

Since stripping the slash is no longer necessary, remove the
PATHSPEC_STRIP_SUBMODULE_SLASH_CHEAP flag.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-05-12 14:23:46 +09:00
Junio C Hamano
d92dbf1934 Merge branch 'jk/submodule-init-segv-fix'
Fix a segv in 'submodule init' when url is not given for a submodule.

* jk/submodule-init-segv-fix:
  submodule_init: die cleanly on submodules without url defined
2017-05-01 14:14:44 +09:00
Jeff King
627fde1025 submodule_init: die cleanly on submodules without url defined
When we init a submodule, we try to die when it has no URL
defined:

  url = xstrdup(sub->url);
  if (!url)
	  die(...);

But that's clearly nonsense. xstrdup() will never return
NULL, and if sub->url is NULL, we'll segfault.

These two bits of code need to be flipped, so we check
sub->url before looking at it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-24 19:00:15 -07:00
Junio C Hamano
52d77af463 Merge branch 'bw/submodule-is-active'
Error message fix.

* bw/submodule-is-active:
  submodule--helper: fix typo in is_active error message
2017-04-19 21:37:18 -07:00
Stefan Beller
9af7ec30cb submodule--helper: fix typo in is_active error message
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-13 18:05:23 -07:00
Brandon Williams
93481a6b89 submodule--helper: add push-check subcommand
Add the 'push-check' subcommand to submodule--helper which is used to
check if the provided remote and refspec can be used as part of a push
operation in the submodule.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-04-11 00:45:25 -07:00
Junio C Hamano
a93dcb0a56 Merge branch 'bw/submodule-is-active'
"what URL do we want to update this submodule?" and "are we
interested in this submodule?" are split into two distinct
concepts, and then the way used to express the latter got extended,
paving a way to make it easier to manage a project with many
submodules and make it possible to later extend use of multiple
worktrees for a project with submodules.

* bw/submodule-is-active:
  submodule add: respect submodule.active and submodule.<name>.active
  submodule--helper init: set submodule.<name>.active
  clone: teach --recurse-submodules to optionally take a pathspec
  submodule init: initialize active submodules
  submodule: decouple url and submodule interest
  submodule--helper clone: check for configured submodules using helper
  submodule sync: use submodule--helper is-active
  submodule sync: skip work for inactive submodules
  submodule status: use submodule--helper is-active
  submodule--helper: add is-active subcommand
2017-03-30 14:07:14 -07:00
Junio C Hamano
e394fa01d6 Merge branch 'sb/checkout-recurse-submodules'
"git checkout" is taught the "--recurse-submodules" option.

* sb/checkout-recurse-submodules:
  builtin/read-tree: add --recurse-submodules switch
  builtin/checkout: add --recurse-submodules switch
  entry.c: create submodules when interesting
  unpack-trees: check if we can perform the operation for submodules
  unpack-trees: pass old oid to verify_clean_submodule
  update submodules: add submodule_move_head
  submodule.c: get_super_prefix_or_empty
  update submodules: move up prepare_submodule_repo_env
  submodules: introduce check to see whether to touch a submodule
  update submodules: add a config option to determine if submodules are updated
  update submodules: add submodule config parsing
  make is_submodule_populated gently
  lib-submodule-update.sh: define tests for recursing into submodules
  lib-submodule-update.sh: replace sha1 by hash
  lib-submodule-update: teach test_submodule_content the -C <dir> flag
  lib-submodule-update.sh: do not use ./. as submodule remote
  lib-submodule-update.sh: reorder create_lib_submodule_repo
  submodule--helper.c: remove duplicate code
  connect_work_tree_and_git_dir: safely create leading directories
2017-03-28 14:05:58 -07:00
Brandon Williams
1f8d711548 submodule--helper init: set submodule.<name>.active
When initializing a submodule set the submodule.<name>.active config to
true if the module hasn't already been configured to be active by some
other means (e.g. a pathspec set in submodule.active).

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 09:51:23 -07:00
Brandon Williams
3e7eaed016 submodule init: initialize active submodules
Teach `submodule init` to initialize submodules which have been
configured to be active by setting 'submodule.active' with a pathspec.

Now if no path arguments are given and 'submodule.active' is configured,
`init` will initialize all submodules which have been configured to be
active.  If no path arguments are given and 'submodule.active' is not
configured, then `init` will retain the old behavior of initializing all
submodules.

This allows users to record more complex patterns as it saves retyping
them whenever you invoke update.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 09:51:23 -07:00
Brandon Williams
ee92ab992f submodule--helper clone: check for configured submodules using helper
Use the 'is_submodule_initialized()' helper to check for configured
submodules instead of manually checking for the submodule's URL in the
config.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-17 11:06:09 -07:00
Brandon Williams
5c2bd8b77a submodule--helper: add is-active subcommand
The definition of which submodules are of interest by the user
is tied to the configuration submodule.<name>.url; when it is
set to a non-empty string, it is of interest.  We'd want to be
able to later change this definition, but there are many places
that explicitly check this condition in the scripted Porcelain.

Introduce the "is-active" subcommand to "submodule--helper", so
that the exact definition of what submodule is of interest can
be centrally defined (and changed in later steps).  In a few
patches that follow, this helper is used to replace the explicit
checks of the configuration variable in scripts.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-17 11:06:09 -07:00
Valery Tolstov
4682085cd4 submodule--helper.c: remove duplicate code
Remove code fragment from module_clone that duplicates functionality
of connect_work_tree_and_git_dir in dir.c

Signed-off-by: Valery Tolstov <me@vtolstov.org>
Reviewed-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-15 18:15:53 -07:00
Stefan Beller
d1b3b81aab submodule init: warn about falling back to a local path
When a submodule is initialized, the config variable 'submodule.<name>.url'
is set depending on the value of the same variable in the .gitmodules
file. When the URL indicates to be relative, then the url is computed
relative to its default remote. The default remote cannot be determined
accurately in all cases, such that it falls back to 'origin'.

The 'origin' remote may not exist, though. In that case we give up looking
for a suitable remote and we'll just assume it to be a local relative path.

This can be confusing to users as there is a lot of guessing involved,
which is not obvious to the user.

So in the corner case of assuming a local autoritative truth, warn the
user to lessen the confusion.

This behavior was introduced in 4d6893200 (submodule add: allow relative
repository path even when no url is set, 2011-06-06), which shared the
code with submodule-init and then ported to C in 3604242f08 (submodule:
port init from shell to C, 2016-04-15).

In case of submodule-add, this behavior makes sense in some use cases[1],
however for submodule-init there does not seem to be an immediate obvious
use case to fall back to a local submodule. However there might be, so
warn instead of die here.

While adding the warning, also clarify the behavior of relative URLs in
the documentation.

[1] e.g. http://stackoverflow.com/questions/8721984/git-ignore-files-for-public-repository-but-not-for-private
"store a secret locally in a submodule, with no intention to publish it"

Reported-by: Shawn Pearce <spearce@spearce.org>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-28 13:46:22 -08:00
Junio C Hamano
6f1c08bdb7 Merge branch 'rs/absolute-pathdup'
Code cleanup.

* rs/absolute-pathdup:
  use absolute_pathdup()
  abspath: add absolute_pathdup()
2017-02-02 13:36:55 -08:00
René Scharfe
0aaad415bc use absolute_pathdup()
Apply the semantic patch for converting callers that duplicate the
result of absolute_path() to call absolute_pathdup() instead, which
avoids an extra string copy to a static buffer.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-27 10:18:15 -08:00
Junio C Hamano
e801be066c Merge branch 'sb/submodule-init'
Error message fix.

* sb/submodule-init:
  submodule update --init: display correct path from submodule
2017-01-23 15:59:21 -08:00
Stefan Beller
6e7c14e65c submodule update --init: display correct path from submodule
In the submodule helper we did not correctly handled the display path
for initializing submodules when both the submodule is inside a
subdirectory as well as the command being invoked from a subdirectory
(as viewed from the superproject).

This was broken in 3604242f08, which was written at a time where
there was no super-prefix available, so we abused the --prefix option
for the same purpose and could get only one case right (the call from
within a subdirectory, not the submodule being in a subdirectory).

Test-provided-by: David Turner <novalis@novalis.org>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-12 14:36:20 -08:00
Junio C Hamano
da2b74eeec Merge branch 'sb/submodule-embed-gitdir'
A new submodule helper "git submodule embedgitdirs" to make it
easier to move embedded .git/ directory for submodules in a
superproject to .git/modules/ (and point the latter with the former
that is turned into a "gitdir:" file) has been added.

* sb/submodule-embed-gitdir:
  worktree: initialize return value for submodule_uses_worktrees
  submodule: add absorb-git-dir function
  move connect_work_tree_and_git_dir to dir.h
  worktree: check if a submodule uses worktrees
  test-lib-functions.sh: teach test_commit -C <dir>
  submodule helper: support super prefix
  submodule: use absolute path for computing relative path connecting
2017-01-10 15:24:28 -08:00
Stefan Beller
f6f8586140 submodule: add absorb-git-dir function
When a submodule has its git dir inside the working dir, the submodule
support for checkout that we plan to add in a later patch will fail.

Add functionality to migrate the git directory to be absorbed
into the superprojects git directory.

The newly added code in this patch is structured such that other areas of
Git can also make use of it. The code in the submodule--helper is a mere
wrapper and option parser for the function
`absorb_git_dir_into_superproject`, that takes care of embedding the
submodules git directory into the superprojects git dir. That function
makes use of the more abstract function for this use case
`relocate_gitdir`, which can be used by e.g. the worktree code eventually
to move around a git directory.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12 15:15:07 -08:00
Vitaly "_Vi" Shukela
bf03b79047 submodule--helper: set alternateLocation for cloned submodules
In 31224cbdc7 (clone: recursive and reference option triggers
submodule alternates, 2016-08-17) a mechanism was added to
have submodules referenced.  It did not address _nested_
submodules, however.

This patch makes all not just the root repository, but also
all submodules (recursively) have submodule.alternateLocation
and submodule.alternateErrorStrategy configured, making Git
search for possible alternates for nested submodules as well.

As submodule's alternate target does not end in .git/objects
(rather .git/modules/qqqqqq/objects), this alternate target
path restriction for in add_possible_reference_from_superproject
relates from "*.git/objects" to just */objects".

New tests have been added to t7408-submodule-reference.

Signed-off-by: Vitaly _Vi Shukela <vi0oss@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-12 09:56:52 -08:00
Stefan Beller
89c8626557 submodule helper: support super prefix
Just like main commands in Git, the submodule helper needs
access to the superproject prefix. Enable this in the git.c
but have its own fuse in the helper code by having a flag to
turn on the super prefix.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-09 14:52:57 -08:00
Junio C Hamano
65aaa29f7d Merge branch 'sb/submodule-ignore-trailing-slash'
A minor regression fix for "git submodule".

* sb/submodule-ignore-trailing-slash:
  t0060: sidestep surprising path mangling results on Windows
  submodule: ignore trailing slash in relative url
  submodule: ignore trailing slash on superproject URL
2016-10-27 14:58:49 -07:00
Junio C Hamano
dec040192f Merge branch 'jk/alt-odb-cleanup'
Codepaths involved in interacting alternate object store have
been cleaned up.

* jk/alt-odb-cleanup:
  alternates: use fspathcmp to detect duplicates
  sha1_file: always allow relative paths to alternates
  count-objects: report alternates via verbose mode
  fill_sha1_file: write into a strbuf
  alternates: store scratch buffer as strbuf
  fill_sha1_file: write "boring" characters
  alternates: use a separate scratch space
  alternates: encapsulate alt->base munging
  alternates: provide helper for allocating alternate
  alternates: provide helper for adding to alternates list
  link_alt_odb_entry: refactor string handling
  link_alt_odb_entry: handle normalize_path errors
  t5613: clarify "too deep" recursion tests
  t5613: do not chdir in main process
  t5613: whitespace/style cleanups
  t5613: use test_must_fail
  t5613: drop test_valid_repo function
  t5613: drop reachable_via function
2016-10-17 13:25:20 -07:00
Jeff King
597f9134de alternates: use a separate scratch space
The alternate_object_database struct uses a single buffer
both for storing the path to the alternate, and as a scratch
buffer for forming object names. This is efficient (since
otherwise we'd end up storing the path twice), but it makes
life hard for callers who just want to know the path to the
alternate. They have to remember to stop reading after
"alt->name - alt->base" bytes, and to subtract one for the
trailing '/'.

It would be much simpler if they could simply access a
NUL-terminated path string. We could encapsulate this in a
function which puts a NUL in the scratch buffer and returns
the string, but that opens up questions about the lifetime
of the result. The first time another caller uses the
alternate, the scratch buffer may get other data tacked onto
it.

Let's instead just store the root path separately from the
scratch buffer. There aren't enough alternates being stored
for the duplicated data to matter for performance, and this
keeps things simple and safe for the callers.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10 13:52:36 -07:00
Stefan Beller
3389e78ec8 submodule: ignore trailing slash in relative url
This is similar to the previous patch, though no user reported a bug and
I could not find a regressive behavior.

However it is a good thing to be strict on the output and for that we
always omit a trailing slash.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10 13:30:31 -07:00
Stefan Beller
087885049e submodule: ignore trailing slash on superproject URL
Before 63e95beb0 (2016-04-15, submodule: port resolve_relative_url from
shell to C), it did not matter if the superprojects URL had a trailing
slash or not. It was just chopped off as one of the first steps
(The "remoteurl=${remoteurl%/}" near the beginning of
resolve_relative_url(), which was removed in said commit).

When porting this to the C version, an off-by-one error was introduced
and we did not check the actual last character to be a slash, but the
NULL delimiter.

Reintroduce the behavior from before 63e95beb0, to ignore the trailing
slash.

Reported-by: <venv21@gmail.com>
Helped-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-10-10 13:30:28 -07:00
Junio C Hamano
f0798e6cdb Merge branch 'rs/cocci'
Code clean-up with help from coccinelle tool continues.

* rs/cocci:
  coccicheck: make transformation for strbuf_addf(sb, "...") more precise
  use strbuf_add_unique_abbrev() for adding short hashes, part 2
  use strbuf_addstr() instead of strbuf_addf() with "%s", part 2
  gitignore: ignore output files of coccicheck make target
2016-10-06 14:53:12 -07:00
René Scharfe
92d52fab3a use strbuf_addstr() instead of strbuf_addf() with "%s", part 2
Replace uses of strbuf_addf() for adding strings with more lightweight
strbuf_addstr() calls.  This is shorter and makes the intent clearer.

bc57b9c0cc already converted three cases,
this patch covers two more.

A semantic patch for Coccinelle is included for easier checking for
new cases that might be introduced in the future.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-27 14:02:40 -07:00
Junio C Hamano
c4dfd2291b Merge branch 'jk/clone-recursive-progress'
"git clone --recurse-submodules" lost the progress eye-candy in
recent update, which has been corrected.

* jk/clone-recursive-progress:
  clone: pass --progress decision to recursive submodules
2016-09-26 16:09:22 -07:00
Junio C Hamano
85f34a929d Merge branch 'rs/cocci'
Code cleanup.

* rs/cocci:
  use strbuf_addstr() for adding constant strings to a strbuf, part 2
  add coccicheck make target
  contrib/coccinelle: fix semantic patch for oid_to_hex_r()
2016-09-26 16:09:14 -07:00
Jeff King
72c5f88311 clone: pass --progress decision to recursive submodules
When cloning with "--recursive", we'd generally expect
submodules to show progress reports if the main clone did,
too.

In older versions of git, this mostly worked out of the
box. Since we show progress by default when stderr is a tty,
and since the child clones inherit the parent stderr, then
both processes would come to the same decision by default.
If the parent clone was asked for "--quiet", we passed down
"--quiet" to the child. However, if stderr was not a tty and
the user specified "--progress", we did not propagate this
to the child.

That's a minor bug, but things got much worse when we
switched recently to submodule--helper's update_clone
command. With that change, the stderr of the child clones
are always connected to a pipe, and we never output
progress at all.

This patch teaches git-submodule and git-submodule--helper
how to pass down an explicit "--progress" flag when cloning.
The clone command then decides to propagate that flag based
on the cloning decision made earlier (which takes into
account isatty(2) of the parent process, existing --progress
or --quiet flags, etc). Since the child processes always run
without a tty on stderr, we don't have to worry about
passing an explicit "--no-progress"; it's the default for
them.

This fixes the recent loss of progress during recursive
clones. And as a bonus, it makes:

  git clone --recursive --progress ... 2>&1 | cat

work by triggering progress explicitly in the children.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-22 11:22:47 -07:00
Junio C Hamano
4af9a7d344 Merge branch 'bc/object-id'
The "unsigned char sha1[20]" to "struct object_id" conversion
continues.  Notable changes in this round includes that ce->sha1,
i.e. the object name recorded in the cache_entry, turns into an
object_id.

It had merge conflicts with a few topics in flight (Christian's
"apply.c split", Dscho's "cat-file --filters" and Jeff Hostetler's
"status --porcelain-v2").  Extra sets of eyes double-checking for
mismerges are highly appreciated.

* bc/object-id:
  builtin/reset: convert to use struct object_id
  builtin/commit-tree: convert to struct object_id
  builtin/am: convert to struct object_id
  refs: add an update_ref_oid function.
  sha1_name: convert get_sha1_mb to struct object_id
  builtin/update-index: convert file to struct object_id
  notes: convert init_notes to use struct object_id
  builtin/rm: convert to use struct object_id
  builtin/blame: convert file to use struct object_id
  Convert read_mmblob to take struct object_id.
  notes-merge: convert struct notes_merge_pair to struct object_id
  builtin/checkout: convert some static functions to struct object_id
  streaming: make stream_blob_to_fd take struct object_id
  builtin: convert textconv_object to use struct object_id
  builtin/cat-file: convert some static functions to struct object_id
  builtin/cat-file: convert struct expand_data to use struct object_id
  builtin/log: convert some static functions to use struct object_id
  builtin/blame: convert struct origin to use struct object_id
  builtin/apply: convert static functions to struct object_id
  cache: convert struct cache_entry to use struct object_id
2016-09-19 13:47:19 -07:00
René Scharfe
a22ae753cb use strbuf_addstr() for adding constant strings to a strbuf, part 2
Replace uses of strbuf_addf() for adding strings with more lightweight
strbuf_addstr() calls.  This makes the intent clearer and avoids
potential issues with printf format specifiers.

02962d3684 already converted six cases,
this patch covers eleven more.

A semantic patch for Coccinelle is included for easier checking for
new cases that might be introduced in the future.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-15 12:23:38 -07:00
Junio C Hamano
02c6c14d6c Merge branch 'sb/submodule-clone-rr'
"git clone --resurse-submodules --reference $path $URL" is a way to
reduce network transfer cost by borrowing objects in an existing
$path repository when cloning the superproject from $URL; it
learned to also peek into $path for presense of corresponding
repositories of submodules and borrow objects from there when able.

* sb/submodule-clone-rr:
  clone: recursive and reference option triggers submodule alternates
  clone: implement optional references
  clone: clarify option_reference as required
  clone: factor out checking for an alternate path
  submodule--helper update-clone: allow multiple references
  submodule--helper module-clone: allow multiple references
  t7408: merge short tests, factor out testing method
  t7408: modernize style
2016-09-08 21:49:50 -07:00
brian m. carlson
99d1a9861a cache: convert struct cache_entry to use struct object_id
Convert struct cache_entry to use struct object_id by applying the
following semantic patch and the object_id transforms from contrib, plus
the actual change to the struct:

@@
struct cache_entry E1;
@@
- E1.sha1
+ E1.oid.hash

@@
struct cache_entry *E1;
@@
- E1->sha1
+ E1->oid.hash

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-09-07 12:59:42 -07:00
Stefan Beller
31224cbdc7 clone: recursive and reference option triggers submodule alternates
When `--recursive` and `--reference` is given, it is reasonable to
expect that the submodules are created with references to the submodules
of the given alternate for the superproject.

  An initial attempt to do this was presented to the mailing list, which
  used flags that are passed around ("--super-reference") that instructed
  the submodule clone to look for a reference in the submodules of the
  referenced superproject. This is not well thought out, as any further
  `submodule update` should also respect the initial setup.

  When a new  submodule is added to the superproject and the alternate
  of the superproject does not know about that submodule yet, we rather
  error out informing the user instead of being unclear if we did or did
  not use a submodules alternate.

To solve this problem introduce new options that store the configuration
for what the user wanted originally.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-17 17:19:11 -07:00
Stefan Beller
5f50f33e87 submodule--helper update-clone: allow multiple references
Allow the user to pass in multiple references to update_clone.
Currently this is only internal API, but once the shell script is
replaced by a C version, this is needed.

This fixes an API bug between the shell script and the helper.
Currently the helper accepts "--reference" "--reference=foo"
as a OPT_STRING whose value happens to be "--reference=foo", and
then uses

        if (suc->reference)
                argv_array_push(&child->args, suc->reference)

where suc->reference _is_ "--reference=foo" when invoking the
underlying "git clone", it cancels out.

With this change we omit one of the "--reference" arguments when
passing references from the shell script to the helper.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 15:00:17 -07:00
Stefan Beller
965dbea09a submodule--helper module-clone: allow multiple references
Allow users to pass in multiple references, just as clone accepts multiple
references as well.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-12 15:00:17 -07:00
Junio C Hamano
18f3ce8841 Merge branch 'rs/child-process-init'
A small code clean-up.

* rs/child-process-init:
  use CHILD_PROCESS_INIT to initialize automatic variables
2016-08-12 09:47:36 -07:00
Junio C Hamano
2f9c615efb Merge branch 'sb/submodule-clone-retry'
Fix-up to an error codepath in a topic already in 'master'.

* sb/submodule-clone-retry:
  submodule--helper: use parallel processor correctly
2016-08-12 09:47:34 -07:00
Junio C Hamano
11b53957ac Merge branch 'sb/submodule-update-dot-branch'
A few updates to "git submodule update".

Use of "| wc -l" break with BSD variant of 'wc'.

* sb/submodule-update-dot-branch:
  t7406: fix breakage on OSX
  submodule update: allow '.' for branch value
  submodule--helper: add remote-branch helper
  submodule-config: keep configured branch around
  submodule--helper: fix usage string for relative-path
  submodule update: narrow scope of local variable
  submodule update: respect depth in subsequent fetches
  t7406: future proof tests with hard coded depth
2016-08-10 12:33:20 -07:00
Stefan Beller
2201ee09b5 submodule--helper: use parallel processor correctly
When developing another patch series I had a temporary state in
which git-clone would segfault, when the call was prepared in
prepare_to_clone_next_submodule. This lead to the call failing,
i.e. in `update_clone_task_finished` the task was scheduled to be
tried again.  The second call to prepare_to_clone_next_submodule
would return 0, as the segfaulted clone did create the .git file
already, such that was not considered to need to be cloned again. I
was seeing the "BUG: ce was a submodule before?\n" message, which
was the correct behavior at the time as my local code was
buggy. When trying to debug this failure, I tried to use printing
messages into the strbuf that is passed around, but these messages
were never printed as the die(..) doesn't flush the `err` strbuf.

When implementing the die() in 665b35ecc (2016-06-09, "submodule--helper:
initial clone learns retry logic"), I considered this condition to be
a severe condition, which should lead to an immediate abort as we do not
trust ourselves any more. However the queued messages in `err` are valuable
so let's not toss them out by immediately dying, but a graceful return.

Another thing to note: The error message itself was misleading. A return
value of 0 doesn't indicate the passed in `ce` is not a submodule any more,
but just that we do not consider cloning it any more.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-09 14:54:16 -07:00
René Scharfe
542aa25d97 use CHILD_PROCESS_INIT to initialize automatic variables
Initialize struct child_process variables already when they're defined.
That's shorter and saves a function call.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-05 15:10:05 -07:00
Stefan Beller
4d7bc52b17 submodule update: allow '.' for branch value
Gerrit has a "superproject subscription" feature[1], that triggers a
commit in a superproject that is subscribed to its submodules.
Conceptually this Gerrit feature can be done on the client side with
Git via (except for raciness, error handling etc):

  while [ true ]; do
    git -C <superproject> submodule update --remote --force
    git -C <superproject> commit -a -m "Update submodules"
    git -C <superproject> push
  done

for each branch in the superproject. To ease the configuration in Gerrit
a special value of "." has been introduced for the submodule.<name>.branch
to mean the same branch as the superproject[2], such that you can create a
new branch on both superproject and the submodule and this feature
continues to work on that new branch.

Now we find projects in the wild with such a .gitmodules file.
The .gitmodules used in these Gerrit projects do not conform
to Gits understanding of how .gitmodules should look like.
This teaches Git to deal gracefully with this syntax as well.

The redefinition of "." does no harm to existing projects unaware of
this change, as "." is an invalid branch name in Git, so we do not
expect such projects to exist.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-03 16:13:22 -07:00
Stefan Beller
92bbe7ccf1 submodule--helper: add remote-branch helper
In a later patch we want to enhance the logic for the branch selection.
Rewrite the current logic to be in C, so we can directly use C when
we enhance the logic.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-03 16:11:35 -07:00
Stefan Beller
2de26ae1dc submodule--helper: fix usage string for relative-path
Internally we call the underscore version of relative_path, but externally
we present an API with no underscores.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-08-01 14:41:53 -07:00
Johannes Sixt
eb09121b74 submodule-helper: fix indexing in clone retry error reporting path
'git submodule--helper update-clone' has logic to retry failed clones
a second time. For this purpose, there is a list of submodules to clone,
and a second list that is filled with the submodules to retry. Within
these lists, the submodules are identified by an index as if both lists
were just appended.

This works nicely except when the second clone attempt fails as well. To
report an error, the identifying index must be adjusted by an offset so
that it can be used as an index into the second list. However, the
calculation uses the logical total length of the lists so that the result
always points one past the end of the second list.

Pick the correct index.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-07-22 13:43:53 -07:00
Junio C Hamano
bb2d8a817d Merge branch 'sb/submodule-clone-retry'
"git submodule update" that drives many "git clone" could
eventually hit flaky servers/network conditions on one of the
submodules; the command learned to retry the attempt.

* sb/submodule-clone-retry:
  submodule update: continue when a clone fails
  submodule--helper: initial clone learns retry logic
2016-07-11 10:31:04 -07:00
Junio C Hamano
3807098cd6 Merge branch 'sb/submodule-recommend-shallowness'
An upstream project can make a recommendation to shallowly clone
some submodules in the .gitmodules file it ships.

* sb/submodule-recommend-shallowness:
  submodule update: learn `--[no-]recommend-shallow` option
  submodule-config: keep shallow recommendation around
2016-06-20 11:01:01 -07:00
Stefan Beller
665b35eccd submodule--helper: initial clone learns retry logic
Each submodule that is attempted to be cloned, will be retried once in
case of failure after all other submodules were cloned. This helps to
mitigate ephemeral server failures and increases chances of a reliable
clone of a repo with hundreds of submodules immensely.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-13 10:28:38 -07:00
Stefan Beller
44431df024 submodule: remove bashism from shell script
Junio pointed out `relative_path` was using bashisms via the
local variables. As the longer term goal is to rewrite most of the
submodule code in C, do it now.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 11:32:53 -07:00
Stefan Beller
b0f4b40846 submodule--helper: offer a consistent API
In 48308681 (2016-02-29, git submodule update: have a dedicated helper
for cloning), the helper communicated errors back only via exit code,
and dance with printing '#unmatched' in case of error was left to
git-submodule.sh as it uses the output of the helper and pipes it into
shell commands. This change makes the helper consistent by never
printing '#unmatched' in the helper but always handling these piping
issues in the actual shell script.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-01 11:31:49 -07:00
Stefan Beller
abed000aca submodule update: learn --[no-]recommend-shallow option
Sometimes the history of a submodule is not considered important by
the projects upstream. To make it easier for downstream users, allow
a boolean field 'submodule.<name>.shallow' in .gitmodules, which can
be used to recommend whether upstream considers the history important.

This field is honored in the initial clone by default, it can be
ignored by giving the `--no-recommend-shallow` option.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-27 10:40:46 -07:00
Junio C Hamano
e059388fb2 Merge branch 'jk/submodule-c-credential'
An earlier addition of "sanitize_submodule_env" with 14111fc4 (git:
submodule honor -c credential.* from command line, 2016-02-29)
turned out to be a convoluted no-op; implement what it wanted to do
correctly, and stop filtering settings given via "git -c var=val".

* jk/submodule-c-credential:
  submodule: stop sanitizing config options
  submodule: use prepare_submodule_repo_env consistently
  submodule--helper: move config-sanitizing to submodule.c
  submodule: export sanitized GIT_CONFIG_PARAMETERS
  t5550: break submodule config test into multiple sub-tests
  t5550: fix typo in $HTTPD_URL
2016-05-17 14:38:25 -07:00
Junio C Hamano
f2c96ceb57 Merge branch 'sb/submodule-init'
Update of "git submodule" to move pieces of logic to C continues.

* sb/submodule-init:
  submodule init: redirect stdout to stderr
  submodule--helper update-clone: abort gracefully on missing .gitmodules
  submodule init: fail gracefully with a missing .gitmodules file
  submodule: port init from shell to C
  submodule: port resolve_relative_url from shell to C
2016-05-17 14:38:17 -07:00
Jeff King
89044baa8b submodule: stop sanitizing config options
The point of having a whitelist of command-line config
options to pass to submodules was two-fold:

  1. It prevented obvious nonsense like using core.worktree
     for multiple repos.

  2. It could prevent surprise when the user did not mean
     for the options to leak to the submodules (e.g.,
     http.sslverify=false).

For case 1, the answer is mostly "if it hurts, don't do
that". For case 2, we can note that any such example has a
matching inverted surprise (e.g., a user who meant
http.sslverify=true to apply everywhere, but it didn't).

So this whitelist is probably not giving us any benefit, and
is already creating a hassle as people propose things to put
on it. Let's just drop it entirely.

Note that we still need to keep a special code path for
"prepare the submodule environment", because we still have
to take care to pass through $GIT_CONFIG_PARAMETERS (and
block the rest of the repo-specific environment variables).

We can do this easily from within the submodule shell
script, which lets us drop the submodule--helper option
entirely (and it's OK to do so because as a "--" program, it
is entirely a private implementation detail).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-06 12:54:27 -07:00
Stefan Beller
c66410ed32 submodule init: redirect stdout to stderr
Reroute the output of stdout to stderr as it is just informative
messages, not to be consumed by machines.

This should not regress any scripts that try to parse the
current output, as the output is already internationalized
and therefore unstable.

We want to init submodules from the helper for `submodule update`
in a later patch and the stdout output of said helper is consumed
by the parts of `submodule update` which are still written in shell.
So we have to be careful which messages are on stdout.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-05-03 09:39:45 -07:00
Stefan Beller
08fdbdb153 submodule--helper update-clone: abort gracefully on missing .gitmodules
When there is no .gitmodules file availabe to initialize a submodule
from, `submodule_from_path` just returns NULL. We need to check for
that and abort gracefully.

When `git submodule update` was implemented in shell, this error out
with the warning

    Submodule path '%s' not initialized
    Maybe you want to use 'update --init'?

Replicate that behavior for now instead of crashing.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-29 10:07:13 -07:00
Stefan Beller
d92028a575 submodule init: fail gracefully with a missing .gitmodules file
When there is no .gitmodules file availabe to initialize a submodule
from, `submodule_from_path` just returns NULL. We need to check for
that and abort gracefully. When `submodule init` was implemented in shell,
a missing .gitmodules file would result in an error message

    No url found for submodule path '%s' in .gitmodules

Replicate that error message for now.

When the .gitmodules file is missing we can probably fail even earlier
for all of the submodules with an improved error message.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-29 10:05:24 -07:00
Jeff King
4638728c63 submodule--helper: move config-sanitizing to submodule.c
These functions should be used by any code which spawns a
submodule process, which may happen in submodule.c (e.g.,
for spawning fetch). Let's move them there and make them
public so that submodule--helper can continue to use them.

Since they're now public, let's also provide a basic overview
of their intended use.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-28 12:15:21 -07:00
Junio C Hamano
deef3cdc08 Merge branch 'sb/submodule-helper-clone-regression-fix'
A partial rewrite of "git submodule" in the 2.7 timeframe changed
the way the gitdir: pointer in the submodules point at the real
repository location to use absolute paths by accident.  This has
been corrected.

* sb/submodule-helper-clone-regression-fix:
  submodule--helper, module_clone: catch fprintf failure
  submodule--helper: do not borrow absolute_path() result for too long
  submodule--helper, module_clone: always operate on absolute paths
  submodule--helper clone: create the submodule path just once
  submodule--helper: fix potential NULL-dereference
  recursive submodules: test for relative paths
2016-04-22 15:45:04 -07:00
Stefan Beller
3604242f08 submodule: port init from shell to C
By having the `submodule init` functionality in C, we can reference it
easier from other parts in the code in later patches. The code is split
up to have one function to initialize one submodule and a calling function
that takes care of the rest, such as argument handling and translating the
arguments to the paths of the submodules.

This is the first submodule subcommand that is fully converted to C
except for the usage string, so this is actually removing a call to
the `submodule--helper list` function, which is supposed to be used in
this transition. Instead we'll make a direct call to `module_list_compute`.

An explanation why we need to edit the prefixes in cmd_update in
git-submodule.sh in this patch:

By having no processing in the shell part, we need to convey the notion
of wt_prefix and prefix to the C parts, which former patches punted on
and did the processing of displaying path in the shell.

`wt_prefix` used to hold the path from the repository root to the current
directory, e.g. wt_prefix would be t/ if the user invoked the
`git submodule` command in ~/repo/t and ~repo is the GIT_DIR.

`prefix` used to hold the relative path from the repository root to the
operation, e.g. if you have recursive submodules, the shell script would
modify the `prefix` in each recursive step by adding the submodule path.

We will pass `wt_prefix` into the C helper via `git -C <dir>` as that
will setup git in the directory the user actually called git-submodule.sh
from. The `prefix` will be passed in via the `--prefix` option.

Having `prefix` and `wt_prefix` relative to the GIT_DIR of the
calling superproject is unfortunate with this patch as the C code doesn't
know about a possible recursion from a superproject via `submodule update
--init --recursive`.

To fix this, we change the meaning of `wt_prefix` to point to the current
project instead of the superproject and `prefix` to include any relative
paths issues in the superproject. That way `prefix` will become the leading
part for displaying paths and `wt_prefix` will be empty in recursive
calls for now.

The new notion of `wt_prefix` and `prefix` still allows us to reconstruct
the calling directory in the superproject by just traveling reverse of
`prefix`.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-16 23:45:18 -07:00
Stefan Beller
63e95beb08 submodule: port resolve_relative_url from shell to C
Later on we want to automatically call `git submodule init` from
other commands, such that the users don't have to initialize the
submodule themselves.  As these other commands are written in C
already, we'd need the init functionality in C, too.  The
`resolve_relative_url` function is a large part of that init
functionality, so start by porting this function to C.

To create the tests in t0060, the function `resolve_relative_url`
was temporarily enhanced to write all inputs and output to disk
when running the test suite. The added tests in this patch are
a small selection thereof.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-16 23:44:01 -07:00
Junio C Hamano
ee30f17805 Merge branch 'sb/submodule-path-misc-bugs' into sb/submodule-init
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.

Any further comments?  Otherwise will merge to 'next'.

* sb/submodule-path-misc-bugs: (600 commits)
  t7407: make expectation as clear as possible
  submodule update: test recursive path reporting from subdirectory
  submodule update: align reporting path for custom command execution
  submodule status: correct path handling in recursive submodules
  submodule update --init: correct path handling in recursive submodules
  submodule foreach: correct path display in recursive submodules
  Git 2.8
  Documentation: fix git-p4 AsciiDoc formatting
  mingw: skip some tests in t9115 due to file name issues
  t1300: fix the new --show-origin tests on Windows
  t1300-repo-config: make it resilient to being run via 'sh -x'
  config --show-origin: report paths with forward slashes
  submodule: fix regression for deinit without submodules
  l10n: pt_PT: Update and add new translations
  l10n: ca.po: update translation
  Git 2.8-rc4
  Documentation: fix broken linkgit to git-config
  Documentation: use ASCII quotation marks in git-p4
  Revert "config.mak.uname: use clang for Mac OS X 10.6"
  git-compat-util: st_add4: work around gcc 4.2.x compiler crash
  ...
2016-04-14 12:47:45 -07:00
Junio C Hamano
7307dd8989 Merge branch 'sb/submodule-helper-clone-regression-fix' into sb/submodule-init
* sb/submodule-helper-clone-regression-fix:
  submodule--helper, module_clone: catch fprintf failure
  submodule--helper: do not borrow absolute_path() result for too long
  submodule--helper, module_clone: always operate on absolute paths
  submodule--helper clone: create the submodule path just once
  submodule--helper: fix potential NULL-dereference
  recursive submodules: test for relative paths
2016-04-14 12:46:11 -07:00
Junio C Hamano
01e1d54418 Merge branch 'jk/submodule-c-credential'
"git -c credential.<var>=<value> submodule" can now be used to
propagate configuration variables related to credential helper
down to the submodules.

* jk/submodule-c-credential:
  git_config_push_parameter: handle empty GIT_CONFIG_PARAMETERS
  git: submodule honor -c credential.* from command line
  quote: implement sq_quotef()
  submodule: fix segmentation fault in submodule--helper clone
  submodule: fix submodule--helper clone usage
  submodule: check argc count for git submodule--helper clone
  submodule: don't pass empty string arguments to submodule--helper clone
2016-04-06 11:39:12 -07:00
Junio C Hamano
bdebbeb334 Merge branch 'sb/submodule-parallel-update'
A major part of "git submodule update" has been ported to C to take
advantage of the recently added framework to run download tasks in
parallel.

* sb/submodule-parallel-update:
  clone: allow an explicit argument for parallel submodule clones
  submodule update: expose parallelism to the user
  submodule helper: remove double 'fatal: ' prefix
  git submodule update: have a dedicated helper for cloning
  run_processes_parallel: rename parameters for the callbacks
  run_processes_parallel: treat output of children as byte array
  submodule update: direct error message to stderr
  fetching submodules: respect `submodule.fetchJobs` config option
  submodule-config: drop check against NULL
  submodule-config: keep update strategy around
2016-04-06 11:39:01 -07:00
Stefan Beller
1f15ba1f3c submodule--helper, module_clone: catch fprintf failure
The return value of fprintf is unchecked, which may lead to
unreported errors. Use fprintf_or_die to report the error to the user.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-01 14:04:33 -07:00
Junio C Hamano
1ea4d9b7c8 submodule--helper: do not borrow absolute_path() result for too long
absolute_path() is designed to allow its callers to take a brief
peek of the result (typically, to be fed to functions like
strbuf_add() and relative_path() as a parameter) without having to
worry about freeing it, but the other side of the coin of that
memory model is that the caller shouldn't rely too much on the
result living forever--there may be a helper function the caller
subsequently calls that makes its own call to absolute_path(),
invalidating the earlier result.

Use xstrdup() to make our own copy, and free(3) it when we are done.
While at it, remove an unnecessary sm_gitdir_rel variable that was
only used to as a parameter to call absolute_path() and never used
again.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-01 14:04:23 -07:00
Stefan Beller
f8eaa0ba98 submodule--helper, module_clone: always operate on absolute paths
When giving relative paths to `relative_path` to compute a relative path
from one directory to another, this may fail in `relative_path`.
Make sure both arguments to `relative_path` are always absolute.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-04-01 12:21:34 -07:00
Stefan Beller
47d5d64879 submodule--helper clone: create the submodule path just once
We make sure that the parent directory of path exists (or create it
otherwise) and then do the same for path + "/.git".

That is equivalent to just making sure that the parent directory of
path + "/.git" exists (or create it otherwise).

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-31 15:19:55 -07:00
Stefan Beller
3c0663e166 submodule--helper: fix potential NULL-dereference
Don't dereference NULL 'path' if it was never assigned.  Also
protect against an empty --path argument.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-31 15:19:01 -07:00
Stefan Beller
84ba959bbd submodule: fix regression for deinit without submodules
Per Cederqvist wrote:
> It used to be possible to run
>
>    git submodule deinit -f .
>
> to remove any submodules, no matter how many submodules you had.  That
> is no longer possible in projects that don't have any submodules at
> all.  The command will fail with:
>
>     error: pathspec '.' did not match any file(s) known to git.

This regression was introduced in 74703a1e4d (submodule: rewrite
`module_list` shell function in C, 2015-09-02), as we changed the
order of checking in new module listing to first check whether it is
a gitlin before feeding it to match_pathspec().  It used to be that
a pathspec that does not match any path were diagnosed as an error,
but the new code complains for a pathspec that does not match any
submodule path.

Arguably the new behaviour may give us a better diagnosis, but that
is inconsistent with the suggestion "deinit" gives, and also this
was an unintended accident.  The new behaviour hopefully can be
redesigned and implemented better in future releases, but for now,
switch these two checks to restore the same behavior as before.  In
an empty repository, giving the pathspec '.' will still get the same
"did not match" error, but that is the same bug we had before 1.7.0.

Reported-by: Per Cederqvist <cederp@opera.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-22 19:26:43 -07:00
Jacob Keller
14111fc492 git: submodule honor -c credential.* from command line
Due to the way that the git-submodule code works, it clears all local
git environment variables before entering submodules. This is normally
a good thing since we want to clear settings such as GIT_WORKTREE and
other variables which would affect the operation of submodule commands.
However, GIT_CONFIG_PARAMETERS is special, and we actually do want to
preserve these settings. However, we do not want to preserve all
configuration as many things should be left specific to the parent
project.

Add a git submodule--helper function, sanitize-config, which shall be
used to sanitize GIT_CONFIG_PARAMETERS, removing all key/value pairs
except a small subset that are known to be safe and necessary.

Replace all the calls to clear_local_git_env with a wrapped function
that filters GIT_CONFIG_PARAMETERS using the new helper and then
restores it to the filtered subset after clearing the rest of the
environment.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 12:24:22 -08:00
Jacob Keller
7dad263334 submodule: fix segmentation fault in submodule--helper clone
The git submodule--helper clone command will fail with a segmentation
fault when given a null url or null path variable. Since these are
required for proper functioning of the submodule--helper clone
subcommand, add checks to prevent running and fail gracefully when
missing.

Update the usage string to reflect the requirement that the --url and
--path "options" are required.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 12:24:10 -08:00
Jacob Keller
717416ca87 submodule: fix submodule--helper clone usage
git submodule--helper clone usage stated that paths were added after the
[--] argument. The actual implementation required use of --path argument
and only supports one path at a time. Update the usage string to match
the current implementation.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 12:24:05 -08:00
Jacob Keller
08e0970a86 submodule: check argc count for git submodule--helper clone
Extra unused arguments to git submodule--helper clone subcommand were
being silently ignored. Add a check to the argc count after options
handling to ensure that no extra arguments were left on the argv array.

Signed-off-by: Jacob Keller <jacob.keller@gmail.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 12:24:03 -08:00
Stefan Beller
2335b870fa submodule update: expose parallelism to the user
Expose possible parallelism either via the "--jobs" CLI parameter or
the "submodule.fetchJobs" setting.

By having the variable initialized to -1, we make sure 0 can be passed
into the parallel processing machine, which will then pick as many parallel
workers as there are CPUs.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 11:57:20 -08:00
Stefan Beller
cdc04b65b4 submodule helper: remove double 'fatal: ' prefix
The prefix is added by die(...), so we don't have to do it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 11:57:20 -08:00
Stefan Beller
48308681b0 git submodule update: have a dedicated helper for cloning
This introduces a new helper function in git submodule--helper
which takes care of cloning all submodules, which we want to
parallelize eventually.

Some tests (such as empty URL, update_mode=none) are required in the
helper to make the decision for cloning. These checks have been
moved into the C function as well (no need to repeat them in the
shell script).

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-03-01 11:57:20 -08:00
Stefan Beller
2b56bb7a87 submodule helper list: respect correct path prefix
This is a regression introduced by 74703a1e4d (submodule: rewrite
`module_list` shell function in C, 2015-09-02).

Add a test to ensure we list the right submodule when giving a
specific pathspec.

Reported-By: Caleb Jorden <cjorden@gmail.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-02-24 14:33:02 -08:00
Stefan Beller
ee8838d157 submodule: rewrite module_clone shell function in C
This reimplements the helper function `module_clone` in shell
in C as `clone`. This functionality is needed for converting
`git submodule update` later on, which we want to add threading
to.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-08 15:48:21 -07:00
Stefan Beller
0ea306ef17 submodule: rewrite module_name shell function in C
This implements the helper `name` in C instead of shell,
yielding a nice performance boost.

Before this patch, I measured a time (best out of three):

  $ time ./t7400-submodule-basic.sh  >/dev/null
    real	0m11.066s
    user	0m3.348s
    sys	0m8.534s

With this patch applied I measured (also best out of three)

  $ time ./t7400-submodule-basic.sh  >/dev/null
    real	0m10.063s
    user	0m3.044s
    sys	0m7.487s

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03 14:12:40 -07:00
Stefan Beller
74703a1e4d submodule: rewrite module_list shell function in C
Most of the submodule operations work on a set of submodules.
Calculating and using this set is usually done via:

       module_list "$@" | {
           while read mode sha1 stage sm_path
           do
                # the actual operation
           done
       }

Currently the function `module_list` is implemented in the
git-submodule.sh as a shell script wrapping a perl script.
The rewrite is in C, such that it is faster and can later be
easily adapted when other functions are rewritten in C.

git-submodule.sh, similar to the builtin commands, will navigate
to the top-most directory of the repository and keep the
subdirectory as a variable. As the helper is called from
within the git-submodule.sh script, we are already navigated
to the root level, but the path arguments are still relative
to the subdirectory we were in when calling git-submodule.sh.
That's why there is a `--prefix` option pointing to an alternative
path which to anchor relative path arguments.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03 14:12:40 -07:00