Commit graph

44 commits

Author SHA1 Message Date
Linus Arver
548afb0d9a docs: typofixes
These were found with an automated CLI tool [1]. Only the
"Documentation" subfolder (and not source code files) was considered
because the docs are user-facing.

[1]: https://crates.io/crates/typos-cli

Signed-off-by: Linus Arver <linusa@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-06-12 13:52:51 -07:00
William Sprent
00408adeac builtin/sparse-checkout: add check-rules command
There exists no direct way to interrogate git about which paths are
matched by a given set of sparsity rules. It is possible to get this
information from git, but it includes checking out the commit that
contains the paths, applying the sparse checkout patterns and then using
something like 'git ls-files -t' to check if the skip worktree bit is
set. This works in some case, but there are cases where it is awkward or
infeasible to generate a checkout for this purpose.

Exposing the pattern matching of sparse checkout enables more tooling to
be built and avoids a situation where tools that want to reason about
sparse checkouts start containing parallel implementation of the rules.
To accommodate this, add a 'check-rules' subcommand to the
'sparse-checkout' builtin along the lines of the 'git check-ignore' and
'git check-attr' commands. The new command accepts a list of paths on
stdin and outputs just the ones the match the sparse checkout.

To allow for use in a bare repository and to allow for interrogating
about other patterns than the current ones, include a '--rules-file'
option which allows the caller to explicitly pass sparse checkout rules
in the format accepted by 'sparse-checkout set --stdin'.

To allow for reuse of the handling of input patterns for the
'--rules-file' flag, modify 'add_patterns_from_input()' to be able to
read from a 'FILE' instead of just stdin.

To allow for reuse of the logic which decides whether or not rules
should be interpreted as cone-mode patterns, split that part out of
'update_modes()' such that can be called without modifying the config.

An alternative could have been to create a new 'check-sparsity' command.
However, placing it under 'sparse-checkout' allows for a) more easily
re-using the sparse checkout pattern matching and cone/non-code mode
handling, and b) keeps the documentation for the command next to the
experimental warning and the cone-mode discussion.

Signed-off-by: William Sprent <williams@unity3d.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2023-03-27 10:51:12 -07:00
Ævar Arnfjörð Bjarmason
8c9e292dc0 doc txt & -h consistency: add missing options and labels
Fix various issues of SYNOPSIS and -h output syntax where:

 * Options such as --force were missing entirely
 * ...or the short option, such as -f

 * We said "opts" or "options", but could instead enumerate
   the (small) set of supported options

 * Options that were missing entirely (ls-remote's --sort=<key>)

   As we can specify "--sort" multiple times (it's backed by a
   string-list" it should really be "[(--sort=<key>)...]", which is
   what "git for-each-ref" lists it as, but let's leave that issue for
   a subsequent cleanup, and stop at making these consistent. Other
   "ref-filter.h" users share the same issue, e.g. "git-branch.txt".

 * For "verify-tag" and "verify-commit" we were missing the "--raw"
   option.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-13 09:32:56 -07:00
Ævar Arnfjörð Bjarmason
6df5494f73 doc SYNOPSIS: consistently use ' for commands
Most of our commands use ''-quotation only for the name of the command
itself, and not its (optional) arguments. Let's do the same for these.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-13 09:32:54 -07:00
Elijah Newren
5d4b293340 Documentation: some sparsity wording clarifications
Improve the wording for a couple paragraphs in two different manuals
relating to sparse behavior.

Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:39 -07:00
Elijah Newren
a8defed07c git-sparse-checkout.txt: mark non-cone mode as deprecated
While we have no current plans to actually remove --no-cone mode, we
think users would be better off not using it.  Update the documentation
accordingly, including explaining why we think non-cone mode is
problematic for users.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:39 -07:00
Elijah Newren
72fa58ef50 git-sparse-checkout.txt: flesh out pattern set sections a bit
The "Internals -- Cone Pattern Set" section starts off discussing
patterns, despite the fact that cone mode is about avoiding the
patterns.  This made sense back when non-cone mode was the default and
we started by discussing the full pattern set, but now that we are
changing the default, it makes more sense to discuss cone-mode first and
avoid the full discussion of patterns.  Split this section into two, the
first with details about how cone mode operates, and the second
following the full pattern set section and discussing how the cone mode
patterns used under the hood relate to the full pattern set.

While at it, flesh out the "Internals -- Full Pattern Set" section a bit
to include more examples as well.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
5d295dc396 git-sparse-checkout.txt: add a new EXAMPLES section
Since many users like to learn from examples, provide a section in the
manual with example commands that would be used and a brief explanation
of what each does.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
0d86f59a3c git-sparse-checkout.txt: shuffle some sections and mark as internal
With cone mode as the default, it makes sense to discuss it before
non-cone mode.  Also, the new default means we can just use directories
in most cases and users do not need to understand patterns or their
meanings.  Let's take advantage of this to mark several sections as
"INTERNALS", notifying the user that they do not need to know all those
details in order to make use of the sparse-checkout command.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
71ceb816b3 git-sparse-checkout.txt: update docs for deprecation of 'init'
The 'init' subcommand of sparse-checkout was deprecated in ba2f3f58ac
("git-sparse-checkout.txt: update to document init/set/reapply changes",
2021-12-14), but a couple places in the manual still assumed it was the
primary way to use sparse-checkout.  Correct them.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
f69dfef355 git-sparse-checkout.txt: wording updates for the cone mode default
Now that cone mode is the default, we'd like to focus on the arguments
to set/add being directories rather than patterns, and it probably makes
sense to provide an earlier heads up that files from leading directories
get included as well.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Elijah Newren
2d95707a02 sparse-checkout: make --cone the default
Make cone mode the default, and update the documentation accordingly.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-04-21 23:12:38 -07:00
Junio C Hamano
82386b4496 Merge branch 'en/present-despite-skipped'
In sparse-checkouts, files mis-marked as missing from the working tree
could lead to later problems.  Such files were hard to discover, and
harder to correct.  Automatically detecting and correcting the marking
of such files has been added to avoid these problems.

* en/present-despite-skipped:
  repo_read_index: add config to expect files outside sparse patterns
  Accelerate clear_skip_worktree_from_present_files() by caching
  Update documentation related to sparsity and the skip-worktree bit
  repo_read_index: clear SKIP_WORKTREE bit from files present in worktree
  unpack-trees: fix accidental loss of user changes
  t1011: add testcase demonstrating accidental loss of user modifications
2022-03-09 13:38:23 -08:00
Derrick Stolee
7316dc5f6f sparse-checkout: set worktree-config correctly
`git sparse-checkout set/init` enables worktree-specific
configuration[*] by setting extensions.worktreeConfig=true, but neglects
to perform the additional necessary bookkeeping of relocating
`core.bare=true` and `core.worktree` from $GIT_COMMON_DIR/config to
$GIT_COMMON_DIR/config.worktree, as documented in git-worktree.txt. As a
result of this oversight, these settings, which are nonsensical for
secondary worktrees, can cause Git commands to incorrectly consider a
worktree bare (in the case of `core.bare`) or operate on the wrong
worktree (in the case of `core.worktree`). Fix this problem by taking
advantage of the recently-added init_worktree_config() which enables
`extensions.worktreeConfig` and takes care of necessary bookkeeping.

While at it, for backward-compatibility reasons, also stop upgrading the
repository format to "1" since doing so is (unintentionally) not
required to take advantage of `extensions.worktreeConfig`, as explained
by 11664196ac ("Revert "check_repository_format_gently(): refuse
extensions for old repositories"", 2020-07-15).

[*] The main reason to use worktree-specific config for the
sparse-checkout builtin was to avoid enabling sparse-checkout patterns
in one and causing a loss of files in another. If a worktree does not
have a sparse-checkout patterns file, then the sparse-checkout logic
will not kick in on that worktree.

Reported-by: Sean Allred <allred.sean@gmail.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-02-08 09:49:20 -08:00
Elijah Newren
9023535bd3 Update documentation related to sparsity and the skip-worktree bit
Make several small updates, to address a few documentation issues
I spotted:
  * sparse-checkout focused on "patterns" even though the inputs (and
    outputs in the case of `list`) are directories in cone-mode
  * The description section of the sparse-checkout documentation
    was a bit sparse (no pun intended), and focused more on internal
    mechanics rather than end user usage.  This made sense in the
    early days when the command was even more experimental, but let's
    adjust a bit to try to make it more approachable to end users who
    may want to consider using it.  Keep the scary backward
    compatibility warning, though; we're still hard at work trying to
    fix up commands to behave reasonably in sparse checkouts.
  * both read-tree and update-index tried to describe how to use the
    skip-worktree bit, but both predated the sparse-checkout command.
    The sparse-checkout command is a far easier mechanism to use and
    for users trying to reduce the size of their working tree, we
    should recommend users to look at it instead.
  * The update-index documentation pointed out that assume-unchanged
    and skip-worktree sounded similar but had different purposes.
    However, it made no attempt to explain the differences, only to
    point out that they were different.  Explain the differences.
  * The update-index documentation focused much more on (internal?)
    implementation details than on end-user usage.  Try to explain
    its purpose better for users of update-index, rather than
    fellow developers trying to work with the SKIP_WORKTREE bit.
  * Clarify that when core.sparseCheckout=true, we treat a file's
    presence in the working tree as being an override to the
    SKIP_WORKTREE bit (i.e. in sparse checkouts when the file is
    present we ignore the SKIP_WORKTREE bit).

Note that this commit, like many touching documentation, is best viewed
with the `--color-words` option to diff/log.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-01-14 14:44:07 -08:00
Junio C Hamano
2dc94da374 Merge branch 'en/sparse-checkout-set'
The "init" and "set" subcommands in "git sparse-checkout" have been
unified for a better user experience and performance.

* en/sparse-checkout-set:
  sparse-checkout: remove stray trailing space
  clone: avoid using deprecated `sparse-checkout init`
  Documentation: clarify/correct a few sparsity related statements
  git-sparse-checkout.txt: update to document init/set/reapply changes
  sparse-checkout: enable reapply to take --[no-]{cone,sparse-index}
  sparse-checkout: enable `set` to initialize sparse-checkout mode
  sparse-checkout: split out code for tweaking settings config
  sparse-checkout: disallow --no-stdin as an argument to set
  sparse-checkout: add sanity-checks on initial sparsity state
  sparse-checkout: break apart functions for sparse_checkout_(set|add)
  sparse-checkout: pass use_stdin as a parameter instead of as a global
2022-01-03 16:24:15 -08:00
Elijah Newren
d30e2bbe85 Documentation: clarify/correct a few sparsity related statements
Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Victoria Dye <vdye@github.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-15 11:48:22 -08:00
Elijah Newren
ba2f3f58ac git-sparse-checkout.txt: update to document init/set/reapply changes
As noted in the previous commit, using separate `init` and `set` steps
with sparse-checkout result in a number of issues.  The previous commits
made `set` able to handle the work of both commands, and enabled reapply
to tweak the {cone,sparse-index} settings.  Update the documentation to
reflect this, and mark `init` as deprecated.

Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Victoria Dye <vdye@github.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-12-15 11:48:22 -08:00
Junio C Hamano
23c83fc473 Merge branch 'ja/doc-cleanup'
Doc update.

* ja/doc-cleanup:
  init doc: --shared=0xxx does not give umask but perm bits
  doc: git-init: clarify file modes in octal.
  doc: git-http-push: describe the refs as pattern pairs
  doc: uniformize <URL> placeholders' case
  doc: use three dots for indicating repetition instead of star
  doc: git-ls-files: express options as optional alternatives
  doc: use only hyphens as word separators in placeholders
  doc: express grammar placeholders between angle brackets
  doc: split placeholders as individual tokens
  doc: fix git credential synopsis
2021-12-10 14:35:03 -08:00
Jean-Noël Avila
49cbad0edd doc: express grammar placeholders between angle brackets
This discerns user inputs from verbatim options in the synopsis.

Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-11-09 09:39:11 -08:00
Derrick Stolee
55dfcf9591 sparse-checkout: clear tracked sparse dirs
When changing the scope of a sparse-checkout using cone mode, we might
have some tracked directories go out of scope. The current logic removes
the tracked files from within those directories, but leaves the ignored
files within those directories. This is a bit unexpected to users who
have given input to Git saying they don't need those directories
anymore.

This is something that is new to the cone mode pattern type: the user
has explicitly said "I want these directories and _not_ those
directories." The typical sparse-checkout patterns more generally apply
to "I want files with with these patterns" so it is natural to leave
ignored files as they are. This focus on directories in cone mode
provides us an opportunity to change the behavior.

Leaving these ignored files in the sparse directories makes it
impossible to gain performance benefits in the sparse index. When we
track into these directories, we need to know if the files are ignored
or not, which might depend on the _tracked_ .gitignore file(s) within
the sparse directory. This depends on the indexed version of the file,
so the sparse directory must be expanded.

We must take special care to look for untracked, non-ignored files in
these directories before deleting them. We do not want to delete any
meaningful work that the users were doing in those directories and
perhaps forgot to add and commit before switching sparse-checkout
definitions. Since those untracked files might be code files that
generated ignored build output, also do not delete any ignored files
from these directories in that case. The users can recover their state
by resetting their sparse-checkout definition to include that directory
and continue. Alternatively, they can see the warning that is presented
and delete the directory themselves to regain the performance they
expect.

By deleting the sparse directories when changing scope (or running 'git
sparse-checkout reapply') we regain these performance benefits as if the
repository was in a clean state.

Since these ignored files are frequently build output or helper files
from IDEs, the users should not need the files now that the tracked
files are removed. If the tracked files reappear, then they will have
newer timestamps than the build artifacts, so the artifacts will need to
be regenerated anyway.

Use the sparse-index as a data structure in order to find the sparse
directories that can be safely deleted. Re-expand the index to a full
one if it was full before.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Reviewed-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-07 22:41:10 -07:00
Derrick Stolee
122ba1f7b5 sparse-checkout: toggle sparse index from builtin
The sparse index extension is used to signal that index writes should be
in sparse mode. This was only updated using GIT_TEST_SPARSE_INDEX=1.

Add a '--[no-]sparse-index' option to 'git sparse-checkout init' that
specifies if the sparse index should be used. It also updates the index
to use the correct format, either way. Add a warning in the
documentation that the use of a repository extension might reduce
compatibility with third-party tools. 'git sparse-checkout init' already
sets extension.worktreeConfig, which places most sparse-checkout users
outside of the scope of most third-party tools.

Update t1092-sparse-checkout-compatibility.sh to use this CLI instead of
GIT_TEST_SPARSE_INDEX=1.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-03-30 12:57:48 -07:00
Junio C Hamano
81be89e0be Merge branch 'en/sparse-with-submodule-doc'
The effect of sparse checkout settings on submodules is documented.

* en/sparse-with-submodule-doc:
  git-sparse-checkout: clarify interactions with submodules
2020-06-22 15:55:03 -07:00
Elijah Newren
e7d7c73249 git-sparse-checkout: clarify interactions with submodules
Ignoring the sparse-checkout feature momentarily, if one has a submodule and
creates local branches within it with unpushed changes and maybe adds some
untracked files to it, then we would want to avoid accidentally removing such
a submodule.  So, for example with git.git, if you run
   git checkout v2.13.0
then the sha1collisiondetection/ submodule is NOT removed even though it
did not exist as a submodule until v2.14.0.  Similarly, if you only had
v2.13.0 checked out previously and ran
   git checkout v2.14.0
the sha1collisiondetection/ submodule would NOT be automatically
initialized despite being part of v2.14.0.  In both cases, git requires
submodules to be initialized or deinitialized separately.  Further, we
also have special handling for submodules in other commands such as
clean, which requires two --force flags to delete untracked submodules,
and some commands have a --recurse-submodules flag.

sparse-checkout is very similar to checkout, as evidenced by the similar
name -- it adds and removes files from the working copy.  However, for
the same avoid-data-loss reasons we do not want to remove a submodule
from the working copy with checkout, we do not want to do it with
sparse-checkout either.  So submodules need to be separately initialized
or deinitialized; changing sparse-checkout rules should not
automatically trigger the removal or vivification of submodules.

I believe the previous wording in git-sparse-checkout.txt about
submodules was only about this particular issue.  Unfortunately, the
previous wording could be interpreted to imply that submodules should be
considered active regardless of sparsity patterns.  Update the wording
to avoid making such an implication.  It may be helpful to consider two
example situations where the differences in wording become important:

In the future, we want users to be able to run commands like
   git clone --sparse=moduleA --recurse-submodules $REPO_URL
and have sparsity paths automatically set up and have submodules *within
the sparsity paths* be automatically initialized.  We do not want all
submodules in any path to be automatically initialized with that
command.

Similarly, we want to be able to do things like
   git -c sparse.restrictCmds grep --recurse-submodules $REV $PATTERN
and search through $REV for $PATTERN within the recorded sparsity
patterns.  We want it to recurse into submodules within those sparsity
patterns, but do not want to recurse into directories that do not match
the sparsity patterns in search of a possible submodule.

Signed-off-by: Elijah Newren <newren@gmail.com>
Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-06-12 17:21:05 -07:00
Martin Ågren
e26433538a git-sparse-checkout.txt: add missing '
Where we explain the 'reapply' command, we don't properly wrap it in
single quote marks like we do with the other commands: We omit the
closing mark ("'reapply") and this ends up being rendered literally as
"'reapply". Add the missing "'".

Signed-off-by: Martin Ågren <martin.agren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-05-18 13:19:54 -07:00
Elijah Newren
5644ca28cd sparse-checkout: provide a new reapply subcommand
If commands like merge or rebase materialize files as part of their work,
or a previous sparse-checkout command failed to update individual files
due to dirty changes, users may want a command to simply 'reapply' the
sparsity rules.  Provide one.

Reviewed-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-03-27 11:33:31 -07:00
Junio C Hamano
f4d7dfce4d Merge branch 'ds/sparse-add'
"git sparse-checkout" learned a new "add" subcommand.

* ds/sparse-add:
  sparse-checkout: allow one-character directories in cone mode
  sparse-checkout: work with Windows paths
  sparse-checkout: create 'add' subcommand
  sparse-checkout: extract pattern update from 'set' subcommand
  sparse-checkout: extract add_patterns_from_input()
2020-03-05 10:43:02 -08:00
Junio C Hamano
433b8aac2e Merge branch 'ds/sparse-checkout-harden'
Some rough edges in the sparse-checkout feature, especially around
the cone mode, have been cleaned up.

* ds/sparse-checkout-harden:
  sparse-checkout: fix cone mode behavior mismatch
  sparse-checkout: improve docs around 'set' in cone mode
  sparse-checkout: escape all glob characters on write
  sparse-checkout: use C-style quotes in 'list' subcommand
  sparse-checkout: unquote C-style strings over --stdin
  sparse-checkout: write escaped patterns in cone mode
  sparse-checkout: properly match escaped characters
  sparse-checkout: warn on globs in cone patterns
  sparse-checkout: detect short patterns
  sparse-checkout: cone mode does not recognize "**"
  sparse-checkout: fix documentation typo for core.sparseCheckoutCone
  clone: fix --sparse option with URLs
  sparse-checkout: create leading directories
  t1091: improve here-docs
  t1091: use check_files to reduce boilerplate
2020-02-14 12:54:22 -08:00
Derrick Stolee
2631dc879d sparse-checkout: create 'add' subcommand
When using the sparse-checkout feature, a user may want to incrementally
grow their sparse-checkout pattern set. Allow adding patterns using a
new 'add' subcommand. This is not much different from the 'set'
subcommand, because we still want to allow the '--stdin' option and
interpret inputs as directories when in cone mode and patterns
otherwise.

When in cone mode, we are growing the cone. This may actually reduce the
set of patterns when adding directory A when A/B is already a directory
in the cone. Test the different cases: siblings, parents, ancestors.

When not in cone mode, we can only assume the patterns should be
appended to the sparse-checkout file.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-11 09:06:46 -08:00
Derrick Stolee
d2e65f4c90 sparse-checkout: improve docs around 'set' in cone mode
The existing documentation does not clarify how the 'set' subcommand
changes when core.sparseCheckoutCone is enabled. Correct this by
changing some language around the "A/B/C" example. Also include a
description of the input format matching the output of 'git ls-tree
--name-only'.

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-31 13:05:29 -08:00
Jeff King
7aa9ef2fca sparse-checkout: fix documentation typo for core.sparseCheckoutCone
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-24 13:26:54 -08:00
Matheus Tavares
a402723e48 doc: sparse-checkout: mention --cone option
In af09ce2 ("sparse-checkout: init and set in cone mode", 2019-11-21),
the '--cone' option was added to 'git sparse-checkout init'.

Document it.

Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br>
Acked-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-23 13:20:14 -08:00
Junio C Hamano
0f1930cd1b Merge branch 'ds/sparse-cone'
Code cleanup.

* ds/sparse-cone:
  Documentation/git-sparse-checkout.txt: fix a typo
  sparse-checkout: use extern for global variables
2020-01-06 14:17:51 -08:00
Taylor Blau
757ff352bd Documentation/git-sparse-checkout.txt: fix a typo
This typo was introduced in 94c0956b60 (sparse-checkout: create builtin
with 'list' subcommand, 2019-11-21).

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Acked-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-04 14:40:29 -08:00
Derrick Stolee
4fd683b6a3 sparse-checkout: document interactions with submodules
Using 'git submodule (init|deinit)' a user can select a subset of
submodules to populate. This behaves very similar to the sparse-checkout
feature, but those directories contain their own .git directory
including an object database and ref space. To have the sparse-checkout
file also determine if those files should exist would easily cause
problems. Therefore, keeping these features independent in this way
is the best way forward.

Also create a test that demonstrates this behavior to make sure
it doesn't change as the sparse-checkout feature evolves.

Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-30 09:07:20 -08:00
Derrick Stolee
de11951b03 sparse-checkout: list directories in cone mode
When core.sparseCheckoutCone is enabled, the 'git sparse-checkout set'
command takes a list of directories as input, then creates an ordered
list of sparse-checkout patterns such that those directories are
recursively included and all sibling entries along the parent directories
are also included. Listing the patterns is less user-friendly than the
directories themselves.

In cone mode, and as long as the patterns match the expected cone-mode
pattern types, change the output of 'git sparse-checkout list' to only
show the directories that created the patterns.

With this change, the following piped commands would not change the
working directory:

	git sparse-checkout list | git sparse-checkout set --stdin

The only time this would not work is if core.sparseCheckoutCone is
true, but the sparse-checkout file contains patterns that do not
match the expected pattern types for cone mode.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-30 09:07:18 -08:00
Derrick Stolee
190a65f9db sparse-checkout: respect core.ignoreCase in cone mode
When a user uses the sparse-checkout feature in cone mode, they
add patterns using "git sparse-checkout set <dir1> <dir2> ..."
or by using "--stdin" to provide the directories line-by-line over
stdin. This behaviour naturally looks a lot like the way a user
would type "git add <dir1> <dir2> ..."

If core.ignoreCase is enabled, then "git add" will match the input
using a case-insensitive match. Do the same for the sparse-checkout
feature.

Perform case-insensitive checks while updating the skip-worktree
bits during unpack_trees(). This is done by changing the hash
algorithm and hashmap comparison methods to optionally use case-
insensitive methods.

When this is enabled, there is a small performance cost in the
hashing algorithm. To tease out the worst possible case, the
following was run on a repo with a deep directory structure:

	git ls-tree -d -r --name-only HEAD |
		git sparse-checkout set --stdin

The 'set' command was timed with core.ignoreCase disabled or
enabled. For the repo with a deep history, the numbers were

	core.ignoreCase=false: 62s
	core.ignoreCase=true:  74s (+19.3%)

For reproducibility, the equivalent test on the Linux kernel
repository had these numbers:

	core.ignoreCase=false: 3.1s
	core.ignoreCase=true:  3.6s (+16%)

Now, this is not an entirely fair comparison, as most users
will define their sparse cone using more shallow directories,
and the performance improvement from eb42feca97 ("unpack-trees:
hash less in cone mode" 2019-11-21) can remove most of the
hash cost. For a more realistic test, drop the "-r" from the
ls-tree command to store only the first-level directories.
In that case, the Linux kernel repository takes 0.2-0.25s in
each case, and the deep repository takes one second, plus or
minus 0.05s, in each case.

Thus, we _can_ demonstrate a cost to this change, but it is
unlikely to matter to any reasonable sparse-checkout cone.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:01:02 -08:00
Derrick Stolee
99dfa6f970 sparse-checkout: use in-process update for disable subcommand
The 'git sparse-checkout disable' subcommand returns a user to a
full working directory. The old process for doing this required
updating the sparse-checkout file with the "/*" pattern and then
updating the working directory with core.sparseCheckout enabled.
Finally, the sparse-checkout file could be removed and the config
setting disabled.

However, it is valuable to keep a user's sparse-checkout file
intact so they can re-enable the sparse-checkout they previously
used with 'git sparse-checkout init'. This is now possible with
the in-process mechanism for updating the working directory.

Reported-by: Szeder Gábor <szeder.dev@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:45 +09:00
Derrick Stolee
879321eb0b sparse-checkout: add 'cone' mode
The sparse-checkout feature can have quadratic performance as
the number of patterns and number of entries in the index grow.
If there are 1,000 patterns and 1,000,000 entries, this time can
be very significant.

Create a new Boolean config option, core.sparseCheckoutCone, to
indicate that we expect the sparse-checkout file to contain a
more limited set of patterns. This is a separate config setting
from core.sparseCheckout to avoid breaking older clients by
introducing a tri-state option.

The config option does nothing right now, but will be expanded
upon in a later commit.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
72918c1ad9 sparse-checkout: create 'disable' subcommand
The instructions for disabling a sparse-checkout to a full
working directory are complicated and non-intuitive. Add a
subcommand, 'git sparse-checkout disable', to perform those
steps for the user.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
7bffca95ea sparse-checkout: add '--stdin' option to set subcommand
The 'git sparse-checkout set' subcommand takes a list of patterns
and places them in the sparse-checkout file. Then, it updates the
working directory to match those patterns. For a large list of
patterns, the command-line call can get very cumbersome.

Add a '--stdin' option to instead read patterns over standard in.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:44 +09:00
Derrick Stolee
f6039a9423 sparse-checkout: 'set' subcommand
The 'git sparse-checkout set' subcommand takes a list of patterns
as arguments and writes them to the sparse-checkout file. Then, it
updates the working directory using 'git read-tree -mu HEAD'.

The 'set' subcommand will replace the entire contents of the
sparse-checkout file. The write_patterns_and_update() method is
extracted from cmd_sparse_checkout() to make it easier to implement
'add' and/or 'remove' subcommands in the future.

If the core.sparseCheckout config setting is disabled, then enable
the config setting in the worktree config. If we set the config
this way and the sparse-checkout fails, then re-disable the config
setting.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00
Derrick Stolee
bab3c35908 sparse-checkout: create 'init' subcommand
Getting started with a sparse-checkout file can be daunting. Help
users start their sparse enlistment using 'git sparse-checkout init'.
This will set 'core.sparseCheckout=true' in their config, write
an initial set of patterns to the sparse-checkout file, and update
their working directory.

Make sure to use the `extensions.worktreeConfig` setting and write
the sparse checkout config to the worktree-specific config file.
This avoids confusing interactions with other worktrees.

The use of running another process for 'git read-tree' is sub-
optimal. This will be removed in a later change.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00
Derrick Stolee
94c0956b60 sparse-checkout: create builtin with 'list' subcommand
The sparse-checkout feature is mostly hidden to users, as its
only documentation is supplementary information in the docs for
'git read-tree'. In addition, users need to know how to edit the
.git/info/sparse-checkout file with the right patterns, then run
the appropriate 'git read-tree -mu HEAD' command. Keeping the
working directory in sync with the sparse-checkout file requires
care.

Begin an effort to make the sparse-checkout feature a porcelain
feature by creating a new 'git sparse-checkout' builtin. This
builtin will be the preferred mechanism for manipulating the
sparse-checkout file and syncing the working directory.

The documentation provided is adapted from the "git read-tree"
documentation with a few edits for clarity in the new context.
Extra sections are added to hint toward a future change to
a more restricted pattern set.

Helped-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-11-22 16:11:43 +09:00