development/git
development/git
1
0
Fork 0
mirror of https://github.com/git/git synced 2024-08-24 18:26:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'en/sparse-checkout-set'

Browse source 
The "init" and "set" subcommands in "git sparse-checkout" have been
unified for a better user experience and performance.

* en/sparse-checkout-set:
  sparse-checkout: remove stray trailing space
  clone: avoid using deprecated `sparse-checkout init`
  Documentation: clarify/correct a few sparsity related statements
  git-sparse-checkout.txt: update to document init/set/reapply changes
  sparse-checkout: enable reapply to take --[no-]{cone,sparse-index}
  sparse-checkout: enable `set` to initialize sparse-checkout mode
  sparse-checkout: split out code for tweaking settings config
  sparse-checkout: disallow --no-stdin as an argument to set
  sparse-checkout: add sanity-checks on initial sparsity state
  sparse-checkout: break apart functions for sparse_checkout_(set|add)
  sparse-checkout: pass use_stdin as a parameter instead of as a global
This commit is contained in:
Junio C Hamano 2022-01-03 16:24:15 -08:00
parent 4f4b18497a dfac9b609f
commit 2dc94da374
5 changed files with 227 additions and 97 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
Documentation/git-clone.txt
View file

@ -167,10 +167,10 @@ objects from the source repository into a pack in the cloned repository.
configuration variables are created.
--sparse::
Initialize the sparse-checkout file so the working
directory starts with only the files in the root
of the repository. The sparse-checkout file can be
modified to grow the working directory as needed.
Employ a sparse-checkout, with only files in the toplevel
directory initially being present. The
linkgit:git-sparse-checkout[1] command can be used to grow the
working directory as needed.
--filter=<filter-spec>::
Use the partial clone feature and request that the server sends

100
Documentation/git-sparse-checkout.txt
View file

@ -30,28 +30,36 @@ COMMANDS
'list'::
Describe the patterns in the sparse-checkout file.
'init'::
Enable the `core.sparseCheckout` setting. If the
sparse-checkout file does not exist, then populate it with
patterns that match every file in the root directory and
no other directories, then will remove all directories tracked
by Git. Add patterns to the sparse-checkout file to
repopulate the working directory.
'set'::
Enable the necessary config settings
(extensions.worktreeConfig, core.sparseCheckout,
core.sparseCheckoutCone) if they are not already enabled, and
write a set of patterns to the sparse-checkout file from the
list of arguments following the 'set' subcommand. Update the
working directory to match the new patterns.
+
To avoid interfering with other worktrees, it first enables the
`extensions.worktreeConfig` setting and makes sure to set the
`core.sparseCheckout` setting in the worktree-specific config file.
When the `--stdin` option is provided, the patterns are read from
standard in as a newline-delimited list instead of from the arguments.
+
When `--cone` is provided, the `core.sparseCheckoutCone` setting is
also set, allowing for better performance with a limited set of
patterns (see 'CONE PATTERN SET' below).
When `--cone` is passed or `core.sparseCheckoutCone` is enabled, the
input list is considered a list of directories instead of
sparse-checkout patterns. This allows for better performance with a
limited set of patterns (see 'CONE PATTERN SET' below). Note that the
set command will write patterns to the sparse-checkout file to include
all files contained in those directories (recursively) as well as
files that are siblings of ancestor directories. The input format
matches the output of `git ls-tree --name-only`. This includes
interpreting pathnames that begin with a double quote (") as C-style
quoted strings. This may become the default in the future; --no-cone
can be passed to request non-cone mode.
+
Use the `--[no-]sparse-index` option to toggle the use of the sparse
index format. This reduces the size of the index to be more closely
aligned with your sparse-checkout definition. This can have significant
performance advantages for commands such as `git status` or `git add`.
This feature is still experimental. Some commands might be slower with
a sparse index until they are properly integrated with the feature.
Use the `--[no-]sparse-index` option to use a sparse index (the
default is to not use it). A sparse index reduces the size of the
index to be more closely aligned with your sparse-checkout
definition. This can have significant performance advantages for
commands such as `git status` or `git add`. This feature is still
experimental. Some commands might be slower with a sparse index until
they are properly integrated with the feature.
+
**WARNING:** Using a sparse index requires modifying the index in a way
that is not completely understood by external tools. If you have trouble
@ -60,23 +68,6 @@ to rewrite your index to not be sparse. Older versions of Git will not
understand the sparse directory entries index extension and may fail to
interact with your repository until it is disabled.
'set'::
Write a set of patterns to the sparse-checkout file, as given as
a list of arguments following the 'set' subcommand. Update the
working directory to match the new patterns. Enable the
core.sparseCheckout config setting if it is not already enabled.
+
When the `--stdin` option is provided, the patterns are read from
standard in as a newline-delimited list instead of from the arguments.
+
When `core.sparseCheckoutCone` is enabled, the input list is considered a
list of directories instead of sparse-checkout patterns. The command writes
patterns to the sparse-checkout file to include all files contained in those
directories (recursively) as well as files that are siblings of ancestor
directories. The input format matches the output of `git ls-tree --name-only`.
This includes interpreting pathnames that begin with a double quote (") as
C-style quoted strings.
'add'::
Update the sparse-checkout file to include additional patterns.
By default, these patterns are read from the command-line arguments,
@ -93,12 +84,35 @@ C-style quoted strings.
cases, it can make sense to run `git sparse-checkout reapply` later
after cleaning up affected paths (e.g. resolving conflicts, undoing
or committing changes, etc.).
+
The `reapply` command can also take `--[no-]cone` and `--[no-]sparse-index`
flags, with the same meaning as the flags from the `set` command, in order
to change which sparsity mode you are using without needing to also respecify
all sparsity paths.
'disable'::
Disable the `core.sparseCheckout` config setting, and restore the
working directory to include all files. Leaves the sparse-checkout
file intact so a later 'git sparse-checkout init' command may
return the working directory to the same state.
working directory to include all files.
'init'::
Deprecated command that behaves like `set` with no specified paths.
May be removed in the future.
+
Historically, `set` did not handle all the necessary config settings,
which meant that both `init` and `set` had to be called. Invoking
both meant the `init` step would first remove nearly all tracked files
(and in cone mode, ignored files too), then the `set` step would add
many of the tracked files (but not ignored files) back. In addition
to the lost files, the performance and UI of this combination was
poor.
+
Also, historically, `init` would not actually initialize the
sparse-checkout file if it already existed. This meant it was
possible to return to a sparse-checkout without remembering which
paths to pass to a subsequent 'set' or 'add' command. However,
`--cone` and `--sparse-index` options would not be remembered across
the disable command, so the easy restore of calling a plain `init`
decreased in utility.
SPARSE CHECKOUT
---------------
@ -107,7 +121,7 @@ SPARSE CHECKOUT
It uses the skip-worktree bit (see linkgit:git-update-index[1]) to tell
Git whether a file in the working directory is worth looking at. If
the skip-worktree bit is set, then the file is ignored in the working
directory. Git will not populate the contents of those files, which
directory. Git will avoid populating the contents of those files, which
makes a sparse checkout helpful when working in a repository with many
files, but only a few are important to the current user.
@ -117,10 +131,8 @@ directory, it updates the skip-worktree bits in the index based
on this file. The files matching the patterns in the file will
appear in the working directory, and the rest will not.
To enable the sparse-checkout feature, run `git sparse-checkout init` to
initialize a simple sparse-checkout file and enable the `core.sparseCheckout`
config setting. Then, run `git sparse-checkout set` to modify the patterns in
the sparse-checkout file.
To enable the sparse-checkout feature, run `git sparse-checkout set` to
set the patterns you want to use.
To repopulate the working directory with all files, use the
`git sparse-checkout disable` command.

2
builtin/clone.c
View file

@ -633,7 +633,7 @@ static int git_sparse_checkout_init(const char *repo)
{
struct strvec argv = STRVEC_INIT;
int result = 0;
strvec_pushl(&argv, "-C", repo, "sparse-checkout", "init", NULL);
strvec_pushl(&argv, "-C", repo, "sparse-checkout", "set", NULL);
/*
* We must apply the setting in the current process

204
builtin/sparse-checkout.c
View file

@ -56,6 +56,9 @@ static int sparse_checkout_list(int argc, const char **argv)
char *sparse_filename;
int res;
if (!core_apply_sparse_checkout)
die(_("this worktree is not sparse"));
argc = parse_options(argc, argv, NULL,
builtin_sparse_checkout_list_options,
builtin_sparse_checkout_list_usage, 0);
@ -380,6 +383,41 @@ static int set_config(enum sparse_checkout_mode mode)
return 0;
}
static int update_modes(int *cone_mode, int *sparse_index)
{
int mode, record_mode;
/* Determine if we need to record the mode; ensure sparse checkout on */
record_mode = (*cone_mode != -1) || !core_apply_sparse_checkout;
/* If not specified, use previous definition of cone mode */
if (*cone_mode == -1 && core_apply_sparse_checkout)
*cone_mode = core_sparse_checkout_cone;
/* Set cone/non-cone mode appropriately */
core_apply_sparse_checkout = 1;
if (*cone_mode == 1) {
mode = MODE_CONE_PATTERNS;
core_sparse_checkout_cone = 1;
} else {
mode = MODE_ALL_PATTERNS;
}
if (record_mode && set_config(mode))
return 1;
/* Set sparse-index/non-sparse-index mode if specified */
if (*sparse_index >= 0) {
if (set_sparse_index_config(the_repository, *sparse_index) < 0)
die(_("failed to modify sparse-index config"));
/* force an index rewrite */
repo_read_index(the_repository);
the_repository->index->updated_workdir = 1;
}
return 0;
}
static char const * const builtin_sparse_checkout_init_usage[] = {
N_("git sparse-checkout init [--cone] [--[no-]sparse-index]"),
NULL
@ -396,7 +434,6 @@ static int sparse_checkout_init(int argc, const char **argv)
char *sparse_filename;
int res;
struct object_id oid;
int mode;
struct strbuf pattern = STRBUF_INIT;
static struct option builtin_sparse_checkout_init_options[] = {
@ -409,19 +446,14 @@ static int sparse_checkout_init(int argc, const char **argv)
repo_read_index(the_repository);
init_opts.cone_mode = -1;
init_opts.sparse_index = -1;
argc = parse_options(argc, argv, NULL,
builtin_sparse_checkout_init_options,
builtin_sparse_checkout_init_usage, 0);
if (init_opts.cone_mode) {
mode = MODE_CONE_PATTERNS;
core_sparse_checkout_cone = 1;
} else
mode = MODE_ALL_PATTERNS;
if (set_config(mode))
if (update_modes(&init_opts.cone_mode, &init_opts.sparse_index))
return 1;
memset(&pl, 0, sizeof(pl));
@ -429,17 +461,6 @@ static int sparse_checkout_init(int argc, const char **argv)
sparse_filename = get_sparse_checkout_filename();
res = add_patterns_from_file_to_list(sparse_filename, "", 0, &pl, NULL, 0);
if (init_opts.sparse_index >= 0) {
if (set_sparse_index_config(the_repository, init_opts.sparse_index) < 0)
die(_("failed to modify sparse-index config"));
/* force an index rewrite */
repo_read_index(the_repository);
the_repository->index->updated_workdir = 1;
}
core_apply_sparse_checkout = 1;
/* If we already have a sparse-checkout file, use it. */
if (res >= 0) {
free(sparse_filename);
@ -515,17 +536,9 @@ static void strbuf_to_cone_pattern(struct strbuf *line, struct pattern_list *pl)
insert_recursive_pattern(pl, line);
}
static char const * const builtin_sparse_checkout_set_usage[] = {
N_("git sparse-checkout (set|add) (--stdin | <patterns>)"),
NULL
};
static struct sparse_checkout_set_opts {
int use_stdin;
} set_opts;
static void add_patterns_from_input(struct pattern_list *pl,
int argc, const char **argv)
int argc, const char **argv,
int use_stdin)
{
int i;
if (core_sparse_checkout_cone) {
@ -535,7 +548,7 @@ static void add_patterns_from_input(struct pattern_list *pl,
hashmap_init(&pl->parent_hashmap, pl_hashmap_cmp, NULL, 0);
pl->use_cone_patterns = 1;
if (set_opts.use_stdin) {
if (use_stdin) {
struct strbuf unquoted = STRBUF_INIT;
while (!strbuf_getline(&line, stdin)) {
if (line.buf[0] == '"') {
@ -559,7 +572,7 @@ static void add_patterns_from_input(struct pattern_list *pl,
}
}
} else {
if (set_opts.use_stdin) {
if (use_stdin) {
struct strbuf line = STRBUF_INIT;
while (!strbuf_getline(&line, stdin)) {
@ -580,7 +593,8 @@ enum modify_type {
};
static void add_patterns_cone_mode(int argc, const char **argv,
struct pattern_list *pl)
struct pattern_list *pl,
int use_stdin)
{
struct strbuf buffer = STRBUF_INIT;
struct pattern_entry *pe;
@ -588,7 +602,7 @@ static void add_patterns_cone_mode(int argc, const char **argv,
struct pattern_list existing;
char *sparse_filename = get_sparse_checkout_filename();
add_patterns_from_input(pl, argc, argv);
add_patterns_from_input(pl, argc, argv, use_stdin);
memset(&existing, 0, sizeof(existing));
existing.use_cone_patterns = core_sparse_checkout_cone;
@ -614,17 +628,19 @@ static void add_patterns_cone_mode(int argc, const char **argv,
}
static void add_patterns_literal(int argc, const char **argv,
struct pattern_list *pl)
struct pattern_list *pl,
int use_stdin)
{
char *sparse_filename = get_sparse_checkout_filename();
if (add_patterns_from_file_to_list(sparse_filename, "", 0,
pl, NULL, 0))
die(_("unable to load existing sparse-checkout patterns"));
free(sparse_filename);
add_patterns_from_input(pl, argc, argv);
add_patterns_from_input(pl, argc, argv, use_stdin);
}
static int modify_pattern_list(int argc, const char **argv, enum modify_type m)
static int modify_pattern_list(int argc, const char **argv, int use_stdin,
enum modify_type m)
{
int result;
int changed_config = 0;
@ -633,13 +649,13 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m)
switch (m) {
case ADD:
if (core_sparse_checkout_cone)
add_patterns_cone_mode(argc, argv, pl);
add_patterns_cone_mode(argc, argv, pl, use_stdin);
else
add_patterns_literal(argc, argv, pl);
add_patterns_literal(argc, argv, pl, use_stdin);
break;
case REPLACE:
add_patterns_from_input(pl, argc, argv);
add_patterns_from_input(pl, argc, argv, use_stdin);
break;
}
@ -659,41 +675,124 @@ static int modify_pattern_list(int argc, const char **argv, enum modify_type m)
return result;
}
static int sparse_checkout_set(int argc, const char **argv, const char *prefix,
enum modify_type m)
static char const * const builtin_sparse_checkout_add_usage[] = {
N_("git sparse-checkout add (--stdin | <patterns>)"),
NULL
};
static struct sparse_checkout_add_opts {
int use_stdin;
} add_opts;
static int sparse_checkout_add(int argc, const char **argv, const char *prefix)
{
static struct option builtin_sparse_checkout_set_options[] = {
OPT_BOOL(0, "stdin", &set_opts.use_stdin,
static struct option builtin_sparse_checkout_add_options[] = {
OPT_BOOL(0, "stdin", &add_opts.use_stdin,
N_("read patterns from standard in")),
OPT_END(),
};
if (!core_apply_sparse_checkout)
die(_("no sparse-checkout to add to"));
repo_read_index(the_repository);
argc = parse_options(argc, argv, prefix,
builtin_sparse_checkout_add_options,
builtin_sparse_checkout_add_usage,
PARSE_OPT_KEEP_UNKNOWN);
return modify_pattern_list(argc, argv, add_opts.use_stdin, ADD);
}
static char const * const builtin_sparse_checkout_set_usage[] = {
N_("git sparse-checkout set [--[no-]cone] [--[no-]sparse-index] (--stdin | <patterns>)"),
NULL
};
static struct sparse_checkout_set_opts {
int cone_mode;
int sparse_index;
int use_stdin;
} set_opts;
static int sparse_checkout_set(int argc, const char **argv, const char *prefix)
{
int default_patterns_nr = 2;
const char *default_patterns[] = {"/*", "!/*/", NULL};
static struct option builtin_sparse_checkout_set_options[] = {
OPT_BOOL(0, "cone", &set_opts.cone_mode,
N_("initialize the sparse-checkout in cone mode")),
OPT_BOOL(0, "sparse-index", &set_opts.sparse_index,
N_("toggle the use of a sparse index")),
OPT_BOOL_F(0, "stdin", &set_opts.use_stdin,
N_("read patterns from standard in"),
PARSE_OPT_NONEG),
OPT_END(),
};
repo_read_index(the_repository);
set_opts.cone_mode = -1;
set_opts.sparse_index = -1;
argc = parse_options(argc, argv, prefix,
builtin_sparse_checkout_set_options,
builtin_sparse_checkout_set_usage,
PARSE_OPT_KEEP_UNKNOWN);
return modify_pattern_list(argc, argv, m);
if (update_modes(&set_opts.cone_mode, &set_opts.sparse_index))
return 1;
/*
* Cone mode automatically specifies the toplevel directory. For
* non-cone mode, if nothing is specified, manually select just the
* top-level directory (much as 'init' would do).
*/
if (!core_sparse_checkout_cone && argc == 0) {
argv = default_patterns;
argc = default_patterns_nr;
}
return modify_pattern_list(argc, argv, set_opts.use_stdin, REPLACE);
}
static char const * const builtin_sparse_checkout_reapply_usage[] = {
N_("git sparse-checkout reapply"),
N_("git sparse-checkout reapply [--[no-]cone] [--[no-]sparse-index]"),
NULL
};
static struct sparse_checkout_reapply_opts {
int cone_mode;
int sparse_index;
} reapply_opts;
static int sparse_checkout_reapply(int argc, const char **argv)
{
static struct option builtin_sparse_checkout_reapply_options[] = {
OPT_BOOL(0, "cone", &reapply_opts.cone_mode,
N_("initialize the sparse-checkout in cone mode")),
OPT_BOOL(0, "sparse-index", &reapply_opts.sparse_index,
N_("toggle the use of a sparse index")),
OPT_END(),
};
if (!core_apply_sparse_checkout)
die(_("must be in a sparse-checkout to reapply sparsity patterns"));
argc = parse_options(argc, argv, NULL,
builtin_sparse_checkout_reapply_options,
builtin_sparse_checkout_reapply_usage, 0);
repo_read_index(the_repository);
reapply_opts.cone_mode = -1;
reapply_opts.sparse_index = -1;
if (update_modes(&reapply_opts.cone_mode, &reapply_opts.sparse_index))
return 1;
return update_working_directory(NULL);
}
@ -710,6 +809,17 @@ static int sparse_checkout_disable(int argc, const char **argv)
struct pattern_list pl;
struct strbuf match_all = STRBUF_INIT;
/*
* We do not exit early if !core_apply_sparse_checkout; due to the
* ability for users to manually muck things up between
* direct editing of .git/info/sparse-checkout
* running read-tree -m u HEAD or update-index --skip-worktree
* direct toggling of config options
* users might end up with an index with SKIP_WORKTREE bit set on
* some files and not know how to undo it. So, here we just
* forcibly return to a dense checkout regardless of initial state.
*/
argc = parse_options(argc, argv, NULL,
builtin_sparse_checkout_disable_options,
builtin_sparse_checkout_disable_usage, 0);
@ -758,9 +868,9 @@ int cmd_sparse_checkout(int argc, const char **argv, const char *prefix)
if (!strcmp(argv[0], "init"))
return sparse_checkout_init(argc, argv);
if (!strcmp(argv[0], "set"))
return sparse_checkout_set(argc, argv, prefix, REPLACE);
return sparse_checkout_set(argc, argv, prefix);
if (!strcmp(argv[0], "add"))
return sparse_checkout_set(argc, argv, prefix, ADD);
return sparse_checkout_add(argc, argv, prefix);
if (!strcmp(argv[0], "reapply"))
return sparse_checkout_reapply(argc, argv);
if (!strcmp(argv[0], "disable"))

10
t/t1091-sparse-checkout-builtin.sh
View file

@ -41,7 +41,15 @@ test_expect_success 'setup' '
)
'
test_expect_success 'git sparse-checkout list (empty)' '
test_expect_success 'git sparse-checkout list (not sparse)' '
test_must_fail git -C repo sparse-checkout list >list 2>err &&
test_must_be_empty list &&
test_i18ngrep "this worktree is not sparse" err
'
test_expect_success 'git sparse-checkout list (not sparse)' '
git -C repo sparse-checkout set &&
rm repo/.git/info/sparse-checkout &&
git -C repo sparse-checkout list >list 2>err &&
test_must_be_empty list &&
test_i18ngrep "this worktree is not sparse (sparse-checkout file may not exist)" err