During git-aware path completion, e.g. 'git rm dir/fil<TAB>', both
'git ls-files' and 'git diff-index' list all paths in the given 'dir/'
matching certain criteria (cached, modified, untracked, etc.)
appropriate for the given git command, even paths whose names don't
begin with 'fil'. This comes with a considerable performance
penalty when the directory in question contains a lot of paths, but
the current word can be uniquely completed or when only a handful of
those paths match the current word.
Reduce the number of iterations in this codepath from the number of
paths to the number of matching paths by specifying an appropriate
globbing pattern to 'git ls-files' and 'git diff-index' to list only
paths that match the current word to be completed.
Note that both commands treat backslashes as escape characters in
their file arguments, e.g. to preserve the literal meaning of globbing
characters, so we have to double every backslash in the globbing
pattern. This is why one of the path completion tests specifically
checks the completion of a path containing a literal backslash
character (that test still fails, though, because both commands output
such paths enclosed in double quotes and the special characters
escaped; a later patch in this series will deal with those).
This speeds up path completion considerably when there are a lot of
non-matching paths to be filtered out. Uniquely completing a tracked
filename at the top of the worktree in linux.git (over 62k files),
i.e. what's doing all the hard work behind 'git rm Mak<TAB>' to
complete 'Makefile':
Before this patch, best of five, on Linux:
$ time cur=Mak __git_complete_index_file
real 0m2.159s
user 0m1.299s
sys 0m1.089s
After:
real 0m0.033s
user 0m0.023s
sys 0m0.015s
Difference: -98.5%
Speedup: 65.4x
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Our git-aware path completion doesn't work when it has to complete a
word already containing quoted and/or backslash-escaped characters on
the command line. The root cause of the issue is that completion
functions see all words on the command line verbatim, i.e. including
all backslash, single and double quote characters that the shell would
eventually remove when executing the finished command. These
quoting/escaping characters cause different issues depending on which
path component of the word to be completed contains them:
- The quoting/escaping is in the prefix path component(s).
Let's suppose we have a directory called 'New Dir', containing two
untracked files 'file.c' and 'file.o', and we have a gitignore
rule ignoring object files. In this case all of these:
git add New\ Dir/<TAB>
git add "New Dir/<TAB>
git add 'New Dir/<TAB>
should uniquely complete 'file.c' right away, but Bash offers both
'file.c' and 'file.o' instead. The reason for this behavior is
that our completion script uses the prefix directory name like
'git -C "New\ Dir/" ls-files ...", i.e. with the backslash inside
double quotes. Git then tries to enter a directory called
'New\ Dir', which (most likely) fails because such a directory
doesn't exists. As a result our completion script doesn't list
any files, leaves the COMPREPLY array empty, which in turn causes
Bash to fall back to its simple filename completion and lists all
files in that directory, i.e. both 'file.c' and 'file.o'.
- The quoting/escaping is in the path component to be completed.
Let's suppose we have two untracked files 'New File.c' and
'New File.o', and we have a gitignore rule ignoring object files.
In this case all of these:
git add New\ Fi<TAB>
git add "New Fi<TAB>
git add 'New Fi<TAB>
should uniquely complete 'New File.c' right away, but Bash offers
both 'New File.c' and 'New File.o' instead. The reason for this
behavior is that our completion script uses this 'New\ Fi' or
'"New Fi' etc. word to filter matching paths, and of course none
of the potential filenames will match because of the included
backslash or double quote. The end result is the same as above:
the completion script doesn't list any files, Bash falls back to
its filename completion, which then lists the matching object file
as well.
Add the new helper function __git_dequote() [1], which removes (most
of[2]) the quoting and escaping from the word it gets as argument. To
minimize the overhead of calling this function, store its result in
the variable $dequoted_word, supposed to be declared local in the
caller; simply printing the result would require a command
substitution imposing the overhead of fork()ing a subshell. Use this
function in __git_complete_index_file() to dequote the current word,
i.e. the path, to be completed, to avoid the above described
quoting-related issues, thereby fixing two of the failing quoted path
completion tests.
[1] The bash-completion project already has a dequote() function,
which I hoped I could borrow to deal with this, but unfortunately
it doesn't work quite well for this purpose (perhaps that's why
even the bash-completion project only rarely uses it). The main
issue is that their dequote() is implemented as:
eval printf %s "$1" 2> /dev/null
where $1 would contain the word to be completed. While it's a
short and sweet one-liner, the use of 'eval' requires that $1 is a
syntactically valid string, which is not the case when quoting the
path like 'git add "New Dir/<TAB>'. This causes 'eval' to fail,
because it can't find the matching closing double quote, and the
function returns nothing. The result is totally broken behavior,
as if the current word were empty, and the completion script would
then list all files from the current directory. This is why one
of the quoted path completion tests specifically checks the
completion of a path with an opening but without a corresponding
closing double quote character. Furthermore, the 'eval' performs
all kinds of expansions, which may or may not be desired; I think
it's the latter. Finally, using this function would require a
command substitution.
[2] Bash understands the $'string' quoting as well, which "expands to
'string', with backslash-escaped characters replaced as specified
by the ANSI C standard" (quoted from Bash manpage). Since shell
metacharacters, field separators, globbing, etc. can all be easily
entered using standard shell escaping or quoting, this type of
quoting comes in handly when dealing with control characters that
are otherwise difficult both to "type" and to see on the command
line. Because of this difficulty I would assume that people do
avoid pathnames with such control characters anyway, so I didn't
bother implementing it. This function is already way too long as
it is.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Unless the user has 'core.quotePath=false' somewhere in the
configuration, both 'git ls-files' and 'git diff-index' will by
default quote any pathnames that contain bytes with values higher than
0x80, and escape those bytes as '\nnn' octal values. This prevents
completing paths when the current path component to be completed
contains any non-ASCII, most notably UTF-8, characters, because none
of the listed quoted paths will match the current word on the command
line.
Set 'core.quotePath=false' for those 'git ls-files' and 'git
diff-index' invocations, so they won't consider bytes higher than 0x80
as "unusual", and won't quote pathnames containing such characters.
Note that pathnames containing backslash, double quote, or control
characters will still be quoted; a later patch in this series will
deal with those.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Completion functions see all words on the command line verbatim,
including any backslash-escapes, single and double quotes that might
be there. Furthermore, git commands quote pathnames if they contain
certain special characters. All these create various issues when
doing git-aware path completion.
Add a couple of failing tests to demonstrate these issues.
Later patches in this series will discuss these issues in detail as
they fix them.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* 'svn/authors-prog-2' of git://bogomips.org/git-svn:
git-svn: allow empty email-address using authors-prog and authors-file
git-svn: search --authors-prog in PATH too
This reverts commit e4bb62fa1e, reversing
changes made to 468165c1d8.
The topic appears to inflict severe regression in renaming merges,
even though the promise of it was that it would improve them.
We do not yet know which exact change in the topic was wrong, but in
the meantime, let's play it safe and revert it out of 'master'
before real Git-using projects are harmed.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git submodule status" misbehaved on a submodule that has been
removed from the working tree.
* rs/status-with-removed-submodule:
submodule: check for NULL return of get_submodule_ref_store()
Small test-helper programs have been consolidated into a single
binary.
* nd/combined-test-helper: (36 commits)
t/helper: merge test-write-cache into test-tool
t/helper: merge test-wildmatch into test-tool
t/helper: merge test-urlmatch-normalization into test-tool
t/helper: merge test-subprocess into test-tool
t/helper: merge test-submodule-config into test-tool
t/helper: merge test-string-list into test-tool
t/helper: merge test-strcmp-offset into test-tool
t/helper: merge test-sigchain into test-tool
t/helper: merge test-sha1-array into test-tool
t/helper: merge test-scrap-cache-tree into test-tool
t/helper: merge test-run-command into test-tool
t/helper: merge test-revision-walking into test-tool
t/helper: merge test-regex into test-tool
t/helper: merge test-ref-store into test-tool
t/helper: merge test-read-cache into test-tool
t/helper: merge test-prio-queue into test-tool
t/helper: merge test-path-utils into test-tool
t/helper: merge test-online-cpus into test-tool
t/helper: merge test-mktemp into test-tool
t/helper: merge (unused) test-mergesort into test-tool
...
Refactoring the internal global data structure to make it possible
to open multiple repositories, work with and then close them.
Rerolled by Duy on top of a separate preliminary clean-up topic.
The resulting structure of the topics looked very sensible.
* sb/object-store: (27 commits)
sha1_file: allow sha1_loose_object_info to handle arbitrary repositories
sha1_file: allow map_sha1_file to handle arbitrary repositories
sha1_file: allow map_sha1_file_1 to handle arbitrary repositories
sha1_file: allow open_sha1_file to handle arbitrary repositories
sha1_file: allow stat_sha1_file to handle arbitrary repositories
sha1_file: allow sha1_file_name to handle arbitrary repositories
sha1_file: add repository argument to sha1_loose_object_info
sha1_file: add repository argument to map_sha1_file
sha1_file: add repository argument to map_sha1_file_1
sha1_file: add repository argument to open_sha1_file
sha1_file: add repository argument to stat_sha1_file
sha1_file: add repository argument to sha1_file_name
sha1_file: allow prepare_alt_odb to handle arbitrary repositories
sha1_file: allow link_alt_odb_entries to handle arbitrary repositories
sha1_file: add repository argument to prepare_alt_odb
sha1_file: add repository argument to link_alt_odb_entries
sha1_file: add repository argument to read_info_alternates
sha1_file: add repository argument to link_alt_odb_entry
sha1_file: add raw_object_store argument to alt_odb_usable
pack: move approximate object count to object store
...
when refs that do not point at committish are given, "git
filter-branch" gave a misleading error messages. This has been
corrected.
* yk/filter-branch-non-committish-refs:
filter-branch: fix errors caused by refs that point at non-committish
The mechanism to use parse-options API to automate the command line
completion continues to get extended and polished.
* nd/parseopt-completion-more:
completion: use __gitcomp_builtin in _git_cherry
completion: use __gitcomp_builtin in _git_ls_tree
completion: delete option-only completion commands
completion: add --option completion for most builtin commands
completion: factor out _git_xxx calling code
completion: mention the oldest version we need to support
git.c: add hidden option --list-parseopt-builtins
git.c: move cmd_struct declaration up
"git tag --contains no-such-commit" gave a full list of options
after giving an error message.
* ps/contains-id-error-message:
parse-options: do not show usage upon invalid option value
"git stash push -u -- <pathspec>" gave an unnecessary and confusing
error message when there was no tracked files that match the
<pathspec>, which has been fixed.
* tg/stash-untracked-with-pathspec-fix:
stash: drop superfluos pathspec parameter
stash push -u: don't create empty stash
stash push: avoid printing errors
stash: fix nonsense pipeline
The way "git worktree prune" worked internally has been simplified,
by assuming how "git worktree move" moves an existing worktree to a
different place.
* nd/worktree-prune:
worktree prune: improve prune logic when worktree is moved
worktree: delete dead code
gc.txt: more details about what gc does
"git shortlog cruft" aborted with a BUG message when run outside a
Git repository. The command has been taught to complain about
extra and unwanted arguments on its command line instead in such a
case.
* ma/shortlog-revparse:
shortlog: disallow left-over arguments outside repo
shortlog: add usage-string for stdin-reading
git-shortlog.txt: reorder usages
Rename detection logic in "diff" family that is used in "merge" has
learned to guess when all of x/a, x/b and x/c have moved to z/a,
z/b and z/c, it is likely that x/d added in the meantime would also
want to move to z/d by taking the hint that the entire directory
'x' moved to 'z'. A bug causing dirty files involved in a rename
to be overwritten during merge has also been fixed as part of this
work.
* en/rename-directory-detection: (29 commits)
merge-recursive: ensure we write updates for directory-renamed file
merge-recursive: avoid spurious rename/rename conflict from dir renames
directory rename detection: new testcases showcasing a pair of bugs
merge-recursive: fix remaining directory rename + dirty overwrite cases
merge-recursive: fix overwriting dirty files involved in renames
merge-recursive: avoid clobbering untracked files with directory renames
merge-recursive: apply necessary modifications for directory renames
merge-recursive: when comparing files, don't include trees
merge-recursive: check for file level conflicts then get new name
merge-recursive: add computation of collisions due to dir rename & merging
merge-recursive: check for directory level conflicts
merge-recursive: add get_directory_renames()
merge-recursive: make a helper function for cleanup for handle_renames
merge-recursive: split out code for determining diff_filepairs
merge-recursive: make !o->detect_rename codepath more obvious
merge-recursive: fix leaks of allocated renames and diff_filepairs
merge-recursive: introduce new functions to handle rename logic
merge-recursive: move the get_renames() function
directory rename detection: tests for handling overwriting dirty files
directory rename detection: tests for handling overwriting untracked files
...
The email address in --authors-file and --authors-prog can be empty but
git-svn translated it into a fictional email address in the form
jondoe <jondoe@6aafaa21e0fb4338a68ab372a049893d>
containing the SVN repository UUID. Now git-svn behaves like git-commit:
If the email is *explicitly* set to the empty string using '<>', the
commit does not contain an email address, only the name:
jondoe <>
Allowing to remove the email address *intentionally* prevents automatic
systems from sending emails to those fictional addresses and avoids
cluttering the log output with unnecessary stuff.
Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Eric Wong <e@80x24.org>
If we can't find a ref store for a submodule then assume the latter
is not initialized (or was removed). Print a status line accordingly
instead of causing a segmentation fault by passing NULL as the first
parameter of refs_head_ref().
Reported-by: Jeremy Feusi <jeremy@feusi.co>
Reviewed-by: Stefan Beller <sbeller@google.com>
Initial-Test-By: Stefan Beller <sbeller@google.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Hotfix for recently graduated topic that give help to completion
scripts from the Git subcommands that are being completed
* nd/parseopt-completion:
t9902: disable test on the list of merge-strategies under GETTEXT_POISON
completion: clear cached --options when sourcing the completion script
Avoid using pipes downstream of Git commands since the exit codes
of commands upstream of pipes get swallowed, thus potentially
hiding failure of those commands. Instead, capture Git command
output to a file and apply the downstream command(s) to that file.
Signed-off-by: Pratik Karki <predatoramigo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 11395a3b4b (test_must_be_empty: make sure the file exists, not
just empty, 2018-02-27) basically duplicated the 'test_path_is_file'
helper function in 'test_must_be_empty'.
Just call 'test_path_is_file' to avoid this code duplication.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
One of the most interesting thing one can be interested in when
looking at performance test results is possible performance
regressions.
This new option makes it easy to spot such possible regressions.
This new option is named '--sort-by=regression' to make it
possible and easy to add other ways to sort the results, like for
example '--sort-by=utime'.
If we would like to sort according to how much the stime regressed
we could also add a new option called '--sort-by=regression:stime'.
Then '--sort-by=regression' could become a synonym for
'--sort-by=regression:rtime'.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This new helper function will be reused in a subsequent
commit.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
