Commit graph

15949 commits

Author SHA1 Message Date
Taylor Blau
2fa05f31bd t/helper/test-read-graph.c: support commit-graph chains
In 61df89c8e5 (commit-graph: don't early exit(1) on e.g. "git status",
2019-03-25), the former 'load_commit_graph_one' was refactored into
'open_commit_graph' and 'load_commit_graph_one_fd_st' as a means of
avoiding an early-exit from non-library code.

However, 'load_commit_graph_one' does not support commit-graph chains,
and hence the 'read-graph' test tool does not work with them.

Replace 'load_commit_graph_one' with 'read_commit_graph_one' in order to
support commit-graph chains. In the spirit of 61df89c8e5,
'read_commit_graph_one' does not ever 'die()', making it a suitable
replacement here.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-15 09:20:24 -07:00
Taylor Blau
a7df60cac8 commit-graph.h: use odb in 'load_commit_graph_one_fd_st'
Apply a similar treatment as in the previous patch to pass a 'struct
object_directory *' through the 'load_commit_graph_one_fd_st'
initializer, too.

This prevents a potential bug where a pointer comparison is made to a
NULL 'g->odb', which would cause the commit-graph machinery to think
that a pair of commit-graphs belonged to different alternates when in
fact they do not (i.e., in the case of no '--object-dir').

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-04 11:36:51 -08:00
Taylor Blau
ad2dd5bb63 commit-graph.c: remove path normalization, comparison
As of the previous patch, all calls to 'commit-graph.c' functions which
perform path normalization (for e.g., 'get_commit_graph_filename()') are
of the form 'ctx->odb->path', which is always in normalized form.

Now that there are no callers passing non-normalized paths to these
functions, ensure that future callers are bound by the same restrictions
by making these functions take a 'struct object_directory *' instead of
a 'const char *'. To match, replace all calls with arguments of the form
'ctx->odb->path' with 'ctx->odb' To recover the path, functions that
perform path manipulation simply use 'odb->path'.

Further, avoid string comparisons with arguments of the form
'odb->path', and instead prefer raw pointer comparisons, which
accomplish the same effect, but are far less brittle.

This has a pleasant side-effect of making these functions much more
robust to paths that cannot be normalized by 'normalize_path_copy()',
i.e., because they are outside of the current working directory.

For example, prior to this patch, Valgrind reports that the following
uninitialized memory read [1]:

  $ ( cd t && GIT_DIR=../.git valgrind git rev-parse HEAD^ )

because 'normalize_path_copy()' can't normalize '../.git' (since it's
relative to but above of the current working directory) [2].

By using a 'struct object_directory *' directly,
'get_commit_graph_filename()' does not need to normalize, because all
paths are relative to the current working directory since they are
always read from the '->path' of an object directory.

[1]: https://lore.kernel.org/git/20191027042116.GA5801@sigill.intra.peff.net.
[2]: The bug here is that 'get_commit_graph_filename()' returns the
     result of 'normalize_path_copy()' without checking the return
     value.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-02-04 11:36:51 -08:00
Taylor Blau
1793280e91 t5318: don't pass non-object directory to '--object-dir'
In f237c8b6fe (commit-graph: implement git-commit-graph write,
2018-04-02) the test t5318.3 was introduced to ensure that calling 'git
commit-graph write' in a repository with no packfiles does not write any
commit-graph file(s).

To exercise more paths in 'builtin/commit-graph.c', this test passes
'--object-dir' to 'git commit-graph write', but the given argument
refers to the working copy, not the object directory.

Since the commit-graph sub-commands currently swallow these errors, this
does not result in a test failure. But, it is only lucky that the test
ends with no commit-graphs, since there were none to begin with.

In preparation for a future commit where an '--object-dir' argument that
does not match a known object directory will print out a failure, let's
fix the test to still use '--object-dir', but pass the correct location
to the object store instead of '.'.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-31 12:47:40 -08:00
Junio C Hamano
4d924528d8 Revert "Merge branch 'ra/rebase-i-more-options'"
This reverts commit 5d9324e0f4, reversing
changes made to c58ae96fc4.

The topic turns out to be too buggy for real use.

cf. <f2fe7437-8a48-3315-4d3f-8d51fe4bb8f1@gmail.com>
2020-01-12 13:25:18 -08:00
Junio C Hamano
1cf4836865 Merge branch 'js/mingw-loosen-overstrict-tree-entry-checks'
Further tweak to a "no backslash in indexed paths" for Windows port
we applied earlier.

* js/mingw-loosen-overstrict-tree-entry-checks:
  mingw: safeguard better against backslashes in file names
2020-01-10 14:45:27 -08:00
Johannes Schindelin via GitGitGadget
49e268e23e mingw: safeguard better against backslashes in file names
In 224c7d70fa (mingw: only test index entries for backslashes, not tree
entries, 2019-12-31), we relaxed the check for backslashes in tree
entries to check only index entries.

However, the code change was incorrect: it was added to
`add_index_entry_with_check()`, not to `add_index_entry()`, so under
certain circumstances it was possible to side-step the protection.

Besides, the description of that commit purported that all index entries
would be checked when in fact they were only checked when being added to
the index (there are code paths that do not do that, constructing
"transient" index entries).

In any case, it was pointed out in one insightful review at
https://github.com/git-for-windows/git/pull/2437#issuecomment-566771835
that it would be a much better idea to teach `verify_path()` to perform
the check for a backslash. This is safer, even if it comes with two
notable drawbacks:

- `verify_path()` cannot say _what_ is wrong with the path, therefore
  the user will no longer be told that there was a backslash in the
  path, only that the path was invalid.

- The `git apply` command also calls the `verify_path()` function, and
  might have been able to handle Windows-style paths (i.e. with
  backslashes instead of forward slashes). This will no longer be
  possible unless the user (temporarily) sets `core.protectNTFS=false`.

Note that `git add <windows-path>` will _still_ work because
`normalize_path_copy_len()` will convert the backslashes to forward
slashes before hitting the code path that creates an index entry.

The clear advantage is that `verify_path()`'s purpose is to check the
validity of the file name, therefore we naturally tap into all the code
paths that need safeguarding, also implicitly into future code paths.

The benefits of that approach outweigh the downsides, so let's move the
check from `add_index_entry_with_check()` to `verify_path()`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-10 12:29:07 -08:00
Derrick Stolee
a1087c9367 graph: fix lack of color in horizontal lines
In some cases, horizontal lines in rendered graphs can lose their
coloring. This is due to a use of graph_line_addch() instead of
graph_line_write_column(). Using a ternary operator to pick the
character is nice for compact code, but we actually need a column to
provide the color.

Add a test to t4215-log-skewed-merges.sh to prevent regression.

Reported-by: Jeff King <peff@peff.net>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-08 09:37:18 -08:00
Derrick Stolee
0d251c3291 graph: drop assert() for merge with two collapsing parents
When "git log --graph" shows a merge commit that has two collapsing
lines, like:

    | | | | *
    | |_|_|/|
    |/| | |/
    | | |/|
    | |/| |
    | * | |
    * | | |

we trigger an assert():

        graph.c:1228: graph_output_collapsing_line: Assertion
                      `graph->mapping[i - 3] == target' failed.

The assert was introduced by eaf158f8 ("graph API: Use horizontal
lines for more compact graphs", 2009-04-21), which is quite old.
This assert is trying to say that when we complete a horizontal
line with a single slash, it is because we have reached our target.

It is actually the _second_ collapsing line that hits this assert.
The reason we are in this code path is because we are collapsing
the first line, and in that case we are hitting our target now
that the horizontal line is complete. However, the second line
cannot be a horizontal line, so it will collapse without horizontal
lines. In this case, it is inappropriate to assert that we have
reached our target, as we need to continue for another column
before reaching the target. Dropping the assert is safe here.

The new behavior in 0f0f389f12 (graph: tidy up display of
left-skewed merges, 2019-10-15) caused the behavior change that
made this assertion failure possible. In addition to making the
assert possible, it also changed how multiple edges collapse.

In a larger example, the current code will output a collapse
as follows:

	| | | | | | *
	| |_|_|_|_|/|\
	|/| | | | |/ /
	| | | | |/| /
	| | | |/| |/
	| | |/| |/|
	| |/| |/| |
	| | |/| | |
	| | * | | |

However, the intended collapse should allow multiple horizontal lines
as follows:

	| | | | | | *
	| |_|_|_|_|/|\
	|/| | | | |/ /
	| | |_|_|/| /
	| |/| | | |/
	| | | |_|/|
	| | |/| | |
	| | * | | |

This behavior is not corrected by this change, but is noted for a later
update.

Helped-by: Jeff King <peff@peff.net>
Reported-by: Bradley Smith <brad@brad-smith.co.uk>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-08 09:35:07 -08:00
Junio C Hamano
c20d4fd44a Merge branch 'ds/sparse-list-in-cone-mode'
"git sparse-checkout list" subcommand learned to give its output in
a more concise form when the "cone" mode is in effect.

* ds/sparse-list-in-cone-mode:
  sparse-checkout: document interactions with submodules
  sparse-checkout: list directories in cone mode
2020-01-06 14:17:51 -08:00
Junio C Hamano
a578ef9e63 Merge branch 'js/mingw-loosen-overstrict-tree-entry-checks'
An earlier update to Git for Windows declared that a tree object is
invalid if it has a path component with backslash in it, which was
overly strict, which has been corrected.  The only protection the
Windows users need is to prevent such path (or any path that their
filesystem cannot check out) from entering the index.

* js/mingw-loosen-overstrict-tree-entry-checks:
  mingw: only test index entries for backslashes, not tree entries
2020-01-06 14:17:50 -08:00
Johannes Schindelin
224c7d70fa mingw: only test index entries for backslashes, not tree entries
During a clone of a repository that contained a file with a backslash in
its name in the past, as of v2.24.1(2), Git for Windows prints errors
like this:

	error: filename in tree entry contains backslash: '\'

The idea is to prevent Git from even trying to write files with
backslashes in their file names: while these characters are valid in
file names on other platforms, on Windows it is interpreted as directory
separator (which would obviously lead to ambiguities, e.g. when there is
a file `a\b` and there is also a file `a/b`).

Arguably, this is the wrong layer for that error: As long as the user
never checks out the files whose names contain backslashes, there should
not be any problem in the first place.

So let's loosen the requirements: we now leave tree entries with
backslashes in their file names alone, but we do require any entries
that are added to the Git index to contain no backslashes on Windows.

Note: just as before, the check is guarded by `core.protectNTFS` (to
allow overriding the check by toggling that config setting), and it
is _only_ performed on Windows, as the backslash is not a directory
separator elsewhere, even when writing to NTFS-formatted volumes.

An alternative approach would be to try to prevent creating files with
backslashes in their file names. However, that comes with its own set of
problems. For example, `git config -f C:\ProgramData\Git\config ...` is
a very valid way to specify a custom config location, and we obviously
do _not_ want to prevent that. Therefore, the approach chosen in this
patch would appear to be better.

This addresses https://github.com/git-for-windows/git/issues/2435

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-02 12:56:08 -08:00
Junio C Hamano
a82027e9e6 Merge branch 'js/use-test-tool-on-path'
Test fix.

* js/use-test-tool-on-path:
  t3008: find test-tool through path lookup
2020-01-02 12:38:30 -08:00
Junio C Hamano
13432fc6dd Merge branch 'js/mingw-reserved-filenames'
Forbid pathnames that the platform's filesystem cannot represent on
MinGW.

* js/mingw-reserved-filenames:
  mingw: refuse paths containing reserved names
  mingw: short-circuit the conversion of `/dev/null` to UTF-16
2020-01-02 12:38:30 -08:00
Derrick Stolee
4fd683b6a3 sparse-checkout: document interactions with submodules
Using 'git submodule (init|deinit)' a user can select a subset of
submodules to populate. This behaves very similar to the sparse-checkout
feature, but those directories contain their own .git directory
including an object database and ref space. To have the sparse-checkout
file also determine if those files should exist would easily cause
problems. Therefore, keeping these features independent in this way
is the best way forward.

Also create a test that demonstrates this behavior to make sure
it doesn't change as the sparse-checkout feature evolves.

Reported-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-30 09:07:20 -08:00
Derrick Stolee
de11951b03 sparse-checkout: list directories in cone mode
When core.sparseCheckoutCone is enabled, the 'git sparse-checkout set'
command takes a list of directories as input, then creates an ordered
list of sparse-checkout patterns such that those directories are
recursively included and all sibling entries along the parent directories
are also included. Listing the patterns is less user-friendly than the
directories themselves.

In cone mode, and as long as the patterns match the expected cone-mode
pattern types, change the output of 'git sparse-checkout list' to only
show the directories that created the patterns.

With this change, the following piped commands would not change the
working directory:

	git sparse-checkout list | git sparse-checkout set --stdin

The only time this would not work is if core.sparseCheckoutCone is
true, but the sparse-checkout file contains patterns that do not
match the expected pattern types for cone mode.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-30 09:07:18 -08:00
Johannes Sixt
20a67e8ce9 t3008: find test-tool through path lookup
Do not use $GIT_BUILD_DIR without quotes; it may contain spaces and be
split into fields. But it is not necessary to access test-tool with an
absolute path in the first place as it can be found via path lookup.
Remove the explicit path.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-27 14:15:01 -08:00
Junio C Hamano
d2189a721c Merge branch 'en/fill-directory-fixes'
Assorted fixes to the directory traversal API.

* en/fill-directory-fixes:
  dir.c: use st_add3() for allocation size
  dir: consolidate similar code in treat_directory()
  dir: synchronize treat_leading_path() and read_directory_recursive()
  dir: fix checks on common prefix directory
  dir: break part of read_directory_recursive() out for reuse
  dir: exit before wildcard fall-through if there is no wildcard
  dir: remove stray quote character in comment
  Revert "dir.c: make 'git-status --ignored' work within leading directories"
  t3011: demonstrate directory traversal failures
2019-12-25 11:22:02 -08:00
Junio C Hamano
8be0a428d6 Merge branch 'rs/test-cleanup'
Test cleanup.

* rs/test-cleanup:
  t6030: don't create unused file
  t5580: don't create unused file
  t3501: don't create unused file
  t7004: don't create unused file
  t4256: don't create unused file
2019-12-25 11:22:01 -08:00
Junio C Hamano
c0c6a74594 Merge branch 'rs/xdiff-ignore-ws-w-func-context'
Extend test coverage for a recent fix.

* rs/xdiff-ignore-ws-w-func-context:
  t4015: improve coverage of function context test
2019-12-25 11:22:01 -08:00
Junio C Hamano
45b96a6fa1 Merge branch 'js/add-p-in-c'
The effort to move "git-add--interactive" to C continues.

* js/add-p-in-c:
  built-in add -p: show helpful hint when nothing can be staged
  built-in add -p: only show the applicable parts of the help text
  built-in add -p: implement the 'q' ("quit") command
  built-in add -p: implement the '/' ("search regex") command
  built-in add -p: implement the 'g' ("goto") command
  built-in add -p: implement hunk editing
  strbuf: add a helper function to call the editor "on an strbuf"
  built-in add -p: coalesce hunks after splitting them
  built-in add -p: implement the hunk splitting feature
  built-in add -p: show different prompts for mode changes and deletions
  built-in app -p: allow selecting a mode change as a "hunk"
  built-in add -p: handle deleted empty files
  built-in add -p: support multi-file diffs
  built-in add -p: offer a helpful error message when hunk navigation failed
  built-in add -p: color the prompt and the help text
  built-in add -p: adjust hunk headers as needed
  built-in add -p: show colored hunks by default
  built-in add -i: wire up the new C code for the `patch` command
  built-in add -i: start implementing the `patch` functionality in C
2019-12-25 11:22:01 -08:00
Junio C Hamano
bd72a08d6c Merge branch 'ds/sparse-cone'
Management of sparsely checked-out working tree has gained a
dedicated "sparse-checkout" command.

* ds/sparse-cone: (21 commits)
  sparse-checkout: improve OS ls compatibility
  sparse-checkout: respect core.ignoreCase in cone mode
  sparse-checkout: check for dirty status
  sparse-checkout: update working directory in-process for 'init'
  sparse-checkout: cone mode should not interact with .gitignore
  sparse-checkout: write using lockfile
  sparse-checkout: use in-process update for disable subcommand
  sparse-checkout: update working directory in-process
  sparse-checkout: sanitize for nested folders
  unpack-trees: add progress to clear_ce_flags()
  unpack-trees: hash less in cone mode
  sparse-checkout: init and set in cone mode
  sparse-checkout: use hashmaps for cone patterns
  sparse-checkout: add 'cone' mode
  trace2: add region in clear_ce_flags
  sparse-checkout: create 'disable' subcommand
  sparse-checkout: add '--stdin' option to set subcommand
  sparse-checkout: 'set' subcommand
  clone: add --sparse mode
  sparse-checkout: create 'init' subcommand
  ...
2019-12-25 11:21:58 -08:00
Junio C Hamano
f3c520e17f Merge branch 'sg/name-rev-wo-recursion'
Redo "git name-rev" to avoid recursive calls.

* sg/name-rev-wo-recursion:
  name-rev: cleanup name_ref()
  name-rev: eliminate recursion in name_rev()
  name-rev: use 'name->tip_name' instead of 'tip_name'
  name-rev: drop name_rev()'s 'generation' and 'distance' parameters
  name-rev: restructure creating/updating 'struct rev_name' instances
  name-rev: restructure parsing commits and applying date cutoff
  name-rev: pull out deref handling from the recursion
  name-rev: extract creating/updating a 'struct name_rev' into a helper
  t6120: add a test to cover inner conditions in 'git name-rev's name_rev()
  name-rev: use sizeof(*ptr) instead of sizeof(type) in allocation
  name-rev: avoid unnecessary cast in name_ref()
  name-rev: use strbuf_strip_suffix() in get_rev_name()
  t6120-describe: modernize the 'check_describe' helper
  t6120-describe: correct test repo history graph in comment
2019-12-25 11:21:58 -08:00
Junio C Hamano
6514ad40a1 Merge branch 'ra/t5150-depends-on-perl'
Some Porcelain commands are written in Perl, and tests on them are
expected not to work when the platform lacks a working perl.

* ra/t5150-depends-on-perl:
  t5150: skip request-pull test if Perl is disabled
2019-12-25 11:21:58 -08:00
Junio C Hamano
17066bea38 Merge branch 'dl/format-patch-notes-config-fixup'
"git format-patch" can take a set of configured format.notes values
to specify which notes refs to use in the log message part of the
output.  The behaviour of this was not consistent with multiple
--notes command line options, which has been corrected.

* dl/format-patch-notes-config-fixup:
  notes.h: fix typos in comment
  notes: break set_display_notes() into smaller functions
  config/format.txt: clarify behavior of multiple format.notes
  format-patch: move git_config() before repo_init_revisions()
  format-patch: use --notes behavior for format.notes
  notes: extract logic into set_display_notes()
  notes: create init_display_notes() helper
  notes: rename to load_display_notes()
2019-12-25 11:21:58 -08:00
Junio C Hamano
135365dd99 Merge branch 'am/pathspec-f-f-checkout'
A few more commands learned the "--pathspec-from-file" command line
option.

* am/pathspec-f-f-checkout:
  checkout, restore: support the --pathspec-from-file option
  doc: restore: synchronize <pathspec> description
  doc: checkout: synchronize <pathspec> description
  doc: checkout: fix broken text reference
  doc: checkout: remove duplicate synopsis
  add: support the --pathspec-from-file option
  cmd_add: prepare for next patch
2019-12-25 11:21:57 -08:00
Junio C Hamano
ff0cb70d45 Merge branch 'am/pathspec-from-file'
An earlier series to teach "--pathspec-from-file" to "git commit"
forgot to make the option incompatible with "--all", which has been
corrected.

* am/pathspec-from-file:
  commit: forbid --pathspec-from-file --all
2019-12-25 11:21:57 -08:00
Johannes Schindelin
4dc42c6c18 mingw: refuse paths containing reserved names
There are a couple of reserved names that cannot be file names on
Windows, such as `AUX`, `NUL`, etc. For an almost complete list, see
https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file

If one would try to create a directory named `NUL`, it would actually
"succeed", i.e. the call would return success, but nothing would be
created.

Worse, even adding a file extension to the reserved name does not make
it a valid file name. To understand the rationale behind that behavior,
see https://devblogs.microsoft.com/oldnewthing/20031022-00/?p=42073

Let's just disallow them all.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-21 16:09:07 -08:00
Ed Maste
761e3d26bb sparse-checkout: improve OS ls compatibility
On FreeBSD, when executed by root ls enables the '-A' option:

  -A  Include directory entries whose names begin with a dot (`.')
      except for . and ...  Automatically set for the super-user unless
      -I is specified.

As a result the .git directory appeared in the output when run as root.
Simulate no-dotfile ls behaviour using a shell glob.

Helped-by: Eric Wong <e@80x24.org>
Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Ed Maste <emaste@FreeBSD.org>
Acked-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-20 12:47:08 -08:00
Elijah Newren
777b420347 dir: synchronize treat_leading_path() and read_directory_recursive()
Our optimization to avoid calling into read_directory_recursive() when
all pathspecs have a common leading directory mean that we need to match
the logic that read_directory_recursive() would use if we had just
called it from the root.  Since it does more than call treat_path() we
need to copy that same logic.

Alternatively, we could try to change treat_path to return path_recurse
for an untracked directory under the given special circumstances that
this logic checks for, but a simple switch results in many test failures
such as 'git clean -d' not wiping out untracked but empty directories.
To work around that, we'd need the caller of treat_path to check for
path_recurse and sometimes special case it into path_untracked.  In
other words, we'd still have extra logic in both places.

Needing to duplicate logic like this means it is guaranteed someone will
eventually need to make further changes and forget to update both
locations.  It is tempting to just nuke the leading_directory special
casing to avoid such bugs and simplify the code, but unpack_trees'
verify_clean_subdirectory() also calls read_directory() and does so with
a non-empty leading path, so I'm hesitant to try to restructure further.
Add obnoxious warnings to treat_leading_path() and
read_directory_recursive() to try to warn people of such problems.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-19 13:45:47 -08:00
Elijah Newren
b9670c1f5e dir: fix checks on common prefix directory
Many years ago, the directory traversing logic had an optimization that
would always recurse into any directory that was a common prefix of all
the pathspecs without walking the leading directories to get down to
the desired directory.  Thus,
   git ls-files -o .git/                        # case A
would notice that .git/ was a common prefix of all pathspecs (since
it is the only pathspec listed), and then traverse into it and start
showing unknown files under that directory.  Unfortunately, .git/ is not
a directory we should be traversing into, which made this optimization
problematic.  This also affected cases like
   git ls-files -o --exclude-standard t/        # case B
where t/ was in the .gitignore file and thus isn't interesting and
shouldn't be recursed into.  It also affected cases like
   git ls-files -o --directory untracked_dir/   # case C
where untracked_dir/ is indeed untracked and thus interesting, but the
--directory flag means we only want to show the directory itself, not
recurse into it and start listing untracked files below it.

The case B class of bugs were noted and fixed in commits 16e2cfa909
("read_directory(): further split treat_path()", 2010-01-08) and
48ffef966c ("ls-files: fix overeager pathspec optimization",
2010-01-08), with the idea being that we first wanted to check whether
the common prefix was interesting.  The former patch noted that
treat_path() couldn't be used when checking the common prefix because
treat_path() requires a dir_entry() and we haven't read any directories
at the point we are checking the common prefix.  So, that patch split
treat_one_path() out of treat_path().  The latter patch then created a
new treat_leading_path() which duplicated by hand the bits of
treat_path() that couldn't be broken out and then called
treat_one_path() for the remainder.  There were three problems with this
approach:

  * The duplicated logic in treat_leading_path() accidentally missed the
    check for special paths (such as is_dot_or_dotdot and matching
    ".git"), causing case A types of bugs to continue to be an issue.
  * The treat_leading_path() logic assumed we should traverse into
    anything where path_treatment was not path_none, i.e. it perpetuated
    class C types of bugs.
  * It meant we had split logic that needed to kept in sync, running the
    risk that people introduced new inconsistencies (such as in commit
    be8a84c526, which we reverted earlier in this series, or in commit
    df5bcdf83a which we'll fix in a subsequent commit)

Fix most these problems by making treat_leading_path() not only loop
over each leading path component, but calling treat_path() directly on
each.  To do so, we have to create a synthetic dir_entry, but that only
takes a few lines.  Then, pay attention to the path_treatment result we
get from treat_path() and don't treat path_excluded, path_untracked, and
path_recurse all the same as path_recurse.

This leaves one remaining problem, the new inconsistency from commit
df5bcdf83a.  That will be addressed in a subsequent commit.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-19 13:45:47 -08:00
René Scharfe
124a895811 t4015: improve coverage of function context test
Add a test that includes an actual function line in the test file to
check if context is expanded to include the whole function, and add an
ignored change before function context to check if that one stays hidden
while the originally ignored change within function context is shown.

This differs from the existing test, which is concerned with the case
where there is no function line at all in the file (and we might look
past the beginning of the file).

Helped-by: Jeff King <peff@peff.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-19 10:35:21 -08:00
Alexandr Miloslavskiy
509efef789 commit: forbid --pathspec-from-file --all
I forgot this in my previous patch `--pathspec-from-file` for
`git commit` [1]. When both `--pathspec-from-file` and `--all` were
specified, `--all` took precedence and `--pathspec-from-file` was
ignored. Before `--pathspec-from-file` was implemented, this case was
prevented by this check in `parse_and_validate_options()` :

    die(_("paths '%s ...' with -a does not make sense"), argv[0]);

It is unfortunate that these two cases are disconnected. This came as
result of how the code was laid out before my patches, where `pathspec`
is parsed outside of `parse_and_validate_options()`. This branch is
already full of refactoring patches and I did not dare to go for another
one.

Fix by mirroring `die()` for `--pathspec-from-file` as well.

[1] Commit e440fc58 ("commit: support the --pathspec-from-file option" 2019-11-19)

Reported-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-18 14:14:14 -08:00
Elijah Newren
12029dc57d t3434: mark successful test as such
t3434.3 was fixed by commit 917d0d6234 ("Merge branch
'js/rebase-r-safer-label'", 2019-12-05).  t3434 did not exist in
js/rebase-r-safer-label, so could not have marked the test as fixed, and
it was probably not noticed that the merge fixed this test.  Mark it as
fixed now.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-18 13:06:14 -08:00
René Scharfe
675ef6bab8 t6030: don't create unused file
my_bisect_log3.txt was added by c9c4e2d5a2 (bisect: only check merge
bases when needed, 2008-08-22), but hasn't been used then and since.
Get rid of it.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-18 12:32:24 -08:00
René Scharfe
01ed17dc8c t5580: don't create unused file
The file "out" was introduced by 13b57da833 (mingw: verify that paths
are not mistaken for remote nicknames, 2017-05-29), but has not actually
been used then and since.  Get rid of it.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-18 12:32:24 -08:00
René Scharfe
f670adb49b t3501: don't create unused file
The file "out" became unused with fd53b7ffd1 (merge-recursive: improve
add_cacheinfo error handling, 2018-04-19); get rid of it.

Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-18 12:32:24 -08:00
Junio C Hamano
4438a1a59f Merge branch 'js/t3404-indent-fix'
Test cleanup.

* js/t3404-indent-fix:
  t3404: fix indentation
2019-12-16 13:14:47 -08:00
Junio C Hamano
8bc481f4f6 Merge branch 'sg/t9300-robustify'
The test on "fast-import" used to get stuck when "fast-import" died
in the middle.

* sg/t9300-robustify:
  t9300-fast-import: don't hang if background fast-import exits too early
  t9300-fast-import: store the PID in a variable instead of pidfile
2019-12-16 13:08:47 -08:00
Junio C Hamano
011fc2e88e Merge branch 'js/add-i-a-bit-more-tests'
Test coverage update in preparation for further work on "git add -i".

* js/add-i-a-bit-more-tests:
  apply --allow-overlap: fix a corner case
  git add -p: use non-zero exit code when the diff generation failed
  t3701: verify that the diff.algorithm config setting is handled
  t3701: verify the shown messages when nothing can be added
  t3701: add a test for the different `add -p` prompts
  t3701: avoid depending on the TTY prerequisite
  t3701: add a test for advanced split-hunk editing
2019-12-16 13:08:47 -08:00
Junio C Hamano
d1c0fe8d9b Merge branch 'dl/range-diff-with-notes'
Code clean-up.

* dl/range-diff-with-notes:
  range-diff: clear `other_arg` at end of function
  range-diff: mark pointers as const
  t3206: fix incorrect test name
2019-12-16 13:08:46 -08:00
Junio C Hamano
f0070a7df9 Merge branch 'rs/xdiff-ignore-ws-w-func-context'
The "diff" machinery learned not to lose added/removed blank lines
in the context when --ignore-blank-lines and --function-context are
used at the same time.

* rs/xdiff-ignore-ws-w-func-context:
  xdiff: unignore changes in function context
2019-12-16 13:08:32 -08:00
Junio C Hamano
71a7de7a99 Merge branch 'dl/rebase-with-autobase'
"git rebase" did not work well when format.useAutoBase
configuration variable is set, which has been corrected.

* dl/rebase-with-autobase:
  rebase: fix format.useAutoBase breakage
  format-patch: teach --no-base
  t4014: use test_config()
  format-patch: fix indentation
  t3400: demonstrate failure with format.useAutoBase
2019-12-16 13:08:32 -08:00
Junio C Hamano
c9f5fc9114 Merge branch 'dl/test-cleanup'
Test cleanup.

* dl/test-cleanup: (26 commits)
  t7700: stop losing return codes of git commands
  t7700: make references to SHA-1 generic
  t7700: replace egrep with grep
  t7700: consolidate code into test_has_duplicate_object()
  t7700: consolidate code into test_no_missing_in_packs()
  t7700: s/test -f/test_path_is_file/
  t7700: move keywords onto their own line
  t7700: remove spaces after redirect operators
  t7700: drop redirections to /dev/null
  t7501: stop losing return codes of git commands
  t7501: remove spaces after redirect operators
  t5703: stop losing return codes of git commands
  t5703: simplify one-time-sed generation logic
  t5317: use ! grep to check for no matching lines
  t5317: stop losing return codes of git commands
  t4138: stop losing return codes of git commands
  t4015: use test_write_lines()
  t4015: stop losing return codes of git commands
  t3600: comment on inducing SIGPIPE in `git rm`
  t3600: stop losing return codes of git commands
  ...
2019-12-16 13:08:32 -08:00
Junio C Hamano
6d831b8a3e Merge branch 'cs/store-packfiles-in-hashmap'
In a repository with many packfiles, the cost of the procedure that
avoids registering the same packfile twice was unnecessarily high
by using an inefficient search algorithm, which has been corrected.

* cs/store-packfiles-in-hashmap:
  packfile.c: speed up loading lots of packfiles
2019-12-16 13:08:32 -08:00
ryenus
571fb96573 fix-typo: consecutive-word duplications
Correct unintentional duplication(s) of words, such as "the the",
and "can can" etc.

The changes are only applied to cases where it's fixing what is clearly
wrong or prone to misunderstanding, as suggested by the reviewers.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Helped-by: Denton Liu <liu.denton@gmail.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: ryenus <ryenus@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-16 11:53:11 -08:00
Johannes Schindelin
d6cf873340 built-in add -p: implement the '/' ("search regex") command
This patch implements the hunk searching feature in the C version of
`git add -p`.

A test is added to verify that this behavior matches the one of the Perl
version of `git add -p`.

Note that this involves a change of behavior: the Perl version uses (of
course) the Perl flavor of regular expressions, while this patch uses
the regcomp()/regexec(), i.e. POSIX extended regular expressions. In
practice, this behavior change is unlikely to matter.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:37:14 -08:00
Johannes Schindelin
9254bdfb4f built-in add -p: implement the 'g' ("goto") command
With this patch, it is now possible to see a summary of the available
hunks and to navigate between them (by number).

A test is added to verify that this behavior matches the one of the Perl
version of `git add -p`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:37:14 -08:00
Johannes Schindelin
510aeca199 built-in add -p: implement the hunk splitting feature
If this developer's workflow is any indication, then this is *the* most
useful feature of Git's interactive `add `command.

Note: once again, this is not a verbatim conversion from the Perl code
to C: the `hunk_splittable()` function, for example, essentially did all
the work of splitting the hunk, just to find out whether more than one
hunk would have been the result (and then tossed that result into the
trash). In C we instead count the number of resulting hunks (without
actually doing the work of splitting, but just counting the transitions
from non-context lines to context lines), and store that information
with the hunk, and we do that *while* parsing the diff in the first
place.

Another deviation: the built-in `git add -p` was designed with a single
strbuf holding the diff (and another one holding the colored diff, if
that one was asked for) in mind, and hunks essentially store just the
start and end offsets pointing into that strbuf. As a consequence, when
we split hunks, we now use a special mode where the hunk header is
generated dynamically, and only the rest of the hunk is stored using
such start/end offsets. This way, we also avoid the frequent
formatting/re-parsing of the hunk header of the Perl version.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:37:14 -08:00
Derrick Stolee
190a65f9db sparse-checkout: respect core.ignoreCase in cone mode
When a user uses the sparse-checkout feature in cone mode, they
add patterns using "git sparse-checkout set <dir1> <dir2> ..."
or by using "--stdin" to provide the directories line-by-line over
stdin. This behaviour naturally looks a lot like the way a user
would type "git add <dir1> <dir2> ..."

If core.ignoreCase is enabled, then "git add" will match the input
using a case-insensitive match. Do the same for the sparse-checkout
feature.

Perform case-insensitive checks while updating the skip-worktree
bits during unpack_trees(). This is done by changing the hash
algorithm and hashmap comparison methods to optionally use case-
insensitive methods.

When this is enabled, there is a small performance cost in the
hashing algorithm. To tease out the worst possible case, the
following was run on a repo with a deep directory structure:

	git ls-tree -d -r --name-only HEAD |
		git sparse-checkout set --stdin

The 'set' command was timed with core.ignoreCase disabled or
enabled. For the repo with a deep history, the numbers were

	core.ignoreCase=false: 62s
	core.ignoreCase=true:  74s (+19.3%)

For reproducibility, the equivalent test on the Linux kernel
repository had these numbers:

	core.ignoreCase=false: 3.1s
	core.ignoreCase=true:  3.6s (+16%)

Now, this is not an entirely fair comparison, as most users
will define their sparse cone using more shallow directories,
and the performance improvement from eb42feca97 ("unpack-trees:
hash less in cone mode" 2019-11-21) can remove most of the
hash cost. For a more realistic test, drop the "-r" from the
ls-tree command to store only the first-level directories.
In that case, the Linux kernel repository takes 0.2-0.25s in
each case, and the deep repository takes one second, plus or
minus 0.05s, in each case.

Thus, we _can_ demonstrate a cost to this change, but it is
unlikely to matter to any reasonable sparse-checkout cone.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-12-13 12:01:02 -08:00