* by/log-follow:
tests: rename duplicate t4205
Make git log --follow find copies among unmodified files.
Make diffcore_std only can run once before a diff_flush
Add a macro DIFF_QUEUE_CLEAR.
Here we simply make --patch a synonym for -p, whose mnemonic was "patch"
all along.
Signed-off-by: Will Palmer <wmpalmer@gmail.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jk/cached-textconv:
diff: avoid useless filespec population
diff: cache textconv output
textconv: refactor calls to run_textconv
introduce notes-cache interface
make commit_tree a library function
When file renames/copies detection is turned on, the
second diffcore_std will degrade a 'C' pair to a 'R' pair.
And this may happen when we run 'git log --follow' with
hard copies finding. That is, the try_to_follow_renames()
will run diffcore_std to find the copies, and then
'git log' will issue another diffcore_std, which will reduce
'src->rename_used' and recognize this copy as a rename.
This is not what we want.
So, I think we really don't need to run diffcore_std more
than one time.
Signed-off-by: Bo Yang <struggleyb.nku@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor the diff_queue_struct code, this macro help
to reset the structure.
Signed-off-by: Bo Yang <struggleyb.nku@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
xdi_diff_outf() overrides the structure members of its last parameter,
ignoring any value that callers pass in. It's no surprise then that all
callers pass a pointer to an uninitialized structure. They also don't
read it after the call, so the parameter is neither used for input nor
for output. Turn it into a local variable of xdi_diff_outf().
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The diffstat "added" and "changed" fields generally store
line counts; however, for binary files, they store file
sizes. Since we store and print these values as ints, a
diffstat on a file larger than 2G can show a negative size.
Instead, let's use uintmax_t, which should be at least 64
bits on modern platforms.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This teaches the --color-words engine a more general interface that
supports two new modes:
* --word-diff=plain, inspired by the 'wdiff' utility (most similar to
'wdiff -n <old> <new>'): uses delimiters [-removed-] and {+added+}
* --word-diff=porcelain, which generates an ad-hoc machine readable
format:
- each diff unit is prefixed by [-+ ] and terminated by newline as
in unified diff
- newlines in the input are output as a line consisting only of a
tilde '~'
Both of these formats still support color if it is enabled, using it
to highlight the differences. --color-words becomes a synonym for
--word-diff=color, which is the color-only format. Also adds some
compatibility/convenience options.
Thanks to Junio C Hamano and Miles Bader for good ideas.
Signed-off-by: Thomas Rast <trast@student.ethz.ch>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
builtin_diff calls fill_mmfile fairly early, which in turn
calls diff_populate_filespec, which actually retrieves the
file's blob contents into a buffer. Long ago, this was
sensible as we would need to look at the blobs eventually.
These days, however, we may not ever want those blobs if we
end up using a textconv cache, and for large binary files
(exactly the sort for which you might have a textconv
cache), just retrieving the objects can be costly.
This patch just pushes the fill_mmfile call a bit later, so
we can avoid populating the filespec in some cases. There
is one thing to note that looks like a bug but isn't. We
push the fill_mmfile down into the first branch of a
conditional. It seems like we would need it on the other
branch, too, but we don't; fill_textconv does it for us (in
fact, before this, we were just writing over the results of
the fill_mmfile on that branch).
Here's a timing sample on a commit with 45 changed jpgs and
avis. The result is fully textconv cached, but we still
wasted a lot of time just pulling the blobs from storage.
The total size of the blobs (source and dest) is about
180M.
[before]
$ time git show >/dev/null
real 0m0.352s
user 0m0.148s
sys 0m0.200s
[after]
$ time git show >/dev/null
real 0m0.009s
user 0m0.004s
sys 0m0.004s
And that's on a warm cache. On a cold cache, the "after"
case is not much worse, but the "before" case has to do an
extra 180M of I/O.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Running a textconv filter can take a long time. It's
particularly bad for a large file which needs to be spooled
to disk, but even for small files, the fork+exec overhead
can add up for something like "git log -p".
This patch uses the notes-cache mechanism to keep a fast
cache of textconv output. Caches are stored in
refs/notes/textconv/$x, where $x is the userdiff driver
defined in gitattributes.
Caching is enabled only if diff.$x.cachetextconv is true.
In my test repo, on a commit with 45 jpg and avi files
changed and a textconv to show their exif tags:
[before]
$ time git show >/dev/null
real 0m13.724s
user 0m12.057s
sys 0m1.624s
[after, first run]
$ git config diff.mfo.cachetextconv true
$ time git show >/dev/null
real 0m14.252s
user 0m12.197s
sys 0m1.800s
[after, subsequent runs]
$ time git show >/dev/null
real 0m0.352s
user 0m0.148s
sys 0m0.200s
So for a slight (3.8%) cost on the first run, we achieve an
almost 40x speed up on subsequent runs.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch adds a fill_textconv wrapper, which centralizes
some minor logic like error checking and handling the case
of no-textconv.
In addition to dropping the number of lines, this will make
it easier in future patches to handle multiple types of
textconv.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We correctly free() for the normal diff case, but leak for
rewrite diffs.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
To make the code simpler, run_textconv lumps all of its
error checking into one conditional. However, the
short-circuit means that an error in reading will prevent us
from calling finish_command, leaving a zombie child.
Clean up properly after errors.
Based-on-work-by: Jeff King <peff@peff.net>
Signed-off-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jl/submodule-diff-dirtiness:
git status: ignoring untracked files must apply to submodules too
git status: Fix false positive "new commits" output for dirty submodules
Refactor dirty submodule detection in diff-lib.c
git status: Show detailed dirty status of submodules in long format
git diff --submodule: Show detailed dirty status of submodules
Testing if the output "new commits" should appear in the long format of
"git status" is done by comparing the hashes of the diffpair. This always
resulted in printing "new commits" for submodules that contained untracked
or modified content, even if they did not contain new commits. The reason
was that match_stat_with_submodule() did set the "changed" flag for dirty
submodules, resulting in two->sha1 being set to the null_sha1 at the call
sites, which indicates that new commits are present. This is changed so
that when no new commits are present, the same object names are in the
sha1 field for both sides of the filepair, and the working tree side will
have the "dirty_submodule" flag set when appropriate. For a submodule to
be seen as modified even when it just has a dirty work tree, some
conditions had to be extended to also check for the "dirty_submodule"
flag.
Unfortunately the test case that should have found this bug had been
changed incorrectly too. It is fixed and extended to test for other
combinations too.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Moving duplicated code into the new function match_stat_with_submodule().
Replacing the implicit activation of detailed checks for the dirtiness of
submodules when DIFF_FORMAT_PATCH was selected with explicitly setting
the recently added DIFF_OPT_DIRTY_SUBMODULES option in diff_setup_done().
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make git-branch, git-show-branch, git-grep, and all the diff-based
programs accept an optional argument <when> for --color. The argument
is a colorbool: "always", "never", or "auto". If no argument is given,
"always" is used; --no-color is an alias for --color=never. This makes
the command-line interface consistent with other GNU tools, such as `ls'
and `grep', and with the git-config color options. Note that, without
an argument, --color and --no-color work exactly as before.
To implement this, two internal changes were made:
1. Allow the first argument of git_config_colorbool() to be NULL,
in which case it returns -1 if the argument isn't "always", "never",
or "auto".
2. Add OPT_COLOR_FLAG(), OPT__COLOR(), and parse_opt_color_flag_cb()
to the option parsing library. The callback uses
git_config_colorbool(), so color.h is now a dependency
of parse-options.c.
Signed-off-by: Mark Lodato <lodatom@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint-1.6.6:
dwim_ref: fix dangling symref warning
stash pop: remove 'apply' options during 'drop' invocation
diff: make sure --output=/bad/path is caught
Remove hyphen from "git-command" in two error messages
The option -w tells the diff machinery to inspect the contents to set the
exit status, instead of checking the blob object level difference alone.
However, --quiet tells the diff machinery not to look at the contents, which
means DIFF_FROM_CONTENTS has no chance to inspect the change.
Work it around by calling diff_flush_patch() with output sent to /dev/null.
Signed-off-by: Larry D'Anna <larry@elder-gods.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jl/diff-submodule-ignore:
Teach diff --submodule that modified submodule directory is dirty
git diff: Don't test submodule dirtiness with --ignore-submodules
Make ce_uptodate() trustworthy again
Since commit 8e08b4 git diff does append "-dirty" to the work tree side
if the working directory of a submodule contains new or modified files.
Lets do the same when the --submodule option is used.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/fix-tree-walk:
read-tree --debug-unpack
unpack-trees.c: look ahead in the index
unpack-trees.c: prepare for looking ahead in the index
Aggressive three-way merge: fix D/F case
traverse_trees(): handle D/F conflict case sanely
more D/F conflict tests
tests: move convenience regexp to match object names to test-lib.sh
Conflicts:
builtin-read-tree.c
unpack-trees.c
unpack-trees.h
* jl/submodule-diff:
Performance optimization for detection of modified submodules
git status: Show uncommitted submodule changes too when enabled
Teach diff that modified submodule directory is dirty
Show submodules as modified when they contain a dirty work tree
* maint-1.6.4:
Fix mis-backport of t7002
base85: Make the code more obvious instead of explaining the non-obvious
base85: encode_85() does not use the decode table
base85 debug code: Fix length byte calculation
checkout -m: do not try to fall back to --merge from an unborn branch
branch: die explicitly why when calling "git branch [-a|-r] branchname".
textconv: stop leaking file descriptors
commit: --cleanup is a message option
git count-objects: handle packs bigger than 4G
t7102: make the test fail if one of its check fails
* maint-1.6.3:
base85: Make the code more obvious instead of explaining the non-obvious
base85: encode_85() does not use the decode table
base85 debug code: Fix length byte calculation
checkout -m: do not try to fall back to --merge from an unborn branch
branch: die explicitly why when calling "git branch [-a|-r] branchname".
textconv: stop leaking file descriptors
commit: --cleanup is a message option
git count-objects: handle packs bigger than 4G
t7102: make the test fail if one of its check fails
Conflicts:
builtin-commit.c
* maint-1.6.2:
base85: Make the code more obvious instead of explaining the non-obvious
base85: encode_85() does not use the decode table
base85 debug code: Fix length byte calculation
checkout -m: do not try to fall back to --merge from an unborn branch
branch: die explicitly why when calling "git branch [-a|-r] branchname".
textconv: stop leaking file descriptors
commit: --cleanup is a message option
git count-objects: handle packs bigger than 4G
t7102: make the test fail if one of its check fails
Conflicts:
diff.c
In the worst case is_submodule_modified() got called three times for
each submodule. The information we got from scanning the whole
submodule tree the first time can be reused instead.
New parameters have been added to diff_change() and diff_addremove(),
the information is stored in a new member of struct diff_filespec. Its
value is then reused instead of calling is_submodule_modified() again.
When no explicit "-dirty" is needed in the output the call to
is_submodule_modified() is not necessary when the submodules HEAD
already disagrees with the ref of the superproject, as this alone
marks it as modified. To achieve that, get_stat_data() got an extra
argument.
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jk/run-command-use-shell:
t4030, t4031: work around bogus MSYS bash path conversion
diff: run external diff helper with shell
textconv: use shell to run helper
editor: use run_command's shell feature
run-command: optimize out useless shell calls
run-command: convert simple callsites to use_shell
t0021: use $SHELL_PATH for the filter script
run-command: add "use shell" option
A diff run in superproject only compares the name of the commit object
bound at the submodule paths. When we compare with a work tree and the
checked out submodule directory is dirty (e.g. has either staged or
unstaged changes, or has new files the user forgot to add to the index),
show the work tree side as "dirty".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Jens Lehmann <Jens.Lehmann@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* nd/sparse: (25 commits)
t7002: test for not using external grep on skip-worktree paths
t7002: set test prerequisite "external-grep" if supported
grep: do not do external grep on skip-worktree entries
commit: correctly respect skip-worktree bit
ie_match_stat(): do not ignore skip-worktree bit with CE_MATCH_IGNORE_VALID
tests: rename duplicate t1009
sparse checkout: inhibit empty worktree
Add tests for sparse checkout
read-tree: add --no-sparse-checkout to disable sparse checkout support
unpack-trees(): ignore worktree check outside checkout area
unpack_trees(): apply $GIT_DIR/info/sparse-checkout to the final index
unpack-trees(): "enable" sparse checkout and load $GIT_DIR/info/sparse-checkout
unpack-trees.c: generalize verify_* functions
unpack-trees(): add CE_WT_REMOVE to remove on worktree alone
Introduce "sparse checkout"
dir.c: export excluded_1() and add_excludes_from_file_1()
excluded_1(): support exclude files in index
unpack-trees(): carry skip-worktree bit over in merged_entry()
Read .gitignore from index if it is skip-worktree
Avoid writing to buffer in add_excludes_from_file_1()
...
Conflicts:
.gitignore
Documentation/config.txt
Documentation/git-update-index.txt
Makefile
entry.c
t/t7002-grep.sh
* maint-1.6.1:
base85: Make the code more obvious instead of explaining the non-obvious
base85: encode_85() does not use the decode table
base85 debug code: Fix length byte calculation
checkout -m: do not try to fall back to --merge from an unborn branch
branch: die explicitly why when calling "git branch [-a|-r] branchname".
textconv: stop leaking file descriptors
commit: --cleanup is a message option
git count-objects: handle packs bigger than 4G
t7102: make the test fail if one of its check fails
Conflicts:
diff.c
This makes the traversal of index be in sync with the tree traversal.
When unpack_callback() is fed a set of tree entries from trees, it
inspects the name of the entry and checks if the an index entry with
the same name could be hiding behind the current index entry, and
(1) if the name appears in the index as a leaf node, it is also
fed to the n_way_merge() callback function;
(2) if the name is a directory in the index, i.e. there are entries in
that are underneath it, then nothing is fed to the n_way_merge()
callback function;
(3) otherwise, if the name comes before the first eligible entry in the
index, the index entry is first unpacked alone.
When traverse_trees_recursive() descends into a subdirectory, the
cache_bottom pointer is moved to walk index entries within that directory.
All of these are omitted for diff-index, which does not even want to be
fed an index entry and a tree entry with D/F conflicts.
This fixes 3-way read-tree and exposes a bug in other parts of the system
in t6035, test #5. The test prepares these three trees:
O = HEAD^
100644 blob e69de29bb2 a/b-2/c/d
100644 blob e69de29bb2 a/b/c/d
100644 blob e69de29bb2 a/x
A = HEAD
100644 blob e69de29bb2 a/b-2/c/d
100644 blob e69de29bb2 a/b/c/d
100644 blob 587be6b4c3f93f93c489c0111bba5596147a26cb a/x
B = master
120000 blob a36b77384451ea1de7bd340ffca868249626bc52 a/b
100644 blob e69de29bb2 a/b-2/c/d
100644 blob e69de29bb2 a/x
With a clean index that matches HEAD, running
git read-tree -m -u --aggressive $O $A $B
now yields
120000 a36b77384451ea1de7bd340ffca868249626bc52 3 a/b
100644 e69de29bb2 0 a/b-2/c/d
100644 e69de29bb2 1 a/b/c/d
100644 e69de29bb2 2 a/b/c/d
100644 587be6b4c3f93f93c489c0111bba5596147a26cb 0 a/x
which is correct. "master" created "a/b" symlink that did not exist,
and removed "a/b/c/d" while HEAD did not do touch either path.
Before this series, read-tree did not notice the situation and resolved
addition of "a/b" and removal of "a/b/c/d" independently. If A = HEAD had
another path "a/b/c/e" added, this merge should conflict but instead it
silently resolved "a/b" and then immediately overwrote it to add
"a/b/c/e", which was quite bogus.
Tests in t1012 start to work with this.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
