2005-05-21 09:39:09 +00:00
|
|
|
/*
|
|
|
|
* Copyright (C) 2005 Junio C Hamano
|
|
|
|
*/
|
2007-04-29 06:38:52 +00:00
|
|
|
#ifndef DIFFCORE_H
|
|
|
|
#define DIFFCORE_H
|
2005-05-21 09:39:09 +00:00
|
|
|
|
2023-02-24 00:09:25 +00:00
|
|
|
#include "hash.h"
|
2018-08-15 17:54:05 +00:00
|
|
|
|
|
|
|
struct diff_options;
|
2023-02-24 00:09:25 +00:00
|
|
|
struct mem_pool;
|
|
|
|
struct oid_array;
|
2018-09-21 15:57:19 +00:00
|
|
|
struct repository;
|
2021-03-13 22:22:02 +00:00
|
|
|
struct strintmap;
|
2021-02-27 00:30:42 +00:00
|
|
|
struct strmap;
|
2018-09-21 15:57:19 +00:00
|
|
|
struct userdiff_driver;
|
2018-08-15 17:54:05 +00:00
|
|
|
|
2005-05-21 09:39:09 +00:00
|
|
|
/* This header file is internal between diff.c and its diff transformers
|
|
|
|
* (e.g. diffcore-rename, diffcore-pickaxe). Never include this header
|
|
|
|
* in anything else.
|
|
|
|
*/
|
[PATCH] diff: Update -B heuristics.
As Linus pointed out on the mailing list discussion, -B should
break a files that has many inserts even if it still keeps
enough of the original contents, so that the broken pieces can
later be matched with other files by -M or -C. However, if such
a broken pair does not get picked up by -M or -C, we would want
to apply different criteria; namely, regardless of the amount of
new material in the result, the determination of "rewrite"
should be done by looking at the amount of original material
still left in the result. If you still have the original 97
lines from a 100-line document, it does not matter if you add
your own 13 lines to make a 110-line document, or if you add 903
lines to make a 1000-line document. It is not a rewrite but an
in-place edit. On the other hand, if you did lose 97 lines from
the original, it does not matter if you added 27 lines to make a
30-line document or if you added 997 lines to make a 1000-line
document. You did a complete rewrite in either case.
This patch introduces a post-processing phase that runs after
diffcore-rename matches up broken pairs diffcore-break creates.
The purpose of this post-processing is to pick up these broken
pieces and merge them back into in-place modifications. For
this, the score parameter -B option takes is changed into a pair
of numbers, and it takes "-B99/80" format when fully spelled
out. The first number is the minimum amount of "edit" (same
definition as what diffcore-rename uses, which is "sum of
deletion and insertion") that a modification needs to have to be
broken, and the second number is the minimum amount of "delete"
a surviving broken pair must have to avoid being merged back
together. It can be abbreviated to "-B" to use default for
both, "-B9" or "-B9/" to use 90% for "edit" but default (80%)
for merge avoidance, or "-B/75" to use default (99%) "edit" and
75% for merge avoidance.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-06-03 08:40:28 +00:00
|
|
|
|
|
|
|
/* We internally use unsigned short as the score value,
|
|
|
|
* and rely on an int capable to hold 32-bits. -B can take
|
|
|
|
* -Bmerge_score/break_score format and the two scores are
|
|
|
|
* passed around in one int (high 16-bit for merge and low 16-bit
|
|
|
|
* for break).
|
|
|
|
*/
|
2006-01-16 05:08:42 +00:00
|
|
|
#define MAX_SCORE 60000.0
|
[PATCH] Add -B flag to diff-* brothers.
A new diffcore transformation, diffcore-break.c, is introduced.
When the -B flag is given, a patch that represents a complete
rewrite is broken into a deletion followed by a creation. This
makes it easier to review such a complete rewrite patch.
The -B flag takes the same syntax as the -M and -C flags to
specify the minimum amount of non-source material the resulting
file needs to have to be considered a complete rewrite, and
defaults to 99% if not specified.
As the new test t4008-diff-break-rewrite.sh demonstrates, if a
file is a complete rewrite, it is broken into a delete/create
pair, which can further be subjected to the usual rename
detection if -M or -C is used. For example, if file0 gets
completely rewritten to make it as if it were rather based on
file1 which itself disappeared, the following happens:
The original change looks like this:
file0 --> file0' (quite different from file0)
file1 --> /dev/null
After diffcore-break runs, it would become this:
file0 --> /dev/null
/dev/null --> file0'
file1 --> /dev/null
Then diffcore-rename matches them up:
file1 --> file0'
The internal score values are finer grained now. Earlier
maximum of 10000 has been raised to 60000; there is no user
visible changes but there is no reason to waste available bits.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-30 07:08:37 +00:00
|
|
|
#define DEFAULT_RENAME_SCORE 30000 /* rename/copy similarity minimum (50%) */
|
2006-03-04 09:03:53 +00:00
|
|
|
#define DEFAULT_BREAK_SCORE 30000 /* minimum for break to happen (50%) */
|
2010-08-05 16:14:25 +00:00
|
|
|
#define DEFAULT_MERGE_SCORE 36000 /* maximum for break-merge to happen (60%) */
|
[PATCH] diff: Update -B heuristics.
As Linus pointed out on the mailing list discussion, -B should
break a files that has many inserts even if it still keeps
enough of the original contents, so that the broken pieces can
later be matched with other files by -M or -C. However, if such
a broken pair does not get picked up by -M or -C, we would want
to apply different criteria; namely, regardless of the amount of
new material in the result, the determination of "rewrite"
should be done by looking at the amount of original material
still left in the result. If you still have the original 97
lines from a 100-line document, it does not matter if you add
your own 13 lines to make a 110-line document, or if you add 903
lines to make a 1000-line document. It is not a rewrite but an
in-place edit. On the other hand, if you did lose 97 lines from
the original, it does not matter if you added 27 lines to make a
30-line document or if you added 997 lines to make a 1000-line
document. You did a complete rewrite in either case.
This patch introduces a post-processing phase that runs after
diffcore-rename matches up broken pairs diffcore-break creates.
The purpose of this post-processing is to pick up these broken
pieces and merge them back into in-place modifications. For
this, the score parameter -B option takes is changed into a pair
of numbers, and it takes "-B99/80" format when fully spelled
out. The first number is the minimum amount of "edit" (same
definition as what diffcore-rename uses, which is "sum of
deletion and insertion") that a modification needs to have to be
broken, and the second number is the minimum amount of "delete"
a surviving broken pair must have to avoid being merged back
together. It can be abbreviated to "-B" to use default for
both, "-B9" or "-B9/" to use 90% for "edit" but default (80%)
for merge avoidance, or "-B/75" to use default (99%) "edit" and
75% for merge avoidance.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-06-03 08:40:28 +00:00
|
|
|
|
|
|
|
#define MINIMUM_BREAK_SIZE 400 /* do not break a file smaller than this */
|
2005-05-21 09:39:09 +00:00
|
|
|
|
2019-11-17 21:04:40 +00:00
|
|
|
/**
|
|
|
|
* the internal representation for a single file (blob). It records the blob
|
|
|
|
* object name (if known -- for a work tree file it typically is a NUL SHA-1),
|
|
|
|
* filemode and pathname. This is what the `diff_addremove()`, `diff_change()`
|
|
|
|
* and `diff_unmerge()` synthesize and feed `diff_queue()` function with.
|
|
|
|
*/
|
2005-05-21 09:39:09 +00:00
|
|
|
struct diff_filespec {
|
2016-06-24 23:09:23 +00:00
|
|
|
struct object_id oid;
|
2005-05-21 09:39:09 +00:00
|
|
|
char *path;
|
|
|
|
void *data;
|
2006-03-12 11:22:10 +00:00
|
|
|
void *cnt_data;
|
2005-05-21 09:39:09 +00:00
|
|
|
unsigned long size;
|
2007-10-25 18:19:10 +00:00
|
|
|
int count; /* Reference count */
|
2007-10-25 18:20:56 +00:00
|
|
|
int rename_used; /* Count of rename users */
|
2005-05-21 09:39:09 +00:00
|
|
|
unsigned short mode; /* file mode */
|
2016-06-24 23:09:24 +00:00
|
|
|
unsigned oid_valid : 1; /* if true, use oid and trust mode;
|
2005-05-21 09:39:09 +00:00
|
|
|
* if false, use the name and read from
|
|
|
|
* the filesystem.
|
|
|
|
*/
|
2005-06-13 00:23:15 +00:00
|
|
|
#define DIFF_FILE_VALID(spec) (((spec)->mode) != 0)
|
2005-05-21 09:39:09 +00:00
|
|
|
unsigned should_free : 1; /* data should be free()'ed */
|
|
|
|
unsigned should_munmap : 1; /* data should be munmap()'ed */
|
2010-03-04 21:20:33 +00:00
|
|
|
unsigned dirty_submodule : 2; /* For submodules: its work tree is dirty */
|
|
|
|
#define DIRTY_SUBMODULE_UNTRACKED 1
|
|
|
|
#define DIRTY_SUBMODULE_MODIFIED 2
|
2014-01-17 01:19:46 +00:00
|
|
|
unsigned is_stdin : 1;
|
2011-08-20 06:32:51 +00:00
|
|
|
unsigned has_more_entries : 1; /* only appear in combined diff */
|
diff: introduce diff.<driver>.binary
The "diff" gitattribute is somewhat overloaded right now. It
can say one of three things:
1. this file is definitely binary, or definitely not
(i.e., diff or !diff)
2. this file should use an external diff engine (i.e.,
diff=foo, diff.foo.command = custom-script)
3. this file should use particular funcname patterns
(i.e., diff=foo, diff.foo.(x?)funcname = some-regex)
Most of the time, there is no conflict between these uses,
since using one implies that the other is irrelevant (e.g.,
an external diff engine will decide for itself whether the
file is binary).
However, there is at least one conflicting situation: there
is no way to say "use the regular rules to determine whether
this file is binary, but if we do diff it textually, use
this funcname pattern." That is, currently setting diff=foo
indicates that the file is definitely text.
This patch introduces a "binary" config option for a diff
driver, so that one can explicitly set diff.foo.binary. We
default this value to "don't know". That is, setting a diff
attribute to "foo" and using "diff.foo.funcname" will have
no effect on the binaryness of a file. To get the current
behavior, one can set diff.foo.binary to true.
This patch also has one additional advantage: it cleans up
the interface to the userdiff code a bit. Before, calling
code had to know more about whether attributes were false,
true, or unset to determine binaryness. Now that binaryness
is a property of a driver, we can represent these situations
just by passing back a driver struct.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
2008-10-05 21:43:36 +00:00
|
|
|
/* data should be considered "binary"; -1 means "don't know yet" */
|
2014-02-24 00:54:47 +00:00
|
|
|
signed int is_binary : 2;
|
diff_filespec: reorder is_binary field
The middle of the diff_filespec struct contains a mixture of
ints, shorts, and bit-fields, followed by a pointer. On an
x86-64 system with an LP64 or LLP64 data model (i.e., most
of them), the integers and flags end up being padded out by
41 bits to put the pointer at an 8-byte boundary.
After the pointer, we have the "int is_binary" field, which
is only 32 bits. We end up wasting another 32 bits to pad
the struct size up to a multiple of 64 bits.
We can move the is_binary field before the pointer, which
lets the compiler store it where we used to have padding.
This shrinks the top padding to only 9 bits (from the
bit-fields), and eliminates the bottom padding entirely,
dropping the struct size from 88 to 80 bytes.
On a 32-bit system, there is no benefit, but nor should
there be any harm (we only need 4-byte alignment there, so
we were already using only 9 bits of padding).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-17 01:22:56 +00:00
|
|
|
struct userdiff_driver *driver;
|
2005-05-21 09:39:09 +00:00
|
|
|
};
|
|
|
|
|
2018-06-30 09:20:25 +00:00
|
|
|
struct diff_filespec *alloc_filespec(const char *);
|
|
|
|
void free_filespec(struct diff_filespec *);
|
|
|
|
void fill_filespec(struct diff_filespec *, const struct object_id *,
|
|
|
|
int, unsigned short);
|
2005-05-21 09:39:09 +00:00
|
|
|
|
diff: restrict when prefetching occurs
Commit 7fbbcb21b1 ("diff: batch fetching of missing blobs", 2019-04-08)
optimized "diff" by prefetching blobs in a partial clone, but there are
some cases wherein blobs do not need to be prefetched. In these cases,
any command that uses the diff machinery will unnecessarily fetch blobs.
diffcore_std() may read blobs when it calls the following functions:
(1) diffcore_skip_stat_unmatch() (controlled by the config variable
diff.autorefreshindex)
(2) diffcore_break() and diffcore_merge_broken() (for break-rewrite
detection)
(3) diffcore_rename() (for rename detection)
(4) diffcore_pickaxe() (for detecting addition/deletion of specified
string)
Instead of always prefetching blobs, teach diffcore_skip_stat_unmatch(),
diffcore_break(), and diffcore_rename() to prefetch blobs upon the first
read of a missing object. This covers (1), (2), and (3): to cover the
rest, teach diffcore_std() to prefetch if the output type is one that
includes blob data (and hence blob data will be required later anyway),
or if it knows that (4) will be run.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:11:43 +00:00
|
|
|
/*
|
|
|
|
* Prefetch the entries in diff_queued_diff. The parameter is a pointer to a
|
|
|
|
* struct repository.
|
|
|
|
*/
|
|
|
|
void diff_queued_diff_prefetch(void *repository);
|
|
|
|
|
2020-04-07 22:11:41 +00:00
|
|
|
struct diff_populate_filespec_options {
|
|
|
|
unsigned check_size_only : 1;
|
|
|
|
unsigned check_binary : 1;
|
diff: restrict when prefetching occurs
Commit 7fbbcb21b1 ("diff: batch fetching of missing blobs", 2019-04-08)
optimized "diff" by prefetching blobs in a partial clone, but there are
some cases wherein blobs do not need to be prefetched. In these cases,
any command that uses the diff machinery will unnecessarily fetch blobs.
diffcore_std() may read blobs when it calls the following functions:
(1) diffcore_skip_stat_unmatch() (controlled by the config variable
diff.autorefreshindex)
(2) diffcore_break() and diffcore_merge_broken() (for break-rewrite
detection)
(3) diffcore_rename() (for rename detection)
(4) diffcore_pickaxe() (for detecting addition/deletion of specified
string)
Instead of always prefetching blobs, teach diffcore_skip_stat_unmatch(),
diffcore_break(), and diffcore_rename() to prefetch blobs upon the first
read of a missing object. This covers (1), (2), and (3): to cover the
rest, teach diffcore_std() to prefetch if the output type is one that
includes blob data (and hence blob data will be required later anyway),
or if it knows that (4) will be run.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:11:43 +00:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If an object is missing, diff_populate_filespec() will invoke this
|
|
|
|
* callback before attempting to read that object again.
|
|
|
|
*/
|
|
|
|
void (*missing_object_cb)(void *);
|
|
|
|
void *missing_object_data;
|
2020-04-07 22:11:41 +00:00
|
|
|
};
|
|
|
|
int diff_populate_filespec(struct repository *, struct diff_filespec *,
|
|
|
|
const struct diff_populate_filespec_options *);
|
2018-06-30 09:20:25 +00:00
|
|
|
void diff_free_filespec_data(struct diff_filespec *);
|
|
|
|
void diff_free_filespec_blob(struct diff_filespec *);
|
2018-09-21 15:57:19 +00:00
|
|
|
int diff_filespec_is_binary(struct repository *, struct diff_filespec *);
|
2005-05-21 09:39:09 +00:00
|
|
|
|
2019-11-17 21:04:40 +00:00
|
|
|
/**
|
|
|
|
* This records a pair of `struct diff_filespec`; the filespec for a file in
|
|
|
|
* the "old" set (i.e. preimage) is called `one`, and the filespec for a file
|
|
|
|
* in the "new" set (i.e. postimage) is called `two`. A change that represents
|
|
|
|
* file creation has NULL in `one`, and file deletion has NULL in `two`.
|
|
|
|
*
|
|
|
|
* A `filepair` starts pointing at `one` and `two` that are from the same
|
|
|
|
* filename, but `diffcore_std()` can break pairs and match component filespecs
|
|
|
|
* with other filespecs from a different filepair to form new filepair. This is
|
|
|
|
* called 'rename detection'.
|
|
|
|
*/
|
2005-05-21 09:40:01 +00:00
|
|
|
struct diff_filepair {
|
2005-05-21 09:39:09 +00:00
|
|
|
struct diff_filespec *one;
|
|
|
|
struct diff_filespec *two;
|
2005-05-29 23:56:48 +00:00
|
|
|
unsigned short int score;
|
2008-11-02 13:37:28 +00:00
|
|
|
char status; /* M C R A D U etc. (see Documentation/diff-format.txt or DIFF_STATUS_* in diff.h) */
|
[PATCH] Add -B flag to diff-* brothers.
A new diffcore transformation, diffcore-break.c, is introduced.
When the -B flag is given, a patch that represents a complete
rewrite is broken into a deletion followed by a creation. This
makes it easier to review such a complete rewrite patch.
The -B flag takes the same syntax as the -M and -C flags to
specify the minimum amount of non-source material the resulting
file needs to have to be considered a complete rewrite, and
defaults to 99% if not specified.
As the new test t4008-diff-break-rewrite.sh demonstrates, if a
file is a complete rewrite, it is broken into a delete/create
pair, which can further be subjected to the usual rename
detection if -M or -C is used. For example, if file0 gets
completely rewritten to make it as if it were rather based on
file1 which itself disappeared, the following happens:
The original change looks like this:
file0 --> file0' (quite different from file0)
file1 --> /dev/null
After diffcore-break runs, it would become this:
file0 --> /dev/null
/dev/null --> file0'
file1 --> /dev/null
Then diffcore-rename matches them up:
file1 --> file0'
The internal score values are finer grained now. Earlier
maximum of 10000 has been raised to 60000; there is no user
visible changes but there is no reason to waste available bits.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-30 07:08:37 +00:00
|
|
|
unsigned broken_pair : 1;
|
2006-08-03 19:01:01 +00:00
|
|
|
unsigned renamed_pair : 1;
|
2007-01-05 09:25:18 +00:00
|
|
|
unsigned is_unmerged : 1;
|
2014-01-25 06:46:50 +00:00
|
|
|
unsigned done_skip_stat_unmatch : 1;
|
|
|
|
unsigned skip_stat_unmatch_result : 1;
|
2005-05-21 09:39:09 +00:00
|
|
|
};
|
2019-11-17 21:04:40 +00:00
|
|
|
|
2007-01-05 09:25:18 +00:00
|
|
|
#define DIFF_PAIR_UNMERGED(p) ((p)->is_unmerged)
|
2005-05-21 09:39:09 +00:00
|
|
|
|
2006-08-03 19:01:01 +00:00
|
|
|
#define DIFF_PAIR_RENAME(p) ((p)->renamed_pair)
|
2005-05-29 23:56:48 +00:00
|
|
|
|
[PATCH] Add -B flag to diff-* brothers.
A new diffcore transformation, diffcore-break.c, is introduced.
When the -B flag is given, a patch that represents a complete
rewrite is broken into a deletion followed by a creation. This
makes it easier to review such a complete rewrite patch.
The -B flag takes the same syntax as the -M and -C flags to
specify the minimum amount of non-source material the resulting
file needs to have to be considered a complete rewrite, and
defaults to 99% if not specified.
As the new test t4008-diff-break-rewrite.sh demonstrates, if a
file is a complete rewrite, it is broken into a delete/create
pair, which can further be subjected to the usual rename
detection if -M or -C is used. For example, if file0 gets
completely rewritten to make it as if it were rather based on
file1 which itself disappeared, the following happens:
The original change looks like this:
file0 --> file0' (quite different from file0)
file1 --> /dev/null
After diffcore-break runs, it would become this:
file0 --> /dev/null
/dev/null --> file0'
file1 --> /dev/null
Then diffcore-rename matches them up:
file1 --> file0'
The internal score values are finer grained now. Earlier
maximum of 10000 has been raised to 60000; there is no user
visible changes but there is no reason to waste available bits.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-30 07:08:37 +00:00
|
|
|
#define DIFF_PAIR_BROKEN(p) \
|
|
|
|
( (!DIFF_FILE_VALID((p)->one) != !DIFF_FILE_VALID((p)->two)) && \
|
|
|
|
((p)->broken_pair != 0) )
|
|
|
|
|
2005-05-25 22:07:08 +00:00
|
|
|
#define DIFF_PAIR_TYPE_CHANGED(p) \
|
|
|
|
((S_IFMT & (p)->one->mode) != (S_IFMT & (p)->two->mode))
|
|
|
|
|
2005-05-26 09:24:30 +00:00
|
|
|
#define DIFF_PAIR_MODE_CHANGED(p) ((p)->one->mode != (p)->two->mode)
|
|
|
|
|
2018-06-30 09:20:25 +00:00
|
|
|
void diff_free_filepair(struct diff_filepair *);
|
2021-07-30 11:47:41 +00:00
|
|
|
void pool_diff_free_filepair(struct mem_pool *pool,
|
|
|
|
struct diff_filepair *p);
|
2005-05-27 22:50:30 +00:00
|
|
|
|
2018-06-30 09:20:25 +00:00
|
|
|
int diff_unmodified_pair(struct diff_filepair *);
|
[PATCH] Rename/copy detection fix.
The rename/copy detection logic in earlier round was only good
enough to show patch output and discussion on the mailing list
about the diff-raw format updates revealed many problems with
it. This patch fixes all the ones known to me, without making
things I want to do later impossible, mostly related to patch
reordering.
(1) Earlier rename/copy detector determined which one is rename
and which one is copy too early, which made it impossible
to later introduce diffcore transformers to reorder
patches. This patch fixes it by moving that logic to the
very end of the processing.
(2) Earlier output routine diff_flush() was pruning all the
"no-change" entries indiscriminatingly. This was done due
to my false assumption that one of the requirements in the
diff-raw output was not to show such an entry (which
resulted in my incorrect comment about "diff-helper never
being able to be equivalent to built-in diff driver"). My
special thanks go to Linus for correcting me about this.
When we produce diff-raw output, for the downstream to be
able to tell renames from copies, sometimes it _is_
necessary to output "no-change" entries, and this patch
adds diffcore_prune() function for doing it.
(3) Earlier diff_filepair structure was trying to be not too
specific about rename/copy operations, but the purpose of
the structure was to record one or two paths, which _was_
indeed about rename/copy. This patch discards xfrm_msg
field which was trying to be generic for this wrong reason,
and introduces a couple of fields (rename_score and
rename_rank) that are explicitly specific to rename/copy
logic. One thing to note is that the information in a
single diff_filepair structure _still_ does not distinguish
renames from copies, and it is deliberately so. This is to
allow patches to be reordered in later stages.
(4) This patch also adds some tests about diff-raw format
output and makes sure that necessary "no-change" entries
appear on the output.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-23 04:26:09 +00:00
|
|
|
|
2019-11-17 21:04:40 +00:00
|
|
|
/**
|
|
|
|
* This is a collection of filepairs. Notable members are:
|
|
|
|
*
|
|
|
|
* - `queue`:
|
|
|
|
* An array of pointers to `struct diff_filepair`. This dynamically grows as
|
|
|
|
* you add filepairs;
|
|
|
|
*
|
|
|
|
* - `alloc`:
|
|
|
|
* The allocated size of the `queue` array;
|
|
|
|
*
|
|
|
|
* - `nr`:
|
|
|
|
* The number of elements in the `queue` array.
|
|
|
|
*/
|
2005-05-21 09:39:09 +00:00
|
|
|
struct diff_queue_struct {
|
2005-05-21 09:40:01 +00:00
|
|
|
struct diff_filepair **queue;
|
2005-05-21 09:39:09 +00:00
|
|
|
int alloc;
|
|
|
|
int nr;
|
|
|
|
};
|
2019-11-17 21:04:40 +00:00
|
|
|
|
2010-05-07 04:52:27 +00:00
|
|
|
#define DIFF_QUEUE_CLEAR(q) \
|
|
|
|
do { \
|
|
|
|
(q)->queue = NULL; \
|
|
|
|
(q)->nr = (q)->alloc = 0; \
|
2010-08-12 22:11:15 +00:00
|
|
|
} while (0)
|
2005-05-21 09:39:09 +00:00
|
|
|
|
2005-05-22 02:40:36 +00:00
|
|
|
extern struct diff_queue_struct diff_queued_diff;
|
2018-06-30 09:20:25 +00:00
|
|
|
struct diff_filepair *diff_queue(struct diff_queue_struct *,
|
|
|
|
struct diff_filespec *,
|
|
|
|
struct diff_filespec *);
|
|
|
|
void diff_q(struct diff_queue_struct *, struct diff_filepair *);
|
2022-11-02 22:01:40 +00:00
|
|
|
void diff_free_queue(struct diff_queue_struct *q);
|
[PATCH] Rename/copy detection fix.
The rename/copy detection logic in earlier round was only good
enough to show patch output and discussion on the mailing list
about the diff-raw format updates revealed many problems with
it. This patch fixes all the ones known to me, without making
things I want to do later impossible, mostly related to patch
reordering.
(1) Earlier rename/copy detector determined which one is rename
and which one is copy too early, which made it impossible
to later introduce diffcore transformers to reorder
patches. This patch fixes it by moving that logic to the
very end of the processing.
(2) Earlier output routine diff_flush() was pruning all the
"no-change" entries indiscriminatingly. This was done due
to my false assumption that one of the requirements in the
diff-raw output was not to show such an entry (which
resulted in my incorrect comment about "diff-helper never
being able to be equivalent to built-in diff driver"). My
special thanks go to Linus for correcting me about this.
When we produce diff-raw output, for the downstream to be
able to tell renames from copies, sometimes it _is_
necessary to output "no-change" entries, and this patch
adds diffcore_prune() function for doing it.
(3) Earlier diff_filepair structure was trying to be not too
specific about rename/copy operations, but the purpose of
the structure was to record one or two paths, which _was_
indeed about rename/copy. This patch discards xfrm_msg
field which was trying to be generic for this wrong reason,
and introduces a couple of fields (rename_score and
rename_rank) that are explicitly specific to rename/copy
logic. One thing to note is that the information in a
single diff_filepair structure _still_ does not distinguish
renames from copies, and it is deliberately so. This is to
allow patches to be reordered in later stages.
(4) This patch also adds some tests about diff-raw format
output and makes sure that necessary "no-change" entries
appear on the output.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-05-23 04:26:09 +00:00
|
|
|
|
2021-03-13 22:22:03 +00:00
|
|
|
/* dir_rename_relevance: the reason we want rename information for a dir */
|
|
|
|
enum dir_rename_relevance {
|
|
|
|
NOT_RELEVANT = 0,
|
|
|
|
RELEVANT_FOR_ANCESTOR = 1,
|
|
|
|
RELEVANT_FOR_SELF = 2
|
|
|
|
};
|
2021-03-13 22:22:07 +00:00
|
|
|
/* file_rename_relevance: the reason(s) we want rename information for a file */
|
|
|
|
enum file_rename_relevance {
|
|
|
|
RELEVANT_NO_MORE = 0, /* i.e. NOT relevant */
|
|
|
|
RELEVANT_CONTENT = 1,
|
|
|
|
RELEVANT_LOCATION = 2
|
|
|
|
};
|
2021-03-13 22:22:03 +00:00
|
|
|
|
2021-02-27 00:30:43 +00:00
|
|
|
void partial_clear_dir_rename_count(struct strmap *dir_rename_count);
|
|
|
|
|
2018-09-21 15:57:19 +00:00
|
|
|
void diffcore_break(struct repository *, int);
|
2018-06-30 09:20:25 +00:00
|
|
|
void diffcore_rename(struct diff_options *);
|
2021-02-27 00:30:42 +00:00
|
|
|
void diffcore_rename_extended(struct diff_options *options,
|
2021-07-30 11:47:42 +00:00
|
|
|
struct mem_pool *pool,
|
2021-03-13 22:22:02 +00:00
|
|
|
struct strintmap *relevant_sources,
|
|
|
|
struct strintmap *dirs_removed,
|
merge-ort, diffcore-rename: employ cached renames when possible
When there are many renames between the old base of a series of commits
and the new base, the way sequencer.c, merge-recursive.c, and
diffcore-rename.c have traditionally split the work resulted in
redetecting the same renames with each and every commit being
transplanted. To address this, the last several commits have been
creating a cache of rename detection results, determining when it was
safe to use such a cache in subsequent merge operations, adding helper
functions, and so on. See the previous half dozen commit messages for
additional discussion of this optimization, particularly the message a
few commits ago entitled "add code to check for whether cached renames
can be reused". This commit finally ties all of that work together,
modifying the merge algorithm to make use of these cached renames.
For the testcases mentioned in commit 557ac0350d ("merge-ort: begin
performance work; instrument with trace2_region_* calls", 2020-10-28),
this change improves the performance as follows:
Before After
no-renames: 5.665 s ± 0.129 s 5.622 s ± 0.059 s
mega-renames: 11.435 s ± 0.158 s 10.127 s ± 0.073 s
just-one-mega: 494.2 ms ± 6.1 ms 500.3 ms ± 3.8 ms
That's a fairly small improvement, but mostly because the previous
optimizations were so effective for these particular testcases; this
optimization only kicks in when the others don't. If we undid the
basename-guided rename detection and skip-irrelevant-renames
optimizations, then we'd see that this series by itself improved
performance as follows:
Before Basename Series After Just This Series
no-renames: 13.815 s ± 0.062 s 5.697 s ± 0.080 s
mega-renames: 1799.937 s ± 0.493 s 205.709 s ± 0.457 s
Since this optimization kicks in to help accelerate cases where the
previous optimizations do not apply, this last comparison shows that
this cached-renames optimization has the potential to help signficantly
in cases that don't meet the requirements for the other optimizations to
be effective.
The changes made in this optimization also lay some important groundwork
for a future optimization around having collect_merge_info() avoid
recursing into subtrees in more cases.
However, for this optimization to be effective, merge_switch_to_result()
should only be called when the rebase or cherry-pick operation has
either completed or hit a case where the user needs to resolve a
conflict or edit the result. If it is called after every commit, as
sequencer.c does, then the working tree and index are needlessly updated
with every commit and the cached metadata is tossed, defeating this
optimization. Refactoring sequencer.c to only call
merge_switch_to_result() at the end of the operation is a bigger
undertaking, and the practical benefits of this optimization will not be
realized until that work is performed. Since `test-tool fast-rebase`
only updates at the end of the operation, it was used to obtain the
timings above.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-20 06:09:41 +00:00
|
|
|
struct strmap *dir_rename_count,
|
|
|
|
struct strmap *cached_pairs);
|
2018-06-30 09:20:25 +00:00
|
|
|
void diffcore_merge_broken(void);
|
|
|
|
void diffcore_pickaxe(struct diff_options *);
|
|
|
|
void diffcore_order(const char *orderfile);
|
diff: --{rotate,skip}-to=<path>
In the implementation of "git difftool", there is a case where the
user wants to start viewing the diffs at a specific path and
continue on to the rest, optionally wrapping around to the
beginning. Since it is somewhat cumbersome to implement such a
feature as a post-processing step of "git diff" output, let's
support it internally with two new options.
- "git diff --rotate-to=C", when the resulting patch would show
paths A B C D E without the option, would "rotate" the paths to
shows patch to C D E A B instead. It is an error when there is
no patch for C is shown.
- "git diff --skip-to=C" would instead "skip" the paths before C,
and shows patch to C D E. Again, it is an error when there is no
patch for C is shown.
- "git log [-p]" also accepts these two options, but it is not an
error if there is no change to the specified path. Instead, the
set of output paths are rotated or skipped to the specified path
or the first path that sorts after the specified path.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-02-11 19:57:50 +00:00
|
|
|
void diffcore_rotate(struct diff_options *);
|
2005-06-03 08:36:43 +00:00
|
|
|
|
2014-01-20 16:20:38 +00:00
|
|
|
/* low-level interface to diffcore_order */
|
|
|
|
struct obj_order {
|
|
|
|
void *obj; /* setup by caller */
|
|
|
|
|
|
|
|
/* setup/used by order_objects() */
|
|
|
|
int orig_order;
|
|
|
|
int order;
|
|
|
|
};
|
|
|
|
|
|
|
|
typedef const char *(*obj_path_fn_t)(void *obj);
|
|
|
|
|
|
|
|
void order_objects(const char *orderfile, obj_path_fn_t obj_path,
|
|
|
|
struct obj_order *objs, int nr);
|
|
|
|
|
2005-05-24 08:10:48 +00:00
|
|
|
#define DIFF_DEBUG 0
|
|
|
|
#if DIFF_DEBUG
|
|
|
|
void diff_debug_filespec(struct diff_filespec *, int, const char *);
|
|
|
|
void diff_debug_filepair(const struct diff_filepair *, int);
|
|
|
|
void diff_debug_queue(const char *, struct diff_queue_struct *);
|
|
|
|
#else
|
2010-08-12 22:11:15 +00:00
|
|
|
#define diff_debug_filespec(a,b,c) do { /* nothing */ } while (0)
|
|
|
|
#define diff_debug_filepair(a,b) do { /* nothing */ } while (0)
|
|
|
|
#define diff_debug_queue(a,b) do { /* nothing */ } while (0)
|
2005-05-24 08:10:48 +00:00
|
|
|
#endif
|
|
|
|
|
2018-09-21 15:57:19 +00:00
|
|
|
int diffcore_count_changes(struct repository *r,
|
|
|
|
struct diff_filespec *src,
|
2018-06-30 09:20:25 +00:00
|
|
|
struct diff_filespec *dst,
|
|
|
|
void **src_count_p,
|
|
|
|
void **dst_count_p,
|
|
|
|
unsigned long *src_copied,
|
|
|
|
unsigned long *literal_added);
|
2006-03-01 00:01:36 +00:00
|
|
|
|
diff: restrict when prefetching occurs
Commit 7fbbcb21b1 ("diff: batch fetching of missing blobs", 2019-04-08)
optimized "diff" by prefetching blobs in a partial clone, but there are
some cases wherein blobs do not need to be prefetched. In these cases,
any command that uses the diff machinery will unnecessarily fetch blobs.
diffcore_std() may read blobs when it calls the following functions:
(1) diffcore_skip_stat_unmatch() (controlled by the config variable
diff.autorefreshindex)
(2) diffcore_break() and diffcore_merge_broken() (for break-rewrite
detection)
(3) diffcore_rename() (for rename detection)
(4) diffcore_pickaxe() (for detecting addition/deletion of specified
string)
Instead of always prefetching blobs, teach diffcore_skip_stat_unmatch(),
diffcore_break(), and diffcore_rename() to prefetch blobs upon the first
read of a missing object. This covers (1), (2), and (3): to cover the
rest, teach diffcore_std() to prefetch if the output type is one that
includes blob data (and hence blob data will be required later anyway),
or if it knows that (4) will be run.
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-04-07 22:11:43 +00:00
|
|
|
/*
|
|
|
|
* If filespec contains an OID and if that object is missing from the given
|
|
|
|
* repository, add that OID to to_fetch.
|
|
|
|
*/
|
|
|
|
void diff_add_if_missing(struct repository *r,
|
|
|
|
struct oid_array *to_fetch,
|
|
|
|
const struct diff_filespec *filespec);
|
|
|
|
|
2005-05-21 09:39:09 +00:00
|
|
|
#endif
|