read-tree -m can read up to MAX_TREES, which was arbitrarily set to 8 since
August 2007 (4 is needed to deal with 2 merge-base case).
However, the updated unpack_trees() code had an advertised limit of 4
(which it enforced). In reality the code was prepared to take only 3
trees and giving 4 caused it to stomp on its stack. Rename the MAX_TREES
constant to MAX_UNPACK_TREES, move it to the unpack-trees.h common header
file, and use it from both places to avoid future confusion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When skip_unmerged option is not given, unpack_trees() should not just
skip unmerged cache entries but keep them in the result for the caller to
sort them out.
For callers other than diff-index, the incoming index should never be
unmerged, but diff-index is a special case caller.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There were few places where merge errors detected deeper in the call chain
were ignored and not propagated up the callchain to the caller.
Most notably, this caused switching branches with "git checkout" to ignore
a path modified in a work tree are different between the HEAD version and
the commit being switched to, which it internally notices but ignores it,
resulting in an incorrect two-way merge and loss of the change in the work
tree.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This adds a "discard_index(&o->result)" to the failure path, to reclaim
memory from an in-core index we built but ended up not using.
The *big* memory leak comes from the fact that we leak the cache_entry
things left and right. That's a very traditional and deliberate leak:
because we used to build up the cache entries by just mapping them
directly in from the index file (and we emulate that in modern times
by allocating them from one big array), we can't actually free them
one-by-one.
So doing the "discard_index()" will free the hash tables etc, which is
good, and it will free the "istate->alloc" but that is never set on the
result because we don't get the result from the index read. So we don't
actually free the individual cache entries themselves that got created
from the trees.
That's not something new, btw. We never did. But some day we should just
add a flag to the cache_entry() that it's a "free one by one" kind, and
then we could/should do it. In the meantime, this one-liner will fix
*some* of the memory leaks, but not that old traditional one.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We will always unpack into our own internal index, but we will take the
source from wherever specified, and we will optionally write the result
to a specified index (optionally, because not everybody even _wants_ any
result: the index diffing really wants to just walk the tree and index
in parallel).
This ends up removing a fair number more lines than it adds, for the
simple reason that we can now skip all the crud that tried to be
oh-so-careful about maintaining our position in the index as we were
traversing and modifying it. Since we don't actually modify the source
index any more, we can just update the 'o->pos' pointer without worrying
about whether an index entry got removed or replaced or added to.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is just a very mechanical conversion, and makes everybody set it to
'&the_index' before calling, but at least it makes it more explicit
where we work with the index.
The next stage would be to split that index usage up into a 'source' and
a 'destination' index, so that we can unpack into a different index than
we started out from.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This not only deletes more code than it adds, it gets rid of a
singularly hard-to-understand function (unpack_trees_rec()), and
replaces it with a set of smaller and simpler functions that use the
generic tree traversal mechanism to walk over one or more git trees in
parallel.
It's still not the most wonderful interface, and by no means is the new
code easy to understand either, but it's at least a bit less opaque.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* db/checkout: (21 commits)
checkout: error out when index is unmerged even with -m
checkout: show progress when checkout takes long time while switching branches
Add merge-subtree back
checkout: updates to tracking report
builtin-checkout.c: Remove unused prefix arguments in switch_branches path
checkout: work from a subdirectory
checkout: tone down the "forked status" diagnostic messages
Clean up reporting differences on branch switch
builtin-checkout.c: fix possible usage segfault
checkout: notice when the switched branch is behind or forked
Build in checkout
Move code to clean up after a branch change to branch.c
Library function to check for unmerged index entries
Use diff -u instead of diff in t7201
Move create_branch into a library file
Build-in merge-recursive
Add "skip_unmerged" option to unpack_trees.
Discard "deleted" cache entries after using them to update the working tree
Send unpack-trees debugging output to stderr
Add flag to make unpack_trees() not print errors.
...
Conflicts:
Makefile
So I find it irritating when git thinks for a long time without telling me
what's taking so long. And by "long time" I definitely mean less than two
seconds, which is already way too long for me.
This hits me when doing a large pull and the checkout takes a long time,
or when just switching to another branch that is old and again checkout
takes a while.
Now, git read-tree already had support for the "-v" flag that does nice
updates about what's going on, but it was delayed by two seconds, and if
the thing had already done more than half by then it would be quiet even
after that, so in practice it meant that we migth be quiet for up to four
seconds. Much too long.
So this patch changes the timeout to just one second, which makes it much
more palatable to me.
The other thing this patch does is that "git checkout" now doesn't disable
the "-v" flag when doing its thing, and only disables the output when
given the -q flag. When allowing "checkout -m" to fall back to a 3-way
merge, the users will see the error message from straight "checkout",
so we will tell them that we do fall back to make them look less scary.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used to just memcpy() the index entry when we copied the stat() and
SHA1 hash information, which worked well enough back when the index
entry was just an exact bit-for-bit representation of the information on
disk.
However, these days we actually have various management information in
the cache entry too, and we should be careful to not overwrite it when
we copy the stat information from another index entry.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This option allows the caller to reset everything that isn't unmerged,
leaving the unmerged things to be resolved. If, after a merge of
"working" and "HEAD", this is used with "HEAD" (reset, !update), the
result will be that all of the changes from "local" are in the working
tree but not added to the index (either with the index clean but
unchanged, or with the index unmerged, depending on whether there are
conflicts).
This will be used in checkout -m.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Way back in read-tree.c, we used a mode 0 cache entry to indicate that
an entry had been deleted, so that the update code would remove the
working tree file, and we would just skip it when writing out the
index file afterward.
These days, unpack_trees is a library function, and it is still
leaving these entries in the active cache. Furthermore, unpack_trees
doesn't correctly ignore those entries, and who knows what other code
wouldn't expect them to be there, but just isn't yet called after a
call to unpack_trees. To avoid having other code trip over these
entries, have check_updates() remove them after it removes the working
tree files.
While we're at it, simplify the loop in check_updates(), and avoid
passing global variables as parameters to check_updates(): there is
only one call site anyway.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
(This applies only to errors where a plausible operation is impossible due
to the particular data, not to errors resulting from misuse of the merge
functions.)
This will allow builtin-checkout to suppress merge errors if it's
going to try more merging methods.
Additionally, if unpack_trees() returns with an error, but without
printing anything, it will roll back any changes to the index (by
rereading the index, currently). This obviously could be done by the
caller, but chances are that the caller would forget and debugging
this is difficult. Also, future implementations may give unpack_trees() a
more efficient way of undoing its changes than the caller could.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
Return an error from unpack_trees() instead of calling die(), and exit
with an error in read-tree, builtin-commit, and diff-lib. merge-recursive
already expected an error return from unpack_trees, so it doesn't need to
be changed. The merge function can return negative to abort.
This will be used in builtin-checkout -m.
Signed-off-by: Daniel Barkalow <barkalow@iabervon.org>
When we process "foo/" entries in gitignore files on a system
that does not have d_type member in "struct dirent", the earlier
implementation ran lstat(2) separately when matching with
entries that came from the command line, in-tree .gitignore
files, and $GIT_DIR/info/excludes file.
This optimizes it by delaying the lstat(2) call until it becomes
absolutely necessary.
The initial idea for this change was by Jeff King, but I
optimized it further to pass pointers to around.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A pattern "foo/" in the exclude list did not match directory
"foo", but a pattern "foo" did. This attempts to extend the
exclude mechanism so that it would while not matching a regular
file or a symbolic link "foo". In order to differentiate a
directory and non directory, this passes down the type of path
being checked to excluded() function.
A downside is that the recursive directory walk may need to run
lstat(2) more often on systems whose "struct dirent" do not give
the type of the entry; earlier it did not have to do so for an
excluded path, but we now need to figure out if a path is a
directory before deciding to exclude it. This is especially bad
because an idea similar to the earlier CE_UPTODATE optimization
to reduce number of lstat(2) calls would by definition not apply
to the codepaths involved, as (1) directories will not be
registered in the index, and (2) excluded paths will not be in
the index anyway.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This converts the index explicitly on read and write to its on-disk
format, allowing the in-core format to contain more flags, and be
simpler.
In particular, the in-core format is now host-endian (as opposed to the
on-disk one that is network endian in order to be able to be shared
across machines) and as a result we can dispense with all the
htonl/ntohl on accesses to the cache_entry fields.
This will make it easier to make use of various temporary flags that do
not exist in the on-disk format.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
In unpack-trees.c (line 593), we do
..
if (same(old, merge)) {
*merge = *old;
} else {
..
and that "merge" is a cache_entry pointer. If we have a non-zero
FLEX_ARRAY size, it will cause us to copy the first few bytes of the
name too.
That is technically wrong even for FLEX_ARRAY being 1, but you'll never
notice, since the filenames should always be the same with the current
code. But if we do the same thing for a rename, we'd be screwed.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Operations that walk directories or trees, which potentially need to
consult the .gitignore files, used to always try to open the .gitignore
file every time they entered a new directory, even when they ended up
not needing to call excluded() function to see if a path in the
directory is ignored. This was done by push/pop exclude_per_directory()
functions that managed the data in a stack.
This changes the directory walking API to remove the need to call these
two functions. Instead, the directory walk data structure caches the
data used by excluded() function the last time, and lazily reuses it as
much as possible. Among the data the last check used, the ones from
deeper directories that the path we are checking is outside are
discarded, data from the common leading directories are reused, and then
the directories between the common directory and the directory the path
being checked is in are checked for .gitignore file. This is very
similar to the way gitattributes are handled.
This API change also fixes "ls-files -c -i", which called excluded()
without setting up the gitignore data via the old push/pop functions.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/maint-add-sync-stat:
t2200: test more cases of "add -u"
git-add: make the entry stat-clean after re-adding the same contents
ce_match_stat, run_diff_files: use symbolic constants for readability
Conflicts:
builtin-add.c
ce_match_stat() can be told:
(1) to ignore CE_VALID bit (used under "assume unchanged" mode)
and perform the stat comparison anyway;
(2) not to perform the contents comparison for racily clean
entries and report mismatch of cached stat information;
using its "option" parameter. Give them symbolic constants.
Similarly, run_diff_files() can be told not to report anything
on removed paths. Also give it a symbolic constant for that.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since it is now OK to pass a null pointer to display_progress() and
stop_progress() resulting in a no-op, then we can simplify the code
and remove a bunch of lines by not making those calls conditional all
the time.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This allows for better management of progress "object" existence,
as well as making the progress display implementation more independent
from its callers.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Each progress can be on a single line instead of two.
[sp: Changed "Checking files out" to "Checking out files" at
Johannes Sixt's suggestion as it better explains the
action that is taking place]
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
As mentioned, the three-way case *should* be as trivial as the
following. It passes all the tests, and I verified that a conflicting
merge in the 100,000 file horror-case merged correctly (with the conflict
markers) in 0.687 seconds with this, so it works, but I'm lazy and
somebody else should double-check it [jc: followed all three-way merge
codepaths and verified it removes when it should].
Without this patch, the merge took 8.355 seconds, so this patch
really does make a huge difference for merge performance with lots and
lots of files, and we're not talking percentages, we're talking
orders-of-magnitude differences!
Now "unpack_trees()" is just fast enough that we don't need to avoid it
(although it's probably still a good idea to eventually convert it to use
the traverse_trees() infrastructure some day - just to avoid having
extraneous tree traversal functions).
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This trivially optimizes the two-way merge case of git-read-tree too,
which affects switching branches.
When you have tons and tons of files in your repository, but there are
only small differences in the branches (maybe just a couple of files
changed), the biggest cost of the branch switching was actually just the
index calculations.
This fixes it (timings for switching between the "testing" and "master"
branches in the 100,000 file testing-repo-from-hell, where the branches
only differ in one small file).
Before:
[torvalds@woody bummer]$ time git checkout master
real 0m9.919s
user 0m8.461s
sys 0m0.264s
After:
[torvalds@woody bummer]$ time git checkout testing
real 0m0.576s
user 0m0.348s
sys 0m0.228s
so it's easily an order of magnitude different.
This concludes the series. I think we could/should do the three-way merge
too (to speed up merges), but I'm lazy. Somebody else can do it.
The rule is very simple: you need to remove the old entry if:
- you want to remove the file entirely
- you replace it with a "merge conflict" entry (ie a non-stage-0 entry)
and you can avoid removing it if you either
- keep the old one
- or resolve it to a new one.
and these rules should all be valid for the three-way case too.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This optimizes bind_merge() and oneway_merge() to not unnecessarily
remove and re-add the old index entries when they can just get replaced
by updated ones.
This makes these operations much faster for large trees (where "large"
is in the 50,000+ file range), because we don't unnecessarily move index
entries around in the index array all the time.
Using the "bummer" tree (a test-tree with 100,000 files) we get:
Before:
[torvalds@woody bummer]$ time git commit -m"Change one file" 50/500
real 0m9.470s
user 0m8.729s
sys 0m0.476s
After:
[torvalds@woody bummer]$ time git commit -m"Change one file" 50/500
real 0m1.173s
user 0m0.720s
sys 0m0.452s
so for large trees this is easily very noticeable indeed.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This makes no changes to current code, but it allows the individual merge
functions to decide what to do about the old entry. They might decide to
update it in place, rather than force them to always delete and re-add it.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This doesn't actually change any real code, but it changes the interface
to unpack_trees() to take an array of "struct tree_desc" entries, the same
way the tree-walk.c functions do.
The reason for this is that we would be much better off if we can do the
tree-unpacking using the generic "traverse_trees()" functionality instead
of having to the special "unpack" infrastructure.
This really is a pretty minimal diff, just to change the calling
convention. It passes all the tests, and looks sane. There were only two
users of "unpack_trees()": builtin-read-tree and merge-recursive, and I
tried to keep the changes minimal.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sven originally raised this issue:
If you have a submodule checked out and you go back (or
forward) to a revision of the supermodule that contains a
different revision of the submodule and then switch to
another revision, it will complain that the submodule is not
uptodate, because git simply didn't update the submodule in
the first move.
The current policy is to consider it is perfectly normal that
checked-out submodule is out-of-sync wrt the supermodule index.
At least until we introduce a superproject repository
configuration option that says "in this repository, I do care
about this submodule and at any time I move around in the
superproject, recursively check out the submodule to match", it
is a reasonable policy, as we currently do not recursively
checkout the submodules at all. The most extreme case of this
policy is that the superproject index knows about the submodule
but the subdirectory does not even have to be checked out.
The function verify_uptodate(), called during the two-way merge
aka branch switching, is about "make sure the filesystem entity
that corresponds to this cache entry is up to date, lest we lose
the local modifications". As we explicitly allow submodule
checkout to drift from the supermodule index entry, the check
should say "Ok, for submodules, not matching is the norm" for
now.
Later when we have the ability to mark "I care about this
submodule to be always in sync with the superproject" (thereby
implementing automatic recursive checkout and perhaps diff,
among other things), we should check if the submodule in
question is marked as such and perform the current test.
Acked-by: Lars Hjemli <hjemli@gmail.com>
Acked-by: Sven Verdoolaege <skimo@kotnet.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Remove the two write-only fields executable and symlink from struct
tree_entry_list. Also replace usage of the field directory with
S_ISDIR checks on the mode field, and then remove this now obsolete
field, too. Noticed by David Kastrup.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In particular, when moving back to a commit without a given submodule
and then moving back forward to a commit with the given submodule,
we shouldn't complain that updating would lose untracked file in
the submodule, because git currently does not checkout subprojects
during superproject check-out.
Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Earlier in 16a4c61, we taught "read-tree -m -u" not to be
confused when switching from a branch that has a path frotz/filfre
to another branch that has a symlink frotz that points at xyzzy/
directory. The fix was incomplete in that it was still confused
when coming back (i.e. switching from a branch with frotz -> xyzzy/
to another branch with frotz/filfre).
This fix is rather expensive in that for a path that is created
we would need to see if any of the leading component of that
path exists as a symbolic link in the filesystem (in which case,
we know that path itself does not exist, and the fact we already
decided to check it out tells us that in the index we already
know that symbolic link is going away as there is no D/F
conflict).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* maint-1.5.1:
annotate: make it work from subdirectories.
git-config: Correct asciidoc documentation for --int/--bool
t1300: Add tests for git-config --bool --get
unpack-trees.c: verify_uptodate: remove dead code
Use PATH_MAX instead of TEMPFILE_PATH_LEN
branch: fix segfault when resolving an invalid HEAD
When switching from a branch with both x86_64/boot/Makefile and
i386/boot/Makefile to another branch that has x86_64/boot as a
symlink pointing at ../i386/boot, the code incorrectly removed
i386/boot/Makefile.
This was because we first removed everything under x86_64/boot
to make room to create a symbolic link x86_64/boot, then removed
x86_64/boot/Makefile which no longer exists but now is pointing
at i386/boot/Makefile, thanks to the symlink we just created.
This fixes it by using the has_symlink_leading_path() function
introduced previously for git-apply in the checkout codepath.
Earlier, "git checkout" was broken in t4122 test due to this
bug, and the test had an extra "git reset --hard" as a
workaround, which is removed because it is not needed anymore.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Let's start displaying progress only if more than 50% of total number
of files remains to be checked out after 2 seconds.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If the progress bar ends up in a box, better provide a title for it too.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Instead of having this code duplicated in multiple places, let's have
a common interface for progress display. If someday someone wishes to
display a cheezy progress bar instead then only one file will have to
be changed.
Note: I left merge-recursive.c out since it has a strange notion of
progress as it apparently increase the expected total number as it goes.
Someone with more intimate knowledge of what that is supposed to mean
might look at converting it to the common progress interface.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This fixes three buglets in threeway_merge() regarding D/F
conflict entries.
* After finishing with path D and handling path D/F, some stages
have D/F conflict entry which are obviously non-NULL. For the
purpose of determining if the path D/F is missing in the
ancestor, they should not be taken into account.
* D/F conflict entry is a marker to say "this stage does _not_
have the path", so do not send them to keep_entry().
Signed-off-by: Junio C Hamano <junkio@cox.net>
Case #10 is not handled with unpack-trees.c:threeway_merge()
internally, unless under the agressive rule, and it is not a
bug. As the test expects, ND (one side did not do anything,
other side deleted) case was meant to be handled by the caller's
policy (e.g. git-merge-one-file or git-merge-recursive).
Signed-off-by: Junio C Hamano <junkio@cox.net>
This loosens the over-eager verify_absent() check that gets
upset to find directory D in the current working tree when
switching to a branch that has a file there. The check needs to
make sure that we do not lose precious working tree files as a
result of removing directory D and replacing it with the file
from the other branch, which is a tad expensive but this is a
less common case.
Signed-off-by: Junio C Hamano <junkio@cox.net>
When switching from one tree to another, we should not send a
marker that says "this file does not exist in the new tree -- I
am a placeholder to tell you that, and not a real blob" down to
merged_entry() as the result of the merge.
This variable keeps track of which entry in the original index
the traversal is looking at, and belongs to the unpack_trees_options
structure along with other traversal status information.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Other decision functions, deleted_entry() and merged_entry() take one as
their parameter, and this function should. I'll be introducing a separate
index to build the result in, and am planning to pass it as the part of the
structure.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This removes slightly more lines than it adds, but the real reason for
doing this is that future optimizations will require more setup of the
tree descriptor, and so we want to do it in one place.
Also renamed the "desc.buf" field to "desc.buffer" just to trigger
compiler errors for old-style manual initializations, making sure I
didn't miss anything.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a mechanical clean-up of the way *.c files include
system header files.
(1) sources under compat/, platform sha-1 implementations, and
xdelta code are exempt from the following rules;
(2) the first #include must be "git-compat-util.h" or one of
our own header file that includes it first (e.g. config.h,
builtin.h, pkt-line.h);
(3) system headers that are included in "git-compat-util.h"
need not be included in individual C source files.
(4) "git-compat-util.h" does not have to include subsystem
specific header files (e.g. expat.h).
Signed-off-by: Junio C Hamano <junkio@cox.net>
* jc/read-tree-ignore:
read-tree: document --exclude-per-directory
Loosen "working file will be lost" check in Porcelain-ish
read-tree: further loosen "working file will be lost" check.
