Commit graph

52358 commits

Author SHA1 Message Date
brian m. carlson 2770ccbdb2 commit: express tree entry constants in terms of the_hash_algo
Specify these constants in terms of the size of the hash algorithm
currently in use.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 14:27:39 -07:00
brian m. carlson d9cd734990 hex: switch to using the_hash_algo
Instead of using the GIT_SHA1_* constants, switch to using the_hash_algo
to convert object IDs to and from hex format.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 14:27:39 -07:00
brian m. carlson 83e4b7571c tree-walk: replace hard-coded constants with the_hash_algo
Remove the hard-coded 20-based values and replace them with uses of
the_hash_algo.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 14:27:39 -07:00
brian m. carlson 509f6f62a4 cache: update object ID functions for the_hash_algo
Most of our code has been converted to use struct object_id for object
IDs.  However, there are some places that still have not, and there are
a variety of places that compare equivalently sized hashes that are not
object IDs.  All of these hashes are artifacts of the internal hash
algorithm in use, and when we switch to NewHash for object storage, all
of these uses will also switch.

Update the hashcpy, hashclr, and hashcmp functions to use the_hash_algo,
since they are used in a variety of places to copy and manipulate
buffers that need to move data into or out of struct object_id.  This
has the effect of making the corresponding oid* functions use
the_hash_algo as well.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 14:27:38 -07:00
Jeff King 022d2ac1f3 blame: prefer xsnprintf to strcpy for colors
Our color buffers are all COLOR_MAXLEN, which fits the
largest possible color. So we can never overflow the buffer
by copying an existing color. However, using strcpy() makes
it harder to audit the code-base for calls that _are_
problems. We should use something like xsnprintf(), which
shows the reader that we expect this never to fail (and
provides a run-time assertion if it does, just in case).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 13:59:39 -07:00
Jonathan Tan 8c4cc32689 tag: don't warn if target is missing but promised
deref_tag() prints a warning if the object that a tag refers to does not
exist. However, when a partial clone has an annotated tag from its
promisor remote, but not the object that it refers to, printing a
warning on such a tag is incorrect.

This occurs, for example, when the checkout that happens after a partial
clone causes some objects to be fetched - and as part of the fetch, all
local refs are read. The test included in this patch demonstrates this
situation.

Therefore, do not print a warning in this case.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 12:56:14 -07:00
Jonathan Tan dc0a13f681 revision: tolerate promised targets of tags
In handle_commit(), it is fatal for an annotated tag to point to a
non-existent object. --exclude-promisor-objects should relax this rule
and allow non-existent objects that are promisor objects, but this is
not the case. Update handle_commit() to tolerate this situation.

This was observed when cloning from a repository with an annotated tag
pointing to a blob. The test included in this patch demonstrates this
case.

Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 12:56:14 -07:00
brian m. carlson ab5e67d751 sequencer: pass absolute GIT_WORK_TREE to exec commands
The sequencer currently passes GIT_DIR, but not GIT_WORK_TREE, to exec
commands.  In that configuration, we assume that whatever directory
we're in is the top level of the work tree, and git rev-parse
--show-toplevel responds accordingly.  However, when we're in a
subdirectory, that isn't correct: we respond with the subdirectory as
the top level, resulting in unexpected behavior.

Ensure that we pass GIT_WORK_TREE as well as GIT_DIR so that git
operations within subdirectories work correctly.

Note that we are guaranteed to have a work tree in this case: the
relevant sequencer functions are called only from revert, cherry-pick,
and rebase--helper; all of these commands require a working tree.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 11:16:45 -07:00
Aaron Schrab 02cfd14920 sequencer: use configured comment character
Use the configured comment character when generating comments about
branches in a todo list.  Failure to honor this configuration causes a
failure to parse the resulting todo list.

Setting core.commentChar to "auto" will not be honored here, and the
previously configured or default value will be used instead. But, since
the todo list will consist of only generated content, there should not
be any non-comment lines beginning with that character.

Signed-off-by: Aaron Schrab <aaron@schrab.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 11:04:56 -07:00
Jeff King 64eb14d310 fsck: downgrade gitmodulesParse default to "info"
We added an fsck check in ed8b10f631 (fsck: check
.gitmodules content, 2018-05-02) as a defense against the
vulnerability from 0383bbb901 (submodule-config: verify
submodule names as paths, 2018-04-30). With the idea that
up-to-date hosting sites could protect downstream unpatched
clients that fetch from them.

As part of that defense, we reject any ".gitmodules" entry
that is not syntactically valid. The theory is that if we
cannot even parse the file, we cannot accurately check it
for vulnerabilities. And anybody with a broken .gitmodules
file would eventually want to know anyway.

But there are a few reasons this is a bad tradeoff in
practice:

 - for this particular vulnerability, the client has to be
   able to parse the file. So you cannot sneak an attack
   through using a broken file, assuming the config parsers
   for the process running fsck and the eventual victim are
   functionally equivalent.

 - a broken .gitmodules file is not necessarily a problem.
   Our fsck check detects .gitmodules in _any_ tree, not
   just at the root. And the presence of a .gitmodules file
   does not necessarily mean it will be used; you'd have to
   also have gitlinks in the tree. The cgit repository, for
   example, has a file named .gitmodules from a
   pre-submodule attempt at sharing code, but does not
   actually have any gitlinks.

 - when the fsck check is used to reject a push, it's often
   hard to work around. The pusher may not have full control
   over the destination repository (e.g., if it's on a
   hosting server, they may need to contact the hosting
   site's support). And the broken .gitmodules may be too
   far back in history for rewriting to be feasible (again,
   this is an issue for cgit).

So we're being unnecessarily restrictive without actually
improving the security in a meaningful way. It would be more
convenient to downgrade this check to "info", which means
we'd still comment on it, but not reject a push. Site admins
can already do this via config, but we should ship sensible
defaults.

There are a few counterpoints to consider in favor of
keeping the check as an error:

 - the first point above assumes that the config parsers for
   the victim and the fsck process are equivalent. This is
   pretty true now, but as time goes on will become less so.
   Hosting sites are likely to upgrade their version of Git,
   whereas vulnerable clients will be stagnant (if they did
   upgrade, they'd cease to be vulnerable!). So in theory we
   may see drift over time between what two config parsers
   will accept.

   In practice, this is probably OK. The config format is
   pretty established at this point and shouldn't change a
   lot. And the farther we get from the announcement of the
   vulnerability, the less interesting this extra layer of
   protection becomes. I.e., it was _most_ valuable on day
   0, when everybody's client was still vulnerable and
   hosting sites could protect people. But as time goes on
   and people upgrade, the population of vulnerable clients
   becomes smaller and smaller.

 - In theory this could protect us from other
   vulnerabilities in the future. E.g., .gitmodules are the
   only way for a malicious repository to feed data to the
   config parser, so this check could similarly protect
   clients from a future (to-be-found) bug there.

   But that's trading a hypothetical case for real-world
   pain today. If we do find such a bug, the hosting site
   would need to be updated to fix it, too. At which point
   we could figure out whether it's possible to detect
   _just_ the malicious case without hurting existing
   broken-but-not-evil cases.

 - Until recently, we hadn't made any restrictions on
   .gitmodules content. So now in tightening that we're
   hitting cases where certain things used to work, but
   don't anymore. There's some moderate pain now. But as
   time goes on, we'll see more (and more varied) cases that
   will make tightening harder in the future. So there's
   some argument for putting rules in place _now_, before
   users grow more cases that violate them.

   Again, this is trading pain now for hypothetical benefit
   in the future. And if we try hard in the future to keep
   our tightening to a minimum (i.e., rejecting true
   maliciousness without hurting broken-but-not-evil repos),
   then that reduces even the hypothetical benefit.

Considering both sets of arguments, it makes sense to loosen
this check for now.

Note that we have to tweak the test in t7415 since fsck will
no longer consider this a fatal error. But we still check
that it reports the warning, and that we don't get the
spurious error from the config code.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 10:57:23 -07:00
Jeff King 0d68764d94 fsck: split ".gitmodules too large" error from parse failure
Since ed8b10f631 (fsck: check .gitmodules content,
2018-05-02), we'll report a gitmodulesParse error for two
conditions:

  - a .gitmodules entry is not syntactically valid

  - a .gitmodules entry is larger than core.bigFileThreshold

with the intent that we can detect malicious files and
protect downstream clients. E.g., from the issue in
0383bbb901 (submodule-config: verify submodule names as
paths, 2018-04-30).

But these conditions are actually quite different with
respect to that bug:

 - a syntactically invalid file cannot trigger the problem,
   as the victim would barf before hitting the problematic
   code

 - a too-big .gitmodules _can_ trigger the problem. Even
   though it is obviously silly to have a 500MB .gitmodules
   file, the submodule code will happily parse it if you
   have enough memory.

So it may be reasonable to configure their severity
separately. Let's add a new class for the "too large" case
to allow that.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 10:57:22 -07:00
Derrick Stolee b18ef13a3f coccinelle: update commit.cocci
A recent patch series renamed the get_commit_tree_from_graph method but
forgot to update the coccinelle script that exempted it from rules
regarding accesses to 'maybe_tree'. This fixes that oversight to bring
the coccinelle scripts back to a good state.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-16 10:54:31 -07:00
Junio C Hamano 650161a277 t3404: fix use of "VAR=VAL cmd" with a shell function
Bash may take it happily but running test with dash reveals a breakage.

This was not discovered for a long time as no tests after this test
depended on GIT_AUTHOR_NAME to be reverted correctly back to the
original value after this step is done.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 13:31:57 -07:00
Ben Peart 5a06a20e0c handle lower case drive letters on Windows
On Windows, if a tool calls SetCurrentDirectory with a lower case drive
letter, the subsequent call to GetCurrentDirectory will return the same
lower case drive letter. Powershell, for example, does not normalize the
path. If that happens, test-drop-caches will error out as it does not
correctly to handle lower case drive letters.

Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 12:11:05 -07:00
William Chargin 6b3351e799 sha1-name.c: for ":/", find detached HEAD commits
This patch broadens the set of commits matched by ":/<pattern>" to
include commits reachable from HEAD but not any named ref. This avoids
surprising behavior when working with a detached HEAD and trying to
refer to a commit that was recently created and only exists within the
detached state.

If multiple worktrees exist, only the current worktree's HEAD is
considered reachable. This is consistent with the existing behavior for
other per-worktree refs: e.g., bisect refs are considered reachable, but
only within the relevant worktree.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: William Chargin <wchargin@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 12:07:25 -07:00
Ramsay Jones 6b82db9b42 t6036: fix broken && chain in sub-shell
Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 11:48:52 -07:00
SZEDER Gábor e8b3b2e275 t/lib-httpd: avoid occasional failures when checking access.log
The last test of 't5561-http-backend.sh', 'server request log matches
test results' may fail occasionally, because the order of entries in
Apache's access log doesn't match the order of requests sent in the
previous tests, although all the right requests are there.  I saw it
fail on Travis CI five times in the span of about half a year, when
the order of two subsequent requests was flipped, and could trigger
the failure with a modified Git.  However, I was unable to trigger it
with stock Git on my machine.  Three tests in
't5541-http-push-smart.sh' and 't5551-http-fetch-smart.sh' check
requests in the log the same way, so they might be prone to a similar
occasional failure as well.

When a test sends a HTTP request, it can continue execution after
'git-http-backend' fulfilled that request, but Apache writes the
corresponding access log entry only after 'git-http-backend' exited.
Some time inevitably passes between fulfilling the request and writing
the log entry, and, under unfavourable circumstances, enough time
might pass for the subsequent request to be sent and fulfilled by a
different Apache thread or process, and then Apache writes access log
entries racily.

This effect can be exacerbated by adding a bit of variable delay after
the request is fulfilled but before 'git-http-backend' exits, e.g.
like this:

  diff --git a/http-backend.c b/http-backend.c
  index f3dc218b2..bbf4c125b 100644
  --- a/http-backend.c
  +++ b/http-backend.c
  @@ -709,5 +709,7 @@ int cmd_main(int argc, const char **argv)
   					   max_request_buffer);

   	cmd->imp(&hdr, cmd_arg);
  +	if (getpid() % 2)
  +		sleep(1);
   	return 0;
   }

This delay considerably increases the chances of log entries being
written out of order, and in turn makes t5561's last test fail almost
every time.  Alas, it doesn't seem to be enough to trigger a similar
failure in t5541 and t5551.

So, since we can't just rely on the order of access log entries always
corresponding the order of requests, make checking the access log more
deterministic by sorting (simply lexicographically) both the stripped
access log entries and the expected entries before the comparison with
'test_cmp'.  This way the order of log entries won't matter and
occasional out-of-order entries won't trigger a test failure, but the
comparison will still notice any unexpected or missing log entries.

OTOH, this sorting will make it harder to identify from which test an
unexpected log entry came from or which test's request went missing.
Therefore, in case of an error include the comparison of the unsorted
log enries in the test output as well.

And since all this should be performed in four tests in three test
scripts, put this into a new helper function 'check_access_log' in
't/lib-httpd.sh'.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 10:40:31 -07:00
SZEDER Gábor 6940a06022 t/lib-httpd: add the strip_access_log() helper function
Four tests in three httpd-related test scripts check the contents of
Apache's 'access.log', and they all do so by running 'sed' with the
exact same script consisting of four s/// commands to strip
uninteresting log fields and to vertically align the requested URLs.

Extract this into a common helper function 'strip_access_log' in
'lib-httpd.sh', and use it in all of those tests.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 10:40:31 -07:00
SZEDER Gábor a704c6439a t5541: clean up truncating access log
In the second test of 't5541-http-push-smart.sh', 'no empty path
components' we truncate Apache's access log by running:

  echo >.../access.log

There are two issues with this approach:

  - This doesn't leave an empty file behind, like a proper truncation
    would, but a file with a lone newline in it.  Consequently, a
    later test checking the log's contents must consider this improper
    truncation and include an empty line in the expected content.

  - This truncation is done in the middle of the test, because,
    quoting the in-code comment, "we do this [truncation] before the
    actual comparison to ensure the log is cleared" even when
    subsequent 'test_cmp' fails.  Alas, this is not quite robust
    enough, as it is conceivable that 'git clone' fails after already
    having sent a request, in which case the access log would not be
    truncated and would leave stray log entries behind.

Since there is no need for that newline at all, drop the 'echo' from
the truncation and adjust the expected content accordingly.
Furthermore, make sure that the truncation is performed no matter
whether and how 'git clone' fails unexpectedly by specifying it as a
'test_when_finished' command.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-12 10:40:26 -07:00
Johannes Schindelin caafecfcf1 rebase --rebase-merges: adjust man page for octopus support
Now that we support octopus merges in the `--rebase-merges` mode,
we should give users who actually read the manuals a chance to know
about this fact.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 14:52:33 -07:00
Johannes Schindelin 2b6ad0f4bc rebase --rebase-merges: add support for octopus merges
Previously, we introduced the `merge` command for use in todo lists,
to allow to recreate and modify branch topology.

For ease of implementation, and to make review easier, the initial
implementation only supported merge commits with exactly two parents.

This patch adds support for octopus merges, making use of the
just-introduced `-F <file>` option for the `git merge` command: to keep
things simple, we spawn a new Git command instead of trying to call a
library function, also opening an easier door to enhance `rebase
--rebase-merges` to optionally use a merge strategy different from
`recursive` for regular merges: this feature would use the same code
path as octopus merges and simply spawn a `git merge`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 14:52:30 -07:00
Johannes Schindelin 920f22e6bc merge: allow reading the merge commit message from a file
This is consistent with `git commit` which, like `git merge`, supports
passing the commit message via `-m <msg>` and, unlike `git merge` before
this patch, via `-F <file>`.

It is useful to allow this for scripted use, or for the upcoming patch
to allow (re-)creating octopus merges in `git rebase --rebase-merges`.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 14:47:05 -07:00
Jeff King 3506dc9445 has_uncommitted_changes(): fall back to empty tree
If has_uncommitted_changes() can't resolve HEAD (e.g.,
because it's unborn or corrupt), then we end up calling
run_diff_index() with an empty revs.pending array. This
causes a segfault, as run_diff_index() blindly looks at the
first pending item.

Fixing this raises a question of fault: should
run_diff_index() handle this case, or is the caller wrong to
pass an empty pending list?

Looking at the other callers of run_diff_index(), they
handle this in one of three ways:

 - they resolve the object themselves, and avoid doing the
   diff if it's not valid

 - they resolve the object themselves, and fall back to the
   empty tree

 - they use setup_revisions(), which will die() if the
   object isn't valid

Since this is the only broken caller, that argues that the
fix should go there. Falling back to the empty tree makes
sense here, as we'd claim uncommitted changes if and only if
the index is non-empty. This may be a little funny in the
case of corruption (the corrupt HEAD probably _isn't_
empty), but:

  - we don't actually know the reason here that HEAD didn't
    resolve (the much more likely case is that we have an
    unborn HEAD, in which case the empty tree comparison is
    the right thing)

  - this matches how other code, like "git diff", behaves

While we're thinking about it, let's add an assertion to
run_diff_index(). It should always be passed a single
object, and as this bug shows, it's easy to get it wrong
(and an assertion is easier to hunt down than a segfault, or
a quietly ignored extra tree).

Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 12:12:37 -07:00
Henning Schild fbd0f16610 gpg-interface: make parse_gpg_output static and remove from interface header
Turn parse_gpg_output into a static function, the only outside user was
migrated in an earlier commit.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 10:05:22 -07:00
Henning Schild 3b9291e182 builtin/receive-pack: use check_signature from gpg-interface
The combination of verify_signed_buffer followed by parse_gpg_output is
available as check_signature. Use that instead of implementing it again.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 10:05:20 -07:00
Elijah Newren 587421ebdd t7405: verify 'merge --abort' works after submodule/path conflicts
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:40:04 -07:00
Elijah Newren e81c7d4145 t7405: add a directory/submodule conflict
For a directory/submodule conflict, we want contents from both the
directory and the submodule to be present for the user to use to resolve
the conflict, but we do not want paths under the directory being written
into the submodule and we do not want the merge being confused by paths
under the submodule being in the way.  Add testcases for these situations.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:40:03 -07:00
Elijah Newren 594a8673f2 t7405: add a file/submodule conflict
In the case of a file/submodule conflict, although both cannot exist at
the same path, we expect both to be present somewhere for the user to be
able to resolve the conflict with.  Add a testcase for this.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:40:03 -07:00
Elijah Newren 55f39cf755 merge: fix misleading pre-merge check documentation
builtin/merge.c contains this important requirement for merge strategies:

    ...the index must be in sync with the head commit.  The strategies are
    responsible to ensure this.

However, Documentation/git-merge.txt says:

    ...[merge will] abort if there are any changes registered in the index
    relative to the `HEAD` commit.  (One exception is when the changed
    index entries are in the state that would result from the merge
    already.)

Interestingly, prior to commit c0be8aa06b ("Documentation/git-merge.txt:
Partial rewrite of How Merge Works", 2008-07-19),
Documentation/git-merge.txt said much more:

    ...the index file must match the tree of `HEAD` commit...
    [NOTE]
    This is a bit of a lie.  In certain special cases [explained
    in detail]...
    Otherwise, merge will refuse to do any harm to your repository
    (that is...your working tree...and index are left intact).

So, this suggests that the exceptions existed because there were special
cases where it would case no harm, and potentially be slightly more
convenient for the user.  While the current text in git-merge.txt does
list a condition under which it would be safe to proceed despite the index
not matching HEAD, it does not match what is actually implemented, in
three different ways:

    * The exception is written to describe what unpack-trees allows.  Not
      all merge strategies allow such an exception, though, making this
      description misleading.  'ours' and 'octopus' merges have strictly
      enforced index==HEAD for a while, and the commit previous to this
      one made 'recursive' do so as well.

    * If someone did a three-way content merge on a specific file using
      versions from the relevant commits and staged it prior to running
      merge, then that path would technically satisfy the exception listed
      in git-merge.txt.  unpack-trees.c would still error out on the path,
      though, because it defers the three-way content merge logic to other
      parts of the code (resolve, octopus, or recursive) and has no way of
      checking whether the index entry from before the merge will match
      the end result of the merge.

    * The exception as implemented in unpack-trees actually only checked
      that the index matched the MERGE_HEAD version of the file and that
      HEAD matched the merge base.  Assuming no renames, that would indeed
      provide cases where the index matches the end result we'd get from a
      merge.  But renames means unpack-trees is checking that it instead
      matches something other than what the final result will be, risking
      either erroring out when we shouldn't need to, or not erroring out
      when we should and overwriting the user's staged changes.

In addition to the wording behind this exception being misleading, it is
also somewhat surprising to see how many times the code for the special
cases were wrong or the check to make sure the index matched head was
forgotten altogether:

* Prior to commit ee6566e8d7 ("[PATCH] Rewrite read-tree", 2005-09-05),
  there were many cases where an unclean index entry was allowed (look for
  merged_entry_allow_dirty()); it appears that in those cases, the merge
  would have simply overwritten staged changes with the result of the
  merge.  Thus, the merge result would have been correct, but the user's
  uncommitted changes could be thrown away without warning.

* Prior to commit 160252f816 ("git-merge-ours: make sure our index
  matches HEAD", 2005-11-03), the 'ours' merge strategy did not check
  whether the index matched HEAD.  If it didn't, the resulting merge
  would include all the staged changes, and thus wasn't really an 'ours'
  strategy.

* Prior to commit 3ec62ad9ff ("merge-octopus: abort if index does not
  match HEAD", 2016-04-09), 'octopus' merges did not check whether the
  index matched HEAD, also resulting in any staged changes from before
  the commit silently being folded into the resulting merge.  commit
  a6ee883b8e ("t6044: new merge testcases for when index doesn't match
  HEAD", 2016-04-09) was also added at the same time to try to test to
  make sure all strategies did the necessary checking for the requirement
  that the index match HEAD.  Sadly, it didn't catch all the cases, as
  evidenced by the remainder of this list...

* Prior to commit 65170c07d4 ("merge-recursive: avoid incorporating
  uncommitted changes in a merge", 2017-12-21), merge-recursive simply
  relied on unpack_trees() to do the necessary check, but in one special
  case it avoided calling unpack_trees() entirely and accidentally ended
  up silently including any staged changes from before the merge in the
  resulting merge commit.

* The commit immediately before this one in this series noted that the
  exceptions were written in a way that assumed no renames, making it
  unsafe for merge-recursive to use.  merge-recursive was modified to
  use its own check to enforce that index==HEAD.

This history makes it very tempting to go into builtin/merge.c and replace
the comment that strategies must enforce that index matches HEAD with code
that just enforces it.  At this point, that would only affect the
'resolve' strategy; all other strategies have each been modified to
manually enforce it.  (However, note that index==HEAD is not strictly
enforced for fast-forward merges, as those are not considered a merge
strategy and they trigger in builtin/merge.c before the section in the
code where the relevant comment is found.)

But, even if we don't take the step of just fixing these problems by
enforcing index==HEAD for all strategies, we at least need to update this
misleading documentation in git-merge.txt.  For now, just modify the claim
in Documentation/git-merge.txt to fix the error.  The precise details
around combination of merges strategies and special cases probably is not
relevant to most users, so simply state that exceptions may exist but are
narrow and vary depending upon which merge strategy is in use.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:38:36 -07:00
Elijah Newren eddd1a411d merge-recursive: enforce rule that index matches head before merging
builtin/merge.c says that when we are about to perform a merge:

    ...the index must be in sync with the head commit.  The strategies are
    responsible to ensure this.

merge-recursive has always relied on unpack_trees() to enforce this
requirement, except in the case of an "Already up to date!" merge.
unpack-trees.c does not actually enforce this requirement, though.  It
allows for a pair of exceptions, in cases which it refers to as #14(ALT)
and #2ALT.  Documentation/technical/trivial-merge.txt can be consulted for
the precise meanings of the various case numbers and their meanings for
unpack-trees.c, but we have a high-level description of the intent behind
these two exceptions in a combined and summarized form in
Documentation/git-merge.txt:

    ...[merge will] abort if there are any changes registered in the index
    relative to the `HEAD` commit.  (One exception is when the changed index
    entries are in the state that would result from the merge already.)

While this high-level description does describe conditions under which it
would be safe to allow the index to diverge from HEAD, it does not match
what is actually implemented.  In particular, unpack-trees.c has no
knowledge of renames, and these two exceptions were written assuming that
no renames take place.  Once renames get into the mix, it is no longer
safe to allow the index to not match for #2ALT.  We could modify
unpack-trees to only allow #14(ALT) as an exception, but that would be
more strict than required for the resolve strategy (since the resolve
strategy doesn't handle renames at all).  Therefore, unpack_trees.c seems
like the wrong place to fix this.

Further, if someone fixes the combination of break and rename detection
and modifies merge-recursive to take advantage of the combination, then it
will also no longer be safe to allow the index to not match for #14(ALT)
when the recursive strategy is in use.  Therefore, leaving one of the
exceptions in place with the recursive merge strategy feels like we are
just leaving a latent bug in the code for folks in the future to stumble
across.

It may be possible to fix both unpack-trees and merge-recursive in a way
that implements the exception as stated in Documentation/git-merge.txt,
but it would be somewhat complex, possibly also buggy at first, and
ultimately, not all that valuable.  Instead, just enforce the requirement
stated in builtin/merge.c; error out if the index does not match the HEAD
commit, just like the 'ours' and 'octopus' strategies do.

Some testcase fixups were in order:
  t7611: had many tests designed to show that `git merge --abort` could
	 not always restore the index and working tree to the state they
	 were in before the merge started.  The tests that were associated
	 with having changes in the index before the merge started are no
         longer applicable, so they have been removed.
  t7504: had a few tests that had stray staged changes that were not
         actually part of the test under consideration
  t6044: We no longer expect stray staged changes to sometimes result
         in the merge continuing.  Also, fix a case where a merge
         didn't abort but should have.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:38:36 -07:00
Elijah Newren 7f5271fa15 t6044: add more testcases with staged changes before a merge is invoked
According to Documentation/git-merge.txt,

    ...[merge will] abort if there are any changes registered in the index
    relative to the `HEAD` commit.  (One exception is when the changed
    index entries are in the state that would result from the merge
    already.)

Add some tests showing that this exception, while it does accurately state
what would be a safe condition under which we could allow the merge to
proceed, is not what is actually implemented.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:38:36 -07:00
Elijah Newren e1f8694f33 merge-recursive: fix assumption that head tree being merged is HEAD
`git merge-recursive` does a three-way merge between user-specified trees
base, head, and remote.  Since the user is allowed to specify head, we can
not necesarily assume that head == HEAD.

Modify index_has_changes() to take an extra argument specifying the tree
to compare against.  If NULL, it will compare to HEAD.  We then use this
from merge-recursive to make sure we compare to the user-specified head.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:38:36 -07:00
Elijah Newren 92702392ce merge-recursive: make sure when we say we abort that we actually abort
In commit 65170c07d4 ("merge-recursive: avoid incorporating uncommitted
changes in a merge", 2017-12-21), it was noted that there was a special
case when merge-recursive didn't rely on unpack_trees() to enforce the
index == HEAD requirement, and thus that it needed to do that enforcement
itself.  Unfortunately, it returned the wrong exit status, signalling that
the merge completed but had conflicts, rather than that it was aborted.
Fix the return code, and while we're at it, change the error message to
match what unpack_trees() would have printed.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:38:36 -07:00
Elijah Newren cf69f2af08 t6044: add a testcase for index matching head, when head doesn't match HEAD
The `git merge-recursive` command allows the user to directly specify
three commits to merge -- base, head, and remote.  (More than three can be
specified in the case of multiple merge bases.)  Note that since the user
is allowed to specify head, it need not match HEAD.

Virtually every test and script in the current git.git codebase calls `git
merge-recursive` with head=HEAD, and likely external callers do as well,
which is why this has gone unnoticed.  There is one notable
counter-example: git-stash.sh.  However, git-stash called `git
merge-recursive` with an index that matches the expected merge result,
which happens to be a currently allowed exception to the "index must match
head" rule, so this never triggered an error previously.

Since we would like to tighten up the "index must match head" rule, we
need to make sure we are comparing to the correct head.  Add a testcase
that demonstrates the failure when we check the wrong HEAD.

Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:38:36 -07:00
Max Kirillov b33fdfc34c unpack-trees: do not fail reset because of unmerged skipped entry
After modify/delete merge conflict happens in a file skipped by sparse
checkout, "git reset --merge", which implements the "--abort" actions,
and "git reset --hard" fail with message "Entry * not uptodate. Cannot
update sparse checkout."

As explained in [1], the up-to-date checker mistakenly treats conflicted
entry which does not exist in HEAD as still skipped by sparse checkout.

Use the fix suggested in [1]. Also, add test case which verifies the
issue is fixed.

[1] https://public-inbox.org/git/20180616051444.GA29754@duynguyen.home/

Signed-off-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 09:35:41 -07:00
Elijah Newren 5d1daf30cc t6036: add a failed conflict detection case: regular files, different modes
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 08:40:29 -07:00
Jeff King c5e358d073 sequencer: don't say BUG on bogus input
When cherry-picking a single commit, we go through a special
code path that avoids creating a sequencer todo list at all.
This path expects our revision parsing to turn up exactly
one commit, and dies with a BUG if it doesn't.

But it's actually quite easy to fool. For example:

  $ git cherry-pick --author=no.such.person HEAD
  error: BUG: expected exactly one commit from walk
  fatal: cherry-pick failed

This isn't a bug; it's just bogus input.

The condition to trigger this message actually has two
parts:

  1. We saw no commits. That's the case in the example
     above. Let's drop the "BUG" here to make it clear that
     the input is the problem. And let's also use the phrase
     "empty commit set passed", which matches what we say
     when we do a real revision walk and it turns up empty.

  2. We saw more than one commit. That one _should_ be
     impossible to trigger, since we fed at most one tip and
     provided the no_walk option (and we'll have already
     expanded options like "--branches" that can turn into
     multiple tips). If this ever triggers, it's an
     indication that the conditional added by 7acaaac275
     (revert: allow single-pick in the middle of cherry-pick
     sequence, 2011-12-10) needs to more carefully define
     the single-pick case.

     So this can remain a bug, but we'll upgrade it to use
     the BUG() macro, which would make it easier to detect
     and analyze if it does trigger.

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 08:37:59 -07:00
Jeff King 8530c73915 sequencer: handle empty-set cases consistently
If the user gives us a set that prepare_revision_walk()
takes to be empty, like:

  git cherry-pick base..base

then we report an error. It's nonsense, and there's nothing
to pick.

But if they use revision options that later cull the list,
like:

  git cherry-pick --author=nobody base~2..base

then we quietly create an empty todo list and return
success.

Arguably either behavior is acceptable, but we should
definitely be consistent about it. Reporting an error
seems to match the original intent, which dates all the way
back to 7e2bfd3f99 (revert: allow cherry-picking more than
one commit, 2010-06-02). That in turn was trying to match
the single-commit case that existed before then (and which
continues to issue an error).

Signed-off-by: Jeff King <peff@peff.net>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-11 08:37:47 -07:00
Ben Peart 80a6c2073b convert log_ref_write_fd() to use strbuf
Since we don't care about how many bytes were written, simplify the return
value logic.

log_ref_write_fd() was written long before strbuf was fleshed out. Remove
the old manual buffer management code and replace it with strbuf(). Also
update copy_reflog_msg() which is called only by log_ref_write_fd() to use
strbuf as it keeps things consistent.

Signed-off-by: Ben Peart <Ben.Peart@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-10 14:22:44 -07:00
Beat Bolli 2b647a05d7 utf8.c: avoid char overflow
In ISO C, char constants must be in the range -128..127. Change the BOM
constants to char literals to avoid overflow.

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:38:12 -07:00
Beat Bolli b6d3f5a960 string-list.c: avoid conversion from void * to function pointer
ISO C forbids the conversion of void pointers to function pointers.
Introduce a context struct that encapsulates the function pointer.

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:37:50 -07:00
Beat Bolli 9ad36356dd sequencer.c: avoid empty statements at top level
The macro GIT_PATH_FUNC expands to a function definition that ends with
a closing brace. Remove two extra semicolons.

While at it, fix the example in path.h.

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:37:03 -07:00
Beat Bolli 8302f50e8c convert.c: replace "\e" escapes with "\033".
The "\e" escape is not defined in ISO C.

While on this line, add a missing space after the comma.

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:36:24 -07:00
Junio C Hamano 13f925f3e4 fixup! refs/refs-internal.h: avoid forward declaration of an enum 2018-07-09 14:36:12 -07:00
Beat Bolli 91c2f2040a refs/refs-internal.h: avoid forward declaration of an enum
Include iterator.h to define enum iterator_selection.

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:35:52 -07:00
Junio C Hamano ca5e39683a fixup! connect.h: avoid forward declaration of an enum 2018-07-09 14:35:39 -07:00
Beat Bolli 2e75c8ed85 connect.h: avoid forward declaration of an enum
Include protocol.h to define enum protocol_version.

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:35:19 -07:00
Kim Gybels 12e73a3ce4 gc --auto: release pack files before auto packing
Teach gc --auto to release pack files before auto packing the repository
to prevent failures when removing them.

Also teach the test 'fetching with auto-gc does not lock up' to complain
when it is no longer triggering an auto packing of the repository.

Fixes https://github.com/git-for-windows/git/issues/500

Signed-off-by: Kim Gybels <kgybels@infogroep.be>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:16:10 -07:00
Taylor Blau 9d8db06eb4 grep.c: teach 'git grep --only-matching'
Teach 'git grep --only-matching', a new option to only print the
matching part(s) of a line.

For instance, a line containing the following (taken from README.md:27):

  (`man gitcvs-migration` or `git help cvs-migration` if git is

Is printed as follows:

  $ git grep --line-number --column --only-matching -e git -- \
    README.md | grep ":27"
  README.md:27:7:git
  README.md:27:16:git
  README.md:27:38:git

The patch works mostly as one would expect, with the exception of a few
considerations that are worth mentioning here.

Like GNU grep, this patch ignores --only-matching when --invert (-v) is
given. There is a sensible answer here, but parity with the behavior of
other tools is preferred.

Because a line might contain more than one match, there are special
considerations pertaining to when to print line headers, newlines, and
how to increment the match column offset. The line header and newlines
are handled as a special case within the main loop to avoid polluting
the surrounding code with conditionals that have large blocks.

Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:15:28 -07:00
Beat Bolli 570951eea2 unicode: update the width tables to Unicode 11
Now that Unicode 11 has been announced[0], update the character
width tables to the new version.

[0] http://blog.unicode.org/2018/06/announcing-unicode-standard-version-110.html

Signed-off-by: Beat Bolli <dev+git@drbeat.li>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-09 14:02:51 -07:00