"git push" from a corrupt repository that attempts to push a large
number of refs deadlocked; the thread to relay rejection notices
for these ref updates blocked on writing them to the main thread,
after the main thread at the receiving end notices that the push
failed and decides not to read these notices and return a failure.
* jk/push-client-deadlock-fix:
t5504: drop sigpipe=ok from push tests
fetch-pack: isolate sigpipe in demuxer thread
send-pack: isolate sigpipe in demuxer thread
run-command: teach async threads to ignore SIGPIPE
send-pack: close demux pipe before finishing async process
"git mv old new" did not adjust the path for a submodule that lives
as a subdirectory inside old/ directory correctly.
* sb/mv-submodule-fix:
mv: allow moving nested submodules
Sources to many test helper binaries (and the generated helpers)
have been moved to t/helper/ subdirectory to reduce clutter at the
top level of the tree.
* nd/test-helpers:
test helpers: move test-* to t/helper/ subdirectory
Makefile: clean *.o files we create
The "user.useConfigOnly" configuration variable makes it an error
if users do not explicitly set user.name and user.email. However,
its check was not done early enough and allowed another error to
trigger, reporting that the default value we guessed from the
system setting was unusable. This was a suboptimal end-user
experience as we want the users to set user.name/user.email without
relying on the auto-detection at all.
* da/user-useconfigonly:
ident: give "please tell me" message upon useConfigOnly error
ident: check for useConfigOnly before auto-detection of name/email
* maint:
l10n: fr: don't translate "merge" as a parameter
l10n: fr: change "id de clé" to match "id-clé"
l10n: fr: fix wrongly translated option name
l10n: fr: fix transcation of "dir"
"git submodule" reports the paths of submodules the command
recurses into, but this was incorrect when the command was not run
from the root level of the superproject.
* sb/submodule-path-misc-bugs:
t7407: make expectation as clear as possible
submodule update: test recursive path reporting from subdirectory
submodule update: align reporting path for custom command execution
submodule status: correct path handling in recursive submodules
submodule update --init: correct path handling in recursive submodules
submodule foreach: correct path display in recursive submodules
When "git merge" notices that the merge can be resolved purely at
the tree level (without having to merge blobs) and the resulting
tree happens to already exist in the object store, it forgot to
update the index, which lead to an inconsistent state for later
operations.
* en/merge-trivial-fix:
builtin/merge.c: fix a bug with trivial merges
t7605: add a testcase demonstrating a bug with trivial merges
"merge-octopus" strategy did not ensure that the index is clean
when merge begins.
* en/merge-octopus-fix:
merge-octopus: abort if index does not match HEAD
t6044: new merge testcases for when index doesn't match HEAD
Code restructuring around the "refs" area to prepare for pluggable
refs backends.
* dt/pre-refs-backend: (24 commits)
refs: on symref reflog expire, lock symref not referrent
refs: move resolve_ref_unsafe into common code
show_head_ref(): check the result of resolve_ref_namespace()
check_aliased_update(): check that dst_name is non-NULL
checkout_paths(): remove unneeded flag variable
cmd_merge(): remove unneeded flag variable
fsck_head_link(): remove unneeded flag variable
read_raw_ref(): change flags parameter to unsigned int
files-backend: inline resolve_ref_1() into resolve_ref_unsafe()
read_raw_ref(): manage own scratch space
files-backend: break out ref reading
resolve_ref_1(): eliminate local variable "bad_name"
resolve_ref_1(): reorder code
resolve_ref_1(): eliminate local variable
resolve_ref_unsafe(): ensure flags is always set
resolve_ref_unsafe(): use for loop to count up to MAXDEPTH
resolve_missing_loose_ref(): simplify semantics
t1430: improve test coverage of deletion of badly-named refs
t1430: test for-each-ref in the presence of badly-named refs
t1430: don't rely on symbolic-ref for creating broken symrefs
...
"git rerere" can encounter two or more files with the same conflict
signature that have to be resolved in different ways, but there was
no way to record these separate resolutions.
* jc/rerere-multi:
rerere: adjust 'forget' to multi-variant world order
rerere: split code to call ll_merge() further
rerere: move code related to "forget" together
rerere: gc and clear
rerere: do use multiple variants
t4200: rerere a merge with two identical conflicts
rerere: allow multiple variants to exist
rerere: delay the recording of preimage
rerere: handle leftover rr-cache/$ID directory and postimage files
rerere: scan $GIT_DIR/rr-cache/$ID when instantiating a rerere_id
rerere: split conflict ID further
* 'fr_v2.8.0_r3' of git://github.com/jnavila/git:
l10n: fr: don't translate "merge" as a parameter
l10n: fr: change "id de clé" to match "id-clé"
l10n: fr: fix wrongly translated option name
l10n: fr: fix transcation of "dir"
On Cygwin, object creation uses the "create a temporary and then
rename it to the final name" pattern, not "create a temporary,
hardlink it to the final name and then unlink the temporary"
pattern.
This is necessary to use Git on Windows shared directories, and is
already enabled for the MinGW and plain Windows builds. It also
has been used in Cygwin packaged versions of Git for quite a while.
See http://thread.gmane.org/gmane.comp.version-control.git/291853
($gmane/275680, $gmane/291853).
* ad/cygwin-wants-rename:
config.mak.uname: Cygwin needs OBJECT_CREATION_USES_RENAMES
"git config" had a codepath that tried to pass a NULL to
printf("%s"), which nobody seems to have noticed.
* jk/do-not-printf-NULL:
git_config_set_multivar_in_file: handle "unset" errors
git_config_set_multivar_in_file: all non-zero returns are errors
config: lower-case first word of error strings
The socks5:// proxy support added back in 2.6.4 days was not aware
that socks5h:// proxies behave differently.
* jc/http-socks5h:
http: differentiate socks5:// and socks5h://
Upcoming OpenSSL 1.1.0 will break compilation b updating a few APIs
we use in imap-send, which has been adjusted for the change.
* ky/imap-send-openssl-1.1.0:
configure: remove checking for HMAC_CTX_cleanup
imap-send: avoid deprecated TLSv1_method()
imap-send: check NULL return of SSL_CTX_new()
imap-send: use HMAC() function provided by OpenSSL
Support for CRAM-MD5 authentication method in "git imap-send" did
not work well.
* ky/imap-send:
imap-send: fix CRAM-MD5 response calculation
imap-send: check for NOLOGIN capability only when using LOGIN command
"git commit" misbehaved in a few minor ways when an empty message
is given via -m '', all of which has been corrected.
* ad/commit-have-m-option:
commit: do not ignore an empty message given by -m ''
commit: --amend -m '' silently fails to wipe message
"git send-email" now uses a more readable timestamps when
formulating a message ID.
* ew/send-email-readable-message-id:
send-email: more meaningful Message-ID
A partial rewrite of "git submodule" in the 2.7 timeframe changed
the way the gitdir: pointer in the submodules point at the real
repository location to use absolute paths by accident. This has
been corrected.
* sb/submodule-helper-clone-regression-fix:
submodule--helper, module_clone: catch fprintf failure
submodule--helper: do not borrow absolute_path() result for too long
submodule--helper, module_clone: always operate on absolute paths
submodule--helper clone: create the submodule path just once
submodule--helper: fix potential NULL-dereference
recursive submodules: test for relative paths
These were added by 8bf4bec (add "ok=sigpipe" to
test_must_fail and use it to fix flaky tests, 2015-11-27)
because we would racily die via SIGPIPE when the pack was
rejected by the other side.
But since we have recently de-flaked send-pack, we should be
able to tighten up these tests (including re-adding the
expected output checks).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In commit 9ff18fa (fetch-pack: ignore SIGPIPE in sideband
demuxer, 2016-02-24), we started using sigchain_push() to
ignore SIGPIPE in the async demuxer thread. However, this is
rather clumsy, as it ignores SIGPIPE for the entire process,
including the main thread. At the time we didn't have any
per-thread signal support, but we now we do. Let's use it.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If we get an error from pack-objects, we may exit
send_pack() early, before reading the server's status
response. In such a case, we may racily see SIGPIPE from our
async demuxer (which is trying to write that status back to
us), and we'd prefer to continue pushing the error up the
call stack, rather than taking down the whole process with
signal death.
This is safe to do because our demuxer just calls
recv_sideband, whose data writes are all done with
write_or_die(), which will notice SIGPIPE.
We do also write sideband 2 to stderr, and we would no
longer die on SIGPIPE there (if it were piped in the first
place, and if the piped program went away). But that's
probably a good thing, as it likewise should not abort the
push process at all (neither immediately by signal, nor
eventually by reporting failure back to the main thread).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Async processes can be implemented as separate forked
processes, or as threads (depending on the NO_PTHREADS
setting). In the latter case, if an async thread gets
SIGPIPE, it takes down the whole process. This is obviously
bad if the main process was not otherwise going to die, but
even if we were going to die, it means the main process does
not have a chance to report a useful error message.
There's also the small matter that forked async processes
will not take the main process down on a signal, meaning git
will behave differently depending on the NO_PTHREADS
setting.
This patch fixes it by adding a new flag to "struct async"
to block SIGPIPE just in the async thread. In theory, this
should always be on (which makes async threads behave more
like async processes), but we would first want to make sure
that each async process we spawn is careful about checking
return codes from write() and would not spew endlessly into
a dead pipe. So let's start with it as optional, and we can
enable it for specific sites in future patches.
The natural name for this option would be "ignore_sigpipe",
since that's what it does for the threaded case. But since
that name might imply that we are ignoring it in all cases
(including the separate-process one), let's call it
"isolate_sigpipe". What we are really asking for is
isolation. I.e., not to have our main process taken down by
signals spawned by the async process. How that is
implemented is up to the run-command code.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This fixes a deadlock on the client side when pushing a
large number of refs from a corrupted repo. There's a
reproduction script below, but let's start with a
human-readable explanation.
The client side of a push goes something like this:
1. Start an async process to demux sideband coming from
the server.
2. Run pack-objects to send the actual pack, and wait for
its status via finish_command().
3. If pack-objects failed, abort immediately.
4. If pack-objects succeeded, read the per-ref status from
the server, which is actually coming over a pipe from
the demux process started in step 1.
We run finish_async() to wait for and clean up the demux
process in two places. In step 3, if we see an error, we
want it to end early. And after step 4, it should be done
writing any data and we are just cleaning it up.
Let's focus on the error case first. We hand the output
descriptor to the server over to pack-objects. So by the
time it has returned an error to us, it has closed the
descriptor and the server has gotten EOF. The server will
mark all refs as failed with "unpacker error" and send us
back the status for each (followed by EOF).
This status goes to the demuxer thread, which relays it over
a pipe to the main thread. But the main thread never even
tries reading the status. It's trying to bail because of the
pack-objects error, and is waiting for the demuxer thread to
finish. If there are a small number of refs, that's OK; the
demuxer thread writes into the pipe buffer, sees EOF from
the server, and quits. But if there are a large number of
refs, it may block on write() back to the main thread,
leading to a deadlock (the main thread is waiting for the
demuxer to finish, the demuxer is waiting for the main
thread to read).
We can break this deadlock by closing the pipe between the
demuxer and the main thread before calling finish_async().
Then the demuxer gets a write() error and exits.
The non-error case usually just works, because we will have
read all of the data from the other side. We do close
demux.out already, but we only do so _after_ calling
finish_async(). This is OK because there shouldn't be any
more data coming from the server. But technically we've only
read to a flush packet, and a broken or malicious server
could be sending more cruft. In such a case, we would hit
the same deadlock. Closing the pipe first doesn't affect the
normal case, and means that for a cruft-sending server,
we'll notice a write() error rather than deadlocking.
Note that when write() sees this error, we'll actually
deliver SIGPIPE to the thread, which will take down the
whole process (unless we're compiled with NO_PTHREADS). This
isn't ideal, but it's an improvement over the status quo,
which is deadlocking. And SIGPIPE handling in async threads
is a bigger problem that we can deal with separately.
A simple reproduction for the error case is below. It's
technically racy (we could exit the main process and take
down the async thread with us before it even reads the
status), though in practice it seems to fail pretty
consistently.
git init repo &&
cd repo &&
# make some commits; we need two so we can simulate corruption
# in the history later.
git commit --allow-empty -m one &&
one=$(git rev-parse HEAD) &&
git commit --allow-empty -m two &&
two=$(git rev-parse HEAD) &&
# now make a ton of refs; our goal here is to overflow the pipe buffer
# when reporting the ref status, which will cause the demuxer to block
# on write()
for i in $(seq 20000); do
echo "create refs/heads/this-is-a-really-long-branch-name-$i $two"
done |
git update-ref --stdin &&
# now make a corruption in the history such that pack-objects will fail
rm -vf .git/objects/$(echo $one | sed 's}..}&/}') &&
# and then push the result
git init --bare dst.git &&
git push --mirror dst.git
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We simply need to read the config, is all.
This fixes https://github.com/git-for-windows/git/issues/733
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When directories are moved using `git mv` all files in the directory
have been just moved, but no further action was taken on them. This
was done by assigning the mode = WORKING_DIRECTORY to the files
inside a moved directory.
submodules however need to update their link to the git directory as
well as updates to the .gitmodules file. By removing the condition of
`mode != INDEX` (the remaining modes are BOTH and WORKING_DIRECTORY) for
the required submodule actions, we perform these for submodules in a
moved directory.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A change back in version 2.7 to "git branch" broke display of a
symbolic ref in a non-standard place in the refs/ hierarchy (we
expect symbolic refs to appear in refs/remotes/*/HEAD to point at
the primary branch the remote has, and as .git/HEAD to point at the
branch we locally checked out).
* jk/branch-shortening-funny-symrefs:
branch: fix shortening of non-remote symrefs
When "git worktree" feature is in use, "git branch -m" renamed a
branch that is checked out in another worktree without adjusting
the HEAD symbolic ref for the worktree.
* ky/branch-m-worktree:
set_worktree_head_symref(): fix error message
branch -m: update all per-worktree HEADs
refs: add a new function set_worktree_head_symref
This keeps top dir a bit less crowded. And because these programs are
for testing purposes, it makes sense that they stay somewhere in t/
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The part that removes object files in the 'clean' target predates
various Makefile macros that list object files we create, and
instead removes the objects with shell glob, perpetually requiring
updates whenever a new location that builds object files is added.
Simplify the target by removing $(OBJECTS), which is supposed to
have all the objects we create during the build.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git merge FETCH_HEAD" dereferenced NULL pointer when merging
nothing into an unborn history (which is arguably unusual usage,
which perhaps was the reason why nobody noticed it).
* jv/merge-nothing-into-void:
merge: fix NULL pointer dereference when merging nothing into void
When "git merge --squash" stopped due to conflict, the concluding
"git commit" failed to read in the SQUASH_MSG that shows the log
messages from all the squashed commits.
* ss/commit-squash-msg:
commit: do not lose SQUASH_MSG contents
"git send-email" had trouble parsing alias file in mailrc format
when lines in it had trailing whitespaces on them.
* jk/send-email-rtrim-mailrc-alias:
send-email: ignore trailing whitespace in mailrc alias file
"git mergetool" did not work well with conflicts that both sides
deleted.
* da/mergetool-delete-delete-conflict:
mergetool: honor tempfile configuration when resolving delete conflicts
mergetool: support delete/delete conflicts
The startup_info data, which records if we are working inside a
repository (among other things), are now uniformly available to Git
subcommand implementations, and Git avoids attempting to touch
references when we are not in a repository.
* jk/startup-info:
use setup_git_directory() in test-* programs
grep: turn off gitlink detection for --no-index
mailmap: do not resolve blobs in a non-repository
remote: don't resolve HEAD in non-repository
setup: set startup_info->have_repository more reliably
setup: make startup_info available everywhere
strbuf_getwholeline() did not NUL-terminate the buffer on certain
corner cases in its error codepath.
* jk/getwholeline-getdelim-empty:
strbuf_getwholeline: NUL-terminate getdelim buffer on error
A small memory leak in an error codepath has been plugged in xdiff
code.
* rj/xdiff-prepare-plug-leak-on-error-codepath:
xdiff/xprepare: fix a memory leak
xdiff/xprepare: use the XDF_DIFF_ALG() macro to access flag bits
Fetching of history by naming a commit object name directly didn't
work across remote-curl transport.
* gf/fetch-pack-direct-object-fetch:
fetch-pack: update the documentation for "<refs>..." arguments
fetch-pack: fix object_id of exact sha1
The "--local-env-vars" and "--resolve-git-dir" options of "git
rev-parse" failed to work outside a repository when the command's
option parsing was rewritten in 1.8.5 era.
* jk/rev-parse-local-env-vars:
rev-parse: let some options run outside repository
t1515: add tests for rev-parse out-of-repo helpers
"git config --get-urlmatch", unlike other variants of the "git
config --get" family, did not signal error with its exit status
when there was no matching configuration.
* jk/config-get-urlmatch:
Documentation/git-config: fix --get-all description
Documentation/git-config: use bulleted list for exit codes
config: fail if --get-urlmatch finds no value
