Callers of the run-command API may mark a child as
"clean_on_exit"; it gets added to a list and killed when the
main process dies. Since commit 46df6906f
(execv_dashed_external: wait for child on signal death,
2017-01-06), we respect an extra "wait_after_clean" flag,
which we expect to find in the child_process struct.
When Git is built with NO_PTHREADS, we start "struct
async" processes by forking rather than spawning a thread.
The resulting processes get added to the cleanup list but
they don't have a child_process struct, and the cleanup
function ends up dereferencing NULL.
We should notice this case and assume that the processes do
not need to be waited for (i.e., the same behavior they had
before 46df6906f).
Reported-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The string after_subject is added to a strbuf by pp_title_line() if
it's not NULL. Adding an empty string has the same effect as not
adding anything, but the latter is easier, so don't bother changing
the context member from NULL to "".
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When the 'test_pause' helper function invokes the shell mid-test, it
explicitly redirects the shell's stdout and stderr to file descriptors
3 and 4, which are the stdout and stderr of the tests (i.e. where they
would be connected anyway without those redirections). These file
descriptors are only attached to the terminal in verbose mode, hence
the restriction of 'test_pause' to work only with '-v'.
Redirect the shell's stdout and stderr to the test environment's
original stdout and stderr, allowing it to work properly even in
non-verbose mode, and the restriction can be lifted.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'debug' test helper is supposed to facilitate debugging by running
a command of the test suite under gdb. Unfortunately, its usefulness
is severely limited, because that gdb session is not interactive,
since the test's, and thus gdb's standard input is redirected from
/dev/null (for a good reason, see 781f76b15 (test-lib: redirect stdin
of tests, 2011-12-15)).
Redirect gdb's standard file descriptors from/to the test
environment's stdin, stdout and stderr in the 'debug' helper, thus
creating an interactive gdb session (even in non-verbose mode), which
is much, much more useful.
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The definition of path_name() was removed by 2824e1841 (list-objects:
pass full pathname to callbacks); remove its declaration as well.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Acked-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This function always returns a reference to an object, creating one if
needed, so remove the unnecessary NULL check.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Instead of counting the arguments to see if there are any and then
building the full command use a single loop and add the hook command
just before the first argument. This reduces duplication and overall
code size.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Commit 0281e487fd ("grep: optionally recurse into submodules")
added functions grep_submodule() and grep_submodule_launch() which
use "struct work_item" which is defined only when thread support
is available.
The original implementation of grep_submodule() used the "struct
work_item" in order to gain access to a strbuf to store its output which
was to be printed at a later point in time. This differs from how both
grep_file() and grep_sha1() handle their output. This patch eliminates
the reliance on the "struct work_item" and instead opts to use the
output function stored in the output field of the "struct grep_opt"
object directly, making it behave similarly to both grep_file() and
grep_sha1().
Reported-by: Rahul Bedarkar <rahul.bedarkar@imgtec.com>
Signed-off-by: Brandon Williams <bmwill@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In addition to adding submodule.<name>.url to the config, set
submodule.<name>.active to true unless submodule.active is configured
and the submodule's path matches the configured pathspec.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When initializing a submodule set the submodule.<name>.active config to
true if the module hasn't already been configured to be active by some
other means (e.g. a pathspec set in submodule.active).
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach clone --recurse-submodules to optionally take a pathspec argument
which describes which submodules should be recursively initialized and
cloned. If no pathspec is provided, --recurse-submodules will
recursively initialize and clone all submodules by using a default
pathspec of ".". In order to construct more complex pathspecs,
--recurse-submodules can be given multiple times.
This also configures the 'submodule.active' configuration option to be
the given pathspec, such that any future invocation of `git submodule
update` will keep up with the pathspec.
Additionally the switch '--recurse' is removed from the Documentation as
well as marked hidden in the options array, to streamline the options
for submodules. A simple '--recurse' doesn't convey what is being
recursed, e.g. it could mean directories or trees (c.f. ls-tree) In a
lot of other commands we already have '--recurse-submodules' to mean
recursing into submodules, so advertise this spelling here as the
genuine option.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Teach `submodule init` to initialize submodules which have been
configured to be active by setting 'submodule.active' with a pathspec.
Now if no path arguments are given and 'submodule.active' is configured,
`init` will initialize all submodules which have been configured to be
active. If no path arguments are given and 'submodule.active' is not
configured, then `init` will retain the old behavior of initializing all
submodules.
This allows users to record more complex patterns as it saves retyping
them whenever you invoke update.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently the submodule.<name>.url config option is used to determine if
a given submodule is of interest to the user. This ends up being
cumbersome in a world where we want to have different submodules checked
out in different worktrees or a more generalized mechanism to select
which submodules are of interest.
In a future with worktree support for submodules, there will be multiple
working trees, each of which may only need a subset of the submodules
checked out. The URL (which is where the submodule repository can be
obtained) should not differ between different working trees.
It may also be convenient for users to more easily specify groups of
submodules they are interested in as opposed to running "git submodule
init <path>" on each submodule they want checked out in their working
tree.
To this end two config options are introduced, submodule.active and
submodule.<name>.active. The submodule.active config holds a pathspec
that specifies which submodules should exist in the working tree. The
submodule.<name>.active config is a boolean flag used to indicate if
that particular submodule should exist in the working tree.
Its important to note that submodule.active functions differently than
the other configuration options since it takes a pathspec. This allows
users to adopt at least two new workflows:
1. Submodules can be grouped with a leading directory, such that a
pathspec e.g. 'lib/' would cover all library-ish modules to allow
those who are interested in library-ish modules to set
"submodule.active = lib/" just once to say any and all modules in
'lib/' are interesting.
2. Once the pathspec-attribute feature is invented, users can label
submodules with attributes to group them, so that a broad pathspec
with attribute requirements, e.g. ':(attr:lib)', can be used to say
any and all modules with the 'lib' attribute are interesting.
Since the .gitattributes file, just like the .gitmodules file, is
tracked by the superproject, when a submodule moves in the
superproject tree, the project can adjust which path gets the
attribute in .gitattributes, just like it can adjust which path has
the submodule in .gitmodules.
Neither of these two additional configuration options solve the problem
of wanting different submodules checked out in different worktrees
because multiple worktrees share .git/config. Only once per-worktree
configurations become a reality can this be solved, but this is a
necessary preparatory step for that future.
Given these multiple ways to check if a submodule is of interest, the
more fine-grained submodule.<name>.active option has the highest order
of precedence followed by the pathspec check against submodule.active.
To ensure backwards compatibility, if neither of these options are set,
git falls back to checking the submodule.<name>.url option to determine
if a submodule is interesting.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git revert -m 0 $merge_commit" complained that reverting a merge
needs to say relative to which parent the reversion needs to
happen, as if "-m 0" weren't given. The correct diagnosis is that
"-m 0" does not refer to the first parent ("-m 1" does). This has
been fixed.
* jk/cherry-pick-0-mainline:
cherry-pick: detect bogus arguments to --mainline
The start-up sequence of "git" needs to figure out some configured
settings before it finds and set itself up in the location of the
repository and was quite messy due to its "chicken-and-egg" nature.
The code has been restructured.
* js/early-config:
setup.c: mention unresolved problems
t1309: document cases where we would want early config not to die()
setup_git_directory_gently_1(): avoid die()ing
t1309: test read_early_config()
read_early_config(): really discover .git/
read_early_config(): avoid .git/config hack when unneeded
setup: make read_early_config() reusable
setup: introduce the discover_git_directory() function
setup_git_directory_1(): avoid changing global state
setup: prepare setup_discovered_git_dir() for the root directory
setup_git_directory(): use is_dir_sep() helper
t7006: replace dubious test
Our source code has used the SHA1_HEADER cpp macro after "#include"
in the C code to switch among the SHA-1 implementations. Instead,
list the exact header file names and switch among implementations
using "#ifdef BLK_SHA1/#include "block-sha1/sha1.h"/.../#endif";
this helps some IDE tools.
* bc/sha1-header-selection-with-cpp-macros:
hash.h: move SHA-1 implementation selection into a header file
"git status --porcelain" is supposed to give a stable output, but a
few strings were left as translatable by mistake.
* mg/status-porcelain-no-i18n:
git-status: make porcelain more robust
"git add -p <pathspec>" unnecessarily expanded the pathspec to a
list of individual files that matches the pathspec by running "git
ls-files <pathspec>", before feeding it to "git diff-index" to see
which paths have changes, because historically the pathspec
language supported by "diff-index" was weaker. These days they are
equivalent and there is no reason to internally expand it. This
helps both performance and avoids command line argument limit on
some platforms.
* jk/add-i-use-pathspecs:
add--interactive: do not expand pathspecs with ls-files
"Dumb http" transport used to misparse a nonsense http-alternates
response, which has been fixed.
* jk/http-walker-buffer-underflow-fix:
http-walker: fix buffer underflow processing remote alternates
The pathspec mechanism learned to further limit the paths that
match the pattern to those that have specified attributes attached
via the gitattributes mechanism.
* bw/attr-pathspec:
pathspec: allow escaped query values
pathspec: allow querying for attributes
"git tag --contains" used to (ab)use the object bits to keep track
of the state of object reachability without clearing them after
use; this has been cleaned up and made to use the newer commit-slab
facility.
* jk/ref-filter-flags-cleanup:
ref-filter: use separate cache for contains_tag_algo
ref-filter: die on parse_commit errors
ref-filter: use contains_result enum consistently
ref-filter: move ref_cbdata definition into ref-filter.c
From a working tree of a repository, a new option of "rev-parse"
lets you ask if the repository is used as a submodule of another
project, and where the root level of the working tree of that
project (i.e. your superproject) is.
* sb/rev-parse-show-superproject-root:
rev-parse: add --show-superproject-working-tree
Picking two versions of Git and running tests to make sure the
older one and the newer one interoperate happily has now become
possible.
* jk/interop-test:
t/interop: add test of old clients against modern git-daemon
t: add an interoperability test harness
The experimental "split index" feature has gained a few
configuration variables to make it easier to use.
* cc/split-index-config: (22 commits)
Documentation/git-update-index: explain splitIndex.*
Documentation/config: add splitIndex.sharedIndexExpire
read-cache: use freshen_shared_index() in read_index_from()
read-cache: refactor read_index_from()
t1700: test shared index file expiration
read-cache: unlink old sharedindex files
config: add git_config_get_expiry() from gc.c
read-cache: touch shared index files when used
sha1_file: make check_and_freshen_file() non static
Documentation/config: add splitIndex.maxPercentChange
t1700: add tests for splitIndex.maxPercentChange
read-cache: regenerate shared index if necessary
config: add git_config_get_max_percent_split_change()
Documentation/git-update-index: talk about core.splitIndex config var
Documentation/config: add information for core.splitIndex
t1700: add tests for core.splitIndex
update-index: warn in case of split-index incoherency
read-cache: add and then use tweak_split_index()
split-index: add {add,remove}_split_index() functions
config: add git_config_get_split_index()
...
When using the --recurse-submodules flag with a relative pathspec which
includes "..", an error is produced inside the child process spawned for a
submodule. When creating the pathspec struct in the child, the ".." is
interpreted to mean "go up a directory" which causes an error stating that the
path ".." is outside of the repository.
While it is true that ".." is outside the scope of the submodule, it is
confusing to a user who originally invoked the command where ".." was indeed
still inside the scope of the superproject. Since the child process launched
for the submodule has some context that it is operating underneath a
superproject, this error could be avoided.
This patch fixes the bug by passing the 'prefix' to the child process. Now
each child process that works on a submodule has two points of reference to the
superproject: (1) the 'super_prefix' which is the path from the root of the
superproject down to root of the submodule and (2) the 'prefix' which is the
path from the root of the superproject down to the directory where the user
invoked the git command.
With these two pieces of information a child process can correctly interpret
the pathspecs provided by the user as well as being able to properly format its
output relative to the directory the user invoked the original command from.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using the --recurse-submodules flag with a relative pathspec which
includes "..", an error is produced inside the child process spawned for
a submodule. When creating the pathspec struct in the child, the ".."
is interpreted to mean "go up a directory" which causes an error stating
that the path ".." is outside of the repository.
While it is true that ".." is outside the scope of the submodule, it is
confusing to a user who originally invoked the command where ".." was
indeed still inside the scope of the superproject. Since the child
process launched for the submodule has some context that it is operating
underneath a superproject, this error could be avoided.
This patch fixes the bug by passing the 'prefix' to the child process.
Now each child process that works on a submodule has two points of
reference to the superproject: (1) the 'super_prefix' which is the path
from the root of the superproject down to root of the submodule and (2)
the 'prefix' which is the path from the root of the superproject down to
the directory where the user invoked the git command.
With these two pieces of information a child process can correctly
interpret the pathspecs provided by the user as well as being able to
properly format its output relative to the directory the user invoked
the original command from.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a future patch child processes which act on submodules need a little
more context about the original command that was invoked. This patch
teaches git to use the prefix stored in `GIT_INTERNAL_TOPLEVEL_PREFIX`
instead of the prefix that was potentally found during the git directory
setup process.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Make git-credential-cache follow the XDG base path specification by
default. This increases consistency with other applications and helps
keep clutter out of users' home directories.
Check the old socket location, ~/.git-credential-cache/, and use
~/.git-credential-cache/socket if that directory exists rather than
forcing users who have used `git credential-cache` before to migrate to
the new XDG compliant location.
Otherwise use the socket $XDG_CACHE_HOME/git/credential/socket following
XDG base path specification. Use the subdirectory credential/ in case
other files are cached under $XDG_CACHE_HOME/git/ in the future and to
make the socket's purpose clear.
Signed-off-by: Devin Lehmacher <lehmacdj@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Use the 'is_submodule_initialized()' helper to check for configured
submodules instead of manually checking for the submodule's URL in the
config.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Sync does some work determining what URLs should be used for a submodule
but then throws this work away if the submodule isn't active. Instead
perform the activity check earlier and skip inactive submodule in order
to avoid doing unnecessary work.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The definition of which submodules are of interest by the user
is tied to the configuration submodule.<name>.url; when it is
set to a non-empty string, it is of interest. We'd want to be
able to later change this definition, but there are many places
that explicitly check this condition in the scripted Porcelain.
Introduce the "is-active" subcommand to "submodule--helper", so
that the exact definition of what submodule is of interest can
be centrally defined (and changed in later steps). In a few
patches that follow, this helper is used to replace the explicit
checks of the configuration variable in scripts.
Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We used to use the SHA1 implementation from the OpenSSL library by
default. As we are trying to be careful against collision attacks
after the recent "shattered" announcement, switch the default to
encourage people to use DC_SHA1 implementation instead. Those who
want to use the implementation from OpenSSL can explicitly ask for
it by OPENSSL_SHA1=YesPlease when running "make".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We don't actually have a Git-object collision, so the best
we can do is to run one of the shattered PDFs through
test-sha1. This should trigger the collision check and die.
In a sense this isn't really checking anything that the
upstream sha1collisiondetection project doesn't cover
already. But it at least makes sure that our build correctly
uses the library.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This knob lets you use the sha1dc implementation from:
https://github.com/cr-marcstevens/sha1collisiondetection
which can detect certain types of collision attacks (even
when we only see half of the colliding pair). So it
mitigates any attack which consists of getting the "good"
half of a collision into a trusted repository, and then
later replacing it with the "bad" half. The "good" half is
rejected by the victim's version of Git (and even if they
run an old version of Git, any sha1dc-enabled git will
complain loudly if it ever has to interact with the object).
The big downside is that it's slower than either the openssl
or block-sha1 implementations.
Here are some timings based off of linux.git:
- compute sha1 over whole packfile
sha1dc: 3.580s
blk-sha1: 2.046s (-43%)
openssl: 1.335s (-62%)
- rev-list --all --objects
sha1dc: 33.512s
blk-sha1: 33.514s (+0.0%)
openssl: 33.650s (+0.4%)
- git log --no-merges -10000 -p
sha1dc: 8.124s
blk-sha1: 7.986s (-1.6%)
openssl: 8.203s (+0.9%)
- index-pack --verify
sha1dc: 4m19s
blk-sha1: 2m57s (-32%)
openssl: 2m19s (-42%)
So overall the sha1 computation with collision detection is
about 1.75x slower than block-sha1, and 2.7x slower than
sha1. But of course most operations do more than just sha1.
Normal object access isn't really slowed at all (both the
+/- changes there are well within the run-to-run noise); any
changes are drowned out by the other work Git is doing.
The most-affected operation is `index-pack --verify`, which
is essentially just computing the sha1 on every object. This
is similar to the `index-pack` invocation that the receiver
of a push or fetch would perform. So clearly there's some
extra CPU load here.
There will also be some latency for the user, though keep in
mind that such an operation will generally be network bound
(this is about a 1.2GB packfile). Some of that extra CPU is
"free" in the sense that we use it while the pack is
streaming in anyway. But most of it comes during the
delta-resolution phase, after the whole pack has been
received. So we can imagine that for this (quite large)
push, the user might have to wait an extra 100 seconds over
openssl (which is what we use now). If we assume they can
push to us at 20Mbit/s, that's 480s for a 1.2GB pack, which
is only 20% slower.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Improve readability by using the for_each_string_list_item helper
instead of manually iterating with an integer counter.
Signed-off-by: Stefan Beller <sbeller@google.com>
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since the conversion from shell to C in 56dc3ab04 (sequencer
(rebase -i): implement the 'edit' command, 2017-01-02),
stopping at an "edit" instruction went from:
$ git rebase -i
Stopped at 6ce6b914a... odb_pack_keep(): stop generating keepfile name
You can amend the commit now, with
[...more instructions...]
to:
$ git rebase -i
warning: stopped at 6ce6b914a... odb_pack_keep(): stop generating keepfile name
You can amend the commit now, with
[...more instructions...]
The "warning" implies that it's something unexpected, but
it's not. Let's switch back to the original message.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The safe_hash feature is designed to make sha1dc a drop-in
replacement for sha1, where colliding entries will get a
permuted hash to un-collide them. However, since we're
handling the collision case ourselves, this isn't helpful
(and is actually harmful, as it means you get the wrong
object id if you want to show it in a log message).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We can replace system includes with git-compat-util.h or
cache.h (and should make sure it is included first in all C
files). And we can drop includes from headers entirely, as
every C file should include git-compat-util.h itself.
We will add in new include guards around the header files,
though (otherwise you get into trouble including both
sha1dc/sha1.h and cache.h).
And finally, we'll use the full "sha1dc/" path for including
related files. This isn't strictly necessary, but makes the
expected resolution more obvious.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is pulled straight from:
https://github.com/cr-marcstevens/sha1collisiondetection
with no modifications yet (though I've pulled in only the
subset of files necessary for Git to use).
This is commit 007905a93c973f55b2daed6585f9f6c23545bf66.
Further updates can be done like:
git checkout -b vendor-sha1dc $this_commit
cp /path/to/sha1dc/{LICENSE.txt,lib/*} sha1dc/
git add -A sha1dc
git commit -m "update sha1dc"
git checkout -b update-sha1dc origin
git merge vendor-sha1dc
Thanks to both Marc and Dan for making the code fit our
needs by doing both optimization work, cutting down on the
object size, and doing some syntactic changes to work better
with git. And to Linus for kicking off the "diet" work that
removed some of the unused code.
The license of the sha1dc code is the MIT license, which is
obviously compatible with the GPLv2 of git.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
A new known failure mode is introduced[1], which is actually not
a failure but a feature in read-tree. Unlike checkout for which
the recursive submodule tests were originally written, read-tree does
warn about ignored untracked files that would be overwritten.
For the sake of keeping the test library for submodules generic, just
mark the test as a failure.
[1] KNOWN_FAILURE_SUBMODULE_OVERWRITE_IGNORED_UNTRACKED
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This exposes a flag to recurse into submodules
in builtin/checkout making use of the code implemented
in prior patches.
A new failure mode is introduced in the submodule
update library, as the directory/submodule conflict
is not solved in prior patches.
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
