Commit graph

1096 commits

Author SHA1 Message Date
xiaoqiang zhao 5453b83bdf send-email: --batch-size to work around some SMTP server limit
Some email servers (e.g. smtp.163.com) limit the number emails to be
sent per session (connection) and this will lead to a faliure when
sending many messages.

Teach send-email to disconnect after sending a number of messages
(configurable via the --batch-size=<num> option), wait for a few
seconds (configurable via the --relogin-delay=<seconds> option) and
reconnect, to work around such a limit.

Also add two configuration variables to give these options the default.

Note:

  We will use this as a band-aid for now, but in the longer term, we
  should look at and react to the SMTP error code from the server;
  Xianqiang reports that 450 and 451 are returned by problematic
  servers.

  cf. https://public-inbox.org/git/7993e188.d18d.15c3560bcaf.Coremail.zxq_yx_007@163.com/

Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-07-05 09:09:45 -07:00
Junio C Hamano a93dcb0a56 Merge branch 'bw/submodule-is-active'
"what URL do we want to update this submodule?" and "are we
interested in this submodule?" are split into two distinct
concepts, and then the way used to express the latter got extended,
paving a way to make it easier to manage a project with many
submodules and make it possible to later extend use of multiple
worktrees for a project with submodules.

* bw/submodule-is-active:
  submodule add: respect submodule.active and submodule.<name>.active
  submodule--helper init: set submodule.<name>.active
  clone: teach --recurse-submodules to optionally take a pathspec
  submodule init: initialize active submodules
  submodule: decouple url and submodule interest
  submodule--helper clone: check for configured submodules using helper
  submodule sync: use submodule--helper is-active
  submodule sync: skip work for inactive submodules
  submodule status: use submodule--helper is-active
  submodule--helper: add is-active subcommand
2017-03-30 14:07:14 -07:00
Junio C Hamano 3d6586d100 Merge branch 'km/config-grammofix'
Doc update.

* km/config-grammofix:
  doc/config: grammar fixes for core.{editor,commentChar}
2017-03-27 10:59:29 -07:00
Kyle Meyer e7e183d6ee doc/config: grammar fixes for core.{editor,commentChar}
Signed-off-by: Kyle Meyer <kyle@kyleam.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-23 12:04:08 -07:00
Junio C Hamano b7b57b1434 Merge branch 'ab/push-default-doc-fix'
Doc fix.

* ab/push-default-doc-fix:
  push: mention "push.default=tracking" in the documentation
2017-03-21 15:07:19 -07:00
Junio C Hamano 4b7989b103 Merge branch 'nd/conditional-config-include'
The configuration file learned a new "includeIf.<condition>.path"
that includes the contents of the given path only when the
condition holds.  This allows you to say "include this work-related
bit only in the repositories under my ~/work/ directory".

* nd/conditional-config-include:
  config: add conditional include
  config.txt: reflow the second include.path paragraph
  config.txt: clarify multiple key values in include.path
2017-03-21 15:07:18 -07:00
Ævar Arnfjörð Bjarmason e4e016f65d push: mention "push.default=tracking" in the documentation
Change the documentation for push.tracking=* to re-include a mention
of what "tracking" does.

The "tracking" option was renamed to "upstream" back in
53c4031 ("push.default: Rename 'tracking' to 'upstream'", 2011-02-16),
this section was then subsequently rewritten in 87a70e4 ("config doc:
rewrite push.default section", 2013-06-19) to remove any mention of
"tracking".

Maybe we should just warn or die nowadays if this option is in the
config, but I had some old config of mine use this option, I'd
forgotten that it was a synonym, and nothing in git's documentation
mentioned that.

That's bad, either we shouldn't support it at all, or we should
document what it does. This patch does the latter.

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-20 10:59:43 -07:00
Brandon Williams a086f921a7 submodule: decouple url and submodule interest
Currently the submodule.<name>.url config option is used to determine if
a given submodule is of interest to the user.  This ends up being
cumbersome in a world where we want to have different submodules checked
out in different worktrees or a more generalized mechanism to select
which submodules are of interest.

In a future with worktree support for submodules, there will be multiple
working trees, each of which may only need a subset of the submodules
checked out.  The URL (which is where the submodule repository can be
obtained) should not differ between different working trees.

It may also be convenient for users to more easily specify groups of
submodules they are interested in as opposed to running "git submodule
init <path>" on each submodule they want checked out in their working
tree.

To this end two config options are introduced, submodule.active and
submodule.<name>.active.  The submodule.active config holds a pathspec
that specifies which submodules should exist in the working tree.  The
submodule.<name>.active config is a boolean flag used to indicate if
that particular submodule should exist in the working tree.

Its important to note that submodule.active functions differently than
the other configuration options since it takes a pathspec.  This allows
users to adopt at least two new workflows:

  1. Submodules can be grouped with a leading directory, such that a
     pathspec e.g. 'lib/' would cover all library-ish modules to allow
     those who are interested in library-ish modules to set
     "submodule.active = lib/" just once to say any and all modules in
     'lib/' are interesting.

  2. Once the pathspec-attribute feature is invented, users can label
     submodules with attributes to group them, so that a broad pathspec
     with attribute requirements, e.g. ':(attr:lib)', can be used to say
     any and all modules with the 'lib' attribute are interesting.
     Since the .gitattributes file, just like the .gitmodules file, is
     tracked by the superproject, when a submodule moves in the
     superproject tree, the project can adjust which path gets the
     attribute in .gitattributes, just like it can adjust which path has
     the submodule in .gitmodules.

Neither of these two additional configuration options solve the problem
of wanting different submodules checked out in different worktrees
because multiple worktrees share .git/config.  Only once per-worktree
configurations become a reality can this be solved, but this is a
necessary preparatory step for that future.

Given these multiple ways to check if a submodule is of interest, the
more fine-grained submodule.<name>.active option has the highest order
of precedence followed by the pathspec check against submodule.active.
To ensure backwards compatibility, if neither of these options are set,
git falls back to checking the submodule.<name>.url option to determine
if a submodule is interesting.

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-18 09:51:23 -07:00
Junio C Hamano 94c9b5af70 Merge branch 'cc/split-index-config'
The experimental "split index" feature has gained a few
configuration variables to make it easier to use.

* cc/split-index-config: (22 commits)
  Documentation/git-update-index: explain splitIndex.*
  Documentation/config: add splitIndex.sharedIndexExpire
  read-cache: use freshen_shared_index() in read_index_from()
  read-cache: refactor read_index_from()
  t1700: test shared index file expiration
  read-cache: unlink old sharedindex files
  config: add git_config_get_expiry() from gc.c
  read-cache: touch shared index files when used
  sha1_file: make check_and_freshen_file() non static
  Documentation/config: add splitIndex.maxPercentChange
  t1700: add tests for splitIndex.maxPercentChange
  read-cache: regenerate shared index if necessary
  config: add git_config_get_max_percent_split_change()
  Documentation/git-update-index: talk about core.splitIndex config var
  Documentation/config: add information for core.splitIndex
  t1700: add tests for core.splitIndex
  update-index: warn in case of split-index incoherency
  read-cache: add and then use tweak_split_index()
  split-index: add {add,remove}_split_index() functions
  config: add git_config_get_split_index()
  ...
2017-03-17 13:50:23 -07:00
Nguyễn Thái Ngọc Duy 3efd0bedc6 config: add conditional include
Sometimes a set of repositories want to share configuration settings
among themselves that are distinct from other such sets of repositories.
A user may work on two projects, each of which have multiple
repositories, and use one user.email for one project while using another
for the other.

Setting $GIT_DIR/.config works, but if the penalty of forgetting to
update $GIT_DIR/.config is high (especially when you end up cloning
often), it may not be the best way to go. Having the settings in
~/.gitconfig, which would work for just one set of repositories, would
not well in such a situation. Having separate ${HOME}s may add more
problems than it solves.

Extend the include.path mechanism that lets a config file include
another config file, so that the inclusion can be done only when some
conditions hold. Then ~/.gitconfig can say "include config-project-A
only when working on project-A" for each project A the user works on.

In this patch, the only supported grouping is based on $GIT_DIR (in
absolute path), so you would need to group repositories by directory, or
something like that to take advantage of it.

We already have include.path for unconditional includes. This patch goes
with includeIf.<condition>.path to make it clearer that a condition is
required. The new config has the same backward compatibility approach as
include.path: older git versions that don't understand includeIf will
simply ignore them.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-11 19:56:16 -08:00
Nguyễn Thái Ngọc Duy 1050e9874b config.txt: reflow the second include.path paragraph
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-11 19:49:43 -08:00
Nguyễn Thái Ngọc Duy df0233be96 config.txt: clarify multiple key values in include.path
The phrasing in this paragraph may give an impression that you can only
use it once. Rephrase it a bit.

Helped-by: Philip Oakley <philipoakley@iee.org>
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-11 19:49:30 -08:00
Christian Couder b46013950a Documentation/git-update-index: explain splitIndex.*
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-06 12:09:28 -08:00
Christian Couder b2dd1c5c34 Documentation/config: add splitIndex.sharedIndexExpire
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-06 12:09:28 -08:00
Andreas Heiduk 860cd699c2 Documentation: improve description for core.quotePath
Linking the description for pathname quoting to the configuration
variable "core.quotePath" removes inconstistent and incomplete
sections while also giving two hints how to deal with it: Either with
"-c core.quotePath=false" or with "-z".

Signed-off-by: Andreas Heiduk <asheiduk@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-02 11:40:51 -08:00
Christian Couder e77cf4ee65 Documentation/config: add splitIndex.maxPercentChange
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-01 13:24:22 -08:00
Christian Couder 66f9e7a6e8 Documentation/config: add information for core.splitIndex
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-03-01 13:24:21 -08:00
Junio C Hamano c96bc189b5 Merge branch 'dt/gc-ignore-old-gc-logs'
A "gc.log" file left by a backgrounded "gc --auto" disables further
automatic gc; it has been taught to run at least once a day (by
default) by ignoring a stale "gc.log" file that is too old.

* dt/gc-ignore-old-gc-logs:
  gc: ignore old gc.log files
2017-02-27 13:57:15 -08:00
Junio C Hamano be6ab596a8 Merge branch 'sf/putty-w-args'
The command line options for ssh invocation needs to be tweaked for
some implementations of SSH (e.g. PuTTY plink wants "-P <port>"
while OpenSSH wants "-p <port>" to specify port to connect to), and
the variant was guessed when GIT_SSH environment variable is used
to specify it.  The logic to guess now applies to the command
specified by the newer GIT_SSH_COMMAND and also core.sshcommand
configuration variable, and comes with an escape hatch for users to
deal with misdetected cases.

* sf/putty-w-args:
  connect.c: stop conflating ssh command names and overrides
  connect: Add the envvar GIT_SSH_VARIANT and ssh.variant config
  git_connect(): factor out SSH variant handling
  connect: rename tortoiseplink and putty variables
  connect: handle putty/plink also in GIT_SSH_COMMAND
2017-02-27 13:57:15 -08:00
Junio C Hamano a411726930 Merge branch 'ps/urlmatch-wildcard'
The <url> part in "http.<url>.<variable>" configuration variable
can now be spelled with '*' that serves as wildcard.
E.g. "http.https://*.example.com.proxy" can be used to specify the
proxy used for https://a.example.com, https://b.example.com, etc.,
i.e. any host in the example.com domain.

* ps/urlmatch-wildcard:
  urlmatch: allow globbing for the URL host part
  urlmatch: include host in urlmatch ranking
  urlmatch: split host and port fields in `struct url_info`
  urlmatch: enable normalization of URLs with globs
  mailmap: add Patrick Steinhardt's work address
2017-02-27 13:57:12 -08:00
Junio C Hamano 62fef5c564 Merge branch 'dp/submodule-doc-markup-fix'
Doc fix.

* dp/submodule-doc-markup-fix:
  config.txt: fix formatting of submodule.alternateErrorStrategy section
2017-02-16 14:45:15 -08:00
David Pursehouse 8ab9740d9f config.txt: fix formatting of submodule.alternateErrorStrategy section
Add missing `::` after the title.

Signed-off-by: David Pursehouse <dpursehouse@collab.net>
Acked-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-16 13:46:20 -08:00
David Turner a831c06a2b gc: ignore old gc.log files
A server can end up in a state where there are lots of unreferenced
loose objects (say, because many users are doing a bunch of rebasing
and pushing their rebased branches).  Running "git gc --auto" in
this state would cause a gc.log file to be created, preventing
future auto gcs, causing pack files to pile up.  Since many git
operations are O(n) in the number of pack files, this would lead to
poor performance.

Git should never get itself into a state where it refuses to do any
maintenance, just because at some point some piece of the maintenance
didn't make progress.

Teach Git to ignore gc.log files which are older than (by default)
one day old, which can be tweaked via the gc.logExpiry configuration
variable.  That way, these pack files will get cleaned up, if
necessary, at least once per day.  And operators who find a need for
more-frequent gcs can adjust gc.logExpiry to meet their needs.

There is also some cleanup: a successful manual gc, or a
warning-free auto gc with an old log file, will remove any old
gc.log files.

It might still happen that manual intervention is required
(e.g. because the repo is corrupt), but at the very least it won't
be because Git is too dumb to try again.

Signed-off-by: David Turner <dturner@twosigma.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-13 15:19:11 -08:00
Junio C Hamano fafca0f72a Merge branch 'cw/log-updates-for-all-refs-really'
The "core.logAllRefUpdates" that used to be boolean has been
enhanced to take 'always' as well, to record ref updates to refs
other than the ones that are expected to be updated (i.e. branches,
remote-tracking branches and notes).

* cw/log-updates-for-all-refs-really:
  doc: add note about ignoring '--no-create-reflog'
  update-ref: add test cases for bare repository
  refs: add option core.logAllRefUpdates = always
  config: add markup to core.logAllRefUpdates doc
2017-02-03 11:25:19 -08:00
Junio C Hamano 85279e8649 Merge branch 'nd/log-graph-configurable-colors'
Some people feel the default set of colors used by "git log --graph"
rather limiting.  A mechanism to customize the set of colors has
been introduced.

* nd/log-graph-configurable-colors:
  document behavior of empty color name
  color_parse_mem: allow empty color spec
  log --graph: customize the graph lines with config log.graphColors
  color.c: trim leading spaces in color_parse_mem()
  color.c: fix color_parse_mem() with value_len == 0
2017-02-02 13:36:58 -08:00
Jeff King 512aba261a document behavior of empty color name
Commit 55cccf4bb (color_parse_mem: allow empty color spec,
2017-02-01) clearly defined the behavior of an empty color
config variable. Let's document that, and give a hint about
why it might be useful.

It's important not to say that it makes the item uncolored,
because it doesn't. It just sets no attributes, which means
that any previous attributes continue to take effect.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-02 12:23:16 -08:00
Patrick Steinhardt a272b9e70a urlmatch: allow globbing for the URL host part
The URL matching function computes for two URLs whether they match not.
The match is performed by splitting up the URL into different parts and
then doing an exact comparison with the to-be-matched URL.

The main user of `urlmatch` is the configuration subsystem. It allows to
set certain configurations based on the URL which is being connected to
via keys like `http.<url>.*`. A common use case for this is to set
proxies for only some remotes which match the given URL. Unfortunately,
having exact matches for all parts of the URL can become quite tedious
in some setups. Imagine for example a corporate network where there are
dozens or even hundreds of subdomains, which would have to be configured
individually.

Allow users to write an asterisk '*' in place of any 'host' or
'subdomain' label as part of the host name.  For example,
"http.https://*.example.com.proxy" sets "http.proxy" for all direct
subdomains of "https://example.com", e.g. "https://foo.example.com", but
not "https://foo.bar.example.com".

Signed-off-by: Patrick Steinhardt <patrick.steinhardt@elego.de>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-01 13:22:50 -08:00
Segev Finer dd33e07766 connect: Add the envvar GIT_SSH_VARIANT and ssh.variant config
This environment variable and configuration value allow to
override the autodetection of plink/tortoiseplink in case that
Git gets it wrong.

[jes: wrapped overly-long lines, factored out and changed
get_ssh_variant() to handle_ssh_variant() to accomodate the
change from the putty/tortoiseplink variables to
port_option/needs_batch, adjusted the documentation, free()d
value obtained from the config.]

Signed-off-by: Segev Finer <segev208@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-02-01 10:57:53 -08:00
Junio C Hamano 46ab222616 Merge branch 'jc/abbrev-autoscale-config' into maint
Recent update to the default abbreviation length that auto-scales
lacked documentation update, which has been corrected.

* jc/abbrev-autoscale-config:
  config.abbrev: document the new default that auto-scales
2017-01-31 13:32:06 -08:00
Cornelius Weig 341fb28621 refs: add option core.logAllRefUpdates = always
When core.logallrefupdates is true, we only create a new reflog for refs
that are under certain well-known hierarchies. The reason is that we
know that some hierarchies (like refs/tags) are not meant to change, and
that unknown hierarchies might not want reflogs at all (e.g., a
hypothetical refs/foo might be meant to change often and drop old
history immediately).

However, sometimes it is useful to override this decision and simply log
for all refs, because the safety and audit trail is more important than
the performance implications of keeping the log around.

This patch introduces a new "always" mode for the core.logallrefupdates
option which will log updates to everything under refs/, regardless
where in the hierarchy it is (we still will not log things like
ORIG_HEAD and FETCH_HEAD, which are known to be transient).

Based-on-patch-by: Jeff King <peff@peff.net>
Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-31 10:01:24 -08:00
Cornelius Weig d0c93194ec config: add markup to core.logAllRefUpdates doc
Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-30 13:45:50 -08:00
Nguyễn Thái Ngọc Duy 73c727d69f log --graph: customize the graph lines with config log.graphColors
If you have a 256 colors terminal (or one with true color support), then
the predefined 12 colors seem limited. On the other hand, you don't want
to draw graph lines with every single color in this mode because the two
colors could look extremely similar. This option allows you to hand pick
the colors you want.

Even with standard terminal, if your background color is neither black
or white, then the graph line may match your background and become
hidden. You can exclude your background color (or simply the colors you
hate) with this.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-23 18:32:11 -08:00
Junio C Hamano 1ac244d5b2 Merge branch 'sg/fix-versioncmp-with-common-suffix'
The prereleaseSuffix feature of version comparison that is used in
"git tag -l" did not correctly when two or more prereleases for the
same release were present (e.g. when 2.0, 2.0-beta1, and 2.0-beta2
are there and the code needs to compare 2.0-beta1 and 2.0-beta2).

* sg/fix-versioncmp-with-common-suffix:
  versioncmp: generalize version sort suffix reordering
  versioncmp: factor out helper for suffix matching
  versioncmp: use earliest-longest contained suffix to determine sorting order
  versioncmp: cope with common part overlapping with prerelease suffix
  versioncmp: pass full tagnames to swap_prereleases()
  t7004-tag: add version sort tests to show prerelease reordering issues
  t7004-tag: use test_config helper
  t7004-tag: delete unnecessary tags with test_when_finished
2017-01-23 15:59:21 -08:00
Junio C Hamano 647a1bcf14 Merge branch 'mm/gc-safety-doc' into maint
Doc update.

* mm/gc-safety-doc:
  git-gc.txt: expand discussion of races with other processes
2017-01-17 15:19:11 -08:00
Junio C Hamano f976c89a20 Merge branch 'mm/push-social-engineering-attack-doc' into maint
Doc update on fetching and pushing.

* mm/push-social-engineering-attack-doc:
  doc: mention transfer data leaks in more places
2017-01-17 15:19:10 -08:00
Junio C Hamano 48d23c12e7 Merge branch 'dt/smart-http-detect-server-going-away' into maint
When the http server gives an incomplete response to a smart-http
rpc call, it could lead to client waiting for a full response that
will never come.  Teach the client side to notice this condition
and abort the transfer.

An improvement counterproposal has failed.
cf. <20161114194049.mktpsvgdhex2f4zv@sigill.intra.peff.net>

* dt/smart-http-detect-server-going-away:
  upload-pack: optionally allow fetching any sha1
  remote-curl: don't hang when a server dies before any output
2017-01-17 15:19:03 -08:00
Junio C Hamano 5ce6f51ff7 Merge branch 'jk/http-walker-limit-redirect' into maint
Update the error messages from the dumb-http client when it fails
to obtain loose objects; we used to give sensible error message
only upon 404 but we now forbid unexpected redirects that needs to
be reported with something sensible.

* jk/http-walker-limit-redirect:
  http-walker: complain about non-404 loose object errors
  http: treat http-alternates like redirects
  http: make redirects more obvious
  remote-curl: rename shadowed options variable
  http: always update the base URL for redirects
  http: simplify update_url_from_redirect
2017-01-17 14:49:29 -08:00
SZEDER Gábor c026557a37 versioncmp: generalize version sort suffix reordering
The 'versionsort.prereleaseSuffix' configuration variable, as its name
suggests, is supposed to only deal with tagnames with prerelease
suffixes, and allows sorting those prerelease tags in a user-defined
order before the suffixless main release tag, instead of sorting them
simply lexicographically.

However, the previous changes in this series resulted in an
interesting and useful property of version sort:

  - The empty string as a configured suffix matches all tagnames,
    including tagnames without any suffix, but

  - tagnames containing a "real" configured suffix are still ordered
    according to that real suffix, because any longer suffix takes
    precedence over the empty string.

Exploiting this property we can easily generalize suffix reordering
and specify the order of tags with given suffixes not only before but
even after a main release tag by using the empty suffix to denote the
position of the main release tag, without any algorithm changes:

  $ git -c versionsort.prereleaseSuffix=-alpha \
        -c versionsort.prereleaseSuffix=-beta \
        -c versionsort.prereleaseSuffix="" \
        -c versionsort.prereleaseSuffix=-gamma \
        -c versionsort.prereleaseSuffix=-delta \
        tag -l --sort=version:refname 'v3.0*'
  v3.0-alpha1
  v3.0-beta1
  v3.0
  v3.0-gamma1
  v3.0-delta1

Since 'versionsort.prereleaseSuffix' is not a fitting name for a
configuration variable to control this more general suffix reordering,
introduce the new variable 'versionsort.suffix'.  Still keep the old
configuration variable name as a deprecated alias, though, to avoid
suddenly breaking setups already using it.  Ignore the old variable if
both old and new configuration variables are set, but emit a warning
so users will be aware of it and can fix their configuration.  Extend
the documentation to describe and add a test to check this more
general behavior.

Note: since the empty suffix matches all tagnames, tagnames with
suffixes not included in the configuration are listed together with
the suffixless main release tag, ordered lexicographically right after
that, i.e. before tags with suffixes listed in the configuration
following the empty suffix.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2017-01-12 12:25:24 -08:00
Junio C Hamano 33cf69403c Merge branch 'jc/abbrev-autoscale-config'
Recent update to the default abbreviation length that auto-scales
lacked documentation update, which has been corrected.

* jc/abbrev-autoscale-config:
  config.abbrev: document the new default that auto-scales
2017-01-10 15:24:26 -08:00
Junio C Hamano d984592043 Merge branch 'dt/smart-http-detect-server-going-away'
When the http server gives an incomplete response to a smart-http
rpc call, it could lead to client waiting for a full response that
will never come.  Teach the client side to notice this condition
and abort the transfer.

An improvement counterproposal has failed.
cf. <20161114194049.mktpsvgdhex2f4zv@sigill.intra.peff.net>

* dt/smart-http-detect-server-going-away:
  upload-pack: optionally allow fetching any sha1
  remote-curl: don't hang when a server dies before any output
2017-01-10 15:24:25 -08:00
Junio C Hamano 979b82f19f Merge branch 'mm/gc-safety-doc'
Doc update.

* mm/gc-safety-doc:
  git-gc.txt: expand discussion of races with other processes
2017-01-10 15:24:25 -08:00
Junio C Hamano 5f52e70879 Merge branch 'mm/push-social-engineering-attack-doc'
Doc update on fetching and pushing.

* mm/push-social-engineering-attack-doc:
  doc: mention transfer data leaks in more places
2017-01-10 15:24:24 -08:00
Junio C Hamano 9d540e9726 Merge branch 'bw/transport-protocol-policy'
Finer-grained control of what protocols are allowed for transports
during clone/fetch/push have been enabled via a new configuration
mechanism.

* bw/transport-protocol-policy:
  http: respect protocol.*.allow=user for http-alternates
  transport: add from_user parameter to is_transport_allowed
  http: create function to get curl allowed protocols
  transport: add protocol policy config option
  http: always warn if libcurl version is too old
  lib-proto-disable: variable name fix
2016-12-27 00:11:41 -08:00
Junio C Hamano 48d5014dd4 config.abbrev: document the new default that auto-scales
We somehow forgot to update the "default is 7" in the
documentation.  Also give a way to explicitly ask the auto-scaling
by setting config.abbrev to "auto".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-22 13:17:15 -08:00
Junio C Hamano 8a2882f23e Merge branch 'jk/http-walker-limit-redirect-2.9'
Transport with dumb http can be fooled into following foreign URLs
that the end user does not intend to, especially with the server
side redirects and http-alternates mechanism, which can lead to
security issues.  Tighten the redirection and make it more obvious
to the end user when it happens.

* jk/http-walker-limit-redirect-2.9:
  http: treat http-alternates like redirects
  http: make redirects more obvious
  remote-curl: rename shadowed options variable
  http: always update the base URL for redirects
  http: simplify update_url_from_redirect
2016-12-19 14:45:32 -08:00
Brandon Williams f1762d772e transport: add protocol policy config option
Previously the `GIT_ALLOW_PROTOCOL` environment variable was used to
specify a whitelist of protocols to be used in clone/fetch/push
commands.  This patch introduces new configuration options for more
fine-grained control for allowing/disallowing protocols.  This also has
the added benefit of allowing easier construction of a protocol
whitelist on systems where setting an environment variable is
non-trivial.

Now users can specify a policy to be used for each type of protocol via
the 'protocol.<name>.allow' config option.  A default policy for all
unconfigured protocols can be set with the 'protocol.allow' config
option.  If no user configured default is made git will allow known-safe
protocols (http, https, git, ssh, file), disallow known-dangerous
protocols (ext), and have a default policy of `user` for all other
protocols.

The supported policies are `always`, `never`, and `user`.  The `user`
policy can be used to configure a protocol to be usable when explicitly
used by a user, while disallowing it for commands which run
clone/fetch/push commands without direct user intervention (e.g.
recursive initialization of submodules).  Commands which can potentially
clone/fetch/push from untrusted repositories without user intervention
can export `GIT_PROTOCOL_FROM_USER` with a value of '0' to prevent
protocols configured to the `user` policy from being used.

Fix remote-ext tests to use the new config to allow the ext
protocol to be tested.

Based on a patch by Jeff King <peff@peff.net>

Signed-off-by: Brandon Williams <bmwill@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-15 09:29:13 -08:00
SZEDER Gábor 51acfa9db5 versioncmp: use earliest-longest contained suffix to determine sorting order
When comparing tagnames, it is possible that a tagname contains more
than one of the configured prerelease suffixes around the first
different character.  After fixing a bug in the previous commit such a
tagname is sorted according to the contained suffix which comes first
in the configuration.  This is, however, not quite the right thing to
do in the following corner cases:

  1.   $ git -c versionsort.suffix=-bar
             -c versionsort.suffix=-foo-baz
             -c versionsort.suffix=-foo-bar
             tag -l --sort=version:refname 'v1*'
       v1.0-foo-bar
       v1.0-foo-baz

     The suffix of the tagname 'v1.0-foo-bar' is clearly '-foo-bar',
     so it should be listed last.  However, as it also contains '-bar'
     around the first different character, it is listed first instead,
     because that '-bar' suffix comes first the configuration.

  2. One of the configured suffixes starts with the other:

       $ git -c versionsort.prereleasesuffix=-pre \
             -c versionsort.prereleasesuffix=-prerelease \
             tag -l --sort=version:refname 'v2*'
       v2.0-prerelease1
       v2.0-pre1
       v2.0-pre2

     Here the tagname 'v2.0-prerelease1' should be the last.  When
     comparing 'v2.0-pre1' and 'v2.0-prerelease1' the first different
     characters are '1' and 'r', respectively.  Since this first
     different character must be part of the configured suffix, the
     '-pre' suffix is not recognized in the first tagname.  OTOH, the
     '-prerelease' suffix is properly recognized in
     'v2.0-prerelease1', thus it is listed first.

Improve version sort in these corner cases, and

  - look for a configured prerelease suffix containing the first
    different character or ending right before it, so the '-pre'
    suffixes are recognized in case (2).  This also means that
    when comparing tagnames 'v2.0-pre1' and 'v2.0-pre2',
    swap_prereleases() would find the '-pre' suffix in both, but then
    it will return "undecided" and the caller will do the right thing
    by sorting based in '1' and '2'.

  - If the tagname contains more than one suffix, then give precedence
    to the contained suffix that starts at the earliest offset in the
    tagname to address (1).

  - If there are more than one suffixes starting at that earliest
    position, then give precedence to the longest of those suffixes,
    thus ensuring that in (2) the tagname 'v2.0-prerelease1' won't be
    sorted based on the '-pre' suffix.

Add tests for these corner cases and adjust the documentation
accordingly.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-08 11:11:57 -08:00
SZEDER Gábor b8231660fa versioncmp: cope with common part overlapping with prerelease suffix
Version sort with prerelease reordering sometimes puts tagnames in the
wrong order, when the common part of two compared tagnames overlaps
with the leading character(s) of one or more configured prerelease
suffixes.  Note the position of "v2.1.0-beta-1":

  $ git -c versionsort.prereleaseSuffix=-beta \
        tag -l --sort=version:refname v2.1.*
  v2.1.0-beta-2
  v2.1.0-beta-3
  v2.1.0
  v2.1.0-RC1
  v2.1.0-RC2
  v2.1.0-beta-1
  v2.1.1
  v2.1.2

The reason is that when comparing a pair of tagnames, first
versioncmp() looks for the first different character in a pair of
tagnames, and then the swap_prereleases() helper function looks for a
configured prerelease suffix _starting at_ that character.  Thus, when
in the above example the sorting algorithm happens to compare the
tagnames "v2.1.0-beta-1" and "v2.1.0-RC2", swap_prereleases() tries to
match the suffix "-beta" against "beta-1" to no avail, and the two
tagnames erroneously end up being ordered lexicographically.

To fix this issue change swap_prereleases() to look for configured
prerelease suffixes _containing_ the position of that first different
character.

Care must be taken, when a configured suffix is longer than the
tagnames' common part up to the first different character, to avoid
reading memory before the beginning of the tagnames.  Add a test that
uses an exceptionally long prerelease suffix to check for this, in the
hope that in case of a regression the illegal memory access causes a
segfault in 'git tag' on one of the commonly used platforms (the test
happens to pass successfully on my Linux system with the safety check
removed), or at least makes valgrind complain.

Under some circumstances it's possible that more than one prerelease
suffixes can be found in the same tagname around that first different
character.  With this simple bugfix patch such a tagname is sorted
according to the contained suffix that comes first in the
configuration for now.  This is less than ideal in some cases, and the
following patch will take care of those.

Reported-by: Leho Kraav <leho@conversionready.com>
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-08 11:11:57 -08:00
Jeff King 50d3413740 http: make redirects more obvious
We instruct curl to always follow HTTP redirects. This is
convenient, but it creates opportunities for malicious
servers to create confusing situations. For instance,
imagine Alice is a git user with access to a private
repository on Bob's server. Mallory runs her own server and
wants to access objects from Bob's repository.

Mallory may try a few tricks that involve asking Alice to
clone from her, build on top, and then push the result:

  1. Mallory may simply redirect all fetch requests to Bob's
     server. Git will transparently follow those redirects
     and fetch Bob's history, which Alice may believe she
     got from Mallory. The subsequent push seems like it is
     just feeding Mallory back her own objects, but is
     actually leaking Bob's objects. There is nothing in
     git's output to indicate that Bob's repository was
     involved at all.

     The downside (for Mallory) of this attack is that Alice
     will have received Bob's entire repository, and is
     likely to notice that when building on top of it.

  2. If Mallory happens to know the sha1 of some object X in
     Bob's repository, she can instead build her own history
     that references that object. She then runs a dumb http
     server, and Alice's client will fetch each object
     individually. When it asks for X, Mallory redirects her
     to Bob's server. The end result is that Alice obtains
     objects from Bob, but they may be buried deep in
     history. Alice is less likely to notice.

Both of these attacks are fairly hard to pull off. There's a
social component in getting Mallory to convince Alice to
work with her. Alice may be prompted for credentials in
accessing Bob's repository (but not always, if she is using
a credential helper that caches). Attack (1) requires a
certain amount of obliviousness on Alice's part while making
a new commit. Attack (2) requires that Mallory knows a sha1
in Bob's repository, that Bob's server supports dumb http,
and that the object in question is loose on Bob's server.

But we can probably make things a bit more obvious without
any loss of functionality. This patch does two things to
that end.

First, when we encounter a whole-repo redirect during the
initial ref discovery, we now inform the user on stderr,
making attack (1) much more obvious.

Second, the decision to follow redirects is now
configurable. The truly paranoid can set the new
http.followRedirects to false to avoid any redirection
entirely. But for a more practical default, we will disallow
redirects only after the initial ref discovery. This is
enough to thwart attacks similar to (2), while still
allowing the common use of redirects at the repository
level. Since c93c92f30 (http: update base URLs when we see
redirects, 2013-09-28) we re-root all further requests from
the redirect destination, which should generally mean that
no further redirection is necessary.

As an escape hatch, in case there really is a server that
needs to redirect individual requests, the user can set
http.followRedirects to "true" (and this can be done on a
per-server basis via http.*.followRedirects config).

Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-12-06 12:32:48 -08:00
Junio C Hamano 50b8276ab9 Merge branch 'jk/rebase-config-insn-fmt-docfix' into maint
Documentation fix.

* jk/rebase-config-insn-fmt-docfix:
  doc: fix missing "::" in config list
2016-11-29 13:27:58 -08:00