Commit graph

1375 commits

Author SHA1 Message Date
Junio C Hamano 0e35fcb412 Merge branch 'cc/untracked'
Update the untracked cache subsystem and change its primary UI from
"git update-index" to "git config".

* cc/untracked:
  t7063: add tests for core.untrackedCache
  test-dump-untracked-cache: don't modify the untracked cache
  config: add core.untrackedCache
  dir: simplify untracked cache "ident" field
  dir: add remove_untracked_cache()
  dir: add {new,add}_untracked_cache()
  update-index: move 'uc' var declaration
  update-index: add untracked cache notifications
  update-index: add --test-untracked-cache
  update-index: use enum for untracked cache options
  dir: free untracked cache when removing it
2016-02-10 14:20:06 -08:00
Junio C Hamano ad25723e69 Merge branch 'jk/ref-cache-non-repository-optim'
The underlying machinery used by "ls-files -o" and other commands
have been taught not to create empty submodule ref cache for a
directory that is not a submodule.  This removes a ton of wasted
CPU cycles.

* jk/ref-cache-non-repository-optim:
  resolve_gitlink_ref: ignore non-repository paths
  clean: make is_git_repository a public function
2016-02-03 14:16:07 -08:00
Christian Couder dae6c322fa test-dump-untracked-cache: don't modify the untracked cache
To correctly perform its testing function,
test-dump-untracked-cache should not change the state of the
untracked cache in the index.

As a previous patch makes read_index_from() change the state of
the untracked cache and as test-dump-untracked-cache indirectly
calls this function, we need a mechanism to prevent
read_index_from() from changing the untracked cache state when
it's called from test-dump-untracked-cache.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-27 12:30:07 -08:00
Christian Couder 435ec090ec config: add core.untrackedCache
When we know that mtime on directory as given by the environment
is usable for the purpose of untracked cache, we may want the
untracked cache to be always used without any mtime test or
kernel name check being performed.

Also when we know that mtime is not usable for the purpose of
untracked cache, for example because the repo is shared over a
network file system, we may want the untracked-cache to be
automatically removed from the index.

Allow the user to express such preference by setting the
'core.untrackedCache' configuration variable, which can take
'keep', 'false', or 'true' and default to 'keep'.

When read_index_from() is called, it now adds or removes the
untracked cache in the index to respect the value of this
variable. So it does nothing if the value is `keep` or if the
variable is unset; it adds the untracked cache if the value is
`true`; and it removes the cache if the value is `false`.

`git update-index --[no-|force-]untracked-cache` still adds the
untracked cache to, or removes it, from the index, but this
shows a warning if it goes against the value of
core.untrackedCache, because the next time the index is read
the untracked cache will be added or removed if the
configuration is set to do so.

Also `--untracked-cache` used to check that the underlying
operating system and file system change `st_mtime` field of a
directory if files are added or deleted in that directory. But
because those tests take a long time, `--untracked-cache` no
longer performs them. Instead, there is now
`--test-untracked-cache` to perform the tests. This change
makes `--untracked-cache` the same as `--force-untracked-cache`.

This last change is backward incompatible and should be
mentioned in the release notes.

Helped-by: Duy Nguyen <pclouds@gmail.com>
Helped-by: Torsten Bögershausen <tboegi@web.de>
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>

read-cache: Duy'sfixup

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-27 12:30:00 -08:00
Jeff King ffd036b128 clean: make is_git_repository a public function
We have always had is_git_directory(), for looking at a
specific directory to see if it contains a git repo. In
0179ca7 (clean: improve performance when removing lots of
directories, 2015-06-15), we added is_git_repository() which
checks for a non-bare repository by looking at its ".git"
entry.

However, the fix in 0179ca7 needs to be applied other
places, too. Let's make this new helper globally available.
We need to give it a better name, though, to avoid confusion
with is_git_directory(). This patch does that, documents
both functions with a comment to reduce confusion, and
removes the clean-specific references in the comments.

Based-on-a-patch-by: Andreas Krey <a.krey@gmx.de>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-25 11:41:53 -08:00
Junio C Hamano a736764a7b Merge branch 'jk/clang-pedantic'
A few unportable C construct have been spotted by clang compiler
and have been fixed.

* jk/clang-pedantic:
  bswap: add NO_UNALIGNED_LOADS define
  avoid shifting signed integers 31 bits
2016-01-20 11:43:29 -08:00
Junio C Hamano cc14ea8cf4 Merge branch 'nd/ita-cleanup'
Paths that have been told the index about with "add -N" are not
quite yet in the index, but a few commands behaved as if they
already are in a harmful way.

* nd/ita-cleanup:
  grep: make it clear i-t-a entries are ignored
  add and use a convenience macro ce_intent_to_add()
  blame: remove obsolete comment
2016-01-20 11:43:25 -08:00
Junio C Hamano 4fd1359158 Merge branch 'jk/pack-revindex'
In-core storage of the reverse index for .pack files (which lets
you go from a pack offset to an object name) has been streamlined.

* jk/pack-revindex:
  pack-revindex: store entries directly in packed_git
  pack-revindex: drop hash table
2016-01-20 11:43:23 -08:00
Jeff King 9a93c6686f avoid shifting signed integers 31 bits
We sometimes use 32-bit unsigned integers as bit-fields.
It's fine to access the MSB, because it's unsigned. However,
doing so as "1 << 31" is wrong, because the constant "1" is
a signed int, and we shift into the sign bit, causing
undefined behavior.

We can fix this by using "1U" as the constant.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-01-04 09:51:16 -08:00
Jeff King 9d98bbf578 pack-revindex: store entries directly in packed_git
A pack_revindex struct has two elements: the revindex
entries themselves, and a pointer to the packed_git. We need
both to do lookups, because only the latter knows things
like the number of objects in the pack.

Now that packed_git contains the pack_revindex struct it's
just as easy to pass around the packed_git itself, and we do
not need the extra back-pointer.

We can instead just store the entries directly in the pack.
All functions which took a pack_revindex now just take a
packed_git. We still lazy-load in find_pack_revindex, so
most callers are unaffected.

The exception is the bitmap code, which computes the
revindex and caches the pointer when we load the bitmaps. We
can continue to load, drop the extra cache pointer, and just
access bitmap_git.pack.revindex directly.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-21 14:36:28 -08:00
Jeff King f4015337da pack-revindex: drop hash table
The main entry point to the pack-revindex code is
find_pack_revindex(). This calls revindex_for_pack(), which
lazily computes and caches the revindex for the pack.

We store the cache in a very simple hash table. It's created
by init_pack_revindex(), which inserts an entry for every
packfile we know about, and we never grow or shrink the
hash. If we ever need the revindex for a pack that isn't in
the hash, we die() with an internal error.

This can lead to a race, because we may load more packs
after having called init_pack_revindex(). For example,
imagine we have one process which needs to look at the
revindex for a variety of objects (e.g., cat-file's
"%(objectsize:disk)" format).  Simultaneously, git-gc is
running, which is doing a `git repack -ad`. We might hit a
sequence like:

  1. We need the revidx for some packed object. We call
     find_pack_revindex() and end up in init_pack_revindex()
     to create the hash table for all packs we know about.

  2. We look up another object and can't find it, because
     the repack has removed the pack it's in. We re-scan the
     pack directory and find a new pack containing the
     object. It gets added to our packed_git list.

  3. We call find_pack_revindex() for the new object, which
     hits revindex_for_pack() for our new pack. It can't
     find the packed_git in the revindex hash, and dies.

You could also replace the `repack` above with a push or
fetch to create a new pack, though these are less likely
(you would have to somehow learn about the new objects to
look them up).

Prior to 1a6d8b9 (do not discard revindex when re-preparing
packfiles, 2014-01-15), this was safe, as we threw away the
revindex whenever we re-scanned the pack directory (and thus
re-created the revindex hash on the fly). However, we don't
want to simply revert that commit, as it was solving a
different race.

So we have a few options:

  - We can fix the race in 1a6d8b9 differently, by having
    the bitmap code look in the revindex hash instead of
    caching the pointer. But this would introduce a lot of
    extra hash lookups for common bitmap operations.

  - We could teach the revindex to dynamically add new packs
    to the hash table. This would perform the same, but
    would mean adding extra code to the revindex hash (which
    currently cannot be resized at all).

  - We can get rid of the hash table entirely. There is
    exactly one revindex per pack, so we can just store it
    in the packed_git struct. Since it's initialized lazily,
    it does not add to the startup cost.

    This is the best of both worlds: less code and fewer
    hash table lookups.  The original code likely avoided
    this in the name of encapsulation. But the packed_git
    and reverse_index code are fairly intimate already, so
    it's not much of a loss.

This patch implements the final option. It's a minimal
conversion that retains the pack_revindex struct. No callers
need to change, and we can do further cleanup in a follow-on
patch.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-21 14:36:11 -08:00
Junio C Hamano fbe959dde7 Merge branch 'bc/format-patch-null-from-line'
"format-patch" has learned a new option to zero-out the commit
object name on the mbox "From " line.

* bc/format-patch-null-from-line:
  format-patch: check that header line has expected format
  format-patch: add an option to suppress commit hash
  sha1_file.c: introduce a null_oid constant
2015-12-21 10:59:08 -08:00
brian m. carlson 3e56e7245c sha1_file.c: introduce a null_oid constant
null_oid is the struct object_id equivalent to null_sha1.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-12-14 13:35:54 -08:00
Junio C Hamano 844a9ce472 Merge branch 'bc/object-id'
More transition from "unsigned char[40]" to "struct object_id".

This needed a few merge fixups, but is mostly disentangled from other
topics.

* bc/object-id:
  remote: convert functions to struct object_id
  Remove get_object_hash.
  Convert struct object to object_id
  Add several uses of get_object_hash.
  object: introduce get_object_hash macro.
  ref_newer: convert to use struct object_id
  push_refs_with_export: convert to struct object_id
  get_remote_heads: convert to struct object_id
  parse_fetch: convert to use struct object_id
  add_sought_entry_mem: convert to struct object_id
  Convert struct ref to use object_id.
  sha1_file: introduce has_object_file helper.
2015-12-10 12:36:13 -08:00
Junio C Hamano b1cda70fff Merge branch 'dt/refs-backend-pre-vtable'
Code preparation for pluggable ref backends.

* dt/refs-backend-pre-vtable:
  refs: break out ref conflict checks
  files_log_ref_write: new function
  initdb: make safe_create_dir public
  refs: split filesystem-based refs code into a new file
  refs/refs-internal.h: new header file
  refname_is_safe(): improve docstring
  pack_if_possible_fn(): use ref_type() instead of is_per_worktree_ref()
  copy_msg(): rename to copy_reflog_msg()
  verify_refname_available(): new function
  verify_refname_available(): rename function
2015-12-08 14:14:49 -08:00
Junio C Hamano b5d2d8eef0 Merge branch 'ad/sha1-update-chunked' into maint
Apple's common crypto implementation of SHA1_Update() does not take
more than 4GB at a time, and we now have a compile-time workaround
for it.

* ad/sha1-update-chunked:
  sha1: allow limiting the size of the data passed to SHA1_Update()
  sha1: provide another level of indirection for the SHA-1 functions
2015-12-08 14:05:03 -08:00
Junio C Hamano b50ceab48f Merge branch 'dk/gc-idx-wo-pack' into maint
Having a leftover .idx file without corresponding .pack file in
the repository hurts performance; "git gc" learned to prune them.

We may want to do the same for .bitmap (and notice but not prune
.keep) without corresponding .pack, but that can be a separate
topic.

* dk/gc-idx-wo-pack:
  gc: remove garbage .idx files from pack dir
  t5304: test cleaning pack garbage
  prepare_packed_git(): refactor garbage reporting in pack directory
2015-12-04 11:33:08 -08:00
Junio C Hamano 4672123fe5 Merge branch 'ad/sha1-update-chunked'
Apple's common crypto implementation of SHA1_Update() does not take
more than 4GB at a time, and we now have a compile-time workaround
for it.

* ad/sha1-update-chunked:
  sha1: allow limiting the size of the data passed to SHA1_Update()
  sha1: provide another level of indirection for the SHA-1 functions
2015-12-04 11:19:10 -08:00
brian m. carlson b419aa25d5 sha1_file: introduce has_object_file helper.
Add has_object_file, which is a wrapper around has_sha1_file, but for
struct object_id.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 08:02:05 -05:00
Jeff King 45014beac0 Merge branch 'dk/gc-idx-wo-pack'
Having a leftover .idx file without corresponding .pack file in
the repository hurts performance; "git gc" learned to prune them.

* dk/gc-idx-wo-pack:
  gc: remove garbage .idx files from pack dir
  t5304: test cleaning pack garbage
  prepare_packed_git(): refactor garbage reporting in pack directory
2015-11-20 06:55:34 -05:00
David Turner eb33876c26 initdb: make safe_create_dir public
Soon we will want to create initdb functions for ref backends, and
code from initdb that calls this function needs to move into the files
backend. So this function needs to be public.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
2015-11-20 04:52:01 -05:00
Atousa Pahlevan Duprat 001fd7a90b sha1: allow limiting the size of the data passed to SHA1_Update()
Using the previous commit's inredirection mechanism for SHA1,
support a chunked implementation of SHA1_Update() that limits the
amount of data in the chunk passed to SHA1_Update().

This is enabled by using the Makefile variable SHA1_MAX_BLOCK_SIZE
to specify chunk size.  When using Apple's CommonCrypto library this
is set to 1GiB (the implementation cannot handle more 4GiB).

Signed-off-by: Atousa Pahlevan Duprat <apahlevan@ieee.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-05 10:35:11 -08:00
Atousa Pahlevan Duprat 3bc72fde3f sha1: provide another level of indirection for the SHA-1 functions
The git source uses git_SHA1_Update() and friends to call into the
code that computes the hashes.  Traditionally, we used to map these
directly to underlying implementation of the SHA-1 hash (e.g.
SHA1_Update() from OpenSSL or blk_SHA1_Update() from block-sha1/).

This arrangement however makes it hard to tweak behaviour of the
underlying implementation without fully replacing.  If we want to
introduce a tweaked_SHA1_Update() wrapper to implement the "Update"
in a slightly different way, for example, the implementation of the
wrapper still would want to call into the underlying implementation,
but tweaked_SHA1_Update() cannot call git_SHA1_Update() to get to
the underlying implementation (often but not always SHA1_Update()).

Add another level of indirection that maps platform_SHA1_Update()
and friends to their underlying implementations, and by default make
git_SHA1_Update() and friends map to platform_SHA1_* functions.

Doing it this way will later allow us to map git_SHA1_Update() to
tweaked_SHA1_Update(), and the latter can use platform_SHA1_Update()
in its implementation.

Signed-off-by: Atousa Pahlevan Duprat <apahlevan@ieee.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-11-05 10:35:11 -08:00
Junio C Hamano 3a27eec48e Merge branch 'dt/name-hash-dir-entry-fix' into maint
The name-hash subsystem that is used to cope with case insensitive
filesystems keeps track of directories and their on-filesystem
cases for all the paths in the index by holding a pointer to a
randomly chosen cache entry that is inside the directory (for its
ce->ce_name component).  This pointer was not updated even when the
cache entry was removed from the index, leading to use after free.
This was fixed by recording the path for each directory instead of
borrowing cache entries and restructuring the API somewhat.

* dt/name-hash-dir-entry-fix:
  name-hash: don't reuse cache_entry in dir_entry
2015-11-03 15:32:40 -08:00
Junio C Hamano 1bf986bc9c Merge branch 'mk/submodule-gitdir-path' into maint
The submodule code has been taught to work better with separate
work trees created via "git worktree add".

* mk/submodule-gitdir-path:
  path: implement common_dir handling in git_pathdup_submodule()
  submodule refactor: use strbuf_git_path_submodule() in add_submodule_odb()
2015-11-03 15:32:35 -08:00
Junio C Hamano f836a32ba7 Merge branch 'js/clone-dissociate' into maint
"git clone --dissociate" runs a big "git repack" process at the
end, and it helps to close file descriptors that are open on the
packs and their idx files before doing so on filesystems that
cannot remove a file that is still open.

* js/clone-dissociate:
  clone --dissociate: avoid locking pack files
  sha1_file.c: add a function to release all packs
  sha1_file: consolidate code to close a pack's file descriptor
  t5700: demonstrate a Windows file locking issue with `git clone --dissociate`
2015-11-03 15:32:29 -08:00
Junio C Hamano f89baca1b9 Merge branch 'jk/repository-extension' into maint
Prepare for Git on-disk repository representation to undergo
backward incompatible changes by introducing a new repository
format version "1", with an extension mechanism.

* jk/repository-extension:
  introduce "preciousObjects" repository extension
  introduce "extensions" form of core.repositoryformatversion
2015-11-03 15:32:25 -08:00
Junio C Hamano 69fe31887b Merge branch 'dt/name-hash-dir-entry-fix'
The name-hash subsystem that is used to cope with case insensitive
filesystems keeps track of directories and their on-filesystem
cases for all the paths in the index by holding a pointer to a
randomly chosen cache entry that is inside the directory (for its
ce->ce_name component).  This pointer was not updated even when the
cache entry was removed from the index, leading to use after free.
This was fixed by recording the path for each directory instead of
borrowing cache entries and restructuring the API somewhat.

* dt/name-hash-dir-entry-fix:
  name-hash: don't reuse cache_entry in dir_entry
2015-10-29 13:59:19 -07:00
Junio C Hamano fa46579555 Merge branch 'jk/repository-extension'
Prepare for Git on-disk repository representation to undergo
backward incompatible changes by introducing a new repository
format version "1", with an extension mechanism.

* jk/repository-extension:
  introduce "preciousObjects" repository extension
  introduce "extensions" form of core.repositoryformatversion
2015-10-26 15:55:25 -07:00
David Turner 41284eb0f9 name-hash: don't reuse cache_entry in dir_entry
Stop reusing cache_entry in dir_entry; doing so causes a
use-after-free bug.

During merges, we free entries that we no longer need in the
destination index.  But those entries might have also been stored in
the dir_entry cache, and when a later call to add_to_index found them,
they would be used after being freed.

To prevent this, change dir_entry to store a copy of the name instead
of a pointer to a cache_entry.  This entails some refactoring of code
that expects the cache_entry.

Keith McGuigan <kmcguigan@twitter.com> diagnosed this bug and wrote
the initial patch, but this version does not use any of Keith's code.

Helped-by: Keith McGuigan <kmcguigan@twitter.com>
Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-21 12:47:38 -07:00
Junio C Hamano 78891795df Merge branch 'jk/war-on-sprintf'
Many allocations that is manually counted (correctly) that are
followed by strcpy/sprintf have been replaced with a less error
prone constructs such as xstrfmt.

Macintosh-specific breakage was noticed and corrected in this
reroll.

* jk/war-on-sprintf: (70 commits)
  name-rev: use strip_suffix to avoid magic numbers
  use strbuf_complete to conditionally append slash
  fsck: use for_each_loose_file_in_objdir
  Makefile: drop D_INO_IN_DIRENT build knob
  fsck: drop inode-sorting code
  convert strncpy to memcpy
  notes: document length of fanout path with a constant
  color: add color_set helper for copying raw colors
  prefer memcpy to strcpy
  help: clean up kfmclient munging
  receive-pack: simplify keep_arg computation
  avoid sprintf and strcpy with flex arrays
  use alloc_ref rather than hand-allocating "struct ref"
  color: add overflow checks for parsing colors
  drop strcpy in favor of raw sha1_to_hex
  use sha1_to_hex_r() instead of strcpy
  daemon: use cld->env_array when re-spawning
  stat_tracking_info: convert to argv_array
  http-push: use an argv_array for setup_revisions
  fetch-pack: use argv_array for index-pack / unpack-objects
  ...
2015-10-20 15:24:01 -07:00
Junio C Hamano db5adf24bf Merge branch 'js/clone-dissociate'
"git clone --dissociate" runs a big "git repack" process at the
end, and it helps to close file descriptors that are open on the
packs and their idx files before doing so on filesystems that
cannot remove a file that is still open.

* js/clone-dissociate:
  clone --dissociate: avoid locking pack files
  sha1_file.c: add a function to release all packs
  sha1_file: consolidate code to close a pack's file descriptor
  t5700: demonstrate a Windows file locking issue with `git clone --dissociate`
2015-10-15 15:43:49 -07:00
Junio C Hamano 1c630badac Merge branch 'mk/submodule-gitdir-path'
The submodule code has been taught to work better with separate
work trees created via "git worktree add".

* mk/submodule-gitdir-path:
  path: implement common_dir handling in git_pathdup_submodule()
  submodule refactor: use strbuf_git_path_submodule() in add_submodule_odb()
2015-10-15 15:43:32 -07:00
Johannes Schindelin 38849a8116 sha1_file.c: add a function to release all packs
On Windows, files that are in use cannot be removed or renamed. That
means that we have to release pack files when we are about to, say,
repack them. Let's introduce a convenient function to close all the
pack files and their idx files.

While at it, we consolidate the close windows/close fd/close index
stanza in `free_pack_by_name()` into the `close_pack()` function that
is used by the new `close_all_packs()` function to avoid repeated code.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-10-07 10:47:10 -07:00
Junio C Hamano 7b09c459d3 Merge branch 'jk/date-local'
"git log --date=local" used to only show the normal (default)
format in the local timezone.  The command learned to take 'local'
as an instruction to use the local timezone with other formats,
e.g. "git show --date=rfc-local".

* jk/date-local:
  t6300: add tests for "-local" date formats
  t6300: make UTC and local dates different
  date: make "local" orthogonal to date format
  date: check for "local" before anything else
  t6300: add test for "raw" date format
  t6300: introduce test_date() helper
  fast-import: switch crash-report date to iso8601
  Documentation/rev-list: don't list date formats
  Documentation/git-for-each-ref: don't list date formats
  Documentation/config: don't list date formats
  Documentation/blame-options: don't list date formats
2015-10-05 12:30:13 -07:00
Jeff King 48bcc1c3cc add_packed_git: convert strcpy into xsnprintf
We have the path "foo.idx", and we create a buffer big
enough to hold "foo.pack" and "foo.keep", and then strcpy
straight into it. This isn't a bug (we have enough space),
but it's very hard to tell from the strcpy that this is so.

Let's instead use strip_suffix to take off the ".idx",
record the size of our allocation, and use xsnprintf to make
sure we don't violate our assumptions.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25 10:18:18 -07:00
Jeff King af49c6d091 add reentrant variants of sha1_to_hex and find_unique_abbrev
The sha1_to_hex and find_unique_abbrev functions always
write into reusable static buffers. There are a few problems
with this:

  - future calls overwrite our result. This is especially
    annoying with find_unique_abbrev, which does not have a
    ring of buffers, so you cannot even printf() a result
    that has two abbreviated sha1s.

  - if you want to put the result into another buffer, we
    often strcpy, which looks suspicious when auditing for
    overflows.

This patch introduces sha1_to_hex_r and find_unique_abbrev_r,
which write into a user-provided buffer. Of course this is
just punting on the overflow-auditing, as the buffer
obviously needs to be GIT_SHA1_HEXSZ + 1 bytes. But it is
much easier to audit, since that is a well-known size.

We retain the non-reentrant forms, which just become thin
wrappers around the reentrant ones. This patch also adds a
strbuf variant of find_unique_abbrev, which will be handy in
later patches.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25 10:18:18 -07:00
Jeff King bb3788cebb add git_path_buf helper function
If you have a function that uses git_path a lot, but would
prefer to avoid the static buffers, it's useful to keep a
single scratch buffer locally and reuse it for each call.
You used to be able to do this with git_snpath:

  char buf[PATH_MAX];

  foo(git_snpath(buf, sizeof(buf), "foo"));
  bar(git_snpath(buf, sizeof(buf), "bar"));

but since 1a83c24, git_snpath has been replaced with
strbuf_git_path. This is good, because it removes the
arbitrary PATH_MAX limit. But using strbuf_git_path is more
awkward for two reasons:

  1. It adds to the buffer, rather than replacing it. This
     is consistent with other strbuf functions, but makes
     reuse of a single buffer more tedious.

  2. It doesn't return the buffer, so you can't format
     as part of a function's arguments.

The new git_path_buf solves both of these, so you can use it
like:

  struct strbuf buf = STRBUF_INIT;

  foo(git_path_buf(&buf, "foo"));
  bar(git_path_buf(&buf, "bar"));

  strbuf_release(&buf);

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-25 10:18:18 -07:00
Max Kirillov 11f9dd7191 path: implement common_dir handling in git_pathdup_submodule()
When submodule is a linked worktree, "git diff --submodule" and other
calls which directly access the submodule's object database do not correctly
calculate its path. Fix it by changing the git_pathdup_submodule() behavior,
to use either common or per-worktree directory.

Do it similarly as for parent repository, but ignore the GIT_COMMON_DIR
environment variable, because it would mean common directory for the parent
repository and does not make sense for submodule.

Also add test for functionality which uses this call.

Signed-off-by: Max Kirillov <max@max630.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-14 11:03:46 -07:00
Nguyễn Thái Ngọc Duy 895ff3b2c7 add and use a convenience macro ce_intent_to_add()
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-06 20:01:13 -07:00
Jeff King add00ba2de date: make "local" orthogonal to date format
Most of our "--date" modes are about the format of the date:
which items we show and in what order. But "--date=local" is
a bit of an oddball. It means "show the date in the normal
format, but using the local timezone". The timezone we use
is orthogonal to the actual format, and there is no reason
we could not have "localized iso8601", etc.

This patch adds a "local" boolean field to "struct
date_mode", and drops the DATE_LOCAL element from the
date_mode_type enum (it's now just DATE_NORMAL plus
local=1). The new feature is accessible to users by adding
"-local" to any date mode (e.g., "iso-local"), and we retain
"local" as an alias for "default-local" for backwards
compatibility.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-09-03 15:45:26 -07:00
Junio C Hamano b21089db6a Merge branch 'db/push-sign-if-asked'
The client side codepaths in "git push" have been cleaned up
and the user can request to perform an optional "signed push",
i.e. sign only when the other end accepts signed push.

* db/push-sign-if-asked:
  push: add a config option push.gpgSign for default signed pushes
  push: support signing pushes iff the server supports it
  builtin/send-pack.c: use parse_options API
  config.c: rename git_config_maybe_bool_text and export it as git_parse_maybe_bool
  transport: remove git_transport_options.push_cert
  gitremote-helpers.txt: document pushcert option
  Documentation/git-send-pack.txt: document --signed
  Documentation/git-send-pack.txt: wrap long synopsis line
  Documentation/git-push.txt: document when --signed may fail
2015-08-31 15:39:08 -07:00
Junio C Hamano d75bb73bcf Merge branch 'jc/am-state-fix'
Recent reimplementation of "git am" changed the format of state
files kept in $GIT_DIR/rebase-apply/ without meaning to do so,
primarily because write_file() API was cumbersome to use and it was
easy to mistakenly make text files with incomplete lines.  Update
write_file() interface to make it harder to misuse.

* jc/am-state-fix:
  write_file(): drop caller-supplied LF from calls to create a one-liner file
  write_file_v(): do not leave incomplete line at the end
  write_file(): drop "fatal" parameter
  builtin/am: make sure state files are text
  builtin/am: introduce write_state_*() helper functions
2015-08-31 15:39:03 -07:00
Junio C Hamano 7b7c10bf5e Merge branch 'jk/fix-alias-pager-config-key-warnings'
Because the configuration system does not allow "alias.0foo" and
"pager.0foo" as the configuration key, the user cannot use '0foo'
as a custom command name anyway, but "git 0foo" tried to look these
keys up and emitted useless warnings before saying '0foo is not a
git command'.  These warning messages have been squelched.

* jk/fix-alias-pager-config-key-warnings:
  config: silence warnings for command names with invalid keys
2015-08-31 15:38:57 -07:00
Junio C Hamano 12d6ce1dba write_file(): drop "fatal" parameter
All callers except three passed 1 for the "fatal" parameter to ask
this function to die upon error, but to a casual reader of the code,
it was not all obvious what that 1 meant.  Instead, split the
function into two based on a common write_file_v() that takes the
flag, introduce write_file_gently() as a new way to attempt creating
a file without dying on error, and make three callers to call it.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-24 13:09:02 -07:00
Jeff King 9e9de18f1a config: silence warnings for command names with invalid keys
When we are running the git command "foo", we may have to
look up the config keys "pager.foo" and "alias.foo". These
config schemes are mis-designed, as the command names can be
anything, but the config syntax has some restrictions. For
example:

  $ git foo_bar
  error: invalid key: pager.foo_bar
  error: invalid key: alias.foo_bar
  git: 'foo_bar' is not a git command. See 'git --help'.

You cannot name an alias with an underscore. And if you have
an external command with one, you cannot configure its
pager.

In the long run, we may develop a different config scheme
for these features. But in the near term (and because we'll
need to support the existing scheme indefinitely), we should
at least squelch the error messages shown above.

These errors come from git_config_parse_key. Ideally we
would pass a "quiet" flag to the config machinery, but there
are many layers between the pager code and the key parsing.
Passing a flag through all of those would be an invasive
change.

Instead, let's provide a config function to report on
whether a key is syntactically valid, and have the pager and
alias code skip lookup for bogus keys. We can build this
easily around the existing git_config_parse_key, with two
minor modifications:

  1. We now handle a NULL store_key, to validate but not
     write out the normalized key.

  2. We accept a "quiet" flag to avoid writing to stderr.
     This doesn't need to be a full-blown public "flags"
     field, because we can make the existing implementation
     a static helper function, keeping the mess contained
     inside config.c.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-24 08:52:23 -07:00
Junio C Hamano 8c9155e031 Merge branch 'jk/git-path'
git_path() and mkpath() are handy helper functions but it is easy
to misuse, as the callers need to be careful to keep the number of
active results below 4.  Their uses have been reduced.

* jk/git-path:
  memoize common git-path "constant" files
  get_repo_path: refactor path-allocation
  find_hook: keep our own static buffer
  refs.c: remove_empty_directories can take a strbuf
  refs.c: avoid git_path assignment in lock_ref_sha1_basic
  refs.c: avoid repeated git_path calls in rename_tmp_log
  refs.c: simplify strbufs in reflog setup and writing
  path.c: drop git_path_submodule
  refs.c: remove extra git_path calls from read_loose_refs
  remote.c: drop extraneous local variable from migrate_file
  prefer mkpathdup to mkpath in assignments
  prefer git_pathdup to git_path in some possibly-dangerous cases
  add_to_alternates_file: don't add duplicate entries
  t5700: modernize style
  cache.h: complete set of git_path_submodule helpers
  cache.h: clarify documentation for git_path, et al
2015-08-19 14:48:56 -07:00
Junio C Hamano 51a22ce147 Merge branch 'jc/finalize-temp-file'
Long overdue micro clean-up.

* jc/finalize-temp-file:
  sha1_file.c: rename move_temp_to_file() to finalize_object_file()
2015-08-19 14:48:55 -07:00
Dave Borowitz 9a549d4397 config.c: rename git_config_maybe_bool_text and export it as git_parse_maybe_bool
This helper function does not complain about the config variable
but just silently reports failure to the caller.  It is useful for
callers that need to parse any string that could be boolean or other
string (e.g. tristate yes/no/auto).

Signed-off-by: Dave Borowitz <dborowitz@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-19 12:43:22 -07:00
Junio C Hamano 0a489b0680 prepare_packed_git(): refactor garbage reporting in pack directory
The hook to report "garbage" files in $GIT_OBJECT_DIRECTORY/pack/
could be generic but is too specific to count-object's needs.

Move the part to produce human-readable messages to count-objects,
and refine the interface to callback with the "bits" with values
defined in the cache.h header file, so that other callers (e.g.
prune) can later use the same mechanism to enumerate different
kinds of garbage files and do something intelligent about them,
other than reporting in textual messages.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2015-08-17 09:14:59 -07:00