Commit graph

664 commits

Author SHA1 Message Date
Ronnie Sahlberg
01319837c5 refs.c: remove the onerr argument to ref_transaction_commit
Since all callers now use QUIET_ON_ERR we no longer need to provide an onerr
argument any more. Remove the onerr argument from the ref_transaction_commit
signature.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:42 -07:00
Ronnie Sahlberg
c1703d7634 refs.c: make update_ref_write update a strbuf on failure
Change update_ref_write to also update an error strbuf on failure.
This makes the error available to ref_transaction_commit callers if the
transaction failed due to update_ref_sha1/write_ref_sha1 failures.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:42 -07:00
Ronnie Sahlberg
038d005129 refs.c: make ref_update_reject_duplicates take a strbuf argument for errors
Make ref_update_reject_duplicates return any error that occurs through a
new strbuf argument. This means that when a transaction commit fails in
this function we will now be able to pass a helpful error message back to the
caller.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:42 -07:00
Ronnie Sahlberg
dc615de861 refs.c: log_ref_write should try to return meaningful errno
Making errno from write_ref_sha1() meaningful, which should fix

* a bug in "git checkout -b" where it prints strerror(errno)
  despite errno possibly being zero or clobbered

* a bug in "git fetch"'s s_update_ref, which trusts the result of an
  errno == ENOTDIR check to detect D/F conflicts

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:42 -07:00
Ronnie Sahlberg
76d70dc0c6 refs.c: make resolve_ref_unsafe set errno to something meaningful on error
Making errno when returning from resolve_ref_unsafe() meaningful,
which should fix

 * a bug in lock_ref_sha1_basic, where it assumes EISDIR
   means it failed due to a directory being in the way

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:42 -07:00
Ronnie Sahlberg
d3f6655505 refs.c: commit_packed_refs to return a meaningful errno on failure
Making errno when returning from commit_packed_refs() meaningful,
which should fix

 * a bug in "git clone" where it prints strerror(errno) based on
   errno, despite errno possibly being zero and potentially having
   been clobbered by that point
 * the same kind of bug in "git pack-refs"

and prepares for repack_without_refs() to get a meaningful
error message when commit_packed_refs() fails without falling into
the same bug.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:41 -07:00
Ronnie Sahlberg
470a91ef75 refs.c: make remove_empty_directories always set errno to something sane
Making errno when returning from remove_empty_directories() more
obviously meaningful, which should provide some peace of mind for
people auditing lock_ref_sha1_basic.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:41 -07:00
Ronnie Sahlberg
835e3c992f refs.c: verify_lock should set errno to something meaningful
Making errno when returning from verify_lock() meaningful, which
should almost but not completely fix

 * a bug in "git fetch"'s s_update_ref, which trusts the result of an
   errno == ENOTDIR check to detect D/F conflicts

ENOTDIR makes sense as a sign that a file was in the way of a
directory we wanted to create.  Should "git fetch" also look for
ENOTEMPTY or EEXIST to catch cases where a directory was in the way
of a file to be created?

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:41 -07:00
Ronnie Sahlberg
bd3b02daec refs.c: make sure log_ref_setup returns a meaningful errno
Making errno when returning from log_ref_setup() meaningful,

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:41 -07:00
Ronnie Sahlberg
60bca085c8 refs.c: add an err argument to repack_without_refs
Update repack_without_refs to take an err argument and update it if there
is a failure. Pass the err variable from ref_transaction_commit to this
function so that callers can print a meaningful error message if _commit
fails due to this function.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:41 -07:00
Ronnie Sahlberg
447ff1bf0a lockfile.c: make lock_file return a meaningful errno on failurei
Making errno when returning from lock_file() meaningful, which should
fix

 * an existing almost-bug in lock_ref_sha1_basic where it assumes
   errno==ENOENT is meaningful and could waste some work on retries

 * an existing bug in repack_without_refs where it prints
   strerror(errno) and picks advice based on errno, despite errno
   potentially being zero and potentially having been clobbered by
   that point

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:41 -07:00
Ronnie Sahlberg
995f8746bc refs.c: add a strbuf argument to ref_transaction_commit for error logging
Add a strbuf argument to _commit so that we can pass an error string back to
the caller. So that we can do error logging from the caller instead of from
_commit.

Longer term plan is to first convert all callers to use onerr==QUIET_ON_ERR
and craft any log messages from the callers themselves and finally remove the
onerr argument completely.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:40 -07:00
Ronnie Sahlberg
1b07255c95 refs.c: allow passing NULL to ref_transaction_free
Allow ref_transaction_free(NULL) as a no-op. This makes ref_transaction_free
easier to use and more similar to plain 'free'.

In particular, it lets us rollback unconditionally as part of cleanup code
after setting 'transaction = NULL' if a transaction has been committed or
rolled back already.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:40 -07:00
Ronnie Sahlberg
f1c9350ad7 refs.c: constify the sha arguments for ref_transaction_create|delete|update
ref_transaction_create|delete|update has no need to modify the sha1
arguments passed to it so it should use const unsigned char* instead
of unsigned char*.

Some functions, such as fast_forward_to(), already have its old/new
sha1 arguments as consts. This function will at some point need to
use ref_transaction_update() in which case this change is required.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:40 -07:00
Ronnie Sahlberg
33f9fc5932 refs.c: ref_transaction_commit should not free the transaction
Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:40 -07:00
Ronnie Sahlberg
026bd1d3e2 refs.c: remove ref_transaction_rollback
We do not yet need both a rollback and a free function for transactions.
Remove ref_transaction_rollback and use ref_transaction_free instead.

At a later stage we may reintroduce a rollback function if we want to start
adding reusable transactions and similar.

Reviewed-by: Jonathan Nieder <jrnieder@gmail.com>
Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
2014-07-14 11:54:40 -07:00
Jeff King
8ff226a9d5 add object_as_type helper for casting objects
When we call lookup_commit, lookup_tree, etc, the logic goes
something like:

  1. Look for an existing object struct. If we don't have
     one, allocate and return a new one.

  2. Double check that any object we have is the expected
     type (and complain and return NULL otherwise).

  3. Convert an object with type OBJ_NONE (from a prior
     call to lookup_unknown_object) to the expected type.

We can encapsulate steps 2 and 3 in a helper function which
checks whether we have the expected object type, converts
OBJ_NONE as appropriate, and returns the object.

Not only does this shorten the code, but it also provides
one central location for converting OBJ_NONE objects into
objects of other types. Future patches will use that to
enforce type-specific invariants.

Since this is a refactoring, we would want it to behave
exactly as the current code. It takes a little reasoning to
see that this is the case:

  - for lookup_{commit,tree,etc} functions, we are just
    pulling steps 2 and 3 into a function that does the same
    thing.

  - for the call in peel_object, we currently only do step 3
    (but we want to consolidate it with the others, as
    mentioned above). However, step 2 is a noop here, as the
    surrounding conditional makes sure we have OBJ_NONE
    (which we want to keep to avoid an extraneous call to
    sha1_object_info).

  - for the call in lookup_commit_reference_gently, we are
    currently doing step 2 but not step 3. However, step 3
    is a noop here. The object we got will have just come
    from deref_tag, which must have figured out the type for
    each object in order to know when to stop peeling.
    Therefore the type will never be OBJ_NONE.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-13 18:59:05 -07:00
Junio C Hamano
779c99fd68 Merge branch 'dt/refs-check-refname-component-sse-fix'
Fixes to a topic that is already in 'master'.

* dt/refs-check-refname-component-sse-fix:
  refs: fix valgrind suppression file
  refs.c: handle REFNAME_REFSPEC_PATTERN at end of page
2014-07-10 11:27:55 -07:00
David Turner
6d17dc1dd3 refs.c: handle REFNAME_REFSPEC_PATTERN at end of page
When a ref crosses a memory page boundary, we restart the parsing
at the beginning with the bytewise code.  Pass the original flags
to that code, rather than the current flags.

Reported-By: Øyvind A. Holm <sunny@sunbase.org>
Signed-off-by: David Turner <dturner@twitter.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-07-07 11:05:43 -07:00
Junio C Hamano
6f92e5ff3c Merge branch 'dt/refs-check-refname-component-sse'
Further micro-optimization of a leaf-function.

* dt/refs-check-refname-component-sse:
  refs.c: SSE2 optimizations for check_refname_component
2014-07-02 12:53:07 -07:00
Jeff King
2975c770ca replace has_extension with ends_with
These two are almost the same function, with the exception
that has_extension only matches if there is content before
the suffix. So ends_with(".exe", ".exe") is true, but
has_extension would not be.

This distinction does not matter to any of the callers,
though, and we can just replace uses of has_extension with
ends_with. We prefer the "ends_with" name because it is more
generic, and there is nothing about the function that
requires it to be used for file extensions.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-30 13:43:16 -07:00
David Turner
745224e04a refs.c: SSE2 optimizations for check_refname_component
Optimize check_refname_component using SSE2 on x86_64.

git rev-parse HEAD is a good test-case for this, since it does almost
nothing except parse refs.  For one particular repo with about 60k
refs, almost all packed, the timings are:

Look up table: 29 ms
SSE2:          23 ms

This cuts about 20% off of the runtime.

Ondřej Bílka <neleai@seznam.cz> suggested an SSE2 approach to the
substring searches, which netted a speed boost over the SSE4.2 code I
had initially written.

Signed-off-by: David Turner <dturner@twitter.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-18 10:57:18 -07:00
Junio C Hamano
ae7dd1a492 Merge branch 'dt/refs-check-refname-component-optim'
* dt/refs-check-refname-component-optim:
  refs.c: optimize check_refname_component()
2014-06-16 12:18:52 -07:00
Junio C Hamano
bb0ced7581 Merge branch 'rs/read-ref-at'
* rs/read-ref-at:
  refs.c: change read_ref_at to use the reflog iterators
2014-06-16 12:18:48 -07:00
Junio C Hamano
474df928b1 Merge branch 'jl/remote-rm-prune'
"git remote rm" and "git remote prune" can involve removing many
refs at once, which is not a very efficient thing to do when very
many refs exist in the packed-refs file.

* jl/remote-rm-prune:
  remote prune: optimize "dangling symref" check/warning
  remote: repack packed-refs once when deleting multiple refs
  remote rm: delete remote configuration as the last
2014-06-16 12:17:58 -07:00
Junio C Hamano
f7f349e138 Merge branch 'rs/reflog-exists'
* rs/reflog-exists:
  checkout.c: use ref_exists instead of file_exist
  refs.c: add new functions reflog_exists and delete_reflog
2014-06-06 11:23:04 -07:00
David Turner
dde8a902c7 refs.c: optimize check_refname_component()
In a repository with many refs, check_refname_component can be a major
contributor to the runtime of some git commands. One such command is

git rev-parse HEAD

Timings for one particular repo, with about 60k refs, almost all
packed, are:

Old: 35 ms
New: 29 ms

Many other commands which read refs are also sped up.

Signed-off-by: David Turner <dturner@twitter.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-05 15:24:50 -07:00
Ronnie Sahlberg
4207ed285f refs.c: change read_ref_at to use the reflog iterators
read_ref_at has its own parsing of the reflog file for no really good reason
so lets change this to use the existing reflog iterators. This removes one
instance where we manually unmarshall the reflog file format.

Remove the now redundant ref_msg function.

Log messages for errors are changed slightly. We no longer print the file
name for the reflog, instead we refer to it as 'Log for ref <refname>'.
This might be a minor useability regression, but I don't really think so, since
experienced users would know where the log is anyway and inexperienced users
would not know what to do about/how to repair 'Log ... has gap ...' anyway.

Adapt the t1400 test to handle the change in log messages.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-06-03 11:09:32 -07:00
Jens Lindström
e6bea66db6 remote prune: optimize "dangling symref" check/warning
When 'git remote prune' was used to delete many refs in a repository
with many refs, a lot of time was spent checking for (now) dangling
symbolic refs pointing to the deleted ref, since warn_dangling_symref()
was once per deleted ref to check all other refs in the repository.

Avoid this using the new warn_dangling_symrefs() function which
makes one pass over all refs and checks for all the deleted refs in
one go, after they have all been deleted.

Signed-off-by: Jens Lindström <jl@opera.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-27 12:30:47 -07:00
Jens Lindström
c9e768bb77 remote: repack packed-refs once when deleting multiple refs
When 'git remote rm' or 'git remote prune' were used in a repository
with many refs, and needed to delete many remote-tracking refs, a lot
of time was spent deleting those refs since for each deleted ref,
repack_without_refs() was called to rewrite packed-refs without just
that deleted ref.

To avoid this, call repack_without_refs() first to repack without all
the refs that will be deleted, before calling delete_ref() to delete
each one completely.  The call to repack_without_ref() in delete_ref()
then becomes a no-op, since packed-refs already won't contain any of
the deleted refs.

Signed-off-by: Jens Lindström <jl@opera.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-27 12:30:42 -07:00
Ronnie Sahlberg
4da588357a refs.c: add new functions reflog_exists and delete_reflog
Add two new functions, reflog_exists and delete_reflog, to hide the internal
reflog implementation (that they are files under .git/logs/...) from callers.
Update checkout.c to use these functions in update_refs_for_switch instead of
building pathnames and calling out to file access functions. Update reflog.c
to use these to check if the reflog exists. Now there are still many places
in reflog.c where we are still leaking the reflog storage implementation but
this at least reduces the number of such dependencies by one. Finally
change two places in refs.c itself to use the new function to check if a ref
exists or not isntead of build-path-and-stat(). Now, this is strictly not all
that important since these are in parts of refs that are implementing the
actual file storage backend but on the other hand it will not hurt either.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-05-08 14:31:43 -07:00
Michael Haggerty
6a402338ec ref_transaction_commit(): work with transaction->updates in place
Now that we free the transaction when we are done, there is no need to
make a copy of transaction->updates before working with it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:16 -07:00
Michael Haggerty
84178db76f struct ref_update: add a type field
It used to be that ref_transaction_commit() allocated a temporary
array to hold the types of references while it is working.  Instead,
add a type field to ref_update that ref_transaction_commit() can use
as its scratch space.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:15 -07:00
Michael Haggerty
81c960e4dc struct ref_update: add a lock field
Now that we manage ref_update objects internally, we can use them to
hold some of the scratch space we need when actually carrying out the
updates.  Store the (struct ref_lock *) there.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:15 -07:00
Michael Haggerty
cb198d21d3 ref_transaction_commit(): simplify code using temporary variables
Use temporary variables in the for-loop blocks to simplify expressions
in the rest of the loop.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:15 -07:00
Michael Haggerty
88615910db struct ref_update: store refname as a FLEX_ARRAY
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:15 -07:00
Michael Haggerty
5524e2416e struct ref_update: rename field "ref_name" to "refname"
This is consistent with the usual nomenclature.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:15 -07:00
Michael Haggerty
b5c8ea2afb refs: remove API function update_refs()
It has been superseded by reference transactions.  This also means
that struct ref_update can become private.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:14 -07:00
Michael Haggerty
caa4046c4f refs: add a concept of a reference transaction
Build out the API for dealing with a bunch of reference checks and
changes within a transaction.  Define an opaque ref_transaction type
that is managed entirely within refs.c.  Introduce functions for
beginning a transaction, adding updates to a transaction, and
committing/rolling back a transaction.

This API will soon replace update_refs().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:14 -07:00
Michael Haggerty
595deb8da6 update_refs(): fix constness
The old signature of update_refs() required a
(const struct ref_update **) for its updates_orig argument.  The
"const" is presumably there to promise that the function will not
modify the contents of the structures.

But this declaration does not permit the function to be called with a
(struct ref_update **), which is perfectly legitimate.  C's type
system is not powerful enough to express what we'd like.  So remove
the first "const" from the declaration.

On the other hand, the function *can* promise not to modify the
pointers within the array that is passed to it without inconveniencing
its callers.  So add a "const" that has that effect, making the final
declaration
(struct ref_update * const *).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:11 -07:00
Michael Haggerty
f412411245 refs.h: rename the action_on_err constants
Given that these constants are only being used when updating
references, it is inappropriate to give them such generic names as
"DIE_ON_ERR".  So prefix their names with "UPDATE_REFS_".

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-04-07 12:09:11 -07:00
Junio C Hamano
006f678780 Merge branch 'sh/use-hashcpy'
* sh/use-hashcpy:
  Use hashcpy() when copying object names
2014-03-18 13:51:05 -07:00
Sun He
50546b15ed Use hashcpy() when copying object names
We invented hashcpy() to keep the abstraction of "object name"
behind it.  Use it instead of calling memcpy() with hard-coded
20-byte length when moving object names between pieces of memory.

Leave ppc/sha1.c as-is, because the function is about the SHA-1 hash
algorithm whose output is and will always be 20 bytes.

Helped-by: Michael Haggerty <mhagger@alum.mit.edu>
Helped-by: Duy Nguyen <pclouds@gmail.com>
Signed-off-by: Sun He <sunheehnus@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-03-06 14:03:12 -08:00
Nguyễn Thái Ngọc Duy
eb07894fe0 use wildmatch() directly without fnmatch() wrapper
Make it clear that we don't use fnmatch() anymore.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-02-20 14:15:46 -08:00
Junio C Hamano
d0956cfa8e Merge branch 'mh/safe-create-leading-directories'
Code clean-up and protection against concurrent write access to the
ref namespace.

* mh/safe-create-leading-directories:
  rename_tmp_log(): on SCLD_VANISHED, retry
  rename_tmp_log(): limit the number of remote_empty_directories() attempts
  rename_tmp_log(): handle a possible mkdir/rmdir race
  rename_ref(): extract function rename_tmp_log()
  remove_dir_recurse(): handle disappearing files and directories
  remove_dir_recurse(): tighten condition for removing unreadable dir
  lock_ref_sha1_basic(): if locking fails with ENOENT, retry
  lock_ref_sha1_basic(): on SCLD_VANISHED, retry
  safe_create_leading_directories(): add new error value SCLD_VANISHED
  cmd_init_db(): when creating directories, handle errors conservatively
  safe_create_leading_directories(): introduce enum for return values
  safe_create_leading_directories(): always restore slash at end of loop
  safe_create_leading_directories(): split on first of multiple slashes
  safe_create_leading_directories(): rename local variable
  safe_create_leading_directories(): add explicit "slash" pointer
  safe_create_leading_directories(): reduce scope of local variable
  safe_create_leading_directories(): fix format of "if" chaining
2014-01-27 10:45:33 -08:00
Junio C Hamano
9bb5287098 Merge branch 'mh/retire-ref-fetch-rules'
Code simplification.

* mh/retire-ref-fetch-rules:
  refname_match(): always use the rules in ref_rev_parse_rules
2014-01-27 10:44:07 -08:00
Michael Haggerty
08f555cb82 rename_tmp_log(): on SCLD_VANISHED, retry
If safe_create_leading_directories() fails because a file along the
path unexpectedly vanished, try again from the beginning.  Try at most
4 times.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-21 13:47:28 -08:00
Michael Haggerty
f1e9e9a4db rename_tmp_log(): limit the number of remote_empty_directories() attempts
This doesn't seem to be a likely error, but we've got the counter
anyway, so we might as well use it for an added bit of safety.

Please note that the first call to rename() is optimistic, and it is
normal for it to fail if there is a directory in the way.  So bump the
total number of allowed attempts to 4, to be sure that we can still
have at least 3 retries in the case of a race.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-21 13:47:24 -08:00
Michael Haggerty
ae4a283e3b rename_tmp_log(): handle a possible mkdir/rmdir race
If a directory vanishes while renaming the temporary reflog file,
retry (up to 3 times).  This could happen if another process deletes
the directory created by safe_create_leading_directories() just before
we rename the file into the directory.

As far as I can tell, this race could not occur internal to git.  The
only time that a directory under $GIT_DIR/logs is deleted is if room
has to be made for a log file for a reference with the same name;
for example, in the following sequence:

    git branch foo/bar    # Creates file .git/logs/refs/heads/foo/bar
    git branch -d foo/bar # Deletes file but leaves .git/logs/refs/heads/foo/
    git branch foo        # Deletes .git/logs/refs/heads/foo/

But the only reason the last command deletes the directory is because
it wants to create a file with the same name.  So if another process
(e.g.,

    git branch foo/baz

) wants to create that directory, one of the two is doomed to failure
anyway because of a D/F conflict.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-21 13:47:13 -08:00
Michael Haggerty
fa59ae7971 rename_ref(): extract function rename_tmp_log()
It's about to become a bit more complex.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-21 13:46:59 -08:00
Michael Haggerty
e5c223e98b lock_ref_sha1_basic(): if locking fails with ENOENT, retry
If hold_lock_file_for_update() fails with errno==ENOENT, it might be
because somebody else (for example, a pack-refs process) has just
deleted one of the lockfile's ancestor directories.  So if this
condition is detected, try again (up to 3 times).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-21 13:46:30 -08:00
Michael Haggerty
c4c61c763e lock_ref_sha1_basic(): on SCLD_VANISHED, retry
If safe_create_leading_directories() fails because a file along the
path unexpectedly vanished, try again (up to 3 times).

This can occur if another process is deleting directories at the same
time as we are trying to make them.  For example, "git pack-refs
--all" tries to delete the loose refs and any empty directories that
are left behind.  If a pack-refs process is running, then it might
delete a directory that we need to put a new loose reference in.

If safe_create_leading_directories() thinks this might have happened,
then take its advice and try again (maximum three attempts).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-21 13:46:07 -08:00
Michael Haggerty
54457fe509 refname_match(): always use the rules in ref_rev_parse_rules
We used to use two separate rules for the normal ref resolution
dwimming and dwimming done to decide which remote ref to grab.  The
third parameter to refname_match() selected which rules to use.

When these two rules were harmonized in

    2011-11-04 dd621df9cd refs DWIMmery: use the same rule for both "git fetch" and others

, ref_fetch_rules was #defined to avoid potential breakages for
in-flight topics.

It is now safe to remove the backwards-compatibility code, so remove
refname_match()'s third parameter, make ref_rev_parse_rules private to
refs.c, and remove ref_fetch_rules entirely.

Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-14 13:58:06 -08:00
Junio C Hamano
540cc75f38 Merge branch 'mh/shorten-unambigous-ref'
* mh/shorten-unambigous-ref:
  shorten_unambiguous_ref(): tighten up pointer arithmetic
  gen_scanf_fmt(): delete function and use snprintf() instead
  shorten_unambiguous_ref(): introduce a new local variable
2014-01-13 11:34:08 -08:00
Michael Haggerty
7902fe03f9 shorten_unambiguous_ref(): tighten up pointer arithmetic
As long as we're being pathologically stingy with mallocs, we might as
well do the math right and save 6 (!) bytes.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-09 15:02:36 -08:00
Michael Haggerty
4346663a14 gen_scanf_fmt(): delete function and use snprintf() instead
To replace "%.*s" with "%s", all we have to do is use snprintf()
to interpolate "%s" into the pattern.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-09 14:56:06 -08:00
Michael Haggerty
84d5633f98 shorten_unambiguous_ref(): introduce a new local variable
When filling the scanf_fmts array, use a separate variable to keep
track of the offset to avoid clobbering total_len (which we will need
in the next commit).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2014-01-09 14:52:44 -08:00
Christian Couder
5955654823 replace {pre,suf}fixcmp() with {starts,ends}_with()
Leaving only the function definitions and declarations so that any
new topic in flight can still make use of the old functions, replace
existing uses of the prefixcmp() and suffixcmp() with new API
functions.

The change can be recreated by mechanically applying this:

    $ git grep -l -e prefixcmp -e suffixcmp -- \*.c |
      grep -v strbuf\\.c |
      xargs perl -pi -e '
        s|!prefixcmp\(|starts_with\(|g;
        s|prefixcmp\(|!starts_with\(|g;
        s|!suffixcmp\(|ends_with\(|g;
        s|suffixcmp\(|!ends_with\(|g;
      '

on the result of preparatory changes in this series.

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-12-05 14:13:21 -08:00
Junio C Hamano
e0fd1e3841 Merge branch 'sb/refs-code-cleanup'
* sb/refs-code-cleanup:
  cache: remove unused function 'have_git_dir'
  refs: remove unused function invalidate_ref_cache
2013-11-01 07:38:58 -07:00
Junio C Hamano
149a8134a7 Merge branch 'jk/refs-c-squelch-gcc'
* jk/refs-c-squelch-gcc:
  silence gcc array-bounds warning
2013-10-30 12:11:04 -07:00
Stefan Beller
746593bdca refs: remove unused function invalidate_ref_cache
The function 'invalidate_ref_cache' was introduced in 79c7ca5 (2011-10-17,
invalidate_ref_cache(): rename function from invalidate_cached_refs())
by a rename and elevated to be publicly usable in 8be8bde (2011-10-17,
invalidate_ref_cache(): expose this function in the refs API)

However it is not used anymore, as 8bf90dc (2011-10-17, write_ref_sha1():
only invalidate the loose ref cache) and (much) later 506a760 (2013-04-22,
refs: change how packed refs are deleted) removed any calls to this
function. So it seems as if we don't need that function any more,
good bye!

Signed-off-by: Stefan Beller <stefanbeller@googlemail.com>
Acked-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-28 08:55:56 -07:00
Jeff King
a4165851e7 silence gcc array-bounds warning
In shorten_unambiguous_ref, we build and cache a reverse-map of the
rev-parse rules like this:

  static char **scanf_fmts;
  static int nr_rules;
  if (!nr_rules) {
	  for (; ref_rev_parse_rules[nr_rules]; nr_rules++)
		  ... generate scanf_fmts ...
  }

where ref_rev_parse_rules is terminated with a NULL pointer.
Compiling with "gcc -O2 -Wall" does not cause any problems, but
compiling with "-O3 -Wall" generates:

  $ make CFLAGS='-O3 -Wall' refs.o
  refs.c: In function ‘shorten_unambiguous_ref’:
  refs.c:3379:29: warning: array subscript is above array bounds [-Warray-bounds]
     for (; ref_rev_parse_rules[nr_rules]; nr_rules++)

Curiously, we can silence this by explicitly nr_rules to 0
in the beginning of the loop, even though the compiler
should be able to tell that we follow this code path only
when nr_rules is already 0.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-10-24 15:41:56 -07:00
Ramsay Jones
ce1e846207 refs.c: spell NULL pointer as NULL
A call to update_ref_lock() passes '0' to the 'int *type_p' parameter.
Noticed by sparse.  ("Using plain integer as NULL pointer")

Signed-off-by: Ramsay Jones <ramsay@ramsay1.demon.co.uk>
Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
2013-10-14 16:10:50 -07:00
Junio C Hamano
f406140baa Merge branch 'fc/at-head'
Instead of typing four capital letters "HEAD", you can say "@" now,
e.g. "git log @".

* fc/at-head:
  Add new @ shortcut for HEAD
  sha1-name: pass len argument to interpret_branch_name()
2013-09-20 12:38:10 -07:00
Junio C Hamano
9a86b89941 Merge branch 'bk/refs-multi-update'
Give "update-refs" a "--stdin" option to read multiple update
requests and perform them in an all-or-none fashion.

* bk/refs-multi-update:
  update-ref: add test cases covering --stdin signature
  update-ref: support multiple simultaneous updates
  refs: add update_refs for multiple simultaneous updates
  refs: add function to repack without multiple refs
  refs: factor delete_ref loose ref step into a helper
  refs: factor update_ref steps into helpers
  refs: report ref type from lock_any_ref_for_update
  reset: rename update_refs to reset_refs
2013-09-20 12:36:12 -07:00
Felipe Contreras
9ba89f484e Add new @ shortcut for HEAD
Typing 'HEAD' is tedious, especially when we can use '@' instead.

The reason for choosing '@' is that it follows naturally from the
ref@op syntax (e.g. HEAD@{u}), except we have no ref, and no
operation, and when we don't have those, it makes sens to assume
'HEAD'.

So now we can use 'git show @~1', and all that goody goodness.

Until now '@' was a valid name, but it conflicts with this idea, so
let's make it invalid. Probably very few people, if any, used this name.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-12 14:39:34 -07:00
Junio C Hamano
2233ad4534 Merge branch 'jc/push-cas'
Allow a safer "rewind of the remote tip" push than blind "--force",
by requiring that the overwritten remote ref to be unchanged since
the new history to replace it was prepared.

The machinery is more or less ready.  The "--force" option is again
the big red button to override any safety, thanks to J6t's sanity
(the original round allowed --lockref to defeat --force).

The logic to choose the default implemented here is fragile
(e.g. "git fetch" after seeing a failure will update the
remote-tracking branch and will make the next "push" pass,
defeating the safety pretty easily).  It is suitable only for the
simplest workflows, and it may hurt users more than it helps them.

* jc/push-cas:
  push: teach --force-with-lease to smart-http transport
  send-pack: fix parsing of --force-with-lease option
  t5540/5541: smart-http does not support "--force-with-lease"
  t5533: test "push --force-with-lease"
  push --force-with-lease: tie it all together
  push --force-with-lease: implement logic to populate old_sha1_expect[]
  remote.c: add command line option parser for "--force-with-lease"
  builtin/push.c: use OPT_BOOL, not OPT_BOOLEAN
  cache.h: move remote/connect API out of it
2013-09-09 14:30:29 -07:00
Brad King
98aee92d5c refs: add update_refs for multiple simultaneous updates
Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref.  Then acquire locks on all refs with
verified old values.  Then update or delete all refs accordingly.  Fail
if any one lock cannot be obtained or any one old value does not match.

Though the refs themselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-04 11:10:28 -07:00
Brad King
61cee0dbac refs: add function to repack without multiple refs
Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-04 11:09:55 -07:00
Brad King
2ddb5d170a refs: factor delete_ref loose ref step into a helper
Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-04 11:09:09 -07:00
Brad King
4738a33338 refs: factor update_ref steps into helpers
Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.

While at it, drop "static" from the local "lock" variable as it is not
necessary to keep across invocations.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-04 11:08:36 -07:00
Felipe Contreras
cf99a761d3 sha1-name: pass len argument to interpret_branch_name()
This is useful to make sure we don't step outside the boundaries of what
we are interpreting at the moment. For example while interpreting
foobar@{u}~1, the job of interpret_branch_name() ends right before ~1,
but there's no way to figure that out inside the function, unless the
len argument is passed.

So let's do that.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-09-03 11:33:00 -07:00
Brad King
9bbb0fa1fd refs: report ref type from lock_any_ref_for_update
Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it by passing
NULL for now.

Signed-off-by: Brad King <brad.king@kitware.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-08-30 14:57:28 -07:00
Junio C Hamano
2c2b6646c2 Revert "Add new @ shortcut for HEAD"
This reverts commit cdfd94837b, as it
does not just apply to "@" (and forms with modifiers like @{u}
applied to it), but also affects e.g. "refs/heads/@/foo", which it
shouldn't.

The basic idea of giving a short-hand might be good, and the topic
can be retried later, but let's revert to avoid affecting existing
use cases for now for the upcoming release.
2013-08-14 15:04:24 -07:00
Junio C Hamano
f1093b0f60 Merge branch 'mh/packed-refs-do-one-ref-recursion'
Fix a NULL-pointer dereference during nested iterations over
references (for example, when replace references are being used).

* mh/packed-refs-do-one-ref-recursion:
  do_one_ref(): save and restore value of current_ref
2013-07-31 12:38:12 -07:00
Junio C Hamano
29143fc4e3 Merge branch 'mh/ref-races-optim-invalidate-cached'
* mh/ref-races-optim-invalidate-cached:
  refs: do not invalidate the packed-refs cache unnecessarily
2013-07-24 19:21:02 -07:00
Michael Haggerty
d0cf51e940 do_one_ref(): save and restore value of current_ref
If do_one_ref() is called recursively, then the inner call should not
permanently overwrite the value stored in current_ref by the outer
call.  Aside from the tiny optimization loss, peel_ref() expects the
value of current_ref not to change across a call to peel_entry().  But
in the presence of replace references that assumption could be
violated by a recursive call to do_one_ref:

do_for_each_entry()
  do_one_ref()
    builtin/describe.c:get_name()
      peel_ref()
        peel_entry()
          peel_object ()
            deref_tag_noverify()
              parse_object()
                lookup_replace_object()
                  do_lookup_replace_object()
                    prepare_replace_object()
                      do_for_each_ref()
                        do_for_each_entry()
                          do_for_each_entry_in_dir()
                            do_one_ref()

The inner call to do_one_ref() was unconditionally setting current_ref
to NULL when it was done, causing peel_ref() to perform an invalid
memory access.

So change do_one_ref() to save the old value of current_ref before
overwriting it, and restore the old value afterward rather than
setting it to NULL.

Reported-by: Mantas Mikulėnas <grawity@gmail.com>
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-17 18:19:16 -07:00
Junio C Hamano
47a5918536 cache.h: move remote/connect API out of it
The definition of "struct ref" in "cache.h", a header file so
central to the system, always confused me.  This structure is not
about the local ref used by sha1-name API to name local objects.

It is what refspecs are expanded into, after finding out what refs
the other side has, to define what refs are updated after object
transfer succeeds to what values.  It belongs to "remote.h" together
with "struct refspec".

While we are at it, also move the types and functions related to the
Git transport connection to a new header file connect.h

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-07-08 14:34:24 -07:00
Junio C Hamano
079424a2cf Merge branch 'mh/ref-races'
"git pack-refs" that races with new ref creation or deletion have
been susceptible to lossage of refs under right conditions, which
has been tightened up.

* mh/ref-races:
  for_each_ref: load all loose refs before packed refs
  get_packed_ref_cache: reload packed-refs file when it changes
  add a stat_validity struct
  Extract a struct stat_data from cache_entry
  packed_ref_cache: increment refcount when locked
  do_for_each_entry(): increment the packed refs cache refcount
  refs: manage lifetime of packed refs cache via reference counting
  refs: implement simple transactions for the packed-refs file
  refs: wrap the packed refs cache in a level of indirection
  pack_refs(): split creation of packed refs and entry writing
  repack_without_ref(): split list curation and entry writing
2013-06-30 15:40:05 -07:00
Michael Haggerty
5d478f5ca1 refs: do not invalidate the packed-refs cache unnecessarily
Now that we keep track of the packed-refs file metadata, we can detect
when the packed-refs file has been modified since we last read it, and
we do so automatically every time that get_packed_ref_cache() is
called.  So there is no need to invalidate the cache automatically
when lock_packed_refs() is called; usually the old copy will still be
valid.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Jeff King
98eeb09e8a for_each_ref: load all loose refs before packed refs
If we are iterating through the refs using for_each_ref (or
any of its sister functions), we can get into a race
condition with a simultaneous "pack-refs --prune" that looks
like this:

  0. We have a large number of loose refs, and a few packed
     refs. refs/heads/z/foo is loose, with no matching entry
     in the packed-refs file.

  1. Process A starts iterating through the refs. It loads
     the packed-refs file from disk, then starts lazily
     traversing through the loose ref directories.

  2. Process B, running "pack-refs --prune", writes out the
     new packed-refs file. It then deletes the newly packed
     refs, including refs/heads/z/foo.

  3. Meanwhile, process A has finally gotten to
     refs/heads/z (it traverses alphabetically). It
     descends, but finds nothing there.  It checks its
     cached view of the packed-refs file, but it does not
     mention anything in "refs/heads/z/" at all (it predates
     the new file written by B in step 2).

The traversal completes successfully without mentioning
refs/heads/z/foo at all (the name, of course, isn't
important; but the more refs you have and the farther down
the alphabetical list a ref is, the more likely it is to hit
the race). If refs/heads/z/foo did exist in the packed refs
file at state 0, we would see an entry for it, but it would
show whatever sha1 the ref had the last time it was packed
(which could be an arbitrarily long time ago).

This can be especially dangerous when process A is "git
prune", as it means our set of reachable tips will be
incomplete, and we may erroneously prune objects reachable
from that tip (the same thing can happen if "repack -ad" is
used, as it simply drops unreachable objects that are
packed).

This patch solves it by loading all of the loose refs for
our traversal into our in-memory cache, and then refreshing
the packed-refs cache. Because a pack-refs writer will
always put the new packed-refs file into place before
starting the prune, we know that any loose refs we fail to
see will either truly be missing, or will have already been
put in the packed-refs file by the time we refresh.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Jeff King
ca9199300e get_packed_ref_cache: reload packed-refs file when it changes
Once we read the packed-refs file into memory, we cache it
to save work on future ref lookups. However, our cache may
be out of date with respect to what is on disk if another
process is simultaneously packing the refs. Normally it
is acceptable for us to be a little out of date, since there
is no guarantee whether we read the file before or after the
simultaneous update. However, there is an important special
case: our packed-refs file must be up to date with respect
to any loose refs we read. Otherwise, we risk the following
race condition:

  0. There exists a loose ref refs/heads/master.

  1. Process A starts and looks up the ref "master". It
     first checks $GIT_DIR/master, which does not exist. It
     then loads (and caches) the packed-refs file to see if
     "master" exists in it, which it does not.

  2. Meanwhile, process B runs "pack-refs --all --prune". It
     creates a new packed-refs file which contains
     refs/heads/master, and removes the loose copy at
     $GIT_DIR/refs/heads/master.

  3. Process A continues its lookup, and eventually tries
     $GIT_DIR/refs/heads/master.  It sees that the loose ref
     is missing, and falls back to the packed-refs file. But
     it examines its cached version, which does not have
     refs/heads/master. After trying a few other prefixes,
     it reports master as a non-existent ref.

There are many variants (e.g., step 1 may involve process A
looking up another ref entirely, so even a fully qualified
refname can fail). One of the most interesting ones is if
"refs/heads/master" is already packed. In that case process
A will not see it as missing, but rather will report
whatever value happened to be in the packed-refs file before
process B repacked (which might be an arbitrarily old
value).

We can fix this by making sure we reload the packed-refs
file from disk after looking at any loose refs. That's
unacceptably slow, so we can check its stat()-validity as a
proxy, and read it only when it appears to have changed.

Reading the packed-refs file after performing any loose-ref
system calls is sufficient because we know the ordering of
the pack-refs process: it always makes sure the newly
written packed-refs file is installed into place before
pruning any loose refs. As long as those operations by B
appear in their executed order to process A, by the time A
sees the missing loose ref, the new packed-refs file must be
in place.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
4f6b83e370 packed_ref_cache: increment refcount when locked
Increment the packed_ref_cache reference count while it is locked to
prevent its being freed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
8baf2bb99a do_for_each_entry(): increment the packed refs cache refcount
This function calls a user-supplied callback function which could do
something that causes the packed refs cache to be invalidated.  So
acquire a reference count on the data structure to prevent our copy
from being freed while we are iterating over it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
5f5e2a8868 refs: manage lifetime of packed refs cache via reference counting
In struct packed_ref_cache, keep a count of the number of users of the
data structure.  Only free the packed ref cache when the reference
count goes to zero rather than when the packed ref cache is cleared.
This mechanism will be used to prevent the cache data structure from
being freed while it is being iterated over.

So far, only the reference in struct ref_cache::packed is counted;
other users will be adjusted in separate commits.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
9f69d29770 refs: implement simple transactions for the packed-refs file
Handle simple transactions for the packed-refs file at the
packed_ref_cache level via new functions lock_packed_refs(),
commit_packed_refs(), and rollback_packed_refs().

Only allow the packed ref cache to be modified (via add_packed_ref())
while the packed refs file is locked.

Change clone to add the new references within a transaction.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
2fff781290 refs: wrap the packed refs cache in a level of indirection
As we know, we can solve any problem in this manner.  In this case,
the problem is to avoid freeing a packed refs cache while somebody is
using it.  So add a level of indirection as a prelude to
reference-counting the packed refs cache.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
267f9a8cc8 pack_refs(): split creation of packed refs and entry writing
Split pack_refs() into multiple passes:

* Iterate over loose refs.  For each one that can be turned into a
  packed ref, create a corresponding entry in the packed refs cache.

* Write the packed refs to the packed-refs file.

This change isolates the mutation of the packed-refs file to a single
place.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:17 -07:00
Michael Haggerty
7b40d39638 repack_without_ref(): split list curation and entry writing
The repack_without_ref() function first removes the deleted ref from
the internal packed-refs list, then writes the packed-refs list to
disk, omitting any broken or stale entries.  This patch splits that
second step into multiple passes:

* collect the list of refnames that should be deleted from packed_refs

* delete those refnames from the cache

* write the remainder to the packed-refs file

The purpose of this change is to make the "write the remainder" part
reusable.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-20 15:50:16 -07:00
Michael Haggerty
fcb7c76274 resolve_ref_unsafe(): close race condition reading loose refs
We read loose references in two steps.  The code is roughly:

    lstat()
    if error ENOENT:
        loose ref is missing; look for corresponding packed ref
    else if S_ISLNK:
        readlink()
        if error:
            report failure
    else if S_ISDIR:
        report failure
    else
        open()
        if error:
            report failure
        read()

The problem is that the first filesystem call, to lstat(), is not
atomic with the second filesystem call, to readlink() or open().
Therefore it is possible for another process to change the file
between our two calls, for example:

* If the other process deletes the file, our second call will fail
  with ENOENT, which we *should* interpret as "loose ref is missing;
  look for corresponding packed ref".  This can arise if the other
  process is pack-refs; it might have just written a new packed-refs
  file containing the old contents of the reference then deleted the
  loose ref.

* If the other process changes a symlink into a plain file, our call
  to readlink() will fail with EINVAL, which we *should* respond to by
  trying to open() and read() the file.

The old code treats the reference as missing in both of these cases,
which is incorrect.

So instead, handle errors more selectively: if the result of
readline()/open() is a failure that is inconsistent with the result of
the previous lstat(), then something is fishy.  In this case jump back
and start over again with a fresh call to lstat().

One race is still possible and undetected: another process could
change the file from a regular file into a symlink between the call to
lstat and the call to open().  The open() call would silently follow
the symlink and not know that something is wrong.  This situation
could be detected in two ways:

* On systems that support O_NOFOLLOW, pass that option to the open().

* On other systems, call fstat() on the fd returned by open() and make
  sure that it agrees with the stat info from the original lstat().

However, we don't use symlinks anymore, so this situation is unlikely.
Moreover, it doesn't appear that treating a symlink as a regular file
would have grave consequences; after all, this is exactly how the code
handles non-relative symlinks.  So this commit leaves that race
unaddressed.

Note that this solves only the part of the race within
resolve_ref_unsafe. In the situation described above, we may still be
depending on a cached view of the packed-refs file; that race will be
dealt with in a future patch.

This problem was reported and diagnosed by Jeff King <peff@peff.net>,
and this solution is derived from his patch.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-19 10:23:04 -07:00
Michael Haggerty
2884c06ae7 resolve_ref_unsafe(): handle the case of an SHA-1 within loop
There is only one "break" statement within the loop, which jumps to
the code after the loop that handles the case of a file that holds a
SHA-1.  So move that code from below the loop into the if statement
where the break was previously located.  This makes the logic flow
more local.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-19 10:23:03 -07:00
Michael Haggerty
47f534bf92 resolve_ref_unsafe(): extract function handle_missing_loose_ref()
The nesting was getting a bit out of hand, and it's about to get
worse.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-06-19 10:23:03 -07:00
Junio C Hamano
bb1c8fbcc8 Merge branch 'fc/at-head'
Instead of typing four capital letters "HEAD", you can say "@"
instead.

* fc/at-head:
  sha1_name: compare variable with constant, not constant with variable
  Add new @ shortcut for HEAD
  sha1_name: refactor reinterpret()
  sha1_name: check @{-N} errors sooner
  sha1_name: reorganize get_sha1_basic()
  sha1_name: don't waste cycles in the @-parsing loop
  sha1_name: remove unnecessary braces
  sha1_name: remove no-op
  tests: at-combinations: @{N} versus HEAD@{N}
  tests: at-combinations: increase coverage
  tests: at-combinations: improve nonsense()
  tests: at-combinations: check ref names directly
  tests: at-combinations: simplify setup
2013-06-11 13:31:23 -07:00
Junio C Hamano
2f1ef15070 Merge branch 'mh/packed-refs-various'
Update reading and updating packed-refs file, correcting corner case
bugs.

* mh/packed-refs-various: (33 commits)
  refs: handle the main ref_cache specially
  refs: change do_for_each_*() functions to take ref_cache arguments
  pack_one_ref(): do some cheap tests before a more expensive one
  pack_one_ref(): use write_packed_entry() to do the writing
  pack_one_ref(): use function peel_entry()
  refs: inline function do_not_prune()
  pack_refs(): change to use do_for_each_entry()
  refs: use same lock_file object for both ref-packing functions
  pack_one_ref(): rename "path" parameter to "refname"
  pack-refs: merge code from pack-refs.{c,h} into refs.{c,h}
  pack-refs: rename handle_one_ref() to pack_one_ref()
  refs: extract a function write_packed_entry()
  repack_without_ref(): write peeled refs in the rewritten file
  t3211: demonstrate loss of peeled refs if a packed ref is deleted
  refs: change how packed refs are deleted
  search_ref_dir(): return an index rather than a pointer
  repack_without_ref(): silence errors for dangling packed refs
  t3210: test for spurious error messages for dangling packed refs
  refs: change the internal reference-iteration API
  refs: extract a function peel_entry()
  ...
2013-05-29 14:23:49 -07:00
Felipe Contreras
cdfd94837b Add new @ shortcut for HEAD
Typing 'HEAD' is tedious, especially when we can use '@' instead.

The reason for choosing '@' is that it follows naturally from the
ref@op syntax (e.g. HEAD@{u}), except we have no ref, and no
operation, and when we don't have those, it makes sens to assume
'HEAD'.

So now we can use 'git show @~1', and all that goody goodness.

Until now '@' was a valid name, but it conflicts with this idea, so
let's make it invalid. Probably very few people, if any, used this name.

Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-08 12:13:12 -07:00
Michael Haggerty
9da31cb027 refs: handle the main ref_cache specially
Hold the ref_cache instance for the main repository in a dedicated,
statically-allocated instance to avoid the need for a function call
and a linked-list traversal when it is needed.

Suggested by: Heiko Voigt <hvoigt@hvoigt.net>

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
65cf102bb0 refs: change do_for_each_*() functions to take ref_cache arguments
Change the callers convert submodule names into ref_cache pointers.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
b2a8226d63 pack_one_ref(): do some cheap tests before a more expensive one
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
0f29920f1e pack_one_ref(): use write_packed_entry() to do the writing
Change pack_refs() to work with a file descriptor instead of a FILE*
(making the file-locking code less awkward) and use
write_packed_entry() to do the writing.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
f85354b5c7 pack_one_ref(): use function peel_entry()
Change pack_one_ref() to call peel_entry() rather than using its own
code for peeling references.  Aside from sharing code, this lets it
take advantage of the optimization introduced by 6c4a060d7d.

Please note that we *could* use any peeled values that happen to
already be stored in the ref_entries, which would avoid some object
lookups for references that were already packed.  But doing so would
also propagate any peeling errors across runs of "git pack-refs" and
give no way to recover from such errors.  And "git pack-refs" isn't
run often enough that the performance cost is a problem.  So instead,
add a new option to peel_entry() to force the entry to be re-peeled,
and call it with that option set.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
8d3725b96f refs: inline function do_not_prune()
Function do_not_prune() was redundantly checking REF_ISSYMREF, which
was already tested at the top of pack_one_ref(), so remove that check.
And the rest was trivial, so inline the function.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
12e77559ec pack_refs(): change to use do_for_each_entry()
pack_refs() was not using any of the extra features of for_each_ref(),
so change it to use do_for_each_entry().  This also gives it access to
the ref_entry and in particular its peeled field, which will be taken
advantage of in the next commit.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
d947033037 refs: use same lock_file object for both ref-packing functions
Use a single struct lock_file for both pack_refs() and
repack_without_ref().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
3b4ae6d502 pack_one_ref(): rename "path" parameter to "refname"
Make this function conform to the naming convention established in
65385ef7d4 for the rest of the refs.c file.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
32d462cea8 pack-refs: merge code from pack-refs.{c,h} into refs.{c,h}
pack-refs.c doesn't contain much code, and the code it does contain is
closely related to reference handling.  Moreover, there is some
duplication between pack_refs() and repack_without_ref().  Therefore,
merge pack-refs.c into refs.c and pack-refs.h into refs.h.

The code duplication will be addressed in future commits.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:11 -07:00
Michael Haggerty
fec3137ffc refs: extract a function write_packed_entry()
Extract the I/O code from the "business logic" in repack_ref_fn().
Later there will be another caller for this function.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
694b7a1999 repack_without_ref(): write peeled refs in the rewritten file
When a reference that existed in the packed-refs file is deleted, the
packed-refs file must be rewritten.  Previously, the file was
rewritten without any peeled refs, even if the file contained peeled
refs when it was read.  This was not a bug, because the packed-refs
file header didn't claim that the file contained peeled values.  But
it had a performance cost, because the repository would lose the
benefit of having precomputed peeled references until pack-refs was
run again.

Teach repack_without_ref() to write peeled refs to the packed-refs
file (regardless of whether they were present in the old version of
the file).

This means that if the old version of the packed-refs file was not
fully peeled, then repack_without_ref() will have to peel references.
To avoid the expense of reading lots of loose references, we take two
shortcuts relative to pack-refs:

* If the peeled value of a reference is already known (i.e., because
  it was read from the old version of the packed-refs file), then
  output that peeled value again without any checks.  This is the
  usual code path and should avoid any noticeable overhead.  (This is
  different than pack-refs, which always re-peels references.)

* We don't verify that the packed ref is still current.  It could be
  that a packed references is overridden by a loose reference, in
  which case the packed ref is no longer needed and might even refer
  to an object that has been garbage collected.  But we don't check;
  instead, we just try to peel all references.  If peeling is
  successful, the peeled value is written out (even though it might
  not be needed any more); if not, then the reference is silently
  omitted from the output.

The extra overhead of peeling references in repack_without_ref()
should only be incurred the first time the packed-refs file is written
by a version of Git that knows about the "fully-peeled" attribute.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
506a760db8 refs: change how packed refs are deleted
Add a function remove_ref(), which removes a single entry from a
reference cache.

Use this function to reimplement repack_without_ref().  The old
version iterated over all refs, packing all of them except for the one
to be deleted, then discarded the entire packed reference cache.  The
new version deletes the doomed reference from the cache *before*
iterating.

This has two advantages:

* the code for writing packed-refs becomes simpler, because it doesn't
  have to exclude one of the references.

* it is no longer necessary to discard the packed refs cache after
  deleting a reference: symbolic refs cannot be packed, so packed
  references cannot depend on each other, so the rest of the packed
  refs cache remains valid after a reference is deleted.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
9fc0a64806 search_ref_dir(): return an index rather than a pointer
Change search_ref_dir() to return the index of the sought entry (or -1
on error) rather than a pointer to the entry.  This will make it more
natural to use the function for removing an entry from the list.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
ab292bc4f3 repack_without_ref(): silence errors for dangling packed refs
Stop emitting an error message when deleting a packed reference if we
find another dangling packed reference that is overridden by a loose
reference.  See the previous commit for a longer explanation of the
issue.

We have to be careful to make sure that the invalid packed reference
really *is* overridden by a loose reference; otherwise what we have
found is repository corruption, which we *should* report.

Please note that this approach is vulnerable to a race condition
similar to the race conditions already known to affect packed
references [1]:

* Process 1 tries to peel packed reference X as part of deleting
  another packed reference.  It discovers that X does not refer to a
  valid object (because the object that it referred to has been
  garbage collected).

* Process 2 tries to delete reference X.  It starts by deleting the
  loose reference X.

* Process 1 checks whether there is a loose reference X.  There is not
  (it has just been deleted by process 2), so process 1 reports a
  spurious error "X does not point to a valid object!"

The worst case seems relatively harmless, and the fix is identical to
the fix that will be needed for the other race conditions (namely
holding a lock on the packed-refs file during *all* reference
deletions), so we leave the cleaning up of all of them as a future
project.

[1] http://thread.gmane.org/gmane.comp.version-control.git/211956

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
624cac3514 refs: change the internal reference-iteration API
Establish an internal API for iterating over references, which gives
the callback functions direct access to the ref_entry structure
describing the reference.  (Do not change the iteration API that is
exposed outside of the module.)

Define a new internal callback signature

   int each_ref_entry_fn(struct ref_entry *entry, void *cb_data)

Change do_for_each_ref_in_dir() and do_for_each_ref_in_dirs() to
accept each_ref_entry_fn callbacks, and rename them to
do_for_each_entry_in_dir() and do_for_each_entry_in_dirs(),
respectively.  Adapt their callers accordingly.

Add a new function do_for_each_entry() analogous to do_for_each_ref()
but using the new callback style.

Change do_one_ref() into an each_ref_entry_fn that does some
bookkeeping and then calls a wrapped each_ref_fn.

Reimplement do_for_each_ref() in terms of do_for_each_entry(), using
do_one_ref() as an adapter.

Please note that the responsibility for setting current_ref remains in
do_one_ref(), which means that current_ref is *not* set when iterating
over references via the new internal API.  This is not a disadvantage,
because current_ref is not needed by callers of the internal API (they
receive a pointer to the current ref_entry anyway).  But more
importantly, this change prevents peel_ref() from returning invalid
results in the following scenario:

When iterating via the external API, the iteration always includes
both packed and loose references, and in particular never presents a
packed ref if there is a loose ref with the same name.  The internal
API, on the other hand, gives the option to iterate over only the
packed references.  During such an iteration, there is no check
whether the packed ref might be hidden by a loose ref of the same
name.  But until now the packed ref was recorded in current_ref during
the iteration.  So if peel_ref() were called with the reference name
corresponding to current ref, it would return the peeled version of
the packed ref even though there might be a loose ref that peels to a
different value.  This scenario doesn't currently occur in the code,
but fix it to prevent things from breaking in a very confusing way in
the future.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
9a489f3c17 refs: extract a function peel_entry()
Peel the entry, and as a side effect store the peeled value in the
entry.  Use this function from two places in peel_ref(); a third
caller will be added soon.

Please note that this change can lead to ref_entries for unpacked refs
being peeled.  This has no practical benefit but is harmless.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
2312a79320 peel_ref(): fix return value for non-peelable, not-current reference
The old version was inconsistent: when a reference was
REF_KNOWS_PEELED but with a null peeled value, it returned non-zero
for the current reference but zero for other references.  Change the
behavior for non-current references to match that of current_ref,
which is what callers expect.  Document the behavior.

Current callers only call peel_ref() from within a for_each_ref-style
iteration and only for the current ref; therefore, the buggy code path
was never reached.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
68cf870344 peel_object(): give more specific information in return value
Instead of just returning a success/failure bit, return an enumeration
value that explains the reason for any failure.  This will come in
handy shortly.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
cb2ae1c418 refs: extract function peel_object()
It is a nice, logical unit of work, and putting it in a function
removes the need to use a goto in peel_ref().  Soon it will also have
other uses.

The algorithm is unchanged.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
662428f4e9 refs: extract a function ref_resolves_to_object()
It is a nice unit of work and soon will be needed from multiple
locations.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:10 -07:00
Michael Haggerty
7618fd808a repack_without_ref(): use function get_packed_ref()
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
f361baeb71 peel_ref(): use function get_packed_ref()
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
63331581ab get_packed_ref(): return a ref_entry
Instead of copying the reference's SHA1 into a caller-supplied
variable, just return the ref_entry itself (or NULL if there is no
such entry).  This change will allow the function to be used from
elsewhere.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
b830f6c66b do_for_each_ref_in_dirs(): remove dead code
There is no way to drop out of the while loop.  This code has been
dead since 432ad41e.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
3feb4f0cfb refs: define constant PEELED_LINE_LENGTH
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
7d76fdc829 refs: document how current_ref is used
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
fcce17039c refs: document do_for_each_ref() and do_one_ref()
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
6c6f58dfd2 refs: document the fields of struct ref_value
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Michael Haggerty
89df9c84e4 refs: document flags constants REF_*
Document the bits that can appear in the "flags" parameter passed to
an each_ref_function and/or in the ref_entry::flag field.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-05-01 15:33:09 -07:00
Junio C Hamano
f1ad05f3a5 Merge branch 'jk/fully-peeled-packed-ref' into maint-1.8.1
* jk/fully-peeled-packed-ref:
  pack-refs: add fully-peeled trait
  pack-refs: write peeled entry for non-tags
  use parse_object_or_die instead of die("bad object")
  avoid segfaults on parse_object failure
2013-04-03 08:43:03 -07:00
Junio C Hamano
6beb484f25 Merge branch 'jc/reflog-reverse-walk'
An internal function used to implement "git checkout @{-1}" was
hard to use correctly.

* jc/reflog-reverse-walk:
  refs.c: fix fread error handling
  reflog: add for_each_reflog_ent_reverse() API
  for_each_recent_reflog_ent(): simplify opening of a reflog file
  for_each_reflog_ent(): extract a helper to process a single entry
2013-03-26 13:15:56 -07:00
Junio C Hamano
870987dec7 Merge branch 'jk/fully-peeled-packed-ref'
Not that we do not actively encourage having annotated tags outside
refs/tags/ hierarchy, but they were not advertised correctly to the
ls-remote and fetch with recent version of Git.

* jk/fully-peeled-packed-ref:
  pack-refs: add fully-peeled trait
  pack-refs: write peeled entry for non-tags
  use parse_object_or_die instead of die("bad object")
  avoid segfaults on parse_object failure
2013-03-25 14:01:07 -07:00
John Keeping
e4ca819abf refs.c: fix fread error handling
fread returns the number of items read, with no special error return.

Commit 98f85ff (reflog: add for_each_reflog_ent_reverse() API -
2013-03-08) introduced a call to fread which checks for an error with
"nread < 0" which is tautological since nread is unsigned.  The correct
check in this case (which tries to read a single item) is "nread != 1".

Signed-off-by: John Keeping <john@keeping.me.uk>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-23 23:50:50 -07:00
Michael Haggerty
c29c46fa2e pack-refs: add fully-peeled trait
Older versions of pack-refs did not write peel lines for
refs outside of refs/tags. This meant that on reading the
pack-refs file, we might set the REF_KNOWS_PEELED flag for
such a ref, even though we do not know anything about its
peeled value.

The previous commit updated the writer to always peel, no
matter what the ref is. That means that packed-refs files
written by newer versions of git are fine to be read by both
old and new versions of git. However, we still have the
problem of reading packed-refs files written by older
versions of git, or by other implementations which have not
yet learned the same trick.

The simplest fix would be to always unset the
REF_KNOWS_PEELED flag for refs outside of refs/tags that do
not have a peel line (if it has a peel line, we know it is
valid, but we cannot assume a missing peel line means
anything). But that loses an important optimization, as
upload-pack should not need to load the object pointed to by
refs/heads/foo to determine that it is not a tag.

Instead, we add a "fully-peeled" trait to the packed-refs
file. If it is set, we know that we can trust a missing peel
line to mean that a ref cannot be peeled. Otherwise, we fall
back to assuming nothing.

[commit message and tests by Jeff King <peff@peff.net>]

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-18 08:06:28 -07:00
Junio C Hamano
98f85ff4b6 reflog: add for_each_reflog_ent_reverse() API
"git checkout -" is a short-hand for "git checkout @{-1}" and the
"@{nth}" notation for a negative number is to find nth previous
checkout in the reflog of the HEAD to determine the name of the
branch the user was on.  We would want to find the nth most recent
reflog entry that matches "checkout: moving from X to Y" for this.

Unfortunately, reflog is implemented as an append-only file, and the
API to iterate over its entries, for_each_reflog_ent(), reads the
file in order, giving the entries from the oldest to newer.  For the
purpose of finding nth most recent one, this API forces us to record
the last n entries in a rotating buffer and give the result out only
after we read everything.  To optimize for a common case of finding
the nth most recent one for a small value of n, we also have a side
API for_each_recent_reflog_ent() that starts reading near the end of
the file, but it still has to read the entries in the "wrong" order.
The implementation of understanding @{-1} uses this interface.

This all becomes unnecessary if we add an API to let us iterate over
reflog entries in the reverse order, from the newest to older.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-08 14:00:22 -08:00
Junio C Hamano
7ae07c1bd7 for_each_recent_reflog_ent(): simplify opening of a reflog file
There is no reason to use a temporary variable logfile.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-08 14:00:16 -08:00
Junio C Hamano
9a7a183bd2 for_each_reflog_ent(): extract a helper to process a single entry
Split the logic that takes a single line of reflog entry in a
strbuf, parses the message, and calls the callback function out of
the loop into a separate helper function.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-03-08 13:59:57 -08:00
Junio C Hamano
ce735bf7fd Merge branch 'jc/hidden-refs'
Allow the server side to redact the refs/ namespace it shows to the
client.

Will merge to 'master'.

* jc/hidden-refs:
  upload/receive-pack: allow hiding ref hierarchies
  upload-pack: simplify request validation
  upload-pack: share more code
2013-02-17 15:25:57 -08:00
Junio C Hamano
daebaa7813 upload/receive-pack: allow hiding ref hierarchies
A repository may have refs that are only used for its internal
bookkeeping purposes that should not be exposed to the others that
come over the network.

Teach upload-pack to omit some refs from its initial advertisement
by paying attention to the uploadpack.hiderefs multi-valued
configuration variable.  Do the same to receive-pack via the
receive.hiderefs variable.  As a convenient short-hand, allow using
transfer.hiderefs to set the value to both of these variables.

Any ref that is under the hierarchies listed on the value of these
variable is excluded from responses to requests made by "ls-remote",
"fetch", etc. (for upload-pack) and "push" (for receive-pack).

Because these hidden refs do not count as OUR_REF, an attempt to
fetch objects at the tip of them will be rejected, and because these
refs do not get advertised, "git push :" will not see local branches
that have the same name as them as "matching" ones to be sent.

An attempt to update/delete these hidden refs with an explicit
refspec, e.g. "git push origin :refs/hidden/22", is rejected.  This
is not a new restriction.  To the pusher, it would appear that there
is no such ref, so its push request will conclude with "Now that I
sent you all the data, it is time for you to update the refs.  I saw
that the ref did not exist when I started pushing, and I want the
result to point at this commit".  The receiving end will apply the
compare-and-swap rule to this request and rejects the push with
"Well, your update request conflicts with somebody else; I see there
is such a ref.", which is the right thing to do. Otherwise a push to
a hidden ref will always be "the last one wins", which is not a good
default.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-02-07 13:48:47 -08:00
Junio C Hamano
f12e49ae87 Merge branch 'rs/clarify-entry-cmp-sslice'
* rs/clarify-entry-cmp-sslice:
  refs: use strncmp() instead of strlen() and memcmp()
2013-01-23 21:19:06 -08:00
René Scharfe
c971ddfdcd refs: use strncmp() instead of strlen() and memcmp()
Simplify ref_entry_cmp_sslice() by using strncmp() to compare the
length-limited key and a NUL-terminated entry.  While we're at it,
retain the const attribute of the input pointers.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2013-01-16 09:48:36 -08:00
Junio C Hamano
f70eec8400 Merge branch 'jk/repack-ref-racefix' into maint
* jk/repack-ref-racefix:
  refs: do not use cached refs in repack_without_ref
2013-01-10 14:04:17 -08:00
Junio C Hamano
229096a591 Merge branch 'jk/repack-ref-racefix'
"git pack-refs" that ran in parallel to another process that created
new refs had a nasty race.

* jk/repack-ref-racefix:
  refs: do not use cached refs in repack_without_ref
2013-01-02 10:39:37 -08:00
Jeff King
b3f1280ec7 refs: do not use cached refs in repack_without_ref
When we delete a ref that is packed, we rewrite the whole
packed-refs file and simply omit the ref that no longer
exists. However, we base the rewrite on whatever happens to
be in our refs cache, not what is necessarily on disk. That
opens us up to a race condition if another process is
simultaneously packing the refs, as we will overwrite their
newly-made pack-refs file with our potentially stale data,
losing commits.

You can demonstrate the race like this:

  # setup some repositories
  git init --bare parent &&
  (cd parent && git config core.logallrefupdates true) &&
  git clone parent child &&
  (cd child && git commit --allow-empty -m base)

  # in one terminal, repack the refs repeatedly
  cd parent &&
  while true; do
	git pack-refs --all
  done

  # in another terminal, simultaneously push updates to
  # master, and create and delete an unrelated ref
  cd child &&
  while true; do
	git push origin HEAD:newbranch &&
	git commit --allow-empty -m foo
	us=`git rev-parse master` &&
	git push origin master &&
	git push origin :newbranch &&
	them=`git --git-dir=../parent rev-parse master` &&
	if test "$them" != "$us"; then
		echo >&2 "$them" != "$us"
		exit 1
	fi
  done

In many cases the two processes will conflict over locking
the packed-refs file, and the deletion of newbranch will
simply fail.  But eventually you will hit the race, which
happens like this:

  1. We push a new commit to master. It is already packed
     (from the looping pack-refs call). We write the new
     value (let us call it B) to $GIT_DIR/refs/heads/master,
     but the old value (call it A) remains in the
     packed-refs file.

  2. We push the deletion of newbranch, spawning a
     receive-pack process. Receive-pack advertises all refs
     to the client, causing it to iterate over each ref; it
     caches the packed refs in memory, which points at the
     stale value A.

  3. Meanwhile, a separate pack-refs process is running. It
     runs to completion, updating the packed-refs file to
     point master at B, and deleting $GIT_DIR/refs/heads/master
     which also pointed at B.

  4. Back in the receive-pack process, we get the
     instruction to delete :newbranch. We take a lock on
     packed-refs (which works, as the other pack-refs
     process has already finished). We then rewrite the
     contents using the cached refs, which contain the stale
     value A.

The resulting packed-refs file points master once again at
A. The loose ref which would override it to point at B was
deleted (rightfully) in step 3. As a result, master now
points at A. The only trace that B ever existed in the
parent is in the reflog: the final entry will show master
moving from A to B, even though the ref still points at A
(so you can detect this race after the fact, because the
next reflog entry will move from A to C).

We can fix this by invalidating the packed-refs cache after
we have taken the lock. This means that we will re-read the
packed-refs file, and since we have the lock, we will be
sure that what we read will be atomically up-to-date when we
write (it may be out of date with respect to loose refs, but
that is OK, as loose refs take precedence).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-12-21 08:10:22 -08:00
Junio C Hamano
cf22e272e4 Merge branch 'jh/update-ref-d-through-symref'
"update-ref -d --deref SYM" to delete a ref through a symbolic ref
that points to it did not remove it correctly.

* jh/update-ref-d-through-symref:
  Fix failure to delete a packed ref through a symref
  t1400-update-ref: Add test verifying bug with symrefs in delete_ref()
2012-11-25 18:44:17 -08:00
Jeff King
15ba878a1d Merge branch 'rs/lock-correct-ref-during-delete'
When "update-ref -d --no-deref SYM" tried to delete a symbolic ref
SYM, it incorrectly locked the underlying reference pointed by SYM,
not the symbolic ref itself.

* rs/lock-correct-ref-during-delete:
  refs: lock symref that is to be deleted, not its target
2012-11-09 12:42:28 -05:00
Johan Herland
b274a7146c Fix failure to delete a packed ref through a symref
When deleting a ref through a symref (e.g. using 'git update-ref -d HEAD'
to delete refs/heads/master), we would remove the loose ref, but a packed
version of the same ref would remain, the end result being that instead of
deleting refs/heads/master we would appear to reset it to its state as of
the last repack.

This patch fixes the issue, by making sure we pass the correct ref name
when invoking repack_without_ref() from within delete_ref().

Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-10-21 12:08:42 -07:00
René Scharfe
547d058fb6 refs: lock symref that is to be deleted, not its target
When delete_ref is called on a symref then it locks its target and then
either deletes the target or the symref, depending on whether the flag
REF_NODEREF was set in the parameter delopt.

Instead, simply pass the flag to lock_ref_sha1_basic, which will then
either lock the target or the symref, and delete the locked ref.

This reimplements part of eca35a25 (Fix git branch -m for symrefs.).

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-10-16 10:08:23 -07:00
Jeff King
6c4a060d7d peel_ref: check object type before loading
The point of peel_ref is to dereference tags; if the base
object is not a tag, then we can return early without even
loading the object into memory.

This patch accomplishes that by checking sha1_object_info
for the type. For a packed object, we can get away with just
looking in the pack index. For a loose object, we only need
to inflate the first couple of header bytes.

This is a bit of a gamble; if we do find a tag object, then
we will end up loading the content anyway, and the extra
lookup will have been wasteful. However, if it is not a tag
object, then we save loading the object entirely. Depending
on the ratio of non-tags to tags in the input, this can be a
minor win or minor loss.

However, it does give us one potential major win: if a ref
points to a large blob (e.g., via an unannotated tag), then
we can avoid looking at it entirely.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-10-04 20:34:28 -07:00
Jeff King
e6dbffa67b peel_ref: do not return a null sha1
The idea of the peel_ref function is to dereference tag
objects recursively until we hit a non-tag, and return the
sha1. Conceptually, it should return 0 if it is successful
(and fill in the sha1), or -1 if there was nothing to peel.

However, the current behavior is much more confusing. For a
regular loose ref, the behavior is as described above. But
there is an optimization to reuse the peeled-ref value for a
ref that came from a packed-refs file. If we have such a
ref, we return its peeled value, even if that peeled value
is null (indicating that we know the ref definitely does
_not_ peel).

It might seem like such information is useful to the caller,
who would then know not to bother loading and trying to peel
the object. Except that they should not bother loading and
trying to peel the object _anyway_, because that fallback is
already handled by peel_ref. In other words, the whole point
of calling this function is that it handles those details
internally, and you either get a sha1, or you know that it
is not peel-able.

This patch catches the null sha1 case internally and
converts it into a -1 return value (i.e., there is nothing
to peel). This simplifies callers, which do not need to
bother checking themselves.

Two callers are worth noting:

  - in pack-objects, a comment indicates that there is a
    difference between non-peelable tags and unannotated
    tags. But that is not the case (before or after this
    patch). Whether you get a null sha1 has to do with
    internal details of how peel_ref operated.

  - in show-ref, if peel_ref returns a failure, the caller
    tries to decide whether to try peeling manually based on
    whether the REF_ISPACKED flag is set. But this doesn't
    make any sense. If the flag is set, that does not
    necessarily mean the ref came from a packed-refs file
    with the "peeled" extension. But it doesn't matter,
    because even if it didn't, there's no point in trying to
    peel it ourselves, as peel_ref would already have done
    so. In other words, the fallback peeling is guaranteed
    to fail.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-10-04 20:34:28 -07:00
Jeff King
44da6f69ec peel_ref: use faster deref_tag_noverify
When we are asked to peel a ref to a sha1, we internally call
deref_tag, which will recursively parse each tagged object
until we reach a non-tag. This has the benefit that we will
verify our ability to load and parse the pointed-to object.

However, there is a performance downside: we may not need to
load that object at all (e.g., if we are listing peeled
simply listing peeled refs), or it may be a large object
that should follow a streaming code path (e.g., an annotated
tag of a large blob).

It makes more sense for peel_ref to choose the fast thing
rather than performing the extra check, for two reasons:

  1. We will already sometimes short-circuit the tag parsing
     in favor of a peeled entry from a packed-refs file. So
     we are already favoring speed in some cases, and it is
     not wise for a caller to rely on peel_ref to detect
     corruption.

  2. We already silently ignore much larger corruptions,
     like a ref that points to a non-existent object, or a
     tag object that exists but is corrupted.

  2. peel_ref is not the right place to check for such a
     database corruption. It is returning only the sha1
     anyway, not the actual object. Any callers which use
     that sha1 to load an object will soon discover the
     corruption anyway, so we are really just pushing back
     the discovery to later in the program.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-10-04 20:34:28 -07:00
Junio C Hamano
a7060009e1 Merge branch 'rs/refs-string-slice'
Avoid unnecessary temporary allocations while looking for matching refs
inside refs API.

By René Scharfe (3) and Junio C Hamano (1)
* rs/refs-string-slice:
  refs: do not create ref_entry when searching
  refs: use strings directly in find_containing_dir()
  refs: convert parameter of create_dir_entry() to length-limited string
  refs: convert parameter of search_ref_dir() to length-limited string
2012-05-29 13:09:02 -07:00
Junio C Hamano
38f2b874ff Merge branch 'mh/ref-api-lazy-loose'
The code to lazily read loose refs unnecessarily read the refs in a
subhierarchy by mistake when we free the data for the subhierarchy.

By Michael Haggerty
* mh/ref-api-lazy-loose:
  free_ref_entry(): do not trigger reading of loose refs
2012-05-29 13:08:35 -07:00
Junio C Hamano
ec04a275b4 Merge branch 'mh/ref-api'
Fixes a performance regression in the earlier series.
2012-05-25 12:04:36 -07:00
Michael Haggerty
654ad400c2 Avoid sorting if references are added to ref_cache in order
The old code allowed many references to be efficiently added to a
single directory, because it just appended the references to the
containing directory unsorted without doing any searching (and
therefore without requiring any intermediate sorting).  But the old
code was inefficient when a large number of subdirectories were added
to a directory, because the directory always had to be searched to see
if the new subdirectory already existed, and this search required the
directory to be sorted first.  The same was repeated for every new
subdirectory, so the time scaled like O(N^2), where N is the number of
subdirectories within a single directory.

In practice, references are often added to the ref_cache in
lexicographic order, for example when reading the packed-refs file.
So build some intelligence into add_entry_to_dir() to optimize for the
case of references and/or subdirectories being added in lexicographic
order: if the existing entries were already sorted, and the new entry
comes after the last existing entry, then adjust ref_dir::sorted to
reflect the fact that the ref_dir is still sorted.

Thanks to Peff for pointing out the performance regression that
inspired this change.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-24 12:16:06 -07:00
Junio C Hamano
e1980c9d23 refs: do not create ref_entry when searching
The search_ref_dir() function is about looking up an existing ref_entry in
a sorted array of ref_entry stored in dir->entries, but it still allocates
a new ref_entry and frees it before returning.  This is only because the
call to bsearch(3) was coded in a suboptimal way. Unlike the comparison
function given to qsort(3), the first parameter to its comparison function
does not need to point at an object that is shaped like an element in the
array.

Introduce a new comparison function that takes a counted string as the key
and an element in an array of ref_entry and give it to bsearch(), so that
we do not have to allocate a new ref_entry that we will never return to
the caller anyway.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-22 14:28:03 -07:00
René Scharfe
dd02e72852 refs: use strings directly in find_containing_dir()
Convert the parameter subdirname of search_for_subdir() to a
length-limted string and then simply pass the interesting slice of the
refname from find_containing_dir(), thereby avoiding to duplicate the
string.

Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-22 13:32:26 -07:00
René Scharfe
b9146f517a refs: convert parameter of create_dir_entry() to length-limited string
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-22 13:32:24 -07:00
René Scharfe
40ad937d47 refs: convert parameter of search_ref_dir() to length-limited string
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-22 13:32:21 -07:00
Michael Haggerty
27b5587c76 free_ref_entry(): do not trigger reading of loose refs
Do not call get_ref_dir() from within free_ref_entry(), because that
triggers the reading of loose refs, only for them to be freed
immediately.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-20 15:10:49 -07:00
Junio C Hamano
aa6912b081 Merge branch 'mh/ref-api-lazy-loose'
Refs API is updated to lazily read sub-hierarchies of refs/ namespace,
so that we do not have to grab everything from the filesystem when we
are only interested in listing branches, for example.

By Michael Haggerty (17) and Junio C Hamano (1)
* mh/ref-api-lazy-loose:
  refs: fix find_containing_dir() regression
  refs: read loose references lazily
  read_loose_refs(): eliminate ref_cache argument
  struct ref_dir: store a reference to the enclosing ref_cache
  search_for_subdir(): return (ref_dir *) instead of (ref_entry *)
  get_ref_dir(): add function for getting a ref_dir from a ref_entry
  read_loose_refs(): rename function from get_ref_dir()
  refs: wrap top-level ref_dirs in ref_entries
  find_containing_dir(): use strbuf in implementation of this function
  bisect: copy filename string obtained from git_path()
  do_for_each_reflog(): use a strbuf to hold logfile name
  do_for_each_reflog(): return early on error
  get_ref_dir(): take the containing directory as argument
  refs.c: extract function search_for_subdir()
  get_ref_dir(): require that the dirname argument ends in '/'
  get_ref_dir(): rename "base" parameter to "dirname"
  get_ref_dir(): use a strbuf to hold refname
  get_ref_dir(): return early if directory cannot be read
2012-05-10 10:49:08 -07:00
Junio C Hamano
663c1295d8 refs: fix find_containing_dir() regression
The function used to return NULL when asked to find the containing
directory for a ref that does not exist, allowing the caller to
omit iteration altogether. But a misconversion in an earlier change
"refs.c: extract function search_for_subdir()" started returning the
top-level directory entry, forcing callers to walk everything.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-04 09:17:50 -07:00
Michael Haggerty
28e6a34e25 refs: read loose references lazily
Instead of reading the whole directory of loose references the first
time any are needed, only read them on demand, one directory at a
time.

Use a new ref_entry flag bit REF_INCOMPLETE to indicate that the entry
represents a REF_DIR that hasn't been read yet.  Whenever any entries
from such a directory are needed, read all of the loose references
from that directory.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:36 -07:00
Michael Haggerty
423a1afc0b read_loose_refs(): eliminate ref_cache argument
The ref_cache can now be read from the ref_dir.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:36 -07:00
Michael Haggerty
f006c42a11 struct ref_dir: store a reference to the enclosing ref_cache
This means that a directory ref_entry contains all of the information
needed by read_loose_refs().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:36 -07:00
Michael Haggerty
3f3aa1bc62 search_for_subdir(): return (ref_dir *) instead of (ref_entry *)
That is what all the callers want, so give it to them.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:36 -07:00
Michael Haggerty
d7826d5427 get_ref_dir(): add function for getting a ref_dir from a ref_entry
Convert all accesses of a ref_dir within a ref_entry to use this
function.  This function will later be responsible for reading loose
references from disk on demand.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:36 -07:00
Michael Haggerty
1900b976a4 read_loose_refs(): rename function from get_ref_dir()
The new name better describes the function's purpose, and also makes
the old name available for a more suitable purpose.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:36 -07:00
Michael Haggerty
d12229f532 refs: wrap top-level ref_dirs in ref_entries
Make it turtles all the way down.  This affects the loose and packed
fields of ref_cache instances.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:35 -07:00
Michael Haggerty
5fa0441844 find_containing_dir(): use strbuf in implementation of this function
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:35 -07:00
Michael Haggerty
989c0e5d02 do_for_each_reflog(): use a strbuf to hold logfile name
This simplifies the bookkeeping and allows an (artificial) restriction
on refname component length to be removed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:35 -07:00
Michael Haggerty
93c603fcb7 do_for_each_reflog(): return early on error
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:09 -07:00
Michael Haggerty
9f2fb4a373 get_ref_dir(): take the containing directory as argument
Previously, the "dir" argument to get_ref_dir() was a pointer to the
top-level ref_dir.  Change the function to expect a pointer to the
ref_dir corresponding to dirname.  This allows entries to be added
directly to dir, without having to recurse through the reference trie
each time (i.e., we can use add_entry_to_dir() instead of add_ref()).

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:09 -07:00
Michael Haggerty
f348ac923c refs.c: extract function search_for_subdir()
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:09 -07:00
Michael Haggerty
abc390989f get_ref_dir(): require that the dirname argument ends in '/'
This removes some conditional code and makes it consistent with the
way that direntry names are stored.  Please note that this function is
never used on the top-level .git directory; it is always called for
directories at level .git/refs or deeper.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:09 -07:00
Michael Haggerty
66a3d20b8f get_ref_dir(): rename "base" parameter to "dirname"
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:09 -07:00
Michael Haggerty
72b64b44e7 get_ref_dir(): use a strbuf to hold refname
This simplifies the bookkeeping and allows an (artificial) restriction
on refname component length to be removed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-05-03 13:15:09 -07:00
Junio C Hamano
1be65eda6a Merge branch 'nd/i18n'
More message strings marked for i18n.

By Nguyễn Thái Ngọc Duy (10) and Jonathan Nieder (1)
* nd/i18n:
  help: replace underlining "help -a" headers using hyphens with a blank line
  i18n: bundle: mark strings for translation
  i18n: index-pack: mark strings for translation
  i18n: apply: update say_patch_name to give translators complete sentence
  i18n: apply: mark strings for translation
  i18n: remote: mark strings for translation
  i18n: make warn_dangling_symref() automatically append \n
  i18n: help: mark strings for translation
  i18n: mark relative dates for translation
  strbuf: convenience format functions with \n automatically appended
  Makefile: feed all header files to xgettext
2012-05-02 13:51:35 -07:00
Michael Haggerty
d5fdae6737 get_ref_dir(): return early if directory cannot be read
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-25 10:43:20 -07:00
Nguyễn Thái Ngọc Duy
18986d5377 i18n: make warn_dangling_symref() automatically append \n
This helps remove \n from translatable strings

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-24 14:55:48 -07:00
Michael Haggerty
933ac036d2 do_for_each_ref(): only iterate over the subtree that was requested
If the base argument has a "/" chararacter, then only iterate over the
reference subdir whose name is the part up to the last "/".

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:55:55 -07:00
Michael Haggerty
432ad41e60 refs: store references hierarchically
Store references hierarchically in a tree that matches the
pseudo-directory structure of the reference names.  Add a new kind of
ref_entry (with flag REF_DIR) to represent a whole subdirectory of
references.  Sort ref_dirs one subdirectory at a time.

NOTE: the dirs can now be sorted as a side-effect of other function
calls.  Therefore, it would be problematic to do something from a
each_ref_fn callback that could provoke the sorting of a directory
that is currently being iterated over (i.e., the directory containing
the entry that is being processed or any of its parents).

This is a bit far-fetched, because a directory is always sorted just
before being iterated over.  Therefore, read-only accesses cannot
trigger the sorting of a directory whose iteration has already
started.  But if a callback function would add a reference to a parent
directory of the reference in the iteration, then try to resolve a
reference under that directory, a re-sort could be triggered and cause
the iteration to work incorrectly.

Nevertheless...add a comment in refs.h warning against modifications
during iteration.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:55:55 -07:00
Michael Haggerty
81a79d8e27 sort_ref_dir(): simplify logic
Use the more usual indexing idiom for clarity.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:55:50 -07:00
Michael Haggerty
d3177275ed refs.c: rename ref_array -> ref_dir
This purely textual change is in preparation for storing references
hierarchically, when the old ref_array structure will represent one
"directory" of references.  Rename functions that deal with this
structure analogously, and also rename the structure's "refs" member
to "entries".

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:54:59 -07:00
Michael Haggerty
593f1bb82f struct ref_entry: nest the value part in a union
This change is obviously silly by itself, but it is a step towards
adding a second member to the union.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:54:58 -07:00
Michael Haggerty
dac529e44f check_refname_component(): return 0 for zero-length components
Return 0 (instead of -1) for zero-length components.  Move the
interpretation of zero-length components as illegal to
check_refname_format().

This will make it easier to extend check_refname_format() to also
check whether directory names are valid.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:54:58 -07:00
Michael Haggerty
732134edab free_ref_entry(): new function
Add a function free_ref_entry().  This function will become nontrivial
when ref_entry (soon) becomes polymorphic.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:54:58 -07:00
Michael Haggerty
5a4d494731 names_conflict(): simplify implementation
Save a bunch of lines of code and a couple of strlen() calls.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:54:54 -07:00
Michael Haggerty
d66da478f2 repack_without_ref(): reimplement using do_for_each_ref_in_array()
It costs a bit of boilerplate, but it means that the function can be
ignorant of how cached refs are stored.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:53:28 -07:00
Michael Haggerty
b3fd060f9e do_for_each_ref_in_arrays(): new function
Extract function do_for_each_ref_in_arrays() from do_for_each_ref().
The new function will be a useful building block for storing refs
hierarchically.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:51:52 -07:00
Michael Haggerty
c36b5bc2e4 do_for_each_ref_in_array(): new function
Extract function do_for_each_ref_in_array() from do_for_each_ref().
The new function will be a useful building block for storing refs
hierarchically.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:51:52 -07:00
Michael Haggerty
429213e470 refs: manage current_ref within do_one_ref()
Set and clear current_ref within do_one_ref() instead of setting it
here and leaving it to somebody else to clear it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:51:52 -07:00
Michael Haggerty
bc5fd6d3c2 refs.c: reorder definitions more logically
Reorder definitions in file: first check_refname_format() and helper
functions, then the functions for managing the ref_entry and ref_array
data structures, then ref_cache, then the more "business-logicky"
stuff.  No code is changed.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-04-10 15:51:47 -07:00
Michael Haggerty
cf6672edb1 refs: remove the extra_refs API
The extra_refs provided a kludgy way to create fake references at a
global level in the hope that they would only affect some particular
code path.  The last user of this API been rewritten, so strip this
stuff out before somebody else gets the bad idea of using it.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-02-12 19:50:39 -08:00
Michael Haggerty
30249ee68f add_packed_ref(): new function in the refs API.
Add a new function add_packed_ref() that adds a reference directly to
the in-memory packed reference cache.  This will be useful for
creating local references while cloning.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-17 11:55:04 -08:00
Michael Haggerty
e6ed3ca651 ref_array: keep track of whether references are sorted
Keep track of how many entries at the beginning of a ref_array are already
sorted.  In sort_ref_array(), return early if the the array is already
sorted (i.e., if no new references has been appended to the end of the
list since the last call to sort_ref_array()).

Sort ref_arrays only when needed, namely in search_ref_array() and in
do_for_each_ref().  However, never call sort_ref_array() on the
extra_refs, because extra_refs can contain multiple entries with the same
name and because sort_ref_array() not only sorts, but de-dups its
contents.

This change is currently not useful, because entries are not added to
ref_arrays after they are created.  But in a moment they will be...

Implementation note: we could store a binary "sorted" value instead of
an integer, but storing the number of sorted entries leaves the way
open for a couple of possible future optimizations:

* In sort_ref_array(), sort *only* the unsorted entries, then merge
  them with the sorted entries.  This should be faster if most of the
  entries are already sorted.

* Teach search_ref_array() to do a binary search of any sorted
  entries, and if unsuccessful do a linear search of any unsorted
  entries.  This would avoid the need to sort the list every time that
  search_ref_array() is called, and (given some intelligence about how
  often to sort) could significantly improve the speed in certain
  hypothetical usage patterns.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2012-01-17 11:53:21 -08:00
Junio C Hamano
8d68493f20 Merge branch 'mh/ref-api'
* mh/ref-api:
  add_ref(): take a (struct ref_entry *) parameter
  create_ref_entry(): extract function from add_ref()
  repack_without_ref(): remove temporary
  resolve_gitlink_ref_recursive(): change to work with struct ref_cache
  Pass a (ref_cache *) to the resolve_gitlink_*() helper functions
  resolve_gitlink_ref(): improve docstring
  get_ref_dir(): change signature
  refs: change signatures of get_packed_refs() and get_loose_refs()
  is_dup_ref(): extract function from sort_ref_array()
  add_ref(): add docstring
  parse_ref_line(): add docstring
  is_refname_available(): remove the "quiet" argument
  clear_ref_array(): rename from free_ref_array()
  refs: rename parameters result -> sha1
  refs: rename "refname" variables
  struct ref_entry: document name member

Conflicts:
	cache.h
	refs.c
2011-12-20 13:25:53 -08:00
Nguyễn Thái Ngọc Duy
8cad4744ee Rename resolve_ref() to resolve_ref_unsafe()
resolve_ref() may return a pointer to a shared buffer and can be
overwritten by the next resolve_ref() calls. Callers need to
pay attention, not to keep the pointer when the next call happens.

Rename with "_unsafe" suffix to warn developers (or reviewers) before
introducing new call sites.

This patch is generated using the following command

git grep -l 'resolve_ref(' -- '*.[ch]'|xargs sed -i 's/resolve_ref(/resolve_ref_unsafe(/g'

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13 09:39:46 -08:00
Nguyễn Thái Ngọc Duy
96ec7b1e70 Convert resolve_ref+xstrdup to new resolve_refdup function
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-13 09:26:52 -08:00
Michael Haggerty
dd73ecd1bc add_ref(): take a (struct ref_entry *) parameter
Take a pointer to the ref_entry to add to the array, rather than
creating the ref_entry within the function.  This opens the way to
having multiple kinds of ref_entries.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-12 09:08:53 -08:00
Michael Haggerty
cddc42587c create_ref_entry(): extract function from add_ref()
Separate the creation of the ref_entry from its addition to a ref_array.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-12 09:08:53 -08:00
Michael Haggerty
fe9c7b78c5 repack_without_ref(): remove temporary
Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-12 09:08:53 -08:00
Michael Haggerty
064d51dc40 resolve_gitlink_ref_recursive(): change to work with struct ref_cache
resolve_gitlink_ref() and resolve_gitlink_ref_recursive(), together,
basically duplicated the code in git_path_submodule().  So use that
function instead.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-12 09:08:53 -08:00
Michael Haggerty
b0626608ea Pass a (ref_cache *) to the resolve_gitlink_*() helper functions
And remove some redundant arguments from resolve_gitlink_packed_ref().

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-12-12 09:08:53 -08:00