* db/fetch-pack: (60 commits)
Define compat version of mkdtemp for systems lacking it
Avoid scary errors about tagged trees/blobs during git-fetch
fetch: if not fetching from default remote, ignore default merge
Support 'push --dry-run' for http transport
Support 'push --dry-run' for rsync transport
Fix 'push --all branch...' error handling
Fix compilation when NO_CURL is defined
Added a test for fetching remote tags when there is not tags.
Fix a crash in ls-remote when refspec expands into nothing
Remove duplicate ref matches in fetch
Restore default verbosity for http fetches.
fetch/push: readd rsync support
Introduce remove_dir_recursively()
bundle transport: fix an alloc_ref() call
Allow abbreviations in the first refspec to be merged
Prevent send-pack from segfaulting when a branch doesn't match
Cleanup unnecessary break in remote.c
Cleanup style nit of 'x == NULL' in remote.c
Fix memory leaks when disconnecting transport instances
Ensure builtin-fetch honors {fetch,transfer}.unpackLimit
...
receive-pack is only executed remotely so when
reporting errors, say so.
Signed-off-by: Joakim Tjernlund <Joakim.Tjernlund@transmode.se>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
There is a subtle (but important) linkage between receive-pack and
index-pack that allows index-pack to create a packfile but protect
it from being deleted by a concurrent `git repack -a -d` operation.
The linkage works by having index-pack mark the newly created pack
with a ".keep" file and then it passes the SHA-1 name of that new
packfile to receive-pack along its stdout channel.
The receive-pack process must unkeep the packfile by deleting the
.keep file, but can it can only do so after all elgible refs have
been updated in the receiving repository. This ensures that the
packfile is either kept or its objects are reachable, preventing
a concurrent repacker from deleting the packfile before it can
determine that its objects are actually needed by the repository.
The new builtin-fetch code needs to perform the same actions if
it choose to run index-pack rather than unpack-objects, so I am
moving this code out to its own function where both receive-pack
and fetch-pack are able to invoke it when necessary. The caller
is responsible for deleting the returned ".keep" and freeing the
path if the returned path is not NULL.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-checkout is also adapted to make use of this new option
instead of the handcrafted command sequence.
Signed-off-by: Sven Verdoolaege <skimo@kotnet.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Sergey Vlasov, Andy Parkins and Alex Riesen all pointed out that it
is possible for a single invocation of receive-pack to be given more
refs than the OS might allow us to pass as command line parameters
to a single hook invocation.
We don't want to break these up into multiple invocations (like
xargs might do) as that makes it impossible for the pre-receive
hook to verify multiple related ref updates occur at the same time,
and it makes it harder for post-receive to send out a single batch
notification.
Instead we pass the reference data on a pipe connected to the
hook's stdin, supplying one ref per line to the hook. This way a
single hook invocation can obtain an infinite amount of ref data,
without bumping into any operating system limits.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Since we have decided to change the calling conventions for the
pre-receive and post-receive hooks to take the ref data on stdin
rather than on the command line we cannot use the same logic to
invoke the update hook anymore.
So we take a small step backwards towards what we used to have,
and create a specialized function for executing just the update
hook.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I'm pulling the error handling used to decode the result of
run_command up into a new function so that I can reuse it.
No changes, just a simple code movement.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Bill Lear pointed out that it is easy to send out notifications of
changes with the update hook, but successful execution of the update
hook does not necessarily mean that the ref was actually updated.
Lock contention on the ref or being unable to append to the reflog
may prevent the ref from being changed. Sending out notifications
prior to the ref actually changing is very misleading.
To help this situation I am introducing two new hooks to the
receive-pack flow: pre-receive and post-receive. These new hooks
are invoked only once per receive-pack execution and are passed
three arguments per ref (refname, old-sha1, new-sha1).
The new post-receive hook is ideal for sending out notifications,
as it has the complete list of all refnames that were successfully
updated as well as the old and new SHA-1 values. This allows more
interesting notifications to be sent. Multiple ref updates could
be easily summarized into one email, for example.
The new pre-receive hook is ideal for logging update attempts, as it
is run only once for the entire receive-pack operation. It can also
be used to verify multiple updates happen at once, e.g. an update
to the `maint` head must also be accompained by a new annotated tag.
Lots of documentation improvements for receive-pack are included
in this change, as we want to make sure the new hooks are clearly
explained.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
I discovered we did not send an ng line in the report-status feedback
if the ref was not updated because the repository has the config
option receive.denyNonFastForwards enabled. I think the reason this
happened is that it is simply too easy to forget to set error_string
when returning back a failure from update()
We now return an ng line for a non-fastforward update, which in
turn will cause send-pack to exit with a non-zero exit status.
Hence the modified test.
This refactoring changes update to return a const char* describing
the error, which execute_commands always loads into error_string.
The result is what I think is cleaner code, and allows us to
initialize the error_string member to NULL when we read_head_info.
I want error_string to be NULL in all commands before we call
execute_commands, so that we can reuse the run_hook function to
execute a new pre-receive hook.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a simple refactoring of run_update_hook to allow the function
to be passed the name of the hook it runs and also to build the
argument list from a list of struct commands, rather than just one
struct command.
The refactoring is to support new pre-receive and post-receive
hooks that will be given the entire list of struct commands,
rather than just one struct command. These new hooks will follow
in another patch.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
There is little point in executing the post-update hook if all refs
had an error and were unable to be updated. In this case nothing
new is reachable within the repository, and there is no state change
for the post-update hook to be interested in.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
As the post-update hook is meant to run after we have completed the
receipt of the pushed changes, and it might actually try to kick off
a `repack -a -d`, we should delay on invoking it until after we have
removed the *.keep file on the uploaded pack (if we kept the pack).
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This failure to catch the failure of write_ref_sha1 was noticed
by Bill Lear. The ref will not update if the log file could not
be appended to (due to file permissions problems). Such a failure
should be flagged as a failure to update the ref, so that the client
knows the push did not succeed.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This mechanically converts strncmp() to use prefixcmp(), but only when
the parameters match specific patterns, so that they can be verified
easily. Leftover from this will be fixed in a separate step, including
idiotic conversions like
if (!strncmp("foo", arg, 3))
=>
if (!(-prefixcmp(arg, "foo")))
This was done by using this script in px.perl
#!/usr/bin/perl -i.bak -p
if (/strncmp\(([^,]+), "([^\\"]*)", (\d+)\)/ && (length($2) == $3)) {
s|strncmp\(([^,]+), "([^\\"]*)", (\d+)\)|prefixcmp($1, "$2")|;
}
if (/strncmp\("([^\\"]*)", ([^,]+), (\d+)\)/ && (length($1) == $3)) {
s|strncmp\("([^\\"]*)", ([^,]+), (\d+)\)|(-prefixcmp($2, "$1"))|;
}
and running:
$ git grep -l strncmp -- '*.c' | xargs perl px.perl
Signed-off-by: Junio C Hamano <junkio@cox.net>
Back when only handful commands that created commit and tag were
the only users of committer identity information, it made sense
to explicitly call setup_ident() to pre-fill the default value
from the gecos information. But it is much simpler for programs
to make the call automatic when get_ident() is called these days,
since many more programs want to use the information when updating
the reflog.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The code that uses committer_info() in reflog can barf and die
whenever it is asked to update a ref. And I do not think
calling ignore_missing_committer_name() upfront like recent
receive-pack did in the aplication is a reasonable workaround.
What the patch does.
- git_committer_info() takes one parameter. It used to be "if
this is true, then die() if the name is not available due to
bad GECOS, otherwise issue a warning once but leave the name
empty". The reason was because we wanted to prevent bad
commits from being made by git-commit-tree (and its
callers). The value 0 is only used by "git var -l".
Now it takes -1, 0 or 1. When set to -1, it does not
complain but uses the pw->pw_name when name is not
available. Existing 0 and 1 values mean the same thing as
they used to mean before. 0 means issue warnings and leave
it empty, 1 means barf and die.
- ignore_missing_committer_name() and its existing caller
(receive-pack, to set the reflog) have been removed.
- git-format-patch, to come up with the phoney message ID when
asked to thread, now passes -1 to git_committer_info(). This
codepath uses only the e-mail part, ignoring the name. It
used to barf and die. The other call in the same program
when asked to add signed-off-by line based on committer
identity still passes 1 to make sure it barfs instead of
adding a bogus s-o-b line.
- log_ref_write in refs.c, to come up with the name to record
who initiated the ref update in the reflog, passes -1. It
used to barf and die.
The last change means that git-update-ref, git-branch, and
commit walker backends can now be used in a repository with
reflog by somebody who does not have the user identity required
to make a commit. They all used to barf and die.
I've run tests and all of them seem to pass, and also tried "git
clone" as a user whose GECOS is empty -- git clone works again
now (it was broken when reflog was enabled by default).
But this definitely needs extra sets of eyeballs.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This allows transfer.unpackLimit to specify what these two
configuration variables want to set.
We would probably want to deprecate the two separate variables,
as I do not see much point in specifying them independently.
Signed-off-by: Junio C Hamano <junkio@cox.net>
We currently do not support fetching/cloning from a shallow repository
nor pushing into one. Make sure these are not attempted so that we
do not have to worry about corrupting repositories needlessly.
Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently the update hook invoked by receive-pack has its stdin
connected to the pushing client. The hook shouldn't attempt to
read from this stream, and doing so may consume data that was
meant for receive-pack. Instead we should give the update hook
/dev/null as its stdin, ensuring that it always receives EOF and
doesn't disrupt the protocol if it attempts to read any data.
The post-update hook is similar, as it gets invoked with /dev/null
on stdin to prevent the hook from reading data from the client.
Previously we had invoked it with stdout also connected to /dev/null,
throwing away anything on stdout, to prevent client protocol errors.
Instead we should redirect stdout to stderr, like we do with the
update hook.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
If an update hook outputs to stdout then that output will be sent
back over the wire to the push client as though it were part of
the git protocol. This tends to cause protocol errors on the
client end of the connection, as the hook output is not expected
in that context. Most hook developers work around this by making
sure their hook outputs everything to stderr.
But hooks shouldn't need to perform such special behavior. Instead
we can just dup stderr to stdout prior to invoking the update hook.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
The argc parameter is never used by the run_command_v family of
functions. Instead they require that the passed argv[] be NULL
terminated so they can rely on the operating system's execvp
function to correctly pass the arguments to the new process.
Making the caller pass the argc is just confusing, as the caller
could be mislead into believing that the argc might take precendece
over the argv, or that the argv does not need to be NULL terminated.
So goodbye argc. Don't come back.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a mechanical clean-up of the way *.c files include
system header files.
(1) sources under compat/, platform sha-1 implementations, and
xdelta code are exempt from the following rules;
(2) the first #include must be "git-compat-util.h" or one of
our own header file that includes it first (e.g. config.h,
builtin.h, pkt-line.h);
(3) system headers that are included in "git-compat-util.h"
need not be included in individual C source files.
(4) "git-compat-util.h" does not have to include subsystem
specific header files (e.g. expat.h).
Signed-off-by: Junio C Hamano <junkio@cox.net>
If GIT_COMMITTER_NAME is not available in receive-pack but reflogs
are enabled we would normally die out with an error message asking
the user to correct their environment settings.
Now that reflogs are enabled by default in (what we guessed to be)
non-bare Git repositories this may cause problems for some users
who don't have their full name in the gecos field and who don't
have access to the remote system to correct the problem.
So rather than die()'ing out in receive-pack when we try to log a
ref change and have no committer name we default to the username,
as obtained from the host's password database.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Currently the treshold is 5000. The likelihood of this value to ever be
crossed for a single push is really small making it not really useful.
The optimal treshold for a pure space saving on a filesystem with 4kb
blocks is 3. However this is likely to create many small packs
concentrating a large number of files in a single directory compared to
the same objects which are spread over 256 directories when loose. This
means we would need 512 objects per pack on average to approximagte the
same directory cost (a pack has 2 files because of the index).
But 512 is a really high value just like 5000 since most pushes are
unlikely to have that many objects. So let's try with a value of 100
which should have a good balance between small pushes going to be
exploded into loose objects and large pushes kept as whole packs.
This is not a replacement for periodic repacks of course.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* maint:
receive-pack: do not insist on fast-forward outside refs/heads/
git-mv: search more precisely for source directory in index
Conflicts:
receive-pack.c
This allows you to say
git send-pack $URL :refs/heads/$branch
to delete the named remote branch. The refspec $src:$dst means
replace the destination ref with the object known as $src on the
local side, so this is a natural extension to make an empty $src
mean "No object" to delete the target.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This makes both git-fetch and git-push (fetch-pack and receive-pack)
safe against a possible race with aparallel git-repack -a -d that could
prune the new pack while it is not yet referenced, and remove the .keep
file after refs have been updated.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Since keeping a pushed pack or exploding it into loose objects
should be a local repository decision this teaches receive-pack
to decide if it should call unpack-objects or index-pack --stdin
--fix-thin based on the setting of receive.unpackLimit and the
number of objects contained in the received pack.
If the number of objects (hdr_entries) in the received pack is
below the value of receive.unpackLimit (which is 5000 by default)
then we unpack-objects as we have in the past.
If the hdr_entries >= receive.unpackLimit then we call index-pack and
ask it to include our pid and hostname in the .keep file to make it
easier to identify why a given pack has been kept in the repository.
Currently this leaves every received pack as a kept pack. We really
don't want that as received packs will tend to be small. Instead we
want to delete the .keep file automatically after all refs have
been updated. That is being left as room for future improvement.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* master: (90 commits)
gitweb: Better support for non-CSS aware web browsers
gitweb: Output also empty patches in "commitdiff" view
gitweb: Use git-for-each-ref to generate list of heads and/or tags
for-each-ref: "creator" and "creatordate" fields
Add --global option to git-repo-config.
pack-refs: Store the full name of the ref even when packing only tags.
git-clone documentation didn't mention --origin as equivalent of -o
Minor grammar fixes for git-diff-index.txt
link_temp_to_file: call adjust_shared_perm() only when we created the directory
Remove uneccessarily similar printf() from print_ref_list() in builtin-branch
pack-objects doesn't create random pack names
branch: work in subdirectories.
gitweb: Use 's' regexp modifier to secure against filenames with LF
gitweb: Secure against commit-ish/tree-ish with the same name as path
gitweb: esc_html() author in blame
git-svnimport: support for partial imports
link_temp_to_file: don't leave the path truncated on adjust_shared_perm failure
Move deny_non_fast_forwards handling completely into receive-pack.
revision traversal: --unpacked does not limit commit list anymore.
Continue traversal when rev-list --unpacked finds a packed commit.
...
* lj/refs: (63 commits)
Fix show-ref usagestring
t3200: git-branch testsuite update
sha1_name.c: avoid compilation warnings.
Make git-branch a builtin
ref-log: fix D/F conflict coming from deleted refs.
git-revert with conflicts to behave as git-merge with conflicts
core.logallrefupdates thinko-fix
git-pack-refs --all
core.logallrefupdates create new log file only for branch heads.
Remove bashism from t3210-pack-refs.sh
ref-log: allow ref@{count} syntax.
pack-refs: call fflush before fsync.
pack-refs: use lockfile as everybody else does.
git-fetch: do not look into $GIT_DIR/refs to see if a tag exists.
lock_ref_sha1_basic does not remove empty directories on BSD
Do not create tag leading directories since git update-ref does it.
Check that a tag exists using show-ref instead of looking for the ref file.
Use git-update-ref to delete a tag instead of rm()ing the ref file.
Fix refs.c;:repack_without_ref() clean-up path
Clean up "git-branch.sh" and add remove recursive dir test cases.
...
The 'receive.denynonfastforwards' option has nothing to do with
the repository format version. Since receive-pack already uses
git_config to initialize itself before executing any updates we
can use the normal configuration strategy and isolate the receive
specific variables away from the core variables.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
We aren't using this return code variable for anything so lets
just get rid of it to keep this section of code clean.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
* master: (72 commits)
runstatus: do not recurse into subdirectories if not needed
grep: fix --fixed-strings combined with expression.
grep: free expressions and patterns when done.
Corrected copy-and-paste thinko in ignore executable bit test case.
An illustration of rev-list --parents --pretty=raw
Allow git-checkout when on a non-existant branch.
gitweb: Decode long title for link tooltips
git-svn: Fix fetch --no-ignore-externals with GIT_SVN_NO_LIB=1
Ignore executable bit when adding files if filemode=0.
Remove empty ref directories that prevent creating a ref.
Use const for interpolate arguments
git-archive: update documentation
Deprecate merge-recursive.py
gitweb: fix over-eager application of esc_html().
Allow '(no author)' in git-svn's authors file.
Allow 'svn fetch' on '(no date)' revisions in Subversion.
git-repack: allow git-repack to run in subdirectory
Remove upload-tar and make git-tar-tree a thin wrapper to git-archive
git-tar-tree: Move code for git-archive --format=tar to archive-tar.c
git-tar-tree: Remove duplicate git_config() call
...
This converts receive-pack to use the standard ref locking code
instead of its own. As a side effect, it automatically records
the "push" event to ref-log if enabled.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This adds a "int *flag" parameter to resolve_ref() and makes
for_each_ref() family to call callback function with an extra
"int flag" parameter. They are used to give two bits of
information (REF_ISSYMREF and REF_ISPACKED) about the ref.
Signed-off-by: Junio C Hamano <junkio@cox.net>
This is a long overdue fix to the API for for_each_ref() family
of functions. It allows the callers to specify a callback data
pointer, so that the caller does not have to use static
variables to communicate with the callback funciton.
The updated for_each_ref() family takes a function of type
int (*fn)(const char *, const unsigned char *, void *)
and a void pointer as parameters, and calls the function with
the name of the ref and its SHA-1 with the caller-supplied void
pointer as parameters.
The commit updates two callers, builtin-name-rev.c and
builtin-pack-refs.c as an example.
Signed-off-by: Junio C Hamano <junkio@cox.net>
If receive.denyNonFastforwards is set to true, git-receive-pack will deny
non fast-forwards, i.e. forced updates. Most notably, a push to a repository
which has that flag set will fail.
As a first user, 'git-init-db --shared' sets this flag, since in a shared
setup, you are most unlikely to want forced pushes to succeed.
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de>
Signed-off-by: Junio C Hamano <junkio@cox.net>
This abstracts away the size of the hash values when copying them
from memory location to memory location, much as the introduction
of hashcmp abstracted away hash value comparsion.
A few call sites were using char* rather than unsigned char* so
I added the cast rather than open hashcpy to be void*. This is a
reasonable tradeoff as most call sites already use unsigned char*
and the existing hashcmp is also declared to be unsigned char*.
[jc: Splitted the patch to "master" part, to be followed by a
patch for merge-recursive.c which is not in "master" yet.
Fixed the cast in the latter hunk to combine-diff.c which was
wrong in the original.
Also converted ones left-over in combine-diff.c, diff-lib.c and
upload-pack.c ]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Junio C Hamano <junkio@cox.net>
[jc: I needed to hand merge the changes to the updated codebase,
so the result needs to be checked.]
Signed-off-by: David Rientjes <rientjes@google.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
Mark Wooding noticed there was a type mismatch warning in git.c; this
patch does things slightly differently (mostly tightening const) and
was what I was holding onto, waiting for the setup-revisions change
to be merged into the master branch.
Signed-off-by: Junio C Hamano <junkio@cox.net>
The git suite may not be in PATH (and thus programs such as
git-send-pack could not exec git-rev-list). Thus there is a need for
logic that will locate these programs. Modifying PATH is not
desirable as it result in behavior differing from the user's
intentions, as we may end up prepending "/usr/bin" to PATH.
- git C programs will use exec*_git_cmd() APIs to exec sub-commands.
- exec*_git_cmd() will execute a git program by searching for it in
the following directories:
1. --exec-path (as used by "git")
2. The GIT_EXEC_PATH environment variable.
3. $(gitexecdir) as set in Makefile (default value $(bindir)).
- git wrapper will modify PATH as before to enable shell scripts to
invoke "git-foo" commands.
Ideally, shell scripts should use the git wrapper to become independent
of PATH, and then modifying PATH will not be necessary.
[jc: with minor updates after a brief review.]
Signed-off-by: Michal Ostrowski <mostrows@watson.ibm.com>
Signed-off-by: Junio C Hamano <junkio@cox.net>
