development/git
development/git
1
0
Fork 0
mirror of https://github.com/git/git synced 2024-07-16 10:38:05 +00:00
Code Issues Actions 9 Packages Projects Releases Wiki Activity

Merge branch 'js/bundle'

Browse source 
* js/bundle:
  bundle: reword missing prerequisite error message
  git-bundle: record commit summary in the prerequisite data
  git-bundle: fix 'create --all'
  git-bundle: avoid fork() in verify_bundle()
  git-bundle: assorted fixes
  Add git-bundle: move objects and references by archive
This commit is contained in:
Junio C Hamano 2007-02-28 14:38:36 -08:00
parent 1db8b60b2a 64d99e9c5a
commit 77b50ab009
12 changed files with 611 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -13,6 +13,7 @@ git-archive
git-bisect
git-blame
git-branch
git-bundle
git-cat-file
git-check-ref-format
git-checkout

1
Documentation/cmd-list.perl
View file

@ -70,6 +70,7 @@ sub format_one {
git-bisect mainporcelain
git-blame ancillaryinterrogators
git-branch mainporcelain
git-bundle mainporcelain
git-cat-file plumbinginterrogators
git-checkout-index plumbingmanipulators
git-checkout mainporcelain

139
Documentation/git-bundle.txt Normal file
View file

@ -0,0 +1,139 @@
git-bundle(1)
=============
NAME
----
git-bundle - Move objects and refs by archive
SYNOPSIS
--------
'git-bundle' create <file> [git-rev-list args]
'git-bundle' verify <file>
'git-bundle' list-heads <file> [refname...]
'git-bundle' unbundle <file> [refname...]
DESCRIPTION
-----------
Some workflows require that one or more branches of development on one
machine be replicated on another machine, but the two machines cannot
be directly connected so the interactive git protocols (git, ssh,
rsync, http) cannot be used. This command provides suport for
git-fetch and git-pull to operate by packaging objects and references
in an archive at the originating machine, then importing those into
another repository using gitlink:git-fetch[1] and gitlink:git-pull[1]
after moving the archive by some means (i.e., by sneakernet). As no
direct connection between repositories exists, the user must specify a
basis for the bundle that is held by the destination repository: the
bundle assumes that all objects in the basis are already in the
destination repository.
OPTIONS
-------
create <file>::
Used to create a bundle named 'file'. This requires the
git-rev-list arguments to define the bundle contents.
verify <file>::
Used to check that a bundle file is valid and will apply
cleanly to the current repository. This includes checks on the
bundle format itself as well as checking that the prerequisite
commits exist and are fully linked in the current repository.
git-bundle prints a list of missing commits, if any, and exits
with non-zero status.
list-heads <file>::
Lists the references defined in the bundle. If followed by a
list of references, only references matching those given are
printed out.
unbundle <file>::
Passes the objects in the bundle to gitlink:git-index-pack[1]
for storage in the repository, then prints the names of all
defined references. If a reflist is given, only references
matching those in the given list are printed. This command is
really plumbing, intended to be called only by
gitlink:git-fetch[1].
[git-rev-list-args...]::
A list of arguments, accepatble to git-rev-parse and
git-rev-list, that specify the specific objects and references
to transport. For example, "master~10..master" causes the
current master reference to be packaged along with all objects
added since its 10th ancestor commit. There is no explicit
limit to the number of references and objects that may be
packaged.
[refname...]::
A list of references used to limit the references reported as
available. This is principally of use to git-fetch, which
expects to recieve only those references asked for and not
necessarily everything in the pack (in this case, git-bundle is
acting like gitlink:git-fetch-pack[1]).
SPECIFYING REFERENCES
---------------------
git-bundle will only package references that are shown by
git-show-ref: this includes heads, tags, and remote heads. References
such as master~1 cannot be packaged, but are perfectly suitable for
defining the basis. More than one reference may be packaged, and more
than one basis can be specified. The objects packaged are those not
contained in the union of the given bases. Each basis can be
specified explicitly (e.g., ^master~10), or implicitly (e.g.,
master~10..master, master --since=10.days.ago).
It is very important that the basis used be held by the destination.
It is ok to err on the side of conservatism, causing the bundle file
to contain objects already in the destination as these are ignored
when unpacking at the destination.
EXAMPLE
-------
Assume two repositories exist as R1 on machine A, and R2 on machine B.
For whatever reason, direct connection between A and B is not allowed,
but we can move data from A to B via some mechanism (CD, email, etc).
We want to update R2 with developments made on branch master in R1.
We set a tag in R1 (lastR2bundle) after the previous such transport,
and move it afterwards to help build the bundle.
in R1 on A:
$ git-bundle create mybundle master ^lastR2bundle
$ git tag -f lastR2bundle master
(move mybundle from A to B by some mechanism)
in R2 on B:
$ git-bundle verify mybundle
$ git-fetch mybundle refspec
where refspec is refInBundle:localRef
Also, with something like this in your config:
[remote "bundle"]
url = /home/me/tmp/file.bdl
fetch = refs/heads/*:refs/remotes/origin/*
You can first sneakernet the bundle file to ~/tmp/file.bdl and
then these commands:
$ git ls-remote bundle
$ git fetch bundle
$ git pull bundle
would treat it as if it is talking with a remote side over the
network.
Author
------
Written by Mark Levedahl <mdl123@verizon.net>
GIT
---
Part of the gitlink:git[7] suite

1
Makefile
View file

@ -277,6 +277,7 @@ BUILTIN_OBJS = \
builtin-archive.o \
builtin-blame.o \
builtin-branch.o \
builtin-bundle.o \
builtin-cat-file.o \
builtin-checkout-index.o \
builtin-check-ref-format.o \

424
builtin-bundle.c Normal file
View file

@ -0,0 +1,424 @@
#include "cache.h"
#include "object.h"
#include "commit.h"
#include "diff.h"
#include "revision.h"
#include "list-objects.h"
#include "exec_cmd.h"
/*
* Basic handler for bundle files to connect repositories via sneakernet.
* Invocation must include action.
* This function can create a bundle or provide information on an existing
* bundle supporting git-fetch, git-pull, and git-ls-remote
*/
static const char *bundle_usage="git-bundle (create <bundle> <git-rev-list args> | verify <bundle> | list-heads <bundle> [refname]... | unbundle <bundle> [refname]... )";
static const char bundle_signature[] = "# v2 git bundle\n";
struct ref_list {
unsigned int nr, alloc;
struct ref_list_entry {
unsigned char sha1[20];
char *name;
} *list;
};
static void add_to_ref_list(const unsigned char *sha1, const char *name,
struct ref_list *list)
{
if (list->nr + 1 >= list->alloc) {
list->alloc = alloc_nr(list->nr + 1);
list->list = xrealloc(list->list,
list->alloc * sizeof(list->list[0]));
}
memcpy(list->list[list->nr].sha1, sha1, 20);
list->list[list->nr].name = xstrdup(name);
list->nr++;
}
struct bundle_header {
struct ref_list prerequisites;
struct ref_list references;
};
/* this function returns the length of the string */
static int read_string(int fd, char *buffer, int size)
{
int i;
for (i = 0; i < size - 1; i++) {
int count = xread(fd, buffer + i, 1);
if (count < 0)
return error("Read error: %s", strerror(errno));
if (count == 0) {
i--;
break;
}
if (buffer[i] == '\n')
break;
}
buffer[i + 1] = '\0';
return i + 1;
}
/* returns an fd */
static int read_header(const char *path, struct bundle_header *header) {
char buffer[1024];
int fd = open(path, O_RDONLY);
if (fd < 0)
return error("could not open '%s'", path);
if (read_string(fd, buffer, sizeof(buffer)) < 0 ||
strcmp(buffer, bundle_signature)) {
close(fd);
return error("'%s' does not look like a v2 bundle file", path);
}
while (read_string(fd, buffer, sizeof(buffer)) > 0
&& buffer[0] != '\n') {
int is_prereq = buffer[0] == '-';
int offset = is_prereq ? 1 : 0;
int len = strlen(buffer);
unsigned char sha1[20];
struct ref_list *list = is_prereq ? &header->prerequisites
: &header->references;
char delim;
if (buffer[len - 1] == '\n')
buffer[len - 1] = '\0';
if (get_sha1_hex(buffer + offset, sha1)) {
warn("unrecognized header: %s", buffer);
continue;
}
delim = buffer[40 + offset];
if (!isspace(delim) && (delim != '\0' || !is_prereq))
die ("invalid header: %s", buffer);
add_to_ref_list(sha1, isspace(delim) ?
buffer + 41 + offset : "", list);
}
return fd;
}
/* if in && *in >= 0, take that as input file descriptor instead */
static int fork_with_pipe(const char **argv, int *in, int *out)
{
int needs_in, needs_out;
int fdin[2], fdout[2], pid;
needs_in = in && *in < 0;
if (needs_in) {
if (pipe(fdin) < 0)
return error("could not setup pipe");
*in = fdin[1];
}
needs_out = out && *out < 0;
if (needs_out) {
if (pipe(fdout) < 0)
return error("could not setup pipe");
*out = fdout[0];
}
if ((pid = fork()) < 0) {
if (needs_in) {
close(fdin[0]);
close(fdin[1]);
}
if (needs_out) {
close(fdout[0]);
close(fdout[1]);
}
return error("could not fork");
}
if (!pid) {
if (needs_in) {
dup2(fdin[0], 0);
close(fdin[0]);
close(fdin[1]);
} else if (in) {
dup2(*in, 0);
close(*in);
}
if (needs_out) {
dup2(fdout[1], 1);
close(fdout[0]);
close(fdout[1]);
} else if (out) {
dup2(*out, 1);
close(*out);
}
exit(execv_git_cmd(argv));
}
if (needs_in)
close(fdin[0]);
else if (in)
close(*in);
if (needs_out)
close(fdout[1]);
else if (out)
close(*out);
return pid;
}
static int verify_bundle(struct bundle_header *header)
{
/*
* Do fast check, then if any prereqs are missing then go line by line
* to be verbose about the errors
*/
struct ref_list *p = &header->prerequisites;
struct rev_info revs;
const char *argv[] = {NULL, "--all"};
struct object_array refs;
struct commit *commit;
int i, ret = 0, req_nr;
const char *message = "Repository lacks these prerequisite commits:";
init_revisions(&revs, NULL);
for (i = 0; i < p->nr; i++) {
struct ref_list_entry *e = p->list + i;
struct object *o = parse_object(e->sha1);
if (o) {
o->flags |= BOUNDARY_SHOW;
add_pending_object(&revs, o, e->name);
continue;
}
if (++ret == 1)
error(message);
error("%s %s", sha1_to_hex(e->sha1), e->name);
}
if (revs.pending.nr == 0)
return ret;
req_nr = revs.pending.nr;
setup_revisions(2, argv, &revs, NULL);
memset(&refs, 0, sizeof(struct object_array));
for (i = 0; i < revs.pending.nr; i++) {
struct object_array_entry *e = revs.pending.objects + i;
add_object_array(e->item, e->name, &refs);
}
prepare_revision_walk(&revs);
i = req_nr;
while (i && (commit = get_revision(&revs)))
if (commit->object.flags & BOUNDARY_SHOW)
i--;
for (i = 0; i < req_nr; i++)
if (!(refs.objects[i].item->flags & SHOWN)) {
if (++ret == 1)
error(message);
error("%s %s", sha1_to_hex(refs.objects[i].item->sha1),
refs.objects[i].name);
}
for (i = 0; i < refs.nr; i++)
clear_commit_marks((struct commit *)refs.objects[i].item, -1);
return ret;
}
static int list_heads(struct bundle_header *header, int argc, const char **argv)
{
int i;
struct ref_list *r = &header->references;
for (i = 0; i < r->nr; i++) {
if (argc > 1) {
int j;
for (j = 1; j < argc; j++)
if (!strcmp(r->list[i].name, argv[j]))
break;
if (j == argc)
continue;
}
printf("%s %s\n", sha1_to_hex(r->list[i].sha1),
r->list[i].name);
}
return 0;
}
static void show_commit(struct commit *commit)
{
write_or_die(1, sha1_to_hex(commit->object.sha1), 40);
write_or_die(1, "\n", 1);
if (commit->parents) {
free_commit_list(commit->parents);
commit->parents = NULL;
}
}
static void show_object(struct object_array_entry *p)
{
/* An object with name "foo\n0000000..." can be used to
* confuse downstream git-pack-objects very badly.
*/
const char *ep = strchr(p->name, '\n');
int len = ep ? ep - p->name : strlen(p->name);
write_or_die(1, sha1_to_hex(p->item->sha1), 40);
write_or_die(1, " ", 1);
if (len)
write_or_die(1, p->name, len);
write_or_die(1, "\n", 1);
}
static int create_bundle(struct bundle_header *header, const char *path,
int argc, const char **argv)
{
int bundle_fd = -1;
const char **argv_boundary = xmalloc((argc + 4) * sizeof(const char *));
const char **argv_pack = xmalloc(4 * sizeof(const char *));
int pid, in, out, i, status;
char buffer[1024];
struct rev_info revs;
bundle_fd = (!strcmp(path, "-") ? 1 :
open(path, O_CREAT | O_WRONLY, 0666));
if (bundle_fd < 0)
return error("Could not write to '%s'", path);
/* write signature */
write_or_die(bundle_fd, bundle_signature, strlen(bundle_signature));
/* write prerequisites */
memcpy(argv_boundary + 3, argv + 1, argc * sizeof(const char *));
argv_boundary[0] = "rev-list";
argv_boundary[1] = "--boundary";
argv_boundary[2] = "--pretty=oneline";
argv_boundary[argc + 2] = NULL;
out = -1;
pid = fork_with_pipe(argv_boundary, NULL, &out);
if (pid < 0)
return -1;
while ((i = read_string(out, buffer, sizeof(buffer))) > 0)
if (buffer[0] == '-')
write_or_die(bundle_fd, buffer, i);
while ((i = waitpid(pid, &status, 0)) < 0)
if (errno != EINTR)
return error("rev-list died");
if (!WIFEXITED(status) || WEXITSTATUS(status))
return error("rev-list died %d", WEXITSTATUS(status));
/* write references */
save_commit_buffer = 0;
init_revisions(&revs, NULL);
revs.tag_objects = 1;
revs.tree_objects = 1;
revs.blob_objects = 1;
argc = setup_revisions(argc, argv, &revs, NULL);
if (argc > 1)
return error("unrecognized argument: %s'", argv[1]);
for (i = 0; i < revs.pending.nr; i++) {
struct object_array_entry *e = revs.pending.objects + i;
if (!(e->item->flags & UNINTERESTING)) {
unsigned char sha1[20];
char *ref;
if (dwim_ref(e->name, strlen(e->name), sha1, &ref) != 1)
continue;
write_or_die(bundle_fd, sha1_to_hex(e->item->sha1), 40);
write_or_die(bundle_fd, " ", 1);
write_or_die(bundle_fd, ref, strlen(ref));
write_or_die(bundle_fd, "\n", 1);
free(ref);
}
}
/* end header */
write_or_die(bundle_fd, "\n", 1);
/* write pack */
argv_pack[0] = "pack-objects";
argv_pack[1] = "--all-progress";
argv_pack[2] = "--stdout";
argv_pack[3] = NULL;
in = -1;
out = bundle_fd;
pid = fork_with_pipe(argv_pack, &in, &out);
if (pid < 0)
return error("Could not spawn pack-objects");
close(1);
dup2(in, 1);
close(in);
prepare_revision_walk(&revs);
traverse_commit_list(&revs, show_commit, show_object);
close(1);
while (waitpid(pid, &status, 0) < 0)
if (errno != EINTR)
return -1;
if (!WIFEXITED(status) || WEXITSTATUS(status))
return error ("pack-objects died");
return 0;
}
static int unbundle(struct bundle_header *header, int bundle_fd,
int argc, const char **argv)
{
const char *argv_index_pack[] = {"index-pack", "--stdin", NULL};
int pid, status, dev_null;
if (verify_bundle(header))
return -1;
dev_null = open("/dev/null", O_WRONLY);
pid = fork_with_pipe(argv_index_pack, &bundle_fd, &dev_null);
if (pid < 0)
return error("Could not spawn index-pack");
while (waitpid(pid, &status, 0) < 0)
if (errno != EINTR)
return error("index-pack died");
if (!WIFEXITED(status) || WEXITSTATUS(status))
return error("index-pack exited with status %d",
WEXITSTATUS(status));
return list_heads(header, argc, argv);
}
int cmd_bundle(int argc, const char **argv, const char *prefix)
{
struct bundle_header header;
int nongit = 0;
const char *cmd, *bundle_file;
int bundle_fd = -1;
char buffer[PATH_MAX];
if (argc < 3)
usage(bundle_usage);
cmd = argv[1];
bundle_file = argv[2];
argc -= 2;
argv += 2;
prefix = setup_git_directory_gently(&nongit);
if (prefix && bundle_file[0] != '/') {
snprintf(buffer, sizeof(buffer), "%s/%s", prefix, bundle_file);
bundle_file = buffer;
}
memset(&header, 0, sizeof(header));
if (strcmp(cmd, "create") &&
!(bundle_fd = read_header(bundle_file, &header)))
return 1;
if (!strcmp(cmd, "verify")) {
close(bundle_fd);
if (verify_bundle(&header))
return 1;
fprintf(stderr, "%s is okay\n", bundle_file);
return 0;
}
if (!strcmp(cmd, "list-heads")) {
close(bundle_fd);
return !!list_heads(&header, argc, argv);
}
if (!strcmp(cmd, "create")) {
if (nongit)
die("Need a repository to create a bundle.");
return !!create_bundle(&header, bundle_file, argc, argv);
} else if (!strcmp(cmd, "unbundle")) {
if (nongit)
die("Need a repository to unbundle.");
return !!unbundle(&header, bundle_fd, argc, argv);
} else
usage(bundle_usage);
}

1
builtin.h
View file

@ -19,6 +19,7 @@ extern int cmd_apply(int argc, const char **argv, const char *prefix);
extern int cmd_archive(int argc, const char **argv, const char *prefix);
extern int cmd_blame(int argc, const char **argv, const char *prefix);
extern int cmd_branch(int argc, const char **argv, const char *prefix);
extern int cmd_bundle(int argc, const char **argv, const char *prefix);
extern int cmd_cat_file(int argc, const char **argv, const char *prefix);
extern int cmd_checkout_index(int argc, const char **argv, const char *prefix);
extern int cmd_check_ref_format(int argc, const char **argv, const char *prefix);

7
git-fetch.sh
View file

@ -386,8 +386,15 @@ fetch_main () {
( : subshell because we muck with IFS
IFS=" $LF"
(
if test -f "$remote" ; then
test -n "$shallow_depth" &&
die "shallow clone with bundle is not supported"
git-bundle unbundle "$remote" $rref ||
echo failed "$remote"
else
git-fetch-pack --thin $exec $keep $shallow_depth "$remote" $rref ||
echo failed "$remote"
fi
) |
(
trap '

7
git-ls-remote.sh
View file

@ -89,8 +89,13 @@ rsync://* )
;;
* )
git-peek-remote $exec "$peek_repo" ||
if test -f "$peek_repo" ; then
git bundle list-heads "$peek_repo" ||
echo "failed slurping"
else
git-peek-remote $exec "$peek_repo" ||
echo "failed slurping"
fi
;;
esac |
sort -t ' ' -k 2 |

1
git.c
View file

@ -229,6 +229,7 @@ static void handle_internal_command(int argc, const char **argv, char **envp)
{ "archive", cmd_archive },
{ "blame", cmd_blame, RUN_SETUP },
{ "branch", cmd_branch, RUN_SETUP },
{ "bundle", cmd_bundle },
{ "cat-file", cmd_cat_file, RUN_SETUP },
{ "checkout-index", cmd_checkout_index, RUN_SETUP },
{ "check-ref-format", cmd_check_ref_format },

3
index-pack.c
View file

@ -463,7 +463,8 @@ static void parse_pack_objects(unsigned char *sha1)
/* If input_fd is a file, we should have reached its end now. */
if (fstat(input_fd, &st))
die("cannot fstat packfile: %s", strerror(errno));
if (S_ISREG(st.st_mode) && st.st_size != consumed_bytes)
if (S_ISREG(st.st_mode) &&
lseek(input_fd, 0, SEEK_CUR) - input_len != st.st_size)
die("pack has junk at the end");
if (!nr_deltas)

2
revision.c
View file

@ -482,7 +482,7 @@ static int handle_one_ref(const char *path, const unsigned char *sha1, int flag,
struct all_refs_cb *cb = cb_data;
struct object *object = get_reference(cb->all_revs, path, sha1,
cb->all_flags);
add_pending_object(cb->all_revs, object, "");
add_pending_object(cb->all_revs, object, path);
return 0;
}

28
t/t5510-fetch.sh
View file

@ -35,7 +35,9 @@ test_expect_success "clone and setup child repos" '
echo "URL: ../two/.git/"
echo "Pull: refs/heads/master:refs/heads/two"
echo "Pull: refs/heads/one:refs/heads/one"
} >.git/remotes/two
} >.git/remotes/two &&
cd .. &&
git clone . bundle
'
test_expect_success "fetch test" '
@ -81,4 +83,28 @@ test_expect_success 'fetch following tags' '
'
test_expect_success 'create bundle 1' '
cd "$D" &&
echo >file updated again by origin &&
git commit -a -m "tip" &&
git bundle create bundle1 master^..master
'
test_expect_success 'create bundle 2' '
cd "$D" &&
git bundle create bundle2 master~2..master
'
test_expect_failure 'unbundle 1' '
cd "$D/bundle" &&
git checkout -b some-branch &&
git fetch "$D/bundle1" master:master
'
test_expect_success 'unbundle 2' '
cd "$D/bundle" &&
git fetch ../bundle2 master:master &&
test "tip" = "$(git log -1 --pretty=oneline master | cut -b42-)"
'
test_done