git/fetch-pack.h

110 lines
3.2 KiB
C
Raw Permalink Normal View History

#ifndef FETCH_PACK_H
#define FETCH_PACK_H
#include "string-list.h"
#include "protocol.h"
#include "list-objects-filter-options.h"
fetch: teach independent negotiation (no packfile) Currently, the packfile negotiation step within a Git fetch cannot be done independent of sending the packfile, even though there is at least one application wherein this is useful. Therefore, make it possible for this negotiation step to be done independently. A subsequent commit will use this for one such application - push negotiation. This feature is for protocol v2 only. (An implementation for protocol v0 would require a separate implementation in the fetch, transport, and transport helper code.) In the protocol, the main hindrance towards independent negotiation is that the server can unilaterally decide to send the packfile. This is solved by a "wait-for-done" argument: the server will then wait for the client to say "done". In practice, the client will never say it; instead it will cease requests once it is satisfied. In the client, the main change lies in the transport and transport helper code. fetch_refs_via_pack() performs everything needed - protocol version and capability checks, and the negotiation itself. There are 2 code paths that do not go through fetch_refs_via_pack() that needed to be individually excluded: the bundle transport (excluded through requiring smart_options, which the bundle transport doesn't support) and transport helpers that do not support takeover. If or when we support independent negotiation for protocol v0, we will need to modify these 2 code paths to support it. But for now, report failure if independent negotiation is requested in these cases. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04 21:16:01 +00:00
#include "oidset.h"
struct oid_array;
struct fetch_pack_args {
const char *uploadpack;
int unpacklimit;
int depth;
const char *deepen_since;
const struct string_list *deepen_not;
struct list_objects_filter_options filter_options;
const struct string_list *server_options;
/*
* If not NULL, during packfile negotiation, fetch-pack will send "have"
* lines only with these tips and their ancestors.
*/
const struct oid_array *negotiation_tips;
fetch, upload-pack: --deepen=N extends shallow boundary by N commits In git-fetch, --depth argument is always relative with the latest remote refs. This makes it a bit difficult to cover this use case, where the user wants to make the shallow history, say 3 levels deeper. It would work if remote refs have not moved yet, but nobody can guarantee that, especially when that use case is performed a couple months after the last clone or "git fetch --depth". Also, modifying shallow boundary using --depth does not work well with clones created by --since or --not. This patch fixes that. A new argument --deepen=<N> will add <N> more (*) parent commits to the current history regardless of where remote refs are. Have/Want negotiation is still respected. So if remote refs move, the server will send two chunks: one between "have" and "want" and another to extend shallow history. In theory, the client could send no "want"s in order to get the second chunk only. But the protocol does not allow that. Either you send no want lines, which means ls-remote; or you have to send at least one want line that carries deep-relative to the server.. The main work was done by Dongcan Jiang. I fixed it up here and there. And of course all the bugs belong to me. (*) We could even support --deepen=<N> where <N> is negative. In that case we can cut some history from the shallow clone. This operation (and --depth=<shorter depth>) does not require interaction with remote side (and more complicated to implement as a result). Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Eric Sunshine <sunshine@sunshineco.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Dongcan Jiang <dongcan.jiang@gmail.com> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2016-06-12 10:54:09 +00:00
unsigned deepen_relative:1;
unsigned quiet:1;
unsigned keep_pack:1;
unsigned lock_pack:1;
unsigned use_thin_pack:1;
unsigned fetch_all:1;
unsigned stdin_refs:1;
unsigned diag_url:1;
unsigned verbose:1;
unsigned no_progress:1;
unsigned include_tag:1;
unsigned stateless_rpc:1;
unsigned check_self_contained_and_connected:1;
unsigned self_contained_and_connected:1;
unsigned cloning:1;
unsigned update_shallow:1;
unsigned reject_shallow_remote:1;
unsigned deepen:1;
unsigned refetch:1;
/*
* Indicate that the remote of this request is a promisor remote. The
* pack received does not need all referred-to objects to be present in
* the local object store, and fetch-pack will store the pack received
* together with a ".promisor" file indicating that the aforementioned
* pack is a promisor pack.
*/
unsigned from_promisor:1;
fetch-pack: write shallow, then check connectivity When fetching, connectivity is checked after the shallow file is updated. There are 2 issues with this: (1) the connectivity check is only performed up to ancestors of existing refs (which is not thorough enough if we were deepening an existing ref in the first place), and (2) there is no rollback of the shallow file if the connectivity check fails. To solve (1), update the connectivity check to check the ancestry chain completely in the case of a deepening fetch by refraining from passing "--not --all" when invoking rev-list in connected.c. To solve (2), have fetch_pack() perform its own connectivity check before updating the shallow file. To support existing use cases in which "git fetch-pack" is used to download objects without much regard as to the connectivity of the resulting objects with respect to the existing repository, the connectivity check is only done if necessary (that is, the fetch is not a clone, and the fetch involves shallow/deepen functionality). "git fetch" still performs its own connectivity check, preserving correctness but sometimes performing redundant work. This redundancy is mitigated by the fact that fetch_pack() reports if it has performed a connectivity check itself, and if the transport supports connect or stateless-connect, it will bubble up that report so that "git fetch" knows not to perform the connectivity check in such a case. This was noticed when a user tried to deepen an existing repository by fetching with --no-shallow from a server that did not send all necessary objects - the connectivity check as run by "git fetch" succeeded, but a subsequent "git fsck" failed. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2018-07-02 22:08:43 +00:00
/*
* Because fetch_pack() overwrites the shallow file upon a
* successful deepening non-clone fetch, if this struct
* specifies such a fetch, fetch_pack() needs to perform a
* connectivity check before deciding if a fetch is successful
* (and overwriting the shallow file). fetch_pack() sets this
* field to 1 if such a connectivity check was performed.
*
* This is different from check_self_contained_and_connected
* in that the former allows existing objects in the
* repository to satisfy connectivity needs, whereas the
* latter doesn't.
*/
unsigned connectivity_checked:1;
};
/*
* sought represents remote references that should be updated from.
* On return, the names that were found on the remote will have been
* marked as such.
*/
struct ref *fetch_pack(struct fetch_pack_args *args,
int fd[],
const struct ref *ref,
struct ref **sought,
int nr_sought,
struct oid_array *shallow,
struct string_list *pack_lockfiles,
enum protocol_version version);
fetch: teach independent negotiation (no packfile) Currently, the packfile negotiation step within a Git fetch cannot be done independent of sending the packfile, even though there is at least one application wherein this is useful. Therefore, make it possible for this negotiation step to be done independently. A subsequent commit will use this for one such application - push negotiation. This feature is for protocol v2 only. (An implementation for protocol v0 would require a separate implementation in the fetch, transport, and transport helper code.) In the protocol, the main hindrance towards independent negotiation is that the server can unilaterally decide to send the packfile. This is solved by a "wait-for-done" argument: the server will then wait for the client to say "done". In practice, the client will never say it; instead it will cease requests once it is satisfied. In the client, the main change lies in the transport and transport helper code. fetch_refs_via_pack() performs everything needed - protocol version and capability checks, and the negotiation itself. There are 2 code paths that do not go through fetch_refs_via_pack() that needed to be individually excluded: the bundle transport (excluded through requiring smart_options, which the bundle transport doesn't support) and transport helpers that do not support takeover. If or when we support independent negotiation for protocol v0, we will need to modify these 2 code paths to support it. But for now, report failure if independent negotiation is requested in these cases. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04 21:16:01 +00:00
/*
* Execute the --negotiate-only mode of "git fetch", adding all known common
* commits to acked_commits.
*
* In the capability advertisement that has happened prior to invoking this
* function, the "wait-for-done" capability must be present.
*/
void negotiate_using_fetch(const struct oid_array *negotiation_tips,
const struct string_list *server_options,
int stateless_rpc,
int fd[],
struct oidset *acked_commits);
/*
* Print an appropriate error message for each sought ref that wasn't
* matched. Return 0 if all sought refs were matched, otherwise 1.
*/
int report_unmatched_refs(struct ref **sought, int nr_sought);
/*
* Return true if checks for broken objects in received pack are required.
*/
int fetch_pack_fsck_objects(void);
#endif