development/git
development/git
1
0
Fork 0
mirror of https://github.com/git/git synced 2024-09-13 05:14:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
git/builtin
History
Jeff King bb514de356 pack-objects: improve partial packfile reuse 
The old code to reuse deltas from an existing packfile
just tried to dump a whole segment of the pack verbatim.
That's faster than the traditional way of actually adding
objects to the packing list, but it didn't kick in very
often. This new code is really going for a middle ground:
do _some_ per-object work, but way less than we'd
traditionally do.

The general strategy of the new code is to make a bitmap
of objects from the packfile we'll include, and then
iterate over it, writing out each object exactly as it is
in our on-disk pack, but _not_ adding it to our packlist
(which costs memory, and increases the search space for
deltas).

One complication is that if we're omitting some objects,
we can't set a delta against a base that we're not
sending. So we have to check each object in
try_partial_reuse() to make sure we have its delta.

About performance, in the worst case we might have
interleaved objects that we are sending or not sending,
and we'd have as many chunks as objects. But in practice
we send big chunks.

For instance, packing torvalds/linux on GitHub servers
now reused 6.5M objects, but only needed ~50k chunks.

Helped-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2020-01-23 10:51:50 -08:00
..
add.c add: error appropriately on repository with no commits 2019-04-10 12:52:50 +09:00
am.c Merge branch 'nd/switch-and-restore' 2019-07-09 15:25:44 -07:00
annotate.c
apply.c apply.c: make init_apply_state() take a struct repository 2018-08-13 14:14:44 -07:00
archive.c pack-protocol.txt: accept error packets in any context 2019-01-02 13:05:30 -08:00
bisect--helper.c bisect--helper: verify HEAD could be parsed before continuing 2019-05-28 10:51:01 -07:00
blame.c Merge branch 'br/blame-ignore' 2019-07-19 11:30:20 -07:00
branch.c Merge branch 'po/doc-branch' 2019-07-09 15:25:34 -07:00
bundle.c create_bundle(): drop unused "header" parameter 2019-01-24 12:35:44 -08:00
cat-file.c sha1-file.c: remove the_repo from read_object_with_reference() 2019-06-27 12:45:17 -07:00
check-attr.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
check-ignore.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
check-mailmap.c
check-ref-format.c
checkout-index.c Merge branch 'nd/the-index-final' 2019-02-06 22:05:23 -08:00
checkout.c l10n: reformat some localized strings for v2.23.0 2019-08-06 12:43:43 -07:00
clean.c clean: show an error message when the path is too long 2019-07-19 08:12:44 -07:00
clone.c Merge branch 'mt/dir-iterator-updates' 2019-07-25 13:59:22 -07:00
column.c builtin: consistently pass cmd_* prefix to parse_options 2019-05-13 14:22:53 +09:00
commit-graph.c commit-graph: verify chains with --shallow mode 2019-06-19 20:46:26 -07:00
commit-tree.c commit-tree: utilize parse-options api 2019-03-08 10:31:24 +09:00
commit.c Merge branch 'ra/cherry-pick-revert-skip' 2019-07-19 11:30:21 -07:00
config.c Merge branch 'jk/save-getenv-result' 2019-01-29 12:47:54 -08:00
count-objects.c rename "alternate_object_database" to "object_directory" 2018-11-13 14:22:02 +09:00
credential.c
describe.c hashmap: convert sha1hash() to oidhash() 2019-06-20 10:44:22 -07:00
diff-files.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
diff-index.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
diff-tree.c Merge branch 'en/combined-all-paths' 2019-03-07 09:59:54 +09:00
diff.c Merge branch 'nd/diff-parseopt-4' 2019-04-25 16:41:12 +09:00
difftool.c Merge branch 'js/difftool-no-index' 2019-05-19 16:45:35 +09:00
env--helper.c env--helper: mark a file-local symbol as static 2019-07-11 14:31:04 -07:00
fast-export.c object: convert lookup_object() to use object_id 2019-06-20 10:18:09 -07:00
fetch-pack.c fetch_pack(): drop unused parameters 2019-03-20 18:34:09 +09:00
fetch.c l10n: reformat some localized strings for v2.23.0 2019-08-06 12:43:43 -07:00
fmt-merge-msg.c Merge branch 'nd/the-index' 2018-10-19 13:34:02 +09:00
for-each-ref.c parse_opt_ref_sorting: always use with NONEG flag 2019-03-21 12:03:35 +09:00
fsck.c object: convert lookup_object() to use object_id 2019-06-20 10:18:09 -07:00
gc.c Merge branch 'ds/commit-graph-incremental' 2019-07-19 11:30:20 -07:00
get-tar-commit-id.c builtin/get-tar-commit-id: make hash size independent 2019-04-01 11:57:39 +09:00
grep.c sha1-file.c: remove the_repo from read_object_with_reference() 2019-06-27 12:45:17 -07:00
hash-object.c builtin: consistently pass cmd_* prefix to parse_options 2019-05-13 14:22:53 +09:00
help.c completion: add more parameter value completion 2019-02-20 12:31:56 -08:00
index-pack.c index-pack: prefetch missing REF_DELTA bases 2019-05-15 11:01:40 +09:00
init-db.c Merge branch 'nd/init-relative-template-fix' 2019-06-13 13:18:46 -07:00
interpret-trailers.c interpret-trailers: load default config 2019-06-19 07:12:49 -07:00
log.c log: really flip the --mailmap default 2019-08-02 09:55:03 -07:00
ls-files.c cleanup: fix possible overflow errors in binary search, part 2 2019-06-13 11:28:53 -07:00
ls-remote.c parse_opt_ref_sorting: always use with NONEG flag 2019-03-21 12:03:35 +09:00
ls-tree.c Merge branch 'nd/attr-pathspec-in-tree-walk' 2019-01-14 15:29:28 -08:00
mailinfo.c
mailsplit.c
merge-base.c Merge branch 'pk/rebase-in-c-4-opts' 2018-11-02 11:04:55 +09:00
merge-file.c assert NOARG/NONEG behavior of parse-options callbacks 2018-11-06 12:56:29 +09:00
merge-index.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
merge-ours.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
merge-recursive.c Merge branch 'nd/the-index-final' 2019-02-06 22:05:23 -08:00
merge-tree.c tree-walk.c: remove the_repo from fill_tree_descriptor() 2019-06-27 12:45:17 -07:00
merge.c Merge branch 'ea/merge-code-cleanup' 2019-07-19 11:30:23 -07:00
mktag.c object-store: move object access functions to object-store.h 2018-05-16 11:42:03 +09:00
mktree.c mktree: drop unused length parameter 2019-05-13 14:22:54 +09:00
multi-pack-index.c multi-pack-index: prepare 'repack' subcommand 2019-06-11 10:34:40 -07:00
mv.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
name-rev.c object: convert lookup_object() to use object_id 2019-06-20 10:18:09 -07:00
notes.c Merge branch 'nd/the-index' into md/list-objects-filter-by-depth 2019-01-15 15:38:29 -08:00
pack-objects.c pack-objects: improve partial packfile reuse 2020-01-23 10:51:50 -08:00
pack-redundant.c object-store: rename and expand packed_git's sha1 member 2019-04-01 11:57:38 +09:00
pack-refs.c Honor core.precomposeUnicode in more places 2019-04-26 10:54:03 +09:00
patch-id.c format-patch: make --base patch-id output stable 2019-05-08 19:27:43 +09:00
prune-packed.c Merge branch 'rj/prune-packed-excess-args' 2019-03-07 09:59:55 +09:00
prune.c object: convert lookup_object() to use object_id 2019-06-20 10:18:09 -07:00
pull.c pull: add --[no-]show-forced-updates passthrough 2019-06-21 09:38:29 -07:00
push.c Merge branch 'nd/style-opening-brace' 2019-01-18 13:49:52 -08:00
range-diff.c builtin: consistently pass cmd_* prefix to parse_options 2019-05-13 14:22:53 +09:00
read-tree.c cmd_{read,write}_tree: rename "unused" variable that is used 2019-05-13 14:22:53 +09:00
rebase.c Merge branch 'js/rebase-cleanup' 2019-07-29 12:39:14 -07:00
receive-pack.c Merge branch 'jk/check-connected-with-alternates' 2019-07-19 11:30:21 -07:00
reflog.c Merge branch 'jk/loose-object-cache-oid' 2019-02-06 22:05:27 -08:00
remote-ext.c
remote-fd.c
remote.c remove_all_fetch_refspecs(): drop unused "remote" parameter 2019-05-13 14:22:54 +09:00
repack.c repack: simplify handling of auto-bitmaps and .keep files 2019-07-31 13:26:25 -07:00
replace.c Merge branch 'cc/replace-graft-peel-tags' 2019-05-09 00:37:24 +09:00
rerere.c Merge branch 'nd/the-index' into md/list-objects-filter-by-depth 2019-01-15 15:38:29 -08:00
reset.c Merge branch 'nd/tree-walk-with-repo' 2019-07-19 11:30:21 -07:00
rev-list.c rev-list: teach --no-object-names to enable piping 2019-06-20 13:13:04 -07:00
rev-parse.c cache.h: flip NO_THE_INDEX_COMPATIBILITY_MACROS switch 2019-01-24 11:55:06 -08:00
revert.c Merge branch 'ra/cherry-pick-revert-skip' 2019-07-19 11:30:21 -07:00
rm.c Merge branch 'jc/denoise-rm-to-resolve' 2019-07-25 13:59:24 -07:00
send-pack.c pack-protocol.txt: accept error packets in any context 2019-01-02 13:05:30 -08:00
shortlog.c Merge branch 'nd/show-gitcomp-compilation-fix' into maint 2018-12-15 12:24:33 +09:00
show-branch.c show-branch: drop unused parameter from show_independent() 2019-05-13 14:22:54 +09:00
show-index.c make show-index a builtin 2018-05-29 00:28:22 +09:00
show-ref.c Merge branch 'en/unicode-in-refnames' 2019-05-19 16:45:30 +09:00
stash.c Merge branch 'tg/stash-keep-index-with-removed-paths' 2019-07-25 13:59:23 -07:00
stripspace.c stripspace: allow -s/-c outside git repository 2018-12-26 15:41:47 -08:00
submodule--helper.c Merge branch 'ms/submodule-foreach-fix' 2019-07-09 15:25:46 -07:00
symbolic-ref.c
tag.c tag: add tag.gpgSign config option to force all tags be GPG-signed 2019-06-05 14:39:28 -07:00
unpack-file.c object-store: move object access functions to object-store.h 2018-05-16 11:42:03 +09:00
unpack-objects.c object: convert lookup_object() to use object_id 2019-06-20 10:18:09 -07:00
update-index.c tree-walk.c: remove the_repo from get_tree_entry() 2019-06-27 12:45:17 -07:00
update-ref.c update-ref: allow --no-deref with --stdin 2018-09-12 15:17:17 -07:00
update-server-info.c
upload-archive.c archive: initialize archivers earlier 2018-10-26 10:17:59 +09:00
upload-pack.c builtin: consistently pass cmd_* prefix to parse_options 2019-05-13 14:22:53 +09:00
var.c
verify-commit.c Merge branch 'jk/no-system-includes-in-dot-c' 2019-07-31 14:38:56 -07:00
verify-pack.c
verify-tag.c verify-tag: drop signal.h include 2019-06-19 08:19:21 -07:00
worktree.c Merge branch 'nd/worktree-name-sanitization' 2019-06-13 13:19:40 -07:00
write-tree.c cmd_{read,write}_tree: rename "unused" variable that is used 2019-05-13 14:22:53 +09:00