git/builtin
Junio C Hamano ef49a7a012 zlib: zlib can only process 4GB at a time
The size of objects we read from the repository and data we try to put
into the repository are represented in "unsigned long", so that on larger
architectures we can handle objects that weigh more than 4GB.

But the interface defined in zlib.h to communicate with inflate/deflate
limits avail_in (how many bytes of input are we calling zlib with) and
avail_out (how many bytes of output from zlib are we ready to accept)
fields effectively to 4GB by defining their type to be uInt.

In many places in our code, we allocate a large buffer (e.g. mmap'ing a
large loose object file) and tell zlib its size by assigning the size to
avail_in field of the stream, but that will truncate the high octets of
the real size. The worst part of this story is that we often pass around
z_stream (the state object used by zlib) to keep track of the number of
used bytes in input/output buffer by inspecting these two fields, which
practically limits our callchain to the same 4GB limit.

Wrap z_stream in another structure git_zstream that can express avail_in
and avail_out in unsigned long. For now, just die() when the caller gives
a size that cannot be given to a single zlib call. In later patches in the
series, we would make git_inflate() and git_deflate() internally loop to
give callers an illusion that our "improved" version of zlib interface can
operate on a buffer larger than 4GB in one go.

Signed-off-by: Junio C Hamano <gitster@pobox.com>
2011-06-10 11:52:15 -07:00
..
add.c Merge branch 'jc/fix-add-u-unmerged' into maint 2011-05-16 16:37:33 -07:00
annotate.c
apply.c zlib: zlib can only process 4GB at a time 2011-06-10 11:52:15 -07:00
archive.c i18n: git-archive basic messages 2011-03-09 23:52:57 -08:00
bisect--helper.c
blame.c blame: tolerate bogus e-mail addresses a bit better 2011-04-29 11:09:42 -07:00
branch.c Merge branch 'ab/i18n-st' 2011-04-01 17:55:55 -07:00
bundle.c i18n: git-bundle basic messages 2011-03-09 23:52:57 -08:00
cat-file.c blame,cat-file --textconv: Don't assume mode is ``S_IFREF | 0664'' 2010-09-29 16:53:25 -07:00
check-attr.c
check-ref-format.c Allow "check-ref-format --branch" from subdirectory 2010-08-06 10:01:45 -07:00
checkout-index.c Merge branch 'jn/git-cmd-h-bypass-setup' 2010-12-12 21:49:50 -08:00
checkout.c i18n: mark checkout plural warning for translation 2011-04-12 00:20:28 -07:00
clean.c i18n: git-clean clean.requireForce messages 2011-03-09 23:52:57 -08:00
clone.c Merge branch 'ab/i18n-fixup' into maint 2011-05-31 12:00:27 -07:00
commit-tree.c Sync with 1.7.3.2 2010-10-21 17:16:10 -07:00
commit.c Merge branch 'jc/index-update-if-able' into maint 2011-04-03 12:33:05 -07:00
config.c Merge branch 'lp/config-vername-check' into maint 2011-04-03 12:29:45 -07:00
count-objects.c add description parameter to OPT__VERBOSE 2010-11-15 09:56:51 -08:00
describe.c Merge branch 'lt/default-abbrev' into maint 2011-04-03 12:32:51 -07:00
diff-files.c Convert struct diff_options to use struct pathspec 2011-02-03 12:28:15 -08:00
diff-index.c Submodules: Use "ignore" settings from .gitmodules too for diff and status 2010-08-09 09:11:44 -07:00
diff-tree.c diffcore-rename: fall back to -C when -C -C busts the rename limit 2011-03-22 14:29:07 -07:00
diff.c Merge branch 'jc/rename-degrade-cc-to-c' into maint 2011-05-31 12:00:02 -07:00
fast-export.c Merge branch 'mg/placeholders-are-lowercase' 2011-02-27 21:58:30 -08:00
fetch-pack.c Merge branch 'jc/fetch-progressive-stride' 2011-03-29 14:09:08 -07:00
fetch.c Merge branch 'jl/submodule-fetch-on-demand' 2011-04-04 15:02:01 -07:00
fmt-merge-msg.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
for-each-ref.c for-each-ref: fix objectname:short bug 2010-08-26 16:42:25 -07:00
fsck.c Remove unused variables 2011-03-22 11:43:27 -07:00
gc.c i18n: git-gc "Auto packing the repository" message 2011-03-09 23:52:57 -08:00
grep.c sparse: Fix errors and silence warnings 2011-04-03 10:14:53 -07:00
hash-object.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
help.c Cast execl*() NULL sentinels to (char *) 2010-07-25 23:14:18 -07:00
index-pack.c zlib: zlib can only process 4GB at a time 2011-06-10 11:52:15 -07:00
init-db.c Merge branch 'ab/i18n-fixup' into maint 2011-05-31 12:00:27 -07:00
log.c Merge branch 'jc/rename-degrade-cc-to-c' into maint 2011-05-31 12:00:02 -07:00
ls-files.c Merge branch 'jn/git-cmd-h-bypass-setup' 2010-12-12 21:49:50 -08:00
ls-remote.c get_remote_url(): use the same data source as ls-remote to get remote urls 2011-03-02 12:26:53 -08:00
ls-tree.c Use angles for placeholders consistently 2010-10-08 12:29:52 -07:00
mailinfo.c commit: helper methods to reduce redundant blocks of code 2010-11-04 13:53:34 -07:00
mailsplit.c Use parentheses and `...' where appropriate 2010-10-08 12:31:07 -07:00
merge-base.c Documentation: update to git-merge-base --octopus 2011-04-15 10:13:52 -07:00
merge-file.c Merge branch 'rs/opt-help-text' 2010-11-24 15:55:19 -08:00
merge-index.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
merge-ours.c
merge-recursive.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
merge-tree.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
merge.c Merge branch 'ab/i18n-fixup' into maint 2011-05-31 12:00:27 -07:00
mktag.c Merge branch 'jn/maint-c99-format' 2011-03-23 14:55:46 -07:00
mktree.c
mv.c i18n: git-mv "bad" messages 2011-03-09 23:52:56 -08:00
name-rev.c object.h: Add OBJECT_ARRAY_INIT macro and make use of it. 2010-08-29 22:42:49 -07:00
notes.c Merge branch 'ab/i18n-st' 2011-04-01 17:55:55 -07:00
pack-objects.c zlib: zlib can only process 4GB at a time 2011-06-10 11:52:15 -07:00
pack-redundant.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
pack-refs.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
patch-id.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
prune-packed.c
prune.c add description parameter to OPT__DRY_RUN 2010-11-15 09:57:37 -08:00
push.c Merge branch 'ab/i18n-st' 2011-04-01 17:55:55 -07:00
read-tree.c Make <identifier> lowercase as per CodingGuidelines 2011-02-15 11:53:10 -08:00
receive-pack.c sparse: Fix errors and silence warnings 2011-04-03 10:14:53 -07:00
reflog.c Merge branch 'jc/maint-no-reflog-expire-unreach-for-head' 2010-05-21 04:02:18 -07:00
remote-ext.c Remove unused variables 2011-03-22 11:43:27 -07:00
remote-fd.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
remote.c Merge branch 'jk/maint-remote-mirror-safer' 2011-04-06 10:38:14 -07:00
replace.c
rerere.c rerere forget: deprecate invocation without pathspec 2011-03-02 12:26:18 -08:00
reset.c Merge branch 'ab/i18n-st' 2011-04-01 17:55:55 -07:00
rev-list.c revision.c: introduce --min-parents and --max-parents options 2011-03-23 10:16:44 -07:00
rev-parse.c revision.c: introduce --min-parents and --max-parents options 2011-03-23 10:16:44 -07:00
revert.c revert: allow reverting a root commit 2011-05-16 13:01:45 -07:00
rm.c i18n: git-rm basic messages 2011-03-09 23:52:56 -08:00
send-pack.c Merge branch 'jk/git-connection-deadlock-fix' into maint 2011-05-26 09:33:25 -07:00
shortlog.c i18n: git-shortlog basic messages 2011-03-09 23:52:58 -08:00
show-branch.c commit: Add commit_list prefix in two function names. 2010-11-29 14:01:52 -08:00
show-ref.c Describe various forms of "be quiet" using OPT__QUIET 2010-11-15 10:04:56 -08:00
stripspace.c
symbolic-ref.c Describe various forms of "be quiet" using OPT__QUIET 2010-11-15 10:04:56 -08:00
tag.c Merge branch 'ab/i18n-st' 2011-04-01 17:55:55 -07:00
tar-tree.c
unpack-file.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
unpack-objects.c zlib: zlib can only process 4GB at a time 2011-06-10 11:52:15 -07:00
update-index.c Convert ce_path_match() to use struct pathspec 2011-02-03 14:08:30 -08:00
update-ref.c
update-server-info.c add OPT__FORCE 2010-11-15 10:04:43 -08:00
upload-archive.c
var.c Fix sparse warnings 2011-03-22 10:16:54 -07:00
verify-pack.c
verify-tag.c Merge branch 'mg/maint-tag-rfc1991' 2010-12-08 11:24:13 -08:00
write-tree.c