The code handling the fetching of loose objects in http-push.c and
http-walker.c have been refactored into new methods and a new struct
(object_http_request) in http.c. They are not meant to be invoked
elsewhere.
The new methods in http.c are
- new_http_object_request
- process_http_object_request
- finish_http_object_request
- abort_http_object_request
- release_http_object_request
and the new struct is http_object_request.
RANGER_HEADER_SIZE and no_pragma_header is no longer made available
outside of http.c, since after the above changes, there are no other
instances of usage outside of http.c.
Remove members of the transfer_request struct in http-push.c and
http-walker.c, including filename, real_sha1 and zret, as they are used
no longer used.
Move the methods append_remote_object_url() and get_remote_object_url()
from http-push.c to http.c. Additionally, get_remote_object_url() is no
longer defined only when USE_CURL_MULTI is defined, since
non-USE_CURL_MULTI code in http.c uses it (namely, in
new_http_object_request()).
Refactor code from http-push.c::start_fetch_loose() and
http-walker.c::start_object_fetch_request() that deals with the details
of coming up with the filename to store the retrieved object, resuming
a previously aborted request, and making a new curl request, into a new
function, new_http_object_request().
Refactor code from http-walker.c::process_object_request() into the
function, process_http_object_request().
Refactor code from http-push.c::finish_request() and
http-walker.c::finish_object_request() into a new function,
finish_http_object_request(). It returns the result of the
move_temp_to_file() invocation.
Add a function, release_http_object_request(), which cleans up object
request data. http-push.c and http-walker.c invoke this function
separately; http-push.c::release_request() and
http-walker.c::release_object_request() do not invoke this function.
Add a function, abort_http_object_request(), which unlink()s the object
file and invokes release_http_object_request(). Update
http-walker.c::abort_object_request() to use this.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The code handling the fetching of packs in http-push.c and
http-walker.c have been refactored into new methods and a new struct
(http_pack_request) in http.c. They are not meant to be invoked
elsewhere.
The new methods in http.c are
- new_http_pack_request
- finish_http_pack_request
- release_http_pack_request
and the new struct is http_pack_request.
Add a function, new_http_pack_request(), that deals with the details of
coming up with the filename to store the retrieved packfile, resuming a
previously aborted request, and making a new curl request. Update
http-push.c::start_fetch_packed() and http-walker.c::fetch_pack() to
use this.
Add a function, finish_http_pack_request(), that deals with renaming
the pack, advancing the pack list, and installing the pack. Update
http-push.c::finish_request() and http-walker.c::fetch_pack to use
this.
Update release_request() in http-push.c and http-walker.c to invoke
release_http_pack_request() to clean up pack request helper data.
The local_stream member of the transfer_request struct in http-push.c
has been removed, as the packfile pointer will be managed in the struct
http_pack_request.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
http-push.c and http-walker.c no longer have to use fetch_index or
setup_index; they simply need to use http_get_info_packs, a new http
method, in their fetch_indices implementations.
Move fetch_index() and rename to fetch_pack_index() in http.c; this
method is not meant to be used outside of http.c. It invokes
end_url_with_slash with base_url; apart from that change, the code is
identical.
Move setup_index() and rename to fetch_and_setup_pack_index() in
http.c; this method is not meant to be used outside of http.c.
Do not immediately set ret to 0 in http-walker.c::fetch_indices();
instead do it in the HTTP_MISSING_TARGET case, to make it clear that
the HTTP_OK and HTTP_MISSING_TARGET cases both return 0.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Move RANGE_HEADER_SIZE to http.h.
Create no_pragma_header, the curl header list containing the header
"Pragma:" in http.[ch]. It is allocated in http_init, and freed in
http_cleanup. This replaces the no_pragma_header in http-push.c, and
the no_pragma_header member in walker_data in http-walker.c.
Create http_is_verbose. It is to be used by methods in http.c, and is
modified at the entry points of http.c's users, namely http-push.c
(when parsing options) and http-walker.c (in get_http_walker).
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a70c232 ("http-fetch: do not SEGV after fetching a bad pack idx
file"), changes were made to the setup_index method in http-fetch.c
(known in its present form as http-walker.c after 30ae764 ("Modularize
commit-walker")). Since http-push.c has similar similar code for
processing index files, these changes should apply to http-push.c's
implementation of setup_index as well.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the fetch_index implementations in http-push.c and http-walker.c,
the string returned by sha1_to_hex is assumed to stay immutable.
This patch ensures that hex stays immutable by copying the string
returned by sha1_to_hex (via xstrdup) and frees it subsequently. It
also refactors free()'s and fclose()'s with labels.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
- Use tabs to indent, instead of spaces.
- Do not use curly-braces around a single statement body in
if/while statement;
- Do not start multi-line comment with description on the first
line after "/*", i.e.
/*
* We prefer this over...
*/
/* comments like
* this (notice the first line)
*/
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
As it is breaking the build when USE_CURL_MULTI is not defined.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Previously, requests for remote files were simply added to the queue
(pointed to by request_queue_head) and no transfer actually takes
place (the fill function add_fill_function() is not added until line
2441), even though code that followed may rely on these remote files to
be present (eg. the setup_revisions invocation).
The code that sends out the requests on the request queue is refactored
into the method run_request_queue.
After the get_dav_remote_heads invocation (ie. after fetch requests are
added to the queue), the requests on the queue are sent out through an
invocation to run_request_queue.
This invocation to run_request_queue entails adding a fill function
before pushing checks take place, which may lead to accidental,
unwanted pushes previously.
The flag is_running_queue is introduced to prevent this from occurring.
fill_active_slot is made to check the flag is_running_queue before
the sending of the requests proceeds.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Set slot->local to NULL after doing a fclose() on the file it points
to. This prevents the passing of a FILE* pointer to a fclose()'d file
to ftell() in http.c::run_active_slot().
This issue was raised by Clemens Buchacher on 30th May 2009:
http://www.spinics.net/lists/git/msg104623.html
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This helps to notice when something's going wrong, especially on
systems which lock open files.
I used the following criteria when selecting the code for replacement:
- it was already printing a warning for the unlink failures
- it is in a function which already printing something or is
called from such a function
- it is in a static function, returning void and the function is only
called from a builtin main function (cmd_)
- it is in a function which handles emergency exit (signal handlers)
- it is in a function which is obvously cleaning up the lockfiles
Signed-off-by: Alex Riesen <raa.lkml@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If the repo url or the user email contain XML special characters, the
remote DAV server is likely to reject the LOCK requests because the XML
is then malformed.
Signed-off-by: Mike Hommey <mh@glandium.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* jc/shared-literally:
t1301: loosen test for forced modes
set_shared_perm(): sometimes we know what the final mode bits should look like
move_temp_to_file(): do not forget to chmod() in "Coda hack" codepath
Move chmod(foo, 0444) into move_temp_to_file()
"core.sharedrepository = 0mode" should set, not loosen
When using multi-pass authentication methods, the curl library may
need to rewind the read buffers (depending on how much already has
been fed to the server) used for providing data to HTTP PUT, POST or
PROPFIND, and in order to allow the library to do so, we need to tell
it how by providing either an ioctl callback or a seek callback.
This patch adds an ioctl callback, which should be usable on older
curl versions (since 7.12.3) than the seek callback (introduced in
curl 7.18.0).
Some HTTP servers (such as Apache) give an 401 error reply immediately
after receiving the headers (so no data has been read from the read
buffers, and thus no rewinding is needed), but other servers (such
as Lighttpd) only replies after the whole request has been sent and
all data has been read from the read buffers, making rewinding necessary.
Signed-off-by: Martin Storsjo <martin@martin.st>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When writing out a loose object or a pack (index), move_temp_to_file() is
called to finalize the resulting file. These files (loose files and packs)
should all have permission mode 0444 (modulo adjust_shared_perm()).
Therefore, instead of doing chmod(foo, 0444) explicitly from each callsite
(or even forgetting to chmod() at all), do the chmod() call from within
move_temp_to_file().
Signed-off-by: Johan Herland <johan@herland.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* mg/http-auth:
http-push.c: use a faux remote to pass to http_init
Do not name "repo" struct "remote" in push_http.c
http.c: CURLOPT_NETRC_OPTIONAL is not available in ancient versions of cURL
http authentication via prompts
http_init(): Fix config file parsing
http.c: style cleanups
Conflicts:
http-push.c
Change three occurences of using inconsistent error/warning reporting by
using the relevant error() / warning() calls to be consitent with the
rest of the code.
Signed-off-by: Miklos Vajna <vmiklos@frugalware.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch allows http_push to use http authentication via prompts.
You may notice that there is a remote struct that only contains the
url from the repo struct. This struct is a temporary fix for a larger
issue, but gets http authentication via prompts out the door, and
keeps users from having to store passwords in plain text files.
Signed-off-by: Amos King <amos.l.king@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This patch is a first step in getting http-push to use http authentication
via prompts. The patch renames remote to repo so that it doesn't get
confusing with the same remote that is passed around when using http.
Signed-off-by: Amos King <amos.l.king@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* js/remote-improvements: (23 commits)
builtin-remote.c: no "commented out" code, please
builtin-remote: new show output style for push refspecs
builtin-remote: new show output style
remote: make guess_remote_head() use exact HEAD lookup if it is available
builtin-remote: add set-head subcommand
builtin-remote: teach show to display remote HEAD
builtin-remote: fix two inconsistencies in the output of "show <remote>"
builtin-remote: make get_remote_ref_states() always populate states.tracked
builtin-remote: rename variables and eliminate redundant function call
builtin-remote: remove unused code in get_ref_states
builtin-remote: refactor duplicated cleanup code
string-list: new for_each_string_list() function
remote: make match_refs() not short-circuit
remote: make match_refs() copy src ref before assigning to peer_ref
remote: let guess_remote_head() optionally return all matches
remote: make copy_ref() perform a deep copy
remote: simplify guess_remote_head()
move locate_head() to remote.c
move duplicated ref_newer() to remote.c
move duplicated get_local_heads() to remote.c
...
Conflicts:
builtin-clone.c
http-push.c::finish_request():
request is initialized by the for loop
index-pack.c::free_base_data():
b is initialized by the for loop
merge-recursive.c::process_renames():
move compare to narrower scope, and remove unused assignments to it
remove unused variable renames2
xdiff/xdiffi.c::xdl_recs_cmp():
remove unused variable ec
xdiff/xemit.c::xdl_emit_diff():
xche is always overwritten
Signed-off-by: Benjamin Kramer <benny.kra@googlemail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
ref_newer() appears to have been copied from builtin-send-pack.c to
http-push.c via cut and paste. This patch moves the function and its
helper unmark_and_free() to remote.c. There was a slight difference
between the two implementations, one used TMP_MARK for the mark, the
other used 1. Per Jeff King, I went with TMP_MARK as more correct.
This is in preparation for being able to call it from builtin-remote.c
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
get_local_heads() appears to have been copied from builtin-send-pack.c
to http-push.c via cut and paste. This patch moves the function and its
helper one_local_ref() to remote.c.
The two copies of one_local_ref() were not identical. I used the more
recent version from builtin-send-pack.c after confirming with Jeff King
that it was an oversight that commit 30affa1e did not update both
copies.
This is in preparation for being able to call it from builtin-remote.c
Signed-off-by: Jay Soffian <jaysoffian@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
After 753bc91 ("Remove the requirement opaquelocktoken uri scheme"),
lock tokens are in the URI forms in which they are received from the
server, eg. 'opaquelocktoken:', 'urn:uuid:'.
However, "start_put" (and consequently "start_move"), which attempts to
create a unique temporary file using the UUID of the lock token,
inadvertently uses the lock token in its URI form. These file
operations on the server may not be successful (specifically, in
Windows), due to the colon ':' character from the URI form of the lock
token in the file path.
This patch uses a hash of the lock token instead, guaranteeing only
"safe" characters (a-f, 0-9) are used in the file path.
The token's hash is generated when the lock token is received from the
server in handle_new_lock_ctx, minimizing the number of times of
hashing.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* rc/http-push:
http-push: wrap signature of get_remote_object_url
http-push: add back underscore separator before lock token
http-push.c: get_remote_object_url() is only used under USE_CURL_MULTI
http-push: refactor request url creation
The signature of get_remote_object_url stands at 96 characters (as
pointed out by Dscho); this patch wraps it so that it conforms to the
80 characters guideline.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
817d14a (http-push: refactor request url creation, 2009-01-31) removed the
underscore separator between the object path and the appended lock token.
This patch adds it back.
This would be keeping in line with the aforementioned patch's objective
of refactoring, without changing the behaviour and effect, of the code.
This would also be useful for testing if the lock token has been
indeed appended to the object url.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* sp/runtime-prefix:
Windows: Revert to default paths and convert them by RUNTIME_PREFIX
Compute prefix at runtime if RUNTIME_PREFIX is set
Modify setup_path() to only add git_exec_path() to PATH
Add calls to git_extract_argv0_path() in programs that call git_config_*
git_extract_argv0_path(): Move check for valid argv0 from caller to callee
Refactor git_set_argv0_path() to git_extract_argv0_path()
Move computation of absolute paths from Makefile to runtime (in preparation for RUNTIME_PREFIX)
* jk/signal-cleanup:
t0005: use SIGTERM for sigchain test
pager: do wait_for_pager on signal death
refactor signal handling for cleanup functions
chain kill signals for cleanup functions
diff: refactor tempfile cleanup handling
Windows: Fix signal numbers
Introduce two helper functions append_remote_object_url() and
get_remote_object_url() and use them to remove various places
that allocate and format the URL by hand. These functions generate
a URL that point at the fan-out directory inside the remote object
store (e.g. http://host/path/to/repo/objects/a1/) or at an individual
loose object file.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Programs that use git_config need to find the global configuration.
When runtime prefix computation is enabled, this requires that
git_extract_argv0_path() is called early in the program's main().
This commit adds the necessary calls.
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Acked-by: Johannes Sixt <j6t@kdbg.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
b1c7d4a (http-push: refactor lock-related headers creation for curl
requests, 2009-01-24) had many style violations that slipped through.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
DAV-related headers (more specifically, headers related to the lock token,
namely, If, Lock-Token, and Timeout) for curl requests are created and
allocated individually, eg a "if_header" variable for the "If: " header, a
"timeout_header" variable for the "Timeout: " header.
This patch provides a new function ("get_dav_token_headers") that creates
these header, saving methods from allocating memory, and from issuing a
"curl_slist_append()" call. The temporary string storage given to
curl_slist_append() is freed much earlier than the previous code with this
patch, but this change is safe, because curl_slist_append() keeps a copy
of the given string.
In part, this patch also addresses the fact that commit 753bc91 (Remove
the requirement opaquelocktoken uri scheme) did not update memory
allocations for DAV-related headers.
Signed-off-by: Tay Ray Chuan <rctay89@gmail.com>
Acked-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current code is very inconsistent about which signals
are caught for doing cleanup of temporary files and lock
files. Some callsites checked only SIGINT, while others
checked a variety of death-dealing signals.
This patch factors out those signals to a single function,
and then calls it everywhere. For some sites, that means
this is a simple clean up. For others, it is an improvement
in that they will now properly clean themselves up after a
larger variety of signals.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
If a piece of code wanted to do some cleanup before exiting
(e.g., cleaning up a lockfile or a tempfile), our usual
strategy was to install a signal handler that did something
like this:
do_cleanup(); /* actual work */
signal(signo, SIG_DFL); /* restore previous behavior */
raise(signo); /* deliver signal, killing ourselves */
For a single handler, this works fine. However, if we want
to clean up two _different_ things, we run into a problem.
The most recently installed handler will run, but when it
removes itself as a handler, it doesn't put back the first
handler.
This patch introduces sigchain, a tiny library for handling
a stack of signal handlers. You sigchain_push each handler,
and use sigchain_pop to restore whoever was before you in
the stack.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function lock_remote() sends MKCOL requests to make leading
directories; However, if it does not put a forward slash '/' at the end of
the path, the server sends a 301 redirect.
By leaving the '/' in place, we can avoid this additional step.
Incidentally, at least one version of Curl (7.16.3) does not resend
credentials when it follows a 301 redirect, so this commit also fixes
a bug.
Original patch by Tay Ray Chuan <rctay89@gmail.com>.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When getting the result of remote_ls(), we were advancing the variable
"path" to the relative path inside the repository.
However, then we went on to malloc a bogus amount of memory: we were
subtracting the prefix length _again_, quite possibly getting something
negative, which xmalloc() interprets as really, really much.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
R. Tyler Ballance reported a mysterious transient repository corruption;
after much digging, it turns out that we were not catching and reporting
memory allocation errors from some calls we make to zlib.
This one _just_ wraps things; it doesn't do the "retry on low memory
error" part, at least not yet. It is an independent issue from the
reporting. Some of the errors are expected and passed back to the caller,
but we die when zlib reports it failed to allocate memory for now.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The program calls remote_ls() to get list of files from the server over
HTTP; handle_remote_ls_ctx() is used to parse its response to populate
"struct remote_ls_ctx" that is returned from remote_ls().
The handle_remote_ls_ctx() function assumed that the server returns a
local path in href field, but RFC 4918 (14.7) demand of support full URI
(e.g. "http://localhost:8080/repo.git").
This resulted in push failure (e.g. git-http-push issues a PROPFIND
request to "/repo.git/alhost:8080/repo.git/refs/" to the server).
Signed-off-by: Kirill A. Korinskiy <catap@catap.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The program flow of pushing over http is:
- call lock_remote() to issue a DAV_LOCK request to the server to lock
info/refs and branch refs being pushed into; handle_new_lock_ctx() is
used to parse its response to populate "struct remote_lock" that is
returned from lock_remote();
- send objects;
- call unlock_remote() to drop the lock.
The handle_new_lock_ctx() function assumed that the server will use a
lock token in opaquelocktoken URI scheme, which may have been an Ok
assumption under RFC 2518, but under RFC 4918 which obsoletes the older
standard it is not necessarily true.
This resulted in push failure (often resulted in "cannot lock existing
info/refs" error message) when talking to a server that does not use
opaquelocktoken URI scheme.
Signed-off-by: Kirill A. Korinskiy <catap@catap.ru>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With all calls to alloc_ref() gone, we can remove it and then we're free
to give alloc_ref_from_str() the shorter name. It's a much nicer
interface, as the callers always need to have a name string when they
allocate a ref anyway and don't need to calculate and pass its length+1
any more.
Signed-off-by: Rene Scharfe <rene.scharfe@lsrfire.ath.cx>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
On ARM I have the following compilation errors:
CC fast-import.o
In file included from cache.h:8,
from builtin.h:6,
from fast-import.c:142:
arm/sha1.h:14: error: conflicting types for 'SHA_CTX'
/usr/include/openssl/sha.h:105: error: previous declaration of 'SHA_CTX' was here
arm/sha1.h:16: error: conflicting types for 'SHA1_Init'
/usr/include/openssl/sha.h:115: error: previous declaration of 'SHA1_Init' was here
arm/sha1.h:17: error: conflicting types for 'SHA1_Update'
/usr/include/openssl/sha.h:116: error: previous declaration of 'SHA1_Update' was here
arm/sha1.h:18: error: conflicting types for 'SHA1_Final'
/usr/include/openssl/sha.h:117: error: previous declaration of 'SHA1_Final' was here
make: *** [fast-import.o] Error 1
This is because openssl header files are always included in
git-compat-util.h since commit 684ec6c63c whenever NO_OPENSSL is not
set, which somehow brings in <openssl/sha1.h> clashing with the custom
ARM version. Compilation of git is probably broken on PPC too for the
same reason.
Turns out that the only file requiring openssl/ssl.h and openssl/err.h
is imap-send.c. But only moving those problematic includes there
doesn't solve the issue as it also includes cache.h which brings in the
conflicting local SHA1 header file.
As suggested by Jeff King, the best solution is to rename our references
to SHA1 functions and structure to something git specific, and define those
according to the implementation used.
Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some places use the standard malloc/strdup without checking if the
allocation was successful; they should use xmalloc/xstrdup that
check the memory allocation result.
Signed-off-by: Dotan Barak <dotanba@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>