Commit graph

258 commits

Author SHA1 Message Date
Warner Losh a4a37038bb Reinstate revs 1.35-36 and 1.38. Revisions 1.34 and 1.37 were specifically
the root cause of the bus errors I was experiencing.

Submitted by:	fenner
Tested by:	obrien
Prompted by:	peter
2002-10-30 00:17:16 +00:00
David E. O'Brien b68fbebd5a Fix `pkg_add -r' by backing out revs 1.34-1.38.
Revs 1.37-8 produce a bus error in some environments.
Revs 1.34-6 do not bus error, but write corrupted files.
2002-10-29 12:17:43 +00:00
Dag-Erling Smørgrav cf2d89d98b libfetch is now WARNS5 clean in the non-SSL case. 2002-10-28 10:37:31 +00:00
Dag-Erling Smørgrav 32a4a82829 Fix an off-by-one error (> where >= should have been used) which caused
_fetch_writev() to incorrectly report EPIPE in certain cases.

Also fix a number of const warnings by using __DECONST(), plus a signed /
unsigned comparison by casting the rhs to ssize_t.

Submitted by:	fenner, Craig Rodrigues <rodrigc@attbi.com>
2002-10-28 10:19:03 +00:00
Bill Fenner 8497092d00 Up WARNS to 3 if not building with crypto.
Approved by:	des
2002-10-28 01:41:28 +00:00
Dag-Erling Smørgrav 1a5424b137 Slight amendment to rev 1.34: instead of considering any short read an
error, only report an error if no data was read at all (unless len was
0 to start with).  Otherwise, the final read of practically any transfer
will end in a fatal error.
2002-10-27 17:20:49 +00:00
Dag-Erling Smørgrav 2761348f78 Introduce _fetch_writev(), which is the conn_t version of writev(2). In
the SSL case, it is no different from the old _fetch_write(), but in the
non-SSL case it uses writev(2) to send the entire vector as a single
packet (provided it can fit in one packet).  Implement _fetch_write()
and _fetch_putln() in terms of _fetch_writev().

This should improve performance in the non-SSL case (by reducing protocol
overhead) and solve the problem where too-smart-for-their-own-good
firewalls reject FTP packets that do not end in CRLF.

PR:		bin/44123
Submitted by:	fenner
2002-10-27 16:11:21 +00:00
Dag-Erling Smørgrav 9f788e9c90 Eliminate two cases of undefined behaviour: total in _fetch_write() was
not initialized before use, and _http_growbuf() did not return a value
on success.

Reported by:	Peter Edwards <pmedwards@eircom.net>
MFC after:	2 weeks
2002-10-27 15:43:40 +00:00
Dag-Erling Smørgrav e24f60e74f Back out the previous commit, and fix the bug rather than try to hide its
symptoms: make timeouts and short transfers fatal, and set errno to an
appropriate value (ETIMEDOUT for a timeout, EPIPE for a short transfer).

MFC after:	2 weeks
2002-10-27 15:08:21 +00:00
Nate Lawson 13cc1c8394 The FTP connection caching needs a better interface -- connections are
closed through _fetch_close() which is the only one who knows the connection
REALLY was closed (since ref -> 0).  However, FTP keeps its own local
cached_connection and checks if it is valid by comparing it to NULL.  This
is bogus since it may have been freed elsewhere by _fetch_close().

This change checks if we are closing the cached_connection and the ref is 1
(soon to be 0).  If so, set cached_connection to NULL so we don't
accidentally reuse it.  The REAL fix should be to move connection caching
to the common.c level (_fetch_* functions) and NULL the cache(s) in
_fetch_close().  Then all layers could benefit from caching.
2002-10-25 01:17:32 +00:00
David E. O'Brien 513829ec5e Use a warns setting we can catch regressions with. 2002-10-20 23:50:28 +00:00
Tim J. Robbins 9d649c1fd0 When recycling a cached connection, increment the reference count so that
the heap block does not get freed and reused. This should fix the
pkg_add -r crashes that have been happening for months.
2002-10-20 10:36:27 +00:00
Ruslan Ermilov 721d559c03 HTTP authentication got broken in rev. 1.49. 2002-10-03 10:42:19 +00:00
Peter Wemm 224af215a6 Zap now-unused SHLIB_MINOR 2002-09-28 00:25:32 +00:00
Alfred Perlstein a6756ecc22 Fix an infinite loop when _fetch_read() can return 0 (if the
connection is broken), take this into account and return at this
point.
2002-09-20 21:50:57 +00:00
Bill Fenner 40cfbfd508 Make _fetch_connect() always set the error code.
Tell ftp that _fetch_connect() always sets the error code (http already knew)
2002-09-17 05:54:33 +00:00
Dag-Erling Smørgrav 190c185e22 Surround prototypes with __BEGIN_DECLS / __END_DECLS.
PR:		misc/40399
MFC after:	3 days
2002-07-22 16:11:39 +00:00
Hajimu UMEMOTO f8fa093e82 Cope with 2292bis-01 getaddrinfo (no NI_WITHSCOPEID, always attach
scope identifier).

Approved by:	des
MFC after:	3 weeks
2002-07-02 11:09:02 +00:00
Dag-Erling Smørgrav 66ffb8a371 Reintroduce debugging code that somehow got lost in a previous revision. 2002-06-24 12:18:41 +00:00
Dag-Erling Smørgrav a7a4510fd8 Fix incorrect library ordering. I thought I'd committed this already... 2002-06-21 09:56:38 +00:00
Dag-Erling Smørgrav f8bd33a0ae Don't try to dereference conn when we know it's NULL. 2002-06-19 08:36:00 +00:00
Dag-Erling Smørgrav f606d589b9 Add a reference count to struct fetchconn so we don't prematurely close and
free a cached FTP connection.
2002-06-11 11:27:28 +00:00
Ruslan Ermilov 4093807dd9 libfetch now depends on libcrypto and libssl. 2002-06-06 13:45:46 +00:00
Dag-Erling Smørgrav 3070f6cb06 Make SSL support conditional on NOCRYPT. 2002-06-05 21:35:35 +00:00
Dag-Erling Smørgrav d9615d7da4 During buildworld, "regular" libraries are built before crypto stuff, so
libfetch can't depend on lib{crypto,ssl}.  Move the dependency to fetch
until we can figure out how to fix this.
2002-06-05 21:25:33 +00:00
Dag-Erling Smørgrav 111e251009 Add SSL support + slight cleanup.
Submitted by:	Henry Whincup <henry@techiebod.com> (in principle)
2002-06-05 12:46:36 +00:00
Dag-Erling Smørgrav 9601e333a8 Wrap everything in struct connection, and enforce timeouts everywhere
(except for DNS operations).  Always use funopen() for HTTP, to support
both timeouts and SSL.
2002-06-05 12:19:08 +00:00
Dag-Erling Smørgrav 4dc0da3f3e Add the necessary dependencies for SSL. 2002-06-05 11:38:19 +00:00
Dag-Erling Smørgrav ccdd94bdd9 Rename struct cookie to struct httpio to avoid confusion (it's not an HTTP
cookie) and increase symmetry with equivalent FTP code.
2002-06-05 10:31:01 +00:00
Dag-Erling Smørgrav d3b03a9006 Add comments to struct cookie. 2002-06-05 10:27:24 +00:00
Dag-Erling Smørgrav 3f4823c55d Fix a bug I introduced in the chunk decoder in the previous commit.. 2002-06-05 10:23:19 +00:00
Dag-Erling Smørgrav dea29ca1d5 First step towards SSL support: wrap connections in a 'struct connection'
which contains the socket descriptor, the input buffer and (yet unused)
SSL state variables.  This has the neat side effect of greatly improving
reentrance (though we're not *quite* there yet) and opening the door to
HTTP connection caching.

This commit is inspired by email conversations with and patches from
Henry Whincup <henry@techiebod.com> last fall.
2002-06-05 10:05:03 +00:00
Mark Murray 4cd0119367 Do not use __progname directly (except in [gs]etprogname(3)).
Also, make an internal _getprogname() that is used only inside
libc. For libc, getprogname(3) is a weak symbol in case a
function of the same name is defined in userland.
2002-03-29 22:43:43 +00:00
Poul-Henning Kamp 06229ad2af Modernize my email address 2002-03-25 13:53:46 +00:00
Dag-Erling Smørgrav 5a51c23be3 Switch to a self-starting allocation scheme. 2002-02-05 22:15:16 +00:00
Dag-Erling Smørgrav e19e6098b3 Reindent, and add parentheses to return statements. Some functions in
ftp.c and http.c now have exceedingly long lines due to deep nesting;
this will be corrected by reorganizing the code in a later revision.
2002-02-05 22:13:51 +00:00
Dag-Erling Smørgrav 551858f0b0 Eliminate a redundant loop. 2002-02-05 21:32:16 +00:00
Dag-Erling Smørgrav e6f0a33e68 Check the return value from read() when reading the CR/LF at the end of a
chunk.

PR:		bin/33608
MFC after:	2 weeks
2002-01-20 19:53:12 +00:00
Dag-Erling Smørgrav e0583e0c23 Mark uploads as O_WRONLY, not O_RDONLY.
PR:		misc/34043
MFC after:	2 weeks
2002-01-20 19:52:25 +00:00
Dag-Erling Smørgrav d6811c6851 Remove long-obsolete README. 2002-01-01 16:27:26 +00:00
Dag-Erling Smørgrav a8e9bd8750 In verbose mode, display the full error message from the server, stripping
it of HTML tags and comments.

PR:		bin/32989
MFC after:	1 week
2002-01-01 16:25:29 +00:00
Dag-Erling Smørgrav f67efa37d6 Remove VT100 escapes from debugging messages now that they're enabled by
default.

PR:		32988
MFC after:	3 days
2002-01-01 14:48:09 +00:00
Dag-Erling Smørgrav 3c4b4d83f7 Don't build with NDEBUG.
PR:		bin/32615
MFC after:	1 week
2001-12-09 15:06:38 +00:00
Dag-Erling Smørgrav 7f807cb867 Conditionalize some debugging code that didn't use the DEBUG macro.
MFC after:	1 week
2001-12-09 15:05:58 +00:00
Dag-Erling Smørgrav 7eb2f34d73 Introduce a fetchDebug global. Change the DEBUG macro so it only runs the
debugging code if fetchDebug is set.

PR:		bin/32615
MFC after:	1 week
2001-12-09 15:05:19 +00:00
Dag-Erling Smørgrav 6598654546 Properly handle being redirected to an FTP URL.
Also fix a couple of not-so-minor bugs:
 - missing case for HTTP_SEE_OTHER
 - incorrect definition of HTTP_NEED_PROXY_AUTH

PR:		20259
MFC after:	2 weeks
2001-12-04 01:12:51 +00:00
Dag-Erling Smørgrav 89db7b8c6d Add prototype for _ftp_request(). 2001-12-04 01:10:34 +00:00
Dag-Erling Smørgrav 102a87c1e1 Reorganize to reduce code duplication. 2001-12-04 01:09:19 +00:00
Dag-Erling Smørgrav 5b31422b36 Add NO_WERROR so the build won't die because of discarded qualifiers 2001-10-19 10:08:38 +00:00
Dag-Erling Smørgrav 93ba13c1bb Back out part of previous commit which was gcc-centric 2001-10-19 10:08:05 +00:00
Dag-Erling Smørgrav f573a5fc94 Tons of type, style and warning fixes that have been rotting in my tree for
ages - some of which wouldn't be necessary if gcc wasn't broken or TPTB were
willing to do something (-fno-builtin) about it.
2001-10-18 08:29:26 +00:00
Ruslan Ermilov 32eef9aeb1 mdoc(7) police: Use the new .In macro for #include statements. 2001-10-01 16:09:29 +00:00
Matthew Dillon cecb889f1d Add __FBSDID()s to libfetch 2001-09-30 21:36:09 +00:00
Dag-Erling Smørgrav 98b8c4cb10 Use fseeko() instead of fseek() (u->offset is already an off_t), and mark
some function arguments as unused.
2001-09-05 12:22:28 +00:00
Brian Somers 778de35906 Handle snprintf() returning < 0 (not just -1)
MFC after:	2 weeks
2001-08-20 15:44:17 +00:00
Brian Somers 5f32890558 Handle snrintf overflows.
Spotted by: bde
2001-08-20 13:44:13 +00:00
Brian Somers 2449bf28ad Handle snprintf() returning -1.
MFC after:	2 weeks
2001-08-20 12:50:21 +00:00
Stefan Eßer 73b3e4df87 The fix for schemeless and hostless URLs (rev. 1.27) broke the schemeless
proxy specification, which seems to be valid according to the man page.

Change the logic to consider "hostname:port" a hostname and port instead
of a file URL.

Approved by:	des
2001-07-28 21:28:14 +00:00
Dag-Erling Smørgrav 5b2ad516e9 Unbreak parsing URLs that have a host part but no document part.
While we're here, fix a snprintf() usage warning.
2001-07-17 20:22:33 +00:00
Dima Dorfman 7ebcc426ef Remove whitespace at EOL. 2001-07-15 07:53:42 +00:00
Dima Dorfman 70d51341bf mdoc(7) police: remove extraneous .Pp before and/or after .Sh. 2001-07-09 09:54:33 +00:00
Dag-Erling Smørgrav ab39353eef Handle shemeless, hostless URLs correctly. 2001-07-08 15:59:15 +00:00
Ruslan Ermilov a7e16a9d0a mdoc(7) police: fixed markup and typo. 2001-07-05 11:24:26 +00:00
Dag-Erling Smørgrav 9e2a792c46 Don't call _ftp_closefn() upon EOF in _ftp_readfn(); just return 0. This
fixes a bug in pkg_add(1) (which nobody noticed because of another bug).
2001-06-11 14:12:46 +00:00
Dag-Erling Smørgrav 6a0cf64bce Implement a HTTP_USER_AGENT environment variable.
PR:		27669
Submitted by:	Eoin Lawless <eoin@maths.tcd.ie>
2001-05-27 11:00:36 +00:00
Dag-Erling Smørgrav 06b533d32c Document the authentication callback interface.
Update RFC references (should have done that ages ago...)
2001-05-26 19:38:34 +00:00
Dag-Erling Smørgrav 7ec6a14ea8 Bump major number. 2001-05-26 19:37:26 +00:00
Dag-Erling Smørgrav 6490b215b3 Add rudimentary support for an authentication callback function. 2001-05-26 19:37:15 +00:00
Dag-Erling Smørgrav 2b26f942d3 MAXPATHLEN -> PATH_MAX 2001-05-26 19:36:49 +00:00
Dag-Erling Smørgrav b554dea773 Plug memory leak.
PR:		27506
2001-05-26 17:23:38 +00:00
Dag-Erling Smørgrav ea63a39cef Add some error codes that were added in RFC2616. 2001-05-23 18:52:02 +00:00
Archie Cobbs 38c7e4a631 Apply 'const' liberally.
Fix some other minor glitches.
2001-04-24 00:06:21 +00:00
Dima Dorfman 5e75e35cca Grammar police: "its", not "it's", is the possessive form of "it". 2001-04-15 19:53:47 +00:00
Dag-Erling Smørgrav 525be862e6 If the server's reply to the SIZE command is unparseable, reset us->size
to -1 so the caller will know it's invalid.  This is an MFC candidate.
2001-04-07 15:26:31 +00:00
Ruslan Ermilov 4a558355e5 MAN[1-9] -> MAN. 2001-03-27 17:27:19 +00:00
Dag-Erling Smørgrav d5f175ce90 Use high port range by default, and replace the 'h' option with an 'l' option
that forces the ftp code to use the low (default) port range instead.
2001-03-24 00:28:57 +00:00
Dag-Erling Smørgrav e238d2a8da Don't remember an EINTR, since the caller may want to restart the call. 2001-03-07 05:12:14 +00:00
Dag-Erling Smørgrav 4cee73c80f Support lower-case versions of the proxy environment variables.
PR:		bin/25494
2001-03-07 04:45:55 +00:00
Ruslan Ermilov 9884911506 mdoc(7) police: fixed broken references. 2001-01-16 11:52:00 +00:00
Ruslan Ermilov 32f6256a49 Prepare for mdoc(7)NG. 2001-01-16 09:08:22 +00:00
Dag-Erling Smørgrav c86c6f748b The user name for anonymous ftp is now "anonymous".
Remove the period after the last man page reference.
Add a reference to RFC1635, and sort the RFC references by number.
2001-01-08 13:46:40 +00:00
Dag-Erling Smørgrav bf4dd407e3 Back out previous (accidental) commit. 2001-01-08 13:17:21 +00:00
Dag-Erling Smørgrav 80ed165e91 Use "anonymous" rather than "ftp" as login name for anonymous ftp.
Rather than have a separate (misnamed) FTP_ANONYMOUS_PASSWORD constant, use
FTP_ANONYMOUS_USER (i.e. "anonymous") to construct the anonymous ftp password
if getlogin() fails.
2001-01-08 13:15:14 +00:00
Dag-Erling Smørgrav 659e0d5ef7 Document FTP_LOGIN. 2000-12-22 18:03:21 +00:00
Dag-Erling Smørgrav e72f0de24e Check the FTP_LOGIN environment variable before falling back on
FTP_ANONYMOUS_USER.
2000-12-22 18:01:40 +00:00
Dag-Erling Smørgrav 1ba8497660 Avoid a segfault (due to an unitialized pointer) when parsing URLs that have
no scheme or host part.
2000-12-13 11:21:09 +00:00
Dag-Erling Smørgrav 6c81eb523e Somewhere along the line, I misunderstood the whole FTP_PASSIVE_MODE debate
and had libfetch selecting passive mode even when FTP_PASSIVE_MODE was not
set at all, which is really quite surprising unless you know about it. So
change it to the agreed default behaviour of selecting passive mode if
FTP_PASSIVE_MODE is set, but not "no".
2000-12-06 09:23:27 +00:00
Dag-Erling Smørgrav 79aad71d3b Remove some obsolete comments. 2000-12-01 11:05:15 +00:00
Dag-Erling Smørgrav f9c2053bca Clean up the whitespace encoding code. 2000-12-01 11:04:57 +00:00
Dag-Erling Smørgrav 025bb05a82 Add SCHEME_HTTPS. 2000-12-01 11:04:43 +00:00
Dag-Erling Smørgrav 882974d431 Fix old-style proxy specs: default to FTP if FTP_PROXY was set; only default
to HTTP if HTTP_PROXY was used instead.
2000-11-27 13:42:56 +00:00
Dag-Erling Smørgrav b431558526 Don't try to get the proxy port number from /etc/services. 2000-11-27 13:41:55 +00:00
Dag-Erling Smørgrav d41c0df8f8 Don't go haywire if the server closes the connection in the middle of a
multiline response (proper fix this time).
2000-11-22 14:50:46 +00:00
Dag-Erling Smørgrav 04a80993c1 Revert previous commit, it was somewhat hasty. 2000-11-22 14:44:48 +00:00
Dag-Erling Smørgrav 1baad1a183 Don't go haywire if the server closes the connection during a multiline
response.
2000-11-22 14:30:28 +00:00
Ruslan Ermilov 725ab6287f log 2000-11-22 09:23:54 +00:00
Ruslan Ermilov b82f5db0ac Spell the des's name correctly. 2000-11-14 13:19:26 +00:00
Ruslan Ermilov b5c508fba3 Use Fx macro wherever possible. 2000-11-14 11:20:58 +00:00
Dag-Erling Smørgrav e828ada709 Use the documented (and historical) defaults. Centralize the decision logic
in order to avoid this bug in the future.

Submitted by:	se
2000-11-10 08:43:40 +00:00
Dag-Erling Smørgrav d74a913b68 Use CHECK_FLAG 2000-10-29 15:56:10 +00:00