Commit graph

28 commits

Author SHA1 Message Date
Brian Feldman 378949dbca This fixes the problem of cancelling an FTP transfer resulting in
a bogus error ("parsing URI".) I also fixed an inconsistency in
another fs_status.
1999-06-21 02:38:16 +00:00
John-Mark Gurney 85020e14f0 make md5 checksums actually work instead of simply calculating the length
of an empty buffer...  the output file wasn't readable...  also warn that
we can't checksum on stdout and print out the base64 encoded version of the
md5 checksum...

Site to actually return md5 digest: web.golux.com
Verified that fetch was broken: Ken Coar <Ken.Coar@Golux.Com>
1999-05-18 19:37:37 +00:00
Garrett Wollman 189da04438 Deal with broken Web sites which return 302 responses rather than 404
and an error document when the requested resource does not exist.  Grrr.

Requested by:	asami
1999-02-23 18:51:13 +00:00
Bill Fenner 93eb50393d Don't try to parse a colon in a URL as a port
(e.g. http://www.host.name/foo:bar)

PR:		bin/5072
Submitted by:	Takeshi WATANABE <watanabe@komadori.planet.kobe-u.ac.jp>
1999-02-03 20:24:53 +00:00
Garrett Wollman 9caa4ddf6b Fix logic error in RFC 850 kluge. 1999-01-15 17:10:31 +00:00
Garrett Wollman c146843096 For RFC 850 dates received in HTTP responses, implement the century pivot
described in RFC 2068.  Include a reference to same in the manual page.
1999-01-15 16:56:22 +00:00
Martin Cracauer b451f4524e Add -s option, just report size of file that would be fetched.
Reviewed by:	-current list
1998-12-08 13:00:49 +00:00
Bill Fenner 60a53b7d32 If we know the content-length, only read that number of bytes from
the server.  There exists a broken server which sends a few extra
garbage bytes in response to HTTP/1.1 requests.
1998-10-26 02:39:21 +00:00
Jordan K. Hubbard 0637c2cccd Add -S flag; this is benign since fetch's behavior is unchanged if you
don't use it.  That's why I'm bringing it in during our "code slush"
Submitted by:	Stefan Esser <se@mi.uni-koeln.de>
1998-09-20 00:01:26 +00:00
Stefan Eßer cc522880b4 Fix proxy authorization code:
- failed to use authorization parameters passed in the environment, if
  /dev/tty could not be opened (i.e. if running from cron)
- mixed use of /dev/tty and stdin for prompt and reading of the result
1998-07-12 09:07:36 +00:00
Garrett Wollman eeed58d824 Documentation fix: delete references to T/TCP as they are not relevant
to the case in question.  (Said case being quite clearly defined by
the basic TCP specification, RFC 792.)
1998-05-09 20:50:37 +00:00
John Birrell 5362bb64c5 gcc wants to see long long variables passed to %q printf formats, so
cast to that instead of quad_t.
1998-02-20 05:11:42 +00:00
Bill Fenner 18d1eb6776 Suggest using "-b" or "-t" if the connection is reset (typical symptom
of broken TCP stack).
1997-11-22 01:00:42 +00:00
John Polstra a5bd62c599 Add missing arguments detected by "-Wformat". 1997-11-18 03:27:34 +00:00
Andrey A. Chernov 9e8930055e Change diags to see what part of url actually parsed, see
previous change about #? parms
1997-11-01 05:47:41 +00:00
Andrey A. Chernov 4ad97ec183 Don't search for ':' port in ?# params part
Require to correctly parse http://...cgi?http://...
1997-10-31 23:33:44 +00:00
Martin Cracauer 3e27c094e4 fetch(3) doesn't get asctime(3) format, wrong length assumed
PR:		bin/4625
Submitted by:	"Timo J. Rinne" <tri@pooh.tky.hut.fi>
Obtained from:bin/4625
1997-09-28 11:25:59 +00:00
Andrey A. Chernov f4e4504f06 Add -t option which turns T/TCP off as workaround for some broken servers
Submitted by: Marc Slemko <marcs@znep.com>
1997-08-05 20:18:39 +00:00
Garrett Wollman 12b7829739 Work around a brokenness in the HTTP spec by generating the Host header
for ftp: URLs as well.  This can't possibly be done in the general case,
but since we only claim to support http and ftp, we'll adhere to the
bogus requirement (RFC 2068, s. 14.25) anyway...  >sigh<

Submitted by: =?iso-8859-1?Q?=C5ge_R=F8bekk?= <aagero@aage.priv.no>
1997-07-26 20:18:43 +00:00
Garrett Wollman efd4ad4cd2 Fix the error message for when mirroring and restarting are rejected
due to the output not being a regular file.  Also split the error message
in any error case here to be on two lines to lessen the likelihood of it
being too long to fit on just one.
1997-07-26 20:00:05 +00:00
Garrett Wollman 2ec04c7076 Implement HTTP 1.1's ``chunked'' Transfer-Encoding (ick). This hasn't
been extensively tested, but I now can successfully retrieve
<http://www.apache.org/index.html>, so I guess that's a victory of some
sort.

Also move the initialization of ``autherror'' to hopefully eliminate
the reported loop involving authentication.  Still need to implement
MD5 digest authentication.
1997-07-26 19:25:56 +00:00
Garrett Wollman 0c8cc99e5b Provide a new `-b' flag to work around some broken HTTP/TCP implementations
that can't deal with a half-closed connection.
1997-07-25 19:35:44 +00:00
John-Mark Gurney 817eca811d fix restarting http transfers and understanding response from server.
Slight modification of patch by wollman.

Closes PR#2870

Reviewed by:	wollman
Submitted by:	Marc Slemko <marcs@znep.com>
1997-03-06 20:01:32 +00:00
Bill Fenner 8a2f8e395a Fix FTP_PROXY to use user@host[@port] for FTP proxy and eliminate
undocumented FTP_PROXY_USER
Make FTP file errors contian hostname and path.
Pass the FTP port to libftp.
Partially un-HTMLify error messages returned from server
Handle "HTTP NNN" instead of "HTTP/V.vv NNN" response sent by
	pre-HTTP/1.0 servers

Reviewed by:		wollman
1997-03-05 18:57:16 +00:00
Garrett Wollman 857292949c Fix remote request for HTTP proxies. Should close PR#2670. 1997-02-11 20:46:06 +00:00
Garrett Wollman 76dafb8954 Some bug-fixes, clean-ups, and one new feature:
- Fix the bug with URIs of the form ftp://host/filename.
- Fix some more string-termination bugs in util.c.
- Use safe_malloc() rather than testing the return value of
  regular malloc() in 15 places.
- Implement HTTP authentication, for both servers and proxies.
  Currently only ``basic'' authentication is supported; This Is A Bug
  (but less of one tjhan nmot supporting any authentication).

I think there is only one more feature which is required for full
HTTP/1.1 support, which is Transfer-Encoding: chunked; this should
not be toohard, but it isn't very important, either.
1997-02-05 19:59:18 +00:00
Garrett Wollman 2494e810b8 Some fixes for HTTP:
1) Implement redirects (or try to, at least).
2) Implement automatic retry after 503 errors when Retry-After is given.
3) Implement a -a flag to enable both of these behaviors.
4) Recognize Transfer-Encoding headers and emit a warning that the file
  is likely to be damaged.
5) Bug fix: only write the amount of data we read.
6) Actually document some of these.
7) Fix the usage message to display flags in semi-alphabetical order.
1997-01-31 19:55:51 +00:00
Garrett Wollman 78be319939 Here is my long-threatened revamping of fetch. Jean-Marc probably won't
recognize it any more.  This makes the following significant changes:

- The main body of the program doesn't know a thing about URIs,
  HTTP, or FTP.  This makes it possible to easily plug in other
  protocols.  (The next revision will probably be able to dynamically
  add new recognizers.)

- There are no longer arbitrary timeouts for the protocols.  If you want
  to set one for yourself, use the environment variables.

- FTP proxies are now supported (if I implemented it right).

- The HTTP implementation is much more complete, and can now do restarts,
  preserve modtimes, and mrun in mirror mode.  It's not yet up to 1.1,
  but it's getting there.

- Transaction TCP is now used for sending HTTP requests.  The HTTP/1.1 syntax
  for requesting that the connection be closed after one request is
  implemented.

In all of this, I have doubtless broken somebody.  Please test it and tell me
about the bugs.
1997-01-30 21:43:44 +00:00