Add __FBSDID()s to libfetch

Implement a HTTP_USER_AGENT environment variable.

PR: 27669
Submitted by: Eoin Lawless <eoin@maths.tcd.ie>

Add rudimentary support for an authentication callback function.

Apply 'const' liberally.

Fix some other minor glitches.

Support lower-case versions of the proxy environment variables.

PR: bin/25494

Use the documented (and historical) defaults. Centralize the decision logic
in order to avoid this bug in the future.

Submitted by: se

Use CHECK_FLAG

Rework proxy handling so that proxies can be specified as URLs.
As a side effect, remove a lot of duplicate and now redundant code.

Work around buggy servers such as NCSA httpd which send an incomplete
HTTP-Version on the Status-Line (see RFC2616 sections 3.1 and 6.1).

Fix signedness bogon.

Centralize the default port finding code.
Work around YA Apache bug: don't send port in Host: header if it's the
default port.

Ignore environment variables that are set but empty.

Sort out the size / length confusion. Always try to report the *real* file
size in the url_stat structure, no matter how much of it is actually

Ignore environment variables that are set but empty.

Sort out the size / length confusion. Always try to report the *real* file
size in the url_stat structure, no matter how much of it is actually being
sent, and try to detect inconsistencies between sizes.

Rearrange the request loop to avoid having to add meaningless code just to
silence compiler warnings.

Switch to a more sensible and consistent interface for the _http_parse*()
functions.

show more ...

Don't try to skip to the requested offset if the server returns more data
than requested. Instead, inform the caller of the real offset by modifying
the offset field in the original struct url, and

Don't try to skip to the requested offset if the server returns more data
than requested. Instead, inform the caller of the real offset by modifying
the offset field in the original struct url, and let him decide how to handle
the situation.

show more ...

Introduce fetchXGet*(), which combine the functionalities of fetchGet*() and
fetchStat*(). In most cases, either fetchGet*() or fetchXGet*() is a wrapper
around the other; in all cases, calling fetc

Introduce fetchXGet*(), which combine the functionalities of fetchGet*() and
fetchStat*(). In most cases, either fetchGet*() or fetchXGet*() is a wrapper
around the other; in all cases, calling fetchGet*() is identical to calling
fetchXGet*() with the second argument set to NULL.

show more ...

Don't forget to set file descriptor to -1 after closing it, since the code
outside the loop inspects it to determine whether or not we succeeded in
retrieving the requested document. This fixes a bug

Don't forget to set file descriptor to -1 after closing it, since the code
outside the loop inspects it to determine whether or not we succeeded in
retrieving the requested document. This fixes a bug where fetchGetHTTP()
would return a FILE with an invalid file descriptor if it hit the redirect
limit without locating the requested document.

show more ...

#ifdef DEBUG -> #ifndef NDEBUG

Pointed out by: ache

Reinsert the MIT copyright, which applies to the base64 code and was
left out by accident during the rewrite.

Fix a bug (misplaced continue) that caused redirects to fail. Lots of code
moved around, but the acutal functional changes are small.
Add support for site-internal redirects (where the Location: head

Fix a bug (misplaced continue) that caused redirects to fail. Lots of code
moved around, but the acutal functional changes are small.
Add support for site-internal redirects (where the Location: header gives a
path instead of an absolute URI)

Pointed out by: kuriyama

show more ...

Extensive rewrite of the HTTP code. The new code is significantly cleaner and
more robust, and somewhat more efficient. It also handles authorization and
redirects properly, and supports timeouts lik

Extensive rewrite of the HTTP code. The new code is significantly cleaner and
more robust, and somewhat more efficient. It also handles authorization and
redirects properly, and supports timeouts like the FTP code.

Many thanks to Umemoto-san for his assistance with IPv6 support, both here
and in other parts of libfetch.

show more ...

Rework the authorization code.
Only send absolute URI if connected to a proxy, since Apache doesn't always
understand absolute URIs.
Clean up some of the debugging output.

Fix basic authentication, and add proxy authentication.

Submitted by: se

Don't incorrectly report a protocol error when we get a 302 and redirects
are disabled.

Hackish support for 302 redirect.

IPv6 support.

Better handling of some boundary conditions.

Submitted by: ume