1.\" $Id: fetch.1,v 1.23 1998/05/09 21:39:49 wollman Exp $ 2.Dd July 2, 1996 3.Dt FETCH 1 4.Os FreeBSD 2.2 5.Sh NAME 6.Nm fetch 7.Nd retrieve a file by Uniform Resource Locator 8.Sh SYNOPSIS 9.Nm fetch 10.Op Fl MPabmnpqr 11.Op Fl S Ar size 12.Op Fl o Ar file 13.Ar URL 14.Op Ar ... 15.Nm fetch 16.Op Fl MPRmnpqr 17.Op Fl S Ar size 18.Op Fl o Ar file 19.Op Fl c Ar dir 20.Fl f Ar file 21.Fl h Ar host 22.Sh DESCRIPTION 23.Nm fetch 24allows a user to transfer files from a remote network site using 25either the 26.Tn FTP 27or the 28.Tn HTTP 29protocol. In the first form of the command, the 30.Ar URL 31may be of the form 32.Li http://site.domain/path/to/the/file 33or 34.Li ftp://site.domain/path/to/the/file . 35To denote a local filename to be copied or linked to (see the 36.Fl l 37flag below), the 38.Em file:/path/to/the/file 39URL form is used. See URL SYNTAX, below. 40.Pp 41The second form of the command can be used to get a file using the 42.Tn FTP 43protocol, specifying the file name and the remote host with the 44.Fl h 45and the 46.Fl f 47flags. 48.Pp 49The following options are available: 50.Bl -tag -width Fl 51.It Fl a 52Automatically retry the transfer upon soft failures. 53.It Fl b 54Work around a bug in some 55.Tn HTTP 56servers which fail to correctly implement the 57.Tn TCP 58protocol. 59.It Fl c Ar dir 60The file to retrieve is in directory 61.Ar dir 62on the remote host. 63.It Fl f Ar file 64The file to retrieve is named 65.Ar file 66on the remote host. 67.It Fl h Ar host 68The file to retrieve is located on the host 69.Ar host . 70.It Fl l 71If target is a 72.Ar file:/ 73style of URL, make a link to the target rather than trying 74to copy it. 75.It Fl M 76.It Fl m 77Mirror mode: Set the modification time of the file so that it is 78identical to the modification time of the file at the remote host. 79If the file already exists on the local host and is identical (as 80gauged by size and modification time), no transfer is done. 81.It Fl n 82Don't preserve the modtime of the transfered file, use the current time. 83.It Fl o Ar file 84Set the output file name to 85.Ar file . 86By default, a ``pathname'' is extracted from the specified URI, and 87its basename is used as the name of the output file. A 88.Ar file 89argument of 90.Sq Li \&- 91indicates that results are to be directed to the standard output. 92.It Fl P 93.It Fl p 94Use the passive mode of the 95.Tn FTP 96protocol. This is useful for crossing certain sorts of firewalls. 97.It Fl q 98Quiet mode. Do not report transfer progress on the terminal. 99.It Fl R 100The filenames specified are ``precious'', and should not be deleted 101under any circumstances, even if the transfer failed or was incomplete. 102.It Fl r 103Restart a previously interrupted transfer. 104.It Fl S Ar bytes 105Require file size reported by 106.Tn FTP 107or 108.Tn HTTP 109server to match the value specified with this option. 110On mismatch, a message is printed and the file will not be fetched. 111If the server does not support reporting of file sizes, the option 112will be ignored and the file will be retrieved anyway. 113This option is useful to prevent 114.Nm fetch 115from downloading a file that is either incomplete or the wrong version, 116given the correct size of the file in advance. 117.It Fl t 118Work around a different set of buggy 119.Tn TCP 120implementations. 121.It Fl T Ar seconds 122Set timeout value to 123.Ar seconds. 124Overrides the environment variables 125.Ev FTP_TIMEOUT 126for ftp transfers or 127.Ev HTTP_TIMEOUT 128for http transfers if set. 129.It Fl v 130Increase verbosity. More 131.Fl v Ns \&'s 132result in more information. 133.El 134.Pp 135Many options are also controlled solely by the environment (this is a 136bug). 137.Sh URL SYNTAX 138.Nm 139accepts 140.Tn http 141and 142.Tn ftp 143URL's, as described in RFC1738. For 144.Tn ftp 145URL's, a username and password may be specified, using the syntax 146.Li ftp://user:password@host/. 147If the path is to be absolute, as opposed to relative to the user's 148home directory, it must start with %2F, as in 149.Li ftp://root:mypass@localhost/%2Fetc/passwd . 150.Sh PROXY SERVERS 151Many sites use application gateways (``proxy servers'') in their 152firewalls in order to allow communication across the firewall using a 153trusted protocol. The 154.Nm fetch 155program can use both the 156.Tn FTP 157and the 158.Tn HTTP 159protocol with a proxy server. 160.Tn FTP 161proxy servers can only relay 162.Tn FTP 163requests; 164.Tn HTTP 165proxy servers can relay both 166.Tn FTP 167and 168.Tn HTTP 169requests. 170A proxy server can be configured by defining an environment variable 171named 172.Dq Va PROTO Ns Ev _PROXY , 173where 174.Va PROTO 175is the name of the protocol in upper case. The value of the 176environment variable specifies a hostname, optionally followed by a 177colon and a port number. 178.Pp 179The 180.Tn FTP 181proxy client passes the remote username, host and port as the 182.Tn FTP 183session's username, in the form 184.Do Va remoteuser Ns Li \&@ Ns Va remotehost 185.Op Li \^@ Ns Va port 186.Dc . 187The 188.Tn HTTP 189proxy client simply passes the originally-requested URI to the remote 190server in an 191.Tn HTTP 192.Dq Li GET 193request. HTTP proxy authentication is not yet implemented. 194.Sh HTTP AUTHENTICATION 195The 196.Tn HTTP 197protocol includes support for various methods of authentication. 198Currently, the 199.Dq basic 200method, which provides no security from packet-sniffing or 201man-in-the-middle attacks, is the only method supported in 202.Nm fetch . 203Authentication is enabled by the 204.Ev HTTP_AUTH 205and 206.Ev HTTP_PROXY_AUTH 207environment variables. Both variables have the same format, which 208consists of space-separated list of parameter settings, where each 209setting consists of a colon-separated list of parameters. The first 210two parameters are always the (case-insensitive) authentication scheme 211name and the realm in which authentication is to be performed. If the 212realm is specified as 213.Sq Li \&* , 214then it will match all realms not specified otherwise. 215.Pp 216The 217.Li basic 218authentication scheme uses two additional optional parameters; the 219first is a user name, and the second is the password associated with 220it. If either the password or both parameters are not specified in 221the environment, and the standard input of 222.Nm 223is connected to a terminal, then 224.Nm 225will prompt the user to enter the missing parameters. Thus, if the 226user is known as 227.Dq Li jane 228in the 229.Dq Li WallyWorld 230realm, and has a password of 231.Dq Li QghiLx79 232there, then she might set her 233.Ev HTTP_AUTH 234variable to: 235.Bl -enum -offset indent 236.It 237.Dq Li basic:WallyWorld:jane:QghiLx79 238.It 239.Dq Li basic:WallyWorld:jane , 240or 241.It 242.Dq Li basic:WallyWorld 243.El 244.Pp 245and 246.Nm 247will prompt for the missing information if it is required. She might 248also specify a realm of 249.Dq Li \&* 250instead of 251.Dq Li WallyWorld 252to indicate that the parameters can be applied to any realm. (This is 253most commonly used in a construction such as 254.Dq Li basic:* , 255which indicates to 256.Nm 257that it may offer to do 258.Li basic 259authentication for any realm. 260.Sh ERRORS 261The 262.Nm 263command returns zero on success, or a non-zero value from 264.Aq Pa sysexits.h 265on failure. If multiple URIs are given for retrieval, 266.Nm 267will attempt all of them and return zero only if all succeeded 268(otherwise it will return the error from the last failure). 269.Sh ENVIRONMENT 270.Bl -tag -width FTP_PASSIVE_MODE -offset indent 271.It Ev FTP_TIMEOUT 272maximum time, in seconds, to wait before aborting an 273.Tn FTP 274connection. 275.It Ev FTP_LOGIN 276the login name used for 277.Tn FTP 278transfers (default 279.Dq Li anonymous ) 280.It Ev FTP_PASSIVE_MODE 281force the use of passive mode FTP 282.It Ev FTP_PASSWORD 283the password used for 284.Tn FTP 285transfers (default 286.Dq Va yourname Ns Li \&@ Ns Va yourhost ) 287.It Ev FTP_PROXY 288the address (in the form 289.Do Va hostname Ns 290.Op Li : Ns Va port 291.Dc ) 292of a proxy server which understands 293.Tn FTP 294.It Ev HTTP_AUTH 295defines authentication parameters for 296.Tn HTTP 297.It Ev HTTP_PROXY 298the address (in the form 299.Do Va hostname Ns 300.Op Li : Ns Va port 301.Dc ) 302of a proxy server which understands 303.Tn HTTP 304.It Ev HTTP_PROXY_AUTH 305defines authentication parameters for 306.Tn HTTP 307proxy servers 308.It Ev HTTP_TIMEOUT 309maximum time, in seconds, to wait before aborting an 310.Tn HTTP 311connection. 312.Sh SEE ALSO 313.Xr ftp 1 , 314.Xr tftp 1 315.Sh HISTORY 316The 317.Nm fetch 318command appeared in 319.Fx 2.1.5 . 320.Sh AUTHORS 321The original implementation of 322.Nm 323was done by 324.An Jean-Marc Zucconi . 325It was extensively re-worked for 326.Fx 2.2 327by 328.An Garrett Wollman . 329.Sh BUGS 330There are too many environment variables and command-line options. 331.Pp 332The 333.Fl a 334option is only implemented for certain kinds of 335.Tn HTTP 336failures, and no 337.Tn FTP 338failures. 339.Pp 340Only the 341.Dq basic 342authentication mode is implemented for 343.Tn HTTP . 344This should be replaced by digest authentication. 345.Pp 346Some 347.Tn TCP 348implementations (other than 349.Tn FreeBSD ) 350fail to correctly implement cases where the 351.Dv SYN 352and/or 353.Dv FIN 354control flags are specified in packets which also contain data. 355The 356.Sq Fl t 357flag works around the latter deficiency and the 358.Sq Fl b 359flag works around the former. Since these are errors of the server's 360.Tn TCP 361stack, the best we can do is provide these workarounds. Given a correct 362server, an optimal 363.Tn HTTP 364transfer without 365.Fl t 366and 367.Fl b 368involves a minimum of two round trips (for small replies), one less than 369other implementations. 370