1.\" $Id: fetch.1,v 1.22 1998/05/09 20:50:35 wollman Exp $ 2.Dd July 2, 1996 3.Dt FETCH 1 4.Os FreeBSD 2.2 5.Sh NAME 6.Nm fetch 7.Nd retrieve a file by Uniform Resource Locator 8.Sh SYNOPSIS 9.Nm fetch 10.Op Fl MPabmnpqr 11.Op Fl o Ar file 12.Ar URL 13.Op Ar ... 14.Nm fetch 15.Op Fl MPRmnpqr 16.Op Fl o Ar file 17.Op Fl c Ar dir 18.Fl f Ar file 19.Fl h Ar host 20.Sh DESCRIPTION 21.Nm fetch 22allows a user to transfer files from a remote network site using 23either the 24.Tn FTP 25or the 26.Tn HTTP 27protocol. In the first form of the command, the 28.Ar URL 29may be of the form 30.Li http://site.domain/path/to/the/file 31or 32.Li ftp://site.domain/path/to/the/file . 33To denote a local filename to be copied or linked to (see the 34.Fl l 35flag below), the 36.Em file:/path/to/the/file 37URL form is used. See URL SYNTAX, below. 38.Pp 39The second form of the command can be used to get a file using the 40.Tn FTP 41protocol, specifying the file name and the remote host with the 42.Fl h 43and the 44.Fl f 45flags. 46.Pp 47The following options are available: 48.Bl -tag -width Fl 49.It Fl a 50Automatically retry the transfer upon soft failures. 51.It Fl b 52Work around a bug in some 53.Tn HTTP 54servers which fail to correctly implement the 55.Tn TCP 56protocol. 57.It Fl c Ar dir 58The file to retrieve is in directory 59.Ar dir 60on the remote host. 61.It Fl f Ar file 62The file to retrieve is named 63.Ar file 64on the remote host. 65.It Fl h Ar host 66The file to retrieve is located on the host 67.Ar host . 68.It Fl l 69If target is a 70.Ar file:/ 71style of URL, make a link to the target rather than trying 72to copy it. 73.It Fl M 74.It Fl m 75Mirror mode: Set the modification time of the file so that it is 76identical to the modification time of the file at the remote host. 77If the file already exists on the local host and is identical (as 78gauged by size and modification time), no transfer is done. 79.It Fl n 80Don't preserve the modtime of the transfered file, use the current time. 81.It Fl o Ar file 82Set the output file name to 83.Ar file . 84By default, a ``pathname'' is extracted from the specified URI, and 85its basename is used as the name of the output file. A 86.Ar file 87argument of 88.Sq Li \&- 89indicates that results are to be directed to the standard output. 90.It Fl P 91.It Fl p 92Use the passive mode of the 93.Tn FTP 94protocol. This is useful for crossing certain sorts of firewalls. 95.It Fl q 96Quiet mode. Do not report transfer progress on the terminal. 97.It Fl R 98The filenames specified are ``precious'', and should not be deleted 99under any circumstances, even if the transfer failed or was incomplete. 100.It Fl r 101Restart a previously interrupted transfer. 102.It Fl t 103Work around a different set of buggy 104.Tn TCP 105implementations. 106.It Fl T Ar seconds 107Set timeout value to 108.Ar seconds. 109Overrides the environment variables 110.Ev FTP_TIMEOUT 111for ftp transfers or 112.Ev HTTP_TIMEOUT 113for http transfers if set. 114.It Fl v 115Increase verbosity. More 116.Fl v Ns \&'s 117result in more information. 118.El 119.Pp 120Many options are also controlled solely by the environment (this is a 121bug). 122.Sh URL SYNTAX 123.Nm 124accepts 125.Tn http 126and 127.Tn ftp 128URL's, as described in RFC1738. For 129.Tn ftp 130URL's, a username and password may be specified, using the syntax 131.Li ftp://user:password@host/. 132If the path is to be absolute, as opposed to relative to the user's 133home directory, it must start with %2F, as in 134.Li ftp://root:mypass@localhost/%2Fetc/passwd . 135.Sh PROXY SERVERS 136Many sites use application gateways (``proxy servers'') in their 137firewalls in order to allow communication across the firewall using a 138trusted protocol. The 139.Nm fetch 140program can use both the 141.Tn FTP 142and the 143.Tn HTTP 144protocol with a proxy server. 145.Tn FTP 146proxy servers can only relay 147.Tn FTP 148requests; 149.Tn HTTP 150proxy servers can relay both 151.Tn FTP 152and 153.Tn HTTP 154requests. 155A proxy server can be configured by defining an environment variable 156named 157.Dq Va PROTO Ns Ev _PROXY , 158where 159.Va PROTO 160is the name of the protocol in upper case. The value of the 161environment variable specifies a hostname, optionally followed by a 162colon and a port number. 163.Pp 164The 165.Tn FTP 166proxy client passes the remote username, host and port as the 167.Tn FTP 168session's username, in the form 169.Do Va remoteuser Ns Li \&@ Ns Va remotehost 170.Op Li \^@ Ns Va port 171.Dc . 172The 173.Tn HTTP 174proxy client simply passes the originally-requested URI to the remote 175server in an 176.Tn HTTP 177.Dq Li GET 178request. HTTP proxy authentication is not yet implemented. 179.Sh HTTP AUTHENTICATION 180The 181.Tn HTTP 182protocol includes support for various methods of authentication. 183Currently, the 184.Dq basic 185method, which provides no security from packet-sniffing or 186man-in-the-middle attacks, is the only method supported in 187.Nm fetch . 188Authentication is enabled by the 189.Ev HTTP_AUTH 190and 191.Ev HTTP_PROXY_AUTH 192environment variables. Both variables have the same format, which 193consists of space-separated list of parameter settings, where each 194setting consists of a colon-separated list of parameters. The first 195two parameters are always the (case-insensitive) authentication scheme 196name and the realm in which authentication is to be performed. If the 197realm is specified as 198.Sq Li \&* , 199then it will match all realms not specified otherwise. 200.Pp 201The 202.Li basic 203authentication scheme uses two additional optional parameters; the 204first is a user name, and the second is the password associated with 205it. If either the password or both parameters are not specified in 206the environment, and the standard input of 207.Nm 208is connected to a terminal, then 209.Nm 210will prompt the user to enter the missing parameters. Thus, if the 211user is known as 212.Dq Li jane 213in the 214.Dq Li WallyWorld 215realm, and has a password of 216.Dq Li QghiLx79 217there, then she might set her 218.Ev HTTP_AUTH 219variable to: 220.Bl -enum -offset indent 221.It 222.Dq Li basic:WallyWorld:jane:QghiLx79 223.It 224.Dq Li basic:WallyWorld:jane , 225or 226.It 227.Dq Li basic:WallyWorld 228.El 229.Pp 230and 231.Nm 232will prompt for the missing information if it is required. She might 233also specify a realm of 234.Dq Li \&* 235instead of 236.Dq Li WallyWorld 237to indicate that the parameters can be applied to any realm. (This is 238most commonly used in a construction such as 239.Dq Li basic:* , 240which indicates to 241.Nm 242that it may offer to do 243.Li basic 244authentication for any realm. 245.Sh ERRORS 246The 247.Nm 248command returns zero on success, or a non-zero value from 249.Aq Pa sysexits.h 250on failure. If multiple URIs are given for retrieval, 251.Nm 252will attempt all of them and return zero only if all succeeded 253(otherwise it will return the error from the last failure). 254.Sh ENVIRONMENT 255.Bl -tag -width FTP_PASSIVE_MODE -offset indent 256.It Ev FTP_TIMEOUT 257maximum time, in seconds, to wait before aborting an 258.Tn FTP 259connection. 260.It Ev FTP_LOGIN 261the login name used for 262.Tn FTP 263transfers (default 264.Dq Li anonymous ) 265.It Ev FTP_PASSIVE_MODE 266force the use of passive mode FTP 267.It Ev FTP_PASSWORD 268the password used for 269.Tn FTP 270transfers (default 271.Dq Va yourname Ns Li \&@ Ns Va yourhost ) 272.It Ev FTP_PROXY 273the address (in the form 274.Do Va hostname Ns 275.Op Li : Ns Va port 276.Dc ) 277of a proxy server which understands 278.Tn FTP 279.It Ev HTTP_AUTH 280defines authentication parameters for 281.Tn HTTP 282.It Ev HTTP_PROXY 283the address (in the form 284.Do Va hostname Ns 285.Op Li : Ns Va port 286.Dc ) 287of a proxy server which understands 288.Tn HTTP 289.It Ev HTTP_PROXY_AUTH 290defines authentication parameters for 291.Tn HTTP 292proxy servers 293.It Ev HTTP_TIMEOUT 294maximum time, in seconds, to wait before aborting an 295.Tn HTTP 296connection. 297.Sh SEE ALSO 298.Xr ftp 1 , 299.Xr tftp 1 300.Sh HISTORY 301The 302.Nm fetch 303command appeared in 304.Fx 2.1.5 . 305.Sh AUTHORS 306The original implementation of 307.Nm 308was done by 309.An Jean-Marc Zucconi . 310It was extensively re-worked for 311.Fx 2.2 312by 313.An Garrett Wollman . 314.Sh BUGS 315There are too many environment variables and command-line options. 316.Pp 317The 318.Fl a 319option is only implemented for certain kinds of 320.Tn HTTP 321failures, and no 322.Tn FTP 323failures. 324.Pp 325Only the 326.Dq basic 327authentication mode is implemented for 328.Tn HTTP . 329This should be replaced by digest authentication. 330.Pp 331Some 332.Tn TCP 333implementations (other than 334.Tn FreeBSD ) 335fail to correctly implement cases where the 336.Dv SYN 337and/or 338.Dv FIN 339control flags are specified in packets which also contain data. 340The 341.Sq Fl t 342flag works around the latter deficiency and the 343.Sq Fl b 344flag works around the former. Since these are errors of the server's 345.Tn TCP 346stack, the best we can do is provide these workarounds. Given a correct 347server, an optimal 348.Tn HTTP 349transfer without 350.Fl t 351and 352.Fl b 353involves a minimum of two round trips (for small replies), one less than 354other implementations. 355