1.\" $Id: fetch.1,v 1.25 1998/11/08 23:18:47 des Exp $ 2.Dd July 2, 1996 3.Dt FETCH 1 4.Os FreeBSD 2.2 5.Sh NAME 6.Nm fetch 7.Nd retrieve a file by Uniform Resource Locator 8.Sh SYNOPSIS 9.Nm fetch 10.Op Fl MPablmnpqrtv 11.Op Fl S Ar size 12.Op Fl T Ar timeout 13.Op Fl o Ar file 14.Ar URL 15.Op Ar ... 16.Nm fetch 17.Op Fl MPRlmnpqrv 18.Op Fl S Ar size 19.Op Fl o Ar file 20.Op Fl c Ar dir 21.Fl f Ar file 22.Fl h Ar host 23.Sh DESCRIPTION 24.Nm fetch 25allows a user to transfer files from a remote network site using 26either the 27.Tn FTP 28or the 29.Tn HTTP 30protocol. In the first form of the command, the 31.Ar URL 32may be of the form 33.Li http://site.domain/path/to/the/file 34or 35.Li ftp://site.domain/path/to/the/file . 36To denote a local filename to be copied or linked to (see the 37.Fl l 38flag below), the 39.Em file:/path/to/the/file 40URL form is used. See URL SYNTAX, below. 41.Pp 42The second form of the command can be used to get a file using the 43.Tn FTP 44protocol, specifying the file name and the remote host with the 45.Fl h 46and the 47.Fl f 48flags. 49.Pp 50The following options are available: 51.Bl -tag -width Fl 52.It Fl a 53Automatically retry the transfer upon soft failures. 54.It Fl b 55Work around a bug in some 56.Tn HTTP 57servers which fail to correctly implement the 58.Tn TCP 59protocol. 60.It Fl c Ar dir 61The file to retrieve is in directory 62.Ar dir 63on the remote host. 64.It Fl f Ar file 65The file to retrieve is named 66.Ar file 67on the remote host. 68.It Fl h Ar host 69The file to retrieve is located on the host 70.Ar host . 71.It Fl l 72If target is a 73.Ar file:/ 74style of URL, make a link to the target rather than trying 75to copy it. 76.It Fl M 77.It Fl m 78Mirror mode: Set the modification time of the file so that it is 79identical to the modification time of the file at the remote host. 80If the file already exists on the local host and is identical (as 81gauged by size and modification time), no transfer is done. 82.It Fl n 83Don't preserve the modtime of the transfered file, use the current time. 84.It Fl o Ar file 85Set the output file name to 86.Ar file . 87By default, a ``pathname'' is extracted from the specified URI, and 88its basename is used as the name of the output file. A 89.Ar file 90argument of 91.Sq Li \&- 92indicates that results are to be directed to the standard output. 93.It Fl P 94.It Fl p 95Use the passive mode of the 96.Tn FTP 97protocol. This is useful for crossing certain sorts of firewalls. 98.It Fl q 99Quiet mode. Do not report transfer progress on the terminal. 100.It Fl R 101The filenames specified are ``precious'', and should not be deleted 102under any circumstances, even if the transfer failed or was incomplete. 103.It Fl r 104Restart a previously interrupted transfer. 105.It Fl S Ar bytes 106Require the file size reported by 107.Tn FTP 108or 109.Tn HTTP 110server to match the value specified with this option. 111On mismatch, a message is printed and the file will not be fetched. 112If the server does not support reporting of file sizes, the option 113will be ignored and the file will be retrieved anyway. 114This option is useful to prevent 115.Nm fetch 116from downloading a file that is either incomplete or the wrong version, 117given the correct size of the file in advance. 118.It Fl s 119Ask server for size of file in bytes and print it to stdout. Do not 120actually fetch the file. 121.It Fl T Ar seconds 122Set timeout value to 123.Ar seconds. 124Overrides the environment variables 125.Ev FTP_TIMEOUT 126for ftp transfers or 127.Ev HTTP_TIMEOUT 128for http transfers if set. 129.It Fl t 130Work around a different set of buggy 131.Tn TCP 132implementations. 133.It Fl v 134Increase verbosity. More 135.Fl v Ns \&'s 136result in more information. 137.El 138.Pp 139Many options are also controlled solely by the environment (this is a 140bug). 141.Sh URL SYNTAX 142.Nm 143accepts 144.Tn http 145and 146.Tn ftp 147URL's, as described in RFC1738. For 148.Tn ftp 149URL's, a username and password may be specified, using the syntax 150.Li ftp://user:password@host/. 151If the path is to be absolute, as opposed to relative to the user's 152home directory, it must start with %2F, as in 153.Li ftp://root:mypass@localhost/%2Fetc/passwd . 154.Sh PROXY SERVERS 155Many sites use application gateways (``proxy servers'') in their 156firewalls in order to allow communication across the firewall using a 157trusted protocol. The 158.Nm fetch 159program can use both the 160.Tn FTP 161and the 162.Tn HTTP 163protocol with a proxy server. 164.Tn FTP 165proxy servers can only relay 166.Tn FTP 167requests; 168.Tn HTTP 169proxy servers can relay both 170.Tn FTP 171and 172.Tn HTTP 173requests. 174A proxy server can be configured by defining an environment variable 175named 176.Dq Va PROTO Ns Ev _PROXY , 177where 178.Va PROTO 179is the name of the protocol in upper case. The value of the 180environment variable specifies a hostname, optionally followed by a 181colon and a port number. 182.Pp 183The 184.Tn FTP 185proxy client passes the remote username, host and port as the 186.Tn FTP 187session's username, in the form 188.Do Va remoteuser Ns Li \&@ Ns Va remotehost 189.Op Li \^@ Ns Va port 190.Dc . 191The 192.Tn HTTP 193proxy client simply passes the originally-requested URI to the remote 194server in an 195.Tn HTTP 196.Dq Li GET 197request. HTTP proxy authentication is not yet implemented. 198.Sh HTTP AUTHENTICATION 199The 200.Tn HTTP 201protocol includes support for various methods of authentication. 202Currently, the 203.Dq basic 204method, which provides no security from packet-sniffing or 205man-in-the-middle attacks, is the only method supported in 206.Nm fetch . 207Authentication is enabled by the 208.Ev HTTP_AUTH 209and 210.Ev HTTP_PROXY_AUTH 211environment variables. Both variables have the same format, which 212consists of space-separated list of parameter settings, where each 213setting consists of a colon-separated list of parameters. The first 214two parameters are always the (case-insensitive) authentication scheme 215name and the realm in which authentication is to be performed. If the 216realm is specified as 217.Sq Li \&* , 218then it will match all realms not specified otherwise. 219.Pp 220The 221.Li basic 222authentication scheme uses two additional optional parameters; the 223first is a user name, and the second is the password associated with 224it. If either the password or both parameters are not specified in 225the environment, and the standard input of 226.Nm 227is connected to a terminal, then 228.Nm 229will prompt the user to enter the missing parameters. Thus, if the 230user is known as 231.Dq Li jane 232in the 233.Dq Li WallyWorld 234realm, and has a password of 235.Dq Li QghiLx79 236there, then she might set her 237.Ev HTTP_AUTH 238variable to: 239.Bl -enum -offset indent 240.It 241.Dq Li basic:WallyWorld:jane:QghiLx79 242.It 243.Dq Li basic:WallyWorld:jane , 244or 245.It 246.Dq Li basic:WallyWorld 247.El 248.Pp 249and 250.Nm 251will prompt for the missing information if it is required. She might 252also specify a realm of 253.Dq Li \&* 254instead of 255.Dq Li WallyWorld 256to indicate that the parameters can be applied to any realm. (This is 257most commonly used in a construction such as 258.Dq Li basic:* , 259which indicates to 260.Nm 261that it may offer to do 262.Li basic 263authentication for any realm. 264.Sh ERRORS 265The 266.Nm 267command returns zero on success, or a non-zero value from 268.Aq Pa sysexits.h 269on failure. If multiple URIs are given for retrieval, 270.Nm 271will attempt all of them and return zero only if all succeeded 272(otherwise it will return the error from the last failure). 273.Sh ENVIRONMENT 274.Bl -tag -width FTP_PASSIVE_MODE -offset indent 275.It Ev FTP_TIMEOUT 276maximum time, in seconds, to wait before aborting an 277.Tn FTP 278connection. 279.It Ev FTP_LOGIN 280the login name used for 281.Tn FTP 282transfers (default 283.Dq Li anonymous ) 284.It Ev FTP_PASSIVE_MODE 285force the use of passive mode FTP 286.It Ev FTP_PASSWORD 287the password used for 288.Tn FTP 289transfers (default 290.Dq Va yourname Ns Li \&@ Ns Va yourhost ) 291.It Ev FTP_PROXY 292the address (in the form 293.Do Va hostname Ns 294.Op Li : Ns Va port 295.Dc ) 296of a proxy server which understands 297.Tn FTP 298.It Ev HTTP_AUTH 299defines authentication parameters for 300.Tn HTTP 301.It Ev HTTP_PROXY 302the address (in the form 303.Do Va hostname Ns 304.Op Li : Ns Va port 305.Dc ) 306of a proxy server which understands 307.Tn HTTP 308.It Ev HTTP_PROXY_AUTH 309defines authentication parameters for 310.Tn HTTP 311proxy servers 312.It Ev HTTP_TIMEOUT 313maximum time, in seconds, to wait before aborting an 314.Tn HTTP 315connection. 316.Sh SEE ALSO 317.Xr ftp 1 , 318.Xr tftp 1 319.Sh HISTORY 320The 321.Nm fetch 322command appeared in 323.Fx 2.1.5 . 324.Sh AUTHORS 325The original implementation of 326.Nm 327was done by 328.An Jean-Marc Zucconi . 329It was extensively re-worked for 330.Fx 2.2 331by 332.An Garrett Wollman . 333.Sh BUGS 334There are too many environment variables and command-line options. 335.Pp 336The 337.Fl a 338option is only implemented for certain kinds of 339.Tn HTTP 340failures, and no 341.Tn FTP 342failures. 343.Pp 344Only the 345.Dq basic 346authentication mode is implemented for 347.Tn HTTP . 348This should be replaced by digest authentication. 349.Pp 350Some 351.Tn TCP 352implementations (other than 353.Tn FreeBSD ) 354fail to correctly implement cases where the 355.Dv SYN 356and/or 357.Dv FIN 358control flags are specified in packets which also contain data. 359The 360.Sq Fl t 361flag works around the latter deficiency and the 362.Sq Fl b 363flag works around the former. Since these are errors of the server's 364.Tn TCP 365stack, the best we can do is provide these workarounds. Given a correct 366server, an optimal 367.Tn HTTP 368transfer without 369.Fl t 370and 371.Fl b 372involves a minimum of two round trips (for small replies), one less than 373other implementations. 374