1.\"- 2.\" Copyright (c) 2000-2014 Dag-Erling Smørgrav 3.\" Copyright (c) 2013 Michael Gmelin <freebsd@grem.de> 4.\" All rights reserved. 5.\" Portions Copyright (c) 1999 Massachusetts Institute of Technology; used 6.\" by permission. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer 13.\" in this position and unchanged. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 3. The name of the author may not be used to endorse or promote products 18.\" derived from this software without specific prior written permission. 19.\" 20.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR 21.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 22.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 23.\" IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, 24.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 25.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 26.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 27.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 28.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 29.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30.\" 31.\" $FreeBSD$ 32.\" 33.Dd January 28, 2014 34.Dt FETCH 1 35.Os 36.Sh NAME 37.Nm fetch 38.Nd retrieve a file by Uniform Resource Locator 39.Sh SYNOPSIS 40.Nm 41.Op Fl 146AadFlMmnPpqRrsUv 42.Op Fl -allow-sslv2 43.Op Fl B Ar bytes 44.Op Fl -bind-address= Ns Ar host 45.Op Fl -ca-cert= Ns Ar file 46.Op Fl -ca-path= Ns Ar dir 47.Op Fl -cert= Ns Ar file 48.Op Fl -crl= Ns Ar file 49.Op Fl i Ar file 50.Op Fl -key= Ns Ar file 51.Op Fl N Ar file 52.Op Fl -no-passive 53.Op Fl -no-proxy= Ns Ar list 54.Op Fl -no-sslv3 55.Op Fl -no-tlsv1 56.Op Fl -no-verify-hostname 57.Op Fl -no-verify-peer 58.Op Fl o Ar file 59.Op Fl -referer= Ns Ar URL 60.Op Fl S Ar bytes 61.Op Fl T Ar seconds 62.Op Fl -user-agent= Ns Ar agent-string 63.Op Fl w Ar seconds 64.Ar URL ... 65.Nm 66.Op Fl 146AadFlMmnPpqRrsUv 67.Op Fl B Ar bytes 68.Op Fl -bind-address= Ns Ar host 69.Op Fl -ca-cert= Ns Ar file 70.Op Fl -ca-path= Ns Ar dir 71.Op Fl -cert= Ns Ar file 72.Op Fl -crl= Ns Ar file 73.Op Fl i Ar file 74.Op Fl -key= Ns Ar file 75.Op Fl N Ar file 76.Op Fl -no-passive 77.Op Fl -no-proxy= Ns Ar list 78.Op Fl -no-sslv3 79.Op Fl -no-tlsv1 80.Op Fl -no-verify-hostname 81.Op Fl -no-verify-peer 82.Op Fl o Ar file 83.Op Fl -referer= Ns Ar URL 84.Op Fl S Ar bytes 85.Op Fl T Ar seconds 86.Op Fl -user-agent= Ns Ar agent-string 87.Op Fl w Ar seconds 88.Fl h Ar host Fl f Ar file Oo Fl c Ar dir Oc 89.Sh DESCRIPTION 90The 91.Nm 92utility provides a command-line interface to the 93.Xr fetch 3 94library. 95Its purpose is to retrieve the file(s) pointed to by the URL(s) on the 96command line. 97.Pp 98The following options are available: 99.Bl -tag -width Fl 100.It Fl 1 , -one-file 101Stop and return exit code 0 at the first successfully retrieved file. 102.It Fl 4 , -ipv4-only 103Forces 104.Nm 105to use IPv4 addresses only. 106.It Fl 6 , -ipv6-only 107Forces 108.Nm 109to use IPv6 addresses only. 110.It Fl A , -no-redirect 111Do not automatically follow ``temporary'' (302) redirects. 112Some broken Web sites will return a redirect instead of a not-found 113error when the requested object does not exist. 114.It Fl a , -retry 115Automatically retry the transfer upon soft failures. 116.It Fl -allow-sslv2 117[SSL] 118Allow SSL version 2 when negotiating the connection. 119.It Fl B Ar bytes , Fl -buffer-size= Ns Ar bytes 120Specify the read buffer size in bytes. 121The default is 16,384 bytes. 122Attempts to set a buffer size lower than this will be silently 123ignored. 124The number of reads actually performed is reported at verbosity level 125two or higher (see the 126.Fl v 127flag). 128.It Fl -bind-address= Ns Ar host 129Specifies a hostname or IP address to which sockets used for outgoing 130connections will be bound. 131.It Fl c Ar dir 132The file to retrieve is in directory 133.Ar dir 134on the remote host. 135This option is deprecated and is provided for backward compatibility 136only. 137.It Fl -ca-cert= Ns Ar file 138[SSL] 139Path to certificate bundle containing trusted CA certificates. 140If not specified, 141.Pa /etc/ssl/cert.pem 142is used. 143The file may contain multiple CA certificates. The port 144.Pa security/ca_root_nss 145is a common source of a current CA bundle. 146.It Fl -ca-path= Ns Ar dir 147[SSL] 148The directory 149.Ar dir 150contains trusted CA hashes. 151.It Fl -cert= Ns Ar file 152[SSL] 153.Ar file 154is a PEM encoded client certificate/key which will be used in 155client certificate authentication. 156.It Fl -crl= Ns Ar file 157[SSL] 158Points to certificate revocation list 159.Ar file , 160which has to be in PEM format and may contain peer certificates that have 161been revoked. 162.It Fl d , -direct 163Use a direct connection even if a proxy is configured. 164.It Fl F , -force-restart 165In combination with the 166.Fl r 167flag, forces a restart even if the local and remote files have 168different modification times. 169Implies 170.Fl R . 171.It Fl f Ar file 172The file to retrieve is named 173.Ar file 174on the remote host. 175This option is deprecated and is provided for backward compatibility 176only. 177.It Fl h Ar host 178The file to retrieve is located on the host 179.Ar host . 180This option is deprecated and is provided for backward compatibility 181only. 182.It Fl i Ar file , Fl -if-modified-since= Ns Ar file 183If-Modified-Since mode: the remote file will only be retrieved if it 184is newer than 185.Ar file 186on the local host. 187(HTTP only) 188.It Fl -key= Ns Ar file 189[SSL] 190.Ar file 191is a PEM encoded client key that will be used in client certificate 192authentication in case key and client certificate are stored separately. 193.It Fl l , -symlink 194If the target is a file-scheme URL, make a symbolic link to the target 195rather than trying to copy it. 196.It Fl M 197.It Fl m , -mirror 198Mirror mode: if the file already exists locally and has the same size 199and modification time as the remote file, it will not be fetched. 200Note that the 201.Fl m 202and 203.Fl r 204flags are mutually exclusive. 205.It Fl N Ar file , Fl -netrc= Ns Ar file 206Use 207.Ar file 208instead of 209.Pa ~/.netrc 210to look up login names and passwords for FTP sites. 211See 212.Xr ftp 1 213for a description of the file format. 214This feature is experimental. 215.It Fl n , -no-mtime 216Do not preserve the modification time of the transferred file. 217.It Fl -no-passive 218Forces the FTP code to use active mode. 219.It Fl -no-proxy= Ns Ar list 220Either a single asterisk, which disables the use of proxies 221altogether, or a comma- or whitespace-separated list of hosts for 222which proxies should not be used. 223.It Fl -no-sslv3 224[SSL] 225Don't allow SSL version 3 when negotiating the connection. 226.It Fl -no-tlsv1 227[SSL] 228Don't allow TLS version 1 when negotiating the connection. 229.It Fl -no-verify-hostname 230[SSL] 231Do not verify that the hostname matches the subject of the 232certificate presented by the server. 233.It Fl -no-verify-peer 234[SSL] 235Do not verify the peer certificate against trusted CAs. 236.It Fl o Ar file , Fl output= Ns Ar file 237Set the output file name to 238.Ar file . 239By default, a ``pathname'' is extracted from the specified URI, and 240its basename is used as the name of the output file. 241A 242.Ar file 243argument of 244.Sq Li \&- 245indicates that results are to be directed to the standard output. 246If the 247.Ar file 248argument is a directory, fetched file(s) will be placed within the 249directory, with name(s) selected as in the default behaviour. 250.It Fl P 251.It Fl p , -passive 252Use passive FTP. 253These flags have no effect, since passive FTP is the default, but are 254provided for compatibility with earlier versions where active FTP was 255the default. 256To force active mode, use the 257.Fl -no-passive 258flag or set the 259.Ev FTP_PASSIVE_MODE 260environment variable to 261.Ql NO . 262.It Fl -referer= Ns Ar URL 263Specifies the referrer URL to use for HTTP requests. 264If 265.Ar URL 266is set to 267.Dq auto , 268the document URL will be used as referrer URL. 269.It Fl q , -quiet 270Quiet mode. 271.It Fl R , -keep-output 272The output files are precious, and should not be deleted under any 273circumstances, even if the transfer failed or was incomplete. 274.It Fl r , -restart 275Restart a previously interrupted transfer. 276Note that the 277.Fl m 278and 279.Fl r 280flags are mutually exclusive. 281.It Fl S Ar bytes , Fl -require-size= Ns Ar bytes 282Require the file size reported by the server to match the specified 283value. 284If it does not, a message is printed and the file is not fetched. 285If the server does not support reporting file sizes, this option is 286ignored and the file is fetched unconditionally. 287.It Fl s , -print-size 288Print the size in bytes of each requested file, without fetching it. 289.It Fl T Ar seconds , Fl -timeout= Ns Ar seconds 290Set timeout value to 291.Ar seconds . 292Overrides the environment variables 293.Ev FTP_TIMEOUT 294for FTP transfers or 295.Ev HTTP_TIMEOUT 296for HTTP transfers if set. 297.It Fl U , -passive-portrange-default 298When using passive FTP, allocate the port for the data connection from 299the low (default) port range. 300See 301.Xr ip 4 302for details on how to specify which port range this corresponds to. 303.It Fl -user-agent= Ns Ar agent-string 304Specifies the User-Agent string to use for HTTP requests. 305This can be useful when working with HTTP origin or proxy servers that 306differentiate between user agents. 307.It Fl v , -verbose 308Increase verbosity level. 309.It Fl w Ar seconds , Fl -retry-delay= Ns Ar seconds 310When the 311.Fl a 312flag is specified, wait this many seconds between successive retries. 313.El 314.Pp 315If 316.Nm 317receives a 318.Dv SIGINFO 319signal (see the 320.Cm status 321argument for 322.Xr stty 1 ) , 323the current transfer rate statistics will be written to the 324standard error output, in the same format as the standard completion 325message. 326.Sh ENVIRONMENT 327.Bl -tag -width HTTP_TIMEOUT 328.It Ev FTP_TIMEOUT 329Maximum time, in seconds, to wait before aborting an FTP connection. 330.It Ev HTTP_TIMEOUT 331Maximum time, in seconds, to wait before aborting an HTTP connection. 332.El 333.Pp 334See 335.Xr fetch 3 336for a description of additional environment variables, including 337.Ev FETCH_BIND_ADDRESS , 338.Ev FTP_LOGIN , 339.Ev FTP_PASSIVE_MODE , 340.Ev FTP_PASSWORD , 341.Ev FTP_PROXY , 342.Ev ftp_proxy , 343.Ev HTTP_ACCEPT , 344.Ev HTTP_AUTH , 345.Ev HTTP_PROXY , 346.Ev http_proxy , 347.Ev HTTP_PROXY_AUTH , 348.Ev HTTP_REFERER , 349.Ev HTTP_USER_AGENT , 350.Ev NETRC , 351.Ev NO_PROXY , 352.Ev no_proxy , 353.Ev SSL_ALLOW_SSL2 , 354.Ev SSL_CA_CERT_FILE , 355.Ev SSL_CA_CERT_PATH , 356.Ev SSL_CLIENT_CERT_FILE , 357.Ev SSL_CLIENT_KEY_FILE , 358.Ev SSL_CRL_FILE , 359.Ev SSL_NO_SSL3 , 360.Ev SSL_NO_TLS1 , 361.Ev SSL_NO_VERIFY_HOSTNAME 362and 363.Ev SSL_NO_VERIFY_PEER . 364.Sh EXIT STATUS 365The 366.Nm 367command returns zero on success, or one on failure. 368If multiple URLs are listed on the command line, 369.Nm 370will attempt to retrieve each one of them in turn, and will return 371zero only if they were all successfully retrieved. 372.Pp 373If the 374.Fl i 375argument is used and the remote file is not newer than the 376specified file then the command will still return success, 377although no file is transferred. 378.Sh SEE ALSO 379.Xr fetch 3 380.Sh HISTORY 381The 382.Nm 383command appeared in 384.Fx 2.1.5 . 385This implementation first appeared in 386.Fx 4.1 . 387.Sh AUTHORS 388.An -nosplit 389The original implementation of 390.Nm 391was done by 392.An Jean-Marc Zucconi Aq Mt jmz@FreeBSD.org . 393It was extensively re-worked for 394.Fx 2.2 395by 396.An Garrett Wollman Aq Mt wollman@FreeBSD.org , 397and later completely rewritten to use the 398.Xr fetch 3 399library by 400.An Dag-Erling Sm\(/orgrav Aq Mt des@FreeBSD.org 401and 402.An Michael Gmelin Aq Mt freebsd@grem.de . 403.Sh NOTES 404The 405.Fl b 406and 407.Fl t 408options are no longer supported and will generate warnings. 409They were workarounds for bugs in other OSes which this implementation 410does not trigger. 411.Pp 412One cannot both use the 413.Fl h , 414.Fl c 415and 416.Fl f 417options and specify URLs on the command line. 418