1.\" Copyright (c) 2015 EMC / Isilon Storage Division 2.\" All rights reserved. 3.\" 4.\" Redistribution and use in source and binary forms, with or without 5.\" modification, are permitted provided that the following conditions 6.\" are met: 7.\" 1. Redistributions of source code must retain the above copyright 8.\" notice, this list of conditions and the following disclaimer. 9.\" 2. Redistributions in binary form must reproduce the above copyright 10.\" notice, this list of conditions and the following disclaimer in the 11.\" documentation and/or other materials provided with the distribution. 12.\" 13.\" THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' AND 14.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE 17.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23.\" SUCH DAMAGE. 24.\" 25.\" $FreeBSD$ 26.\" 27.Dd January 14, 2016 28.Dt IOAT 4 29.Os 30.Sh NAME 31.Nm I/OAT 32.Nd Intel I/O Acceleration Technology 33.Sh SYNOPSIS 34To compile this driver into your kernel, 35place the following line in your kernel configuration file: 36.Bd -ragged -offset indent 37.Cd "device ioat" 38.Ed 39.Pp 40Or, to load the driver as a module at boot, place the following line in 41.Xr loader.conf 5 : 42.Bd -literal -offset indent 43ioat_load="YES" 44.Ed 45.Pp 46In 47.Xr loader.conf 5 : 48.Pp 49.Cd hw.ioat.force_legacy_interrupts=0 50.Pp 51In 52.Xr loader.conf 5 or 53.Xr sysctl.conf 5 : 54.Pp 55.Cd hw.ioat.enable_ioat_test=0 56.Cd hw.ioat.debug_level=0 57(only critical errors; maximum of 3) 58.Pp 59.Ft typedef void 60.Fn (*bus_dmaengine_callback_t) "void *arg" "int error" 61.Pp 62.Ft bus_dmaengine_t 63.Fn ioat_get_dmaengine "uint32_t channel_index" 64.Ft void 65.Fn ioat_put_dmaengine "bus_dmaengine_t dmaengine" 66.Ft int 67.Fn ioat_get_hwversion "bus_dmaengine_t dmaengine" 68.Ft size_t 69.Fn ioat_get_max_io_size "bus_dmaengine_t dmaengine" 70.Ft int 71.Fn ioat_set_interrupt_coalesce "bus_dmaengine_t dmaengine" "uint16_t delay" 72.Ft uint16_t 73.Fn ioat_get_max_coalesce_period "bus_dmaengine_t dmaengine" 74.Ft void 75.Fn ioat_acquire "bus_dmaengine_t dmaengine" 76.Ft int 77.Fn ioat_acquire_reserve "bus_dmaengine_t dmaengine" "uint32_t n" "int mflags" 78.Ft void 79.Fn ioat_release "bus_dmaengine_t dmaengine" 80.Ft struct bus_dmadesc * 81.Fo ioat_copy 82.Fa "bus_dmaengine_t dmaengine" 83.Fa "bus_addr_t dst" 84.Fa "bus_addr_t src" 85.Fa "bus_size_t len" 86.Fa "bus_dmaengine_callback_t callback_fn" 87.Fa "void *callback_arg" 88.Fa "uint32_t flags" 89.Fc 90.Ft struct bus_dmadesc * 91.Fo ioat_copy_8k_aligned 92.Fa "bus_dmaengine_t dmaengine" 93.Fa "bus_addr_t dst1" 94.Fa "bus_addr_t dst2" 95.Fa "bus_addr_t src1" 96.Fa "bus_addr_t src2" 97.Fa "bus_dmaengine_callback_t callback_fn" 98.Fa "void *callback_arg" 99.Fa "uint32_t flags" 100.Fc 101.Ft struct bus_dmadesc * 102.Fo ioat_blockfill 103.Fa "bus_dmaengine_t dmaengine" 104.Fa "bus_addr_t dst" 105.Fa "uint64_t fillpattern" 106.Fa "bus_size_t len" 107.Fa "bus_dmaengine_callback_t callback_fn" 108.Fa "void *callback_arg" 109.Fa "uint32_t flags" 110.Fc 111.Ft struct bus_dmadesc * 112.Fo ioat_null 113.Fa "bus_dmaengine_t dmaengine" 114.Fa "bus_dmaengine_callback_t callback_fn" 115.Fa "void *callback_arg" 116.Fa "uint32_t flags" 117.Fc 118.Sh DESCRIPTION 119The 120.Nm 121driver provides a kernel API to a variety of DMA engines on some Intel server 122platforms. 123.Pp 124There is a number of DMA channels per CPU package. 125(Typically 4 or 8.) 126Each may be used independently. 127Operations on a single channel proceed sequentially. 128.Pp 129Blockfill operations can be used to write a 64-bit pattern to memory. 130.Pp 131Copy operations can be used to offload memory copies to the DMA engines. 132.Pp 133Null operations do nothing, but may be used to test the interrupt and callback 134mechanism. 135.Pp 136All operations can optionally trigger an interrupt at completion with the 137.Ar DMA_INT_EN 138flag. 139For example, a user might submit multiple operations to the same channel and 140only enable an interrupt and callback for the last operation. 141.Pp 142The hardware can delay and coalesce interrupts on a given channel for a 143configurable period of time, in microseconds. 144This may be desired to reduce the processing and interrupt overhead per 145descriptor, especially for workflows consisting of many small operations. 146Software can control this on a per-channel basis with the 147.Fn ioat_set_interrupt_coalesce 148API. 149The 150.Fn ioat_get_max_coalesce_period 151API can be used to determine the maximum coalescing period supported by the 152hardware, in microseconds. 153Current platforms support up to a 16.383 millisecond coalescing period. 154Optimal configuration will vary by workflow and desired operation latency. 155.Pp 156All operations are safe to use in a non-blocking context with the 157.Ar DMA_NO_WAIT 158flag. 159(Of course, allocations may fail and operations requested with 160.Ar DMA_NO_WAIT 161may return NULL.) 162.Pp 163Operations that depend on the result of prior operations should use 164.Ar DMA_FENCE . 165For example, such a scenario can happen when two related DMA operations are 166queued. 167First, a DMA copy to one location (A), followed directly by a DMA copy 168from A to B. 169In this scenario, some classes of I/OAT hardware may prefetch A for the second 170operation before it is written by the first operation. 171To avoid reading a stale value in sequences of dependent operations, use 172.Ar DMA_FENCE . 173.Pp 174All operations, as well as 175.Fn ioat_get_dmaengine , 176can return NULL in special circumstances. 177For example, if the 178.Nm 179driver is being unloaded, or the administrator has induced a hardware reset, or 180a usage error has resulted in a hardware error state that needs to be recovered 181from. 182.Pp 183It is invalid to attempt to submit new DMA operations in a 184.Fa bus_dmaengine_callback_t 185context. 186.Sh USAGE 187A typical user will lookup the DMA engine object for a given channel with 188.Fn ioat_get_dmaengine . 189When the user wants to offload a copy, they will first 190.Fn ioat_acquire 191the 192.Ar bus_dmaengine_t 193object for exclusive access to enqueue operations on that channel. 194Optionally, the user can reserve space by using 195.Fn ioat_acquire_reserve 196instead. 197If 198.Fn ioat_acquire_reserve 199succeeds, there is guaranteed to be room for 200.Fa N 201new operations in the internal ring buffer. 202Then, they will submit one or more operations using 203.Fn ioat_blockfill , 204.Fn ioat_copy , 205or 206.Fn ioat_null . 207After queuing one or more individual DMA operations, they will 208.Fn ioat_release 209the 210.Ar bus_dmaengine_t 211to drop their exclusive access to the channel. 212The routine they provided for the 213.Fa callback_fn 214argument will be invoked with the provided 215.Fa callback_arg 216when the operation is complete. 217When they are finished with the 218.Ar bus_dmaengine_t , 219the user should 220.Fn ioat_put_dmaengine . 221.Pp 222Users MUST NOT block between 223.Fn ioat_acquire 224and 225.Fn ioat_release . 226Users SHOULD NOT hold 227.Ar bus_dmaengine_t 228references for a very long time to enable fault recovery and kernel module 229unload. 230.Pp 231For an example of usage, see 232.Pa src/sys/dev/ioat/ioat_test.c . 233.Sh FILES 234.Bl -tag 235.It Pa /dev/ioat_test 236test device for 237.Xr ioatcontrol 8 238.El 239.Sh SEE ALSO 240.Xr ioatcontrol 8 241.Sh HISTORY 242The 243.Nm 244driver first appeared in 245.Fx 11.0 . 246.Sh AUTHORS 247The 248.Nm 249driver was developed by 250.An \&Jim Harris Aq Mt jimharris@FreeBSD.org , 251.An \&Carl Delsey Aq Mt carl.r.delsey@intel.com , 252and 253.An \&Conrad Meyer Aq Mt cem@FreeBSD.org . 254This manual page was written by 255.An \&Conrad Meyer Aq Mt cem@FreeBSD.org . 256.Sh CAVEATS 257Copy operation takes bus addresses as parameters, not virtual addresses. 258.Pp 259Buffers for individual copy operations must be physically contiguous. 260.Pp 261Copies larger than max transfer size (1MB, but may vary by hardware) are not 262supported. 263Future versions will likely support this by breaking up the transfer into 264smaller sizes. 265.Sh BUGS 266The 267.Nm 268driver only supports blockfill, copy, and null operations at this time. 269The driver does not yet support advanced DMA modes, such as XOR, that some 270I/OAT devices support. 271