1*f4330e61SAlexander Motin.\" 2*f4330e61SAlexander Motin.\" Copyright (c) 2019 Rajesh Kumar <rajesh1.kumar@amd.com> 3*f4330e61SAlexander Motin.\" All rights reserved. 4*f4330e61SAlexander Motin.\" 5*f4330e61SAlexander Motin.\" Redistribution and use in source and binary forms, with or without 6*f4330e61SAlexander Motin.\" modification, are permitted provided that the following conditions 7*f4330e61SAlexander Motin.\" are met: 8*f4330e61SAlexander Motin.\" 1. Redistributions of source code must retain the above copyright 9*f4330e61SAlexander Motin.\" notice, this list of conditions and the following disclaimer. 10*f4330e61SAlexander Motin.\" 2. Redistributions in binary form must reproduce the above copyright 11*f4330e61SAlexander Motin.\" notice, this list of conditions and the following disclaimer in the 12*f4330e61SAlexander Motin.\" documentation and/or other materials provided with the distribution. 13*f4330e61SAlexander Motin.\" 14*f4330e61SAlexander Motin.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15*f4330e61SAlexander Motin.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16*f4330e61SAlexander Motin.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17*f4330e61SAlexander Motin.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18*f4330e61SAlexander Motin.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19*f4330e61SAlexander Motin.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20*f4330e61SAlexander Motin.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21*f4330e61SAlexander Motin.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22*f4330e61SAlexander Motin.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23*f4330e61SAlexander Motin.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24*f4330e61SAlexander Motin.\" SUCH DAMAGE. 25*f4330e61SAlexander Motin.\" 26*f4330e61SAlexander Motin.Dd August 29, 2019 27*f4330e61SAlexander Motin.Dt NTB_HW_AMD 4 28*f4330e61SAlexander Motin.Os 29*f4330e61SAlexander Motin.Sh NAME 30*f4330e61SAlexander Motin.Nm ntb_hw_amd 31*f4330e61SAlexander Motin.Nd AMD Non-Transparent Bridge driver 32*f4330e61SAlexander Motin.Sh SYNOPSIS 33*f4330e61SAlexander MotinTo compile this driver into your kernel, 34*f4330e61SAlexander Motinplace the following lines in your kernel configuration file: 35*f4330e61SAlexander Motin.Bd -ragged -offset indent 36*f4330e61SAlexander Motin.Cd "device ntb" 37*f4330e61SAlexander Motin.Cd "device ntb_hw_amd" 38*f4330e61SAlexander Motin.Ed 39*f4330e61SAlexander Motin.Pp 40*f4330e61SAlexander MotinOr, to load the driver as a module at boot, place the following line in 41*f4330e61SAlexander Motin.Xr loader.conf 5 : 42*f4330e61SAlexander Motin.Bd -literal -offset indent 43*f4330e61SAlexander Motinntb_hw_amd_load="YES" 44*f4330e61SAlexander Motin.Ed 45*f4330e61SAlexander Motin.Pp 46*f4330e61SAlexander MotinThe following sysctls are supported in this driver 47*f4330e61SAlexander Motin.Bl -ohang 48*f4330e61SAlexander Motin.It Va dev.ntb_hw.X.info 49*f4330e61SAlexander MotinReading this sysctl will give the basic details like the number of memory 50*f4330e61SAlexander Motinwindows, scratchpads and doorbells exposed by the NTB on the local host to 51*f4330e61SAlexander Motinaccess the devices beyond the bridge. 52*f4330e61SAlexander MotinIt also provides details about the masked doorbells, translation address and 53*f4330e61SAlexander Motinsize limit of each exposed memory window and link status information. 54*f4330e61SAlexander Motin.El 55*f4330e61SAlexander Motin.Sh DESCRIPTION 56*f4330e61SAlexander MotinThe 57*f4330e61SAlexander Motin.Nm ntb_hw_amd 58*f4330e61SAlexander Motindriver provides support for the Non-Transparent Bridge (NTB) hardware in 59*f4330e61SAlexander MotinAMD EPYC processor family. 60*f4330e61SAlexander MotinThe Non-Transparent Bridge does not look as a regular PCI bridge, but as PCI 61*f4330e61SAlexander Motinendpoint device, hiding the devices behind it. 62*f4330e61SAlexander MotinThe driver hides details of hardware on the other side, but exposes memory windows, 63*f4330e61SAlexander Motinscratchpads and doorbells to access the other side via hardware independent 64*f4330e61SAlexander MotinKPI to 65*f4330e61SAlexander Motin.Xr ntb 4 66*f4330e61SAlexander Motinsubsystem. 67*f4330e61SAlexander Motin.Pp 68*f4330e61SAlexander MotinThe hardware provides 2 (both 64-bit) or 3 (one 32-bit and two 64-bit) memory 69*f4330e61SAlexander Motinwindows to the other system's memory, up to 16 scratchpad registers and 16 70*f4330e61SAlexander Motindoorbells to communicate with and interrupt the other system respectively. 71*f4330e61SAlexander Motin.Sh CONFIGURATION 72*f4330e61SAlexander MotinThe NTB configuration should be set by BIOS. 73*f4330e61SAlexander MotinThis includes enabling NTB, choosing topology (only NTB-to-Root Port 74*f4330e61SAlexander Motinmode is supported now), role of the host in the topology. 75*f4330e61SAlexander MotinThis needs to be done on both systems. 76*f4330e61SAlexander Motin.Pp 77*f4330e61SAlexander MotinThe BAR size for memory windows is configured to 1 MiB by default. 78*f4330e61SAlexander Motin.Sh SEE ALSO 79*f4330e61SAlexander Motin.Xr if_ntb 4 , 80*f4330e61SAlexander Motin.Xr ntb 4 , 81*f4330e61SAlexander Motin.Xr ntb_transport 4 82*f4330e61SAlexander Motin.Sh AUTHORS 83*f4330e61SAlexander Motin.An -nosplit 84*f4330e61SAlexander MotinThe 85*f4330e61SAlexander Motin.Nm 86*f4330e61SAlexander Motindriver was developed by AMD and originally written by 87*f4330e61SAlexander Motin.An Rajesh Kumar Aq Mt rajesh1.kumar@amd.com . 88*f4330e61SAlexander MotinReviewed by 89*f4330e61SAlexander Motin.An Alexander Motin Aq Mt mav@FreeBSD.org , 90*f4330e61SAlexander Motin.An Conrad E. Meyer Aq Mt cem@FreeBSD.org 91*f4330e61SAlexander Motinand 92*f4330e61SAlexander Motin.An Warner Losh Aq Mt imp@FreeBSD.org . 93