1*ed9652daSAlexander Motin.\" 2*ed9652daSAlexander Motin.\" Copyright (c) 2017 Alexander Motin <mav@FreeBSD.org> 3*ed9652daSAlexander Motin.\" All rights reserved. 4*ed9652daSAlexander Motin.\" 5*ed9652daSAlexander Motin.\" Redistribution and use in source and binary forms, with or without 6*ed9652daSAlexander Motin.\" modification, are permitted provided that the following conditions 7*ed9652daSAlexander Motin.\" are met: 8*ed9652daSAlexander Motin.\" 1. Redistributions of source code must retain the above copyright 9*ed9652daSAlexander Motin.\" notice, this list of conditions and the following disclaimer. 10*ed9652daSAlexander Motin.\" 2. Redistributions in binary form must reproduce the above copyright 11*ed9652daSAlexander Motin.\" notice, this list of conditions and the following disclaimer in the 12*ed9652daSAlexander Motin.\" documentation and/or other materials provided with the distribution. 13*ed9652daSAlexander Motin.\" 14*ed9652daSAlexander Motin.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15*ed9652daSAlexander Motin.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16*ed9652daSAlexander Motin.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17*ed9652daSAlexander Motin.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18*ed9652daSAlexander Motin.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19*ed9652daSAlexander Motin.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20*ed9652daSAlexander Motin.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21*ed9652daSAlexander Motin.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22*ed9652daSAlexander Motin.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23*ed9652daSAlexander Motin.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24*ed9652daSAlexander Motin.\" SUCH DAMAGE. 25*ed9652daSAlexander Motin.\" 26*ed9652daSAlexander Motin.\" $FreeBSD$ 27*ed9652daSAlexander Motin.\" 28*ed9652daSAlexander Motin.Dd August 30, 2017 29*ed9652daSAlexander Motin.Dt NTB_HW_PLX 4 30*ed9652daSAlexander Motin.Os 31*ed9652daSAlexander Motin.Sh NAME 32*ed9652daSAlexander Motin.Nm ntb_hw_plx 33*ed9652daSAlexander Motin.Nd PLX/Avago/Broadcom Non-Transparent Bridge driver 34*ed9652daSAlexander Motin.Sh SYNOPSIS 35*ed9652daSAlexander MotinTo compile this driver into your kernel, 36*ed9652daSAlexander Motinplace the following lines in your kernel configuration file: 37*ed9652daSAlexander Motin.Bd -ragged -offset indent 38*ed9652daSAlexander Motin.Cd "device ntb" 39*ed9652daSAlexander Motin.Cd "device ntb_hw_plx" 40*ed9652daSAlexander Motin.Ed 41*ed9652daSAlexander Motin.Pp 42*ed9652daSAlexander MotinOr, to load the driver as a module at boot, place the following line in 43*ed9652daSAlexander Motin.Xr loader.conf 5 : 44*ed9652daSAlexander Motin.Bd -literal -offset indent 45*ed9652daSAlexander Motinntb_hw_plx_load="YES" 46*ed9652daSAlexander Motin.Ed 47*ed9652daSAlexander Motin.Pp 48*ed9652daSAlexander MotinThe following tunables are settable from the 49*ed9652daSAlexander Motin.Xr loader 8 : 50*ed9652daSAlexander Motin.Bl -ohang 51*ed9652daSAlexander Motin.It Va hint.ntb_hw. Ns Ar X Ns Va .b2b 52*ed9652daSAlexander MotinBeing set to 1 (default) tells the driver attached to Virtual Interface of the 53*ed9652daSAlexander MotinNTB that it works in NTB-to-NTB (back-to-back) mode, 0 -- NTB-to-Root Port. 54*ed9652daSAlexander MotinDriver attached to Link Interface (visible from Root Port side) switches to 55*ed9652daSAlexander MotinNTB-to-Root Port mode automatically, but one attached to Virtual Interface 56*ed9652daSAlexander Motincan't detect what is on the other side and require external knowledge. 57*ed9652daSAlexander Motin.El 58*ed9652daSAlexander Motin.Sh DESCRIPTION 59*ed9652daSAlexander MotinThe 60*ed9652daSAlexander Motin.Nm 61*ed9652daSAlexander Motindriver provides support for the Non-Transparent Bridge (NTB) hardware in 62*ed9652daSAlexander MotinPLX PCIe bridge chips, which allow up to two of their PCIe ports to be 63*ed9652daSAlexander Motinswitched from transparent to non-transparent bridge mode. 64*ed9652daSAlexander MotinIn this mode bridge looks not as a PCI bridge, but as PCI endpoint device. 65*ed9652daSAlexander MotinThe driver hides hardware details, exposing memory windows, scratchpads 66*ed9652daSAlexander Motinand doorbells of the other side via hardware independent KPI to 67*ed9652daSAlexander Motin.Xr ntb 4 68*ed9652daSAlexander Motinsubsystem. 69*ed9652daSAlexander Motin.Pp 70*ed9652daSAlexander MotinEach PLX NTB provides up to 2 64-bit or 4 32-bit memory windows to the 71*ed9652daSAlexander Motinother system's memory, 6 or 12 scratchpad registers and 16 doorbells to 72*ed9652daSAlexander Motininterrupt the other system. 73*ed9652daSAlexander MotinIn NTB-to-NTB mode one of memory windows (or half of it, if bigger then 1MB) 74*ed9652daSAlexander Motinis consumed by the driver itself to access scratchpad and doorbell registers 75*ed9652daSAlexander Motinof the other side. 76*ed9652daSAlexander Motin.Sh HARDWARE 77*ed9652daSAlexander MotinThe following PLX/Avago/Broadcom chips are supported by the 78*ed9652daSAlexander Motin.Nm 79*ed9652daSAlexander Motindriver: 80*ed9652daSAlexander Motin.Pp 81*ed9652daSAlexander Motin.Bl -bullet -compact 82*ed9652daSAlexander Motin.It 83*ed9652daSAlexander MotinPEX 8713 84*ed9652daSAlexander Motin.It 85*ed9652daSAlexander MotinPEX 8717 86*ed9652daSAlexander Motin.It 87*ed9652daSAlexander MotinPEX 8725 88*ed9652daSAlexander Motin.It 89*ed9652daSAlexander MotinPEX 8733 90*ed9652daSAlexander Motin.It 91*ed9652daSAlexander MotinPEX 8749 92*ed9652daSAlexander Motin.El 93*ed9652daSAlexander Motin.Pp 94*ed9652daSAlexander Motin, but it may also work with other compatible ones. 95*ed9652daSAlexander Motin.Sh CONFIGURATION 96*ed9652daSAlexander MotinThe basic chip configuration should be done by serial EEPROM or via i2c. 97*ed9652daSAlexander MotinIt includes enabling NTB on one or both sides (choosing between NTB-to-NTB 98*ed9652daSAlexander Motin(back-to-back) and NTB-to-Root Port modes) and configuring BARs sizes. 99*ed9652daSAlexander Motin.Pp 100*ed9652daSAlexander MotinThe recommended mode is NTB-to-NTB mode, since while NTB-to-Root Port is 101*ed9652daSAlexander Motingenerally supported by the driver, it require PCI hotplug handling on the 102*ed9652daSAlexander MotinRoot Port, that may be difficult or cause different kinds of problems. 103*ed9652daSAlexander Motin.Sh SEE ALSO 104*ed9652daSAlexander Motin.Xr if_ntb 4 , 105*ed9652daSAlexander Motin.Xr ntb_transport 4 , 106*ed9652daSAlexander Motin.Xr ntb 4 , 107*ed9652daSAlexander Motin.Sh AUTHORS 108*ed9652daSAlexander Motin.An -nosplit 109*ed9652daSAlexander MotinThe 110*ed9652daSAlexander Motin.Nm 111*ed9652daSAlexander Motindriver was written by 112*ed9652daSAlexander Motin.An Alexander Motin Aq Mt mav@FreeBSD.org . 113*ed9652daSAlexander Motin.Sh BUGS 114*ed9652daSAlexander MotinThere is no way to protect your system from malicious behavior on the other 115*ed9652daSAlexander Motinsystem once the link is brought up. 116*ed9652daSAlexander MotinAnyone with root or kernel access on the other system can read or write to 117*ed9652daSAlexander Motinany location on your system. 118*ed9652daSAlexander MotinIn other words, only connect two systems that completely trust each other. 119