1*ed9652daSAlexander Motin.\" 2*ed9652daSAlexander Motin.\" Copyright (c) 2016-2017 Alexander Motin <mav@FreeBSD.org> 3*ed9652daSAlexander Motin.\" All rights reserved. 4*ed9652daSAlexander Motin.\" 5*ed9652daSAlexander Motin.\" Redistribution and use in source and binary forms, with or without 6*ed9652daSAlexander Motin.\" modification, are permitted provided that the following conditions 7*ed9652daSAlexander Motin.\" are met: 8*ed9652daSAlexander Motin.\" 1. Redistributions of source code must retain the above copyright 9*ed9652daSAlexander Motin.\" notice, this list of conditions and the following disclaimer. 10*ed9652daSAlexander Motin.\" 2. Redistributions in binary form must reproduce the above copyright 11*ed9652daSAlexander Motin.\" notice, this list of conditions and the following disclaimer in the 12*ed9652daSAlexander Motin.\" documentation and/or other materials provided with the distribution. 13*ed9652daSAlexander Motin.\" 14*ed9652daSAlexander Motin.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15*ed9652daSAlexander Motin.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16*ed9652daSAlexander Motin.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17*ed9652daSAlexander Motin.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18*ed9652daSAlexander Motin.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19*ed9652daSAlexander Motin.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20*ed9652daSAlexander Motin.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21*ed9652daSAlexander Motin.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22*ed9652daSAlexander Motin.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23*ed9652daSAlexander Motin.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24*ed9652daSAlexander Motin.\" SUCH DAMAGE. 25*ed9652daSAlexander Motin.\" 26*ed9652daSAlexander Motin.\" $FreeBSD$ 27*ed9652daSAlexander Motin.\" 28*ed9652daSAlexander Motin.Dd August 30, 2017 29*ed9652daSAlexander Motin.Dt NTB_HW_INTEL 4 30*ed9652daSAlexander Motin.Os 31*ed9652daSAlexander Motin.Sh NAME 32*ed9652daSAlexander Motin.Nm ntb_hw_intel 33*ed9652daSAlexander Motin.Nd Intel(R) Non-Transparent Bridge driver 34*ed9652daSAlexander Motin.Sh SYNOPSIS 35*ed9652daSAlexander MotinTo compile this driver into your kernel, 36*ed9652daSAlexander Motinplace the following lines in your kernel configuration file: 37*ed9652daSAlexander Motin.Bd -ragged -offset indent 38*ed9652daSAlexander Motin.Cd "device ntb" 39*ed9652daSAlexander Motin.Cd "device ntb_hw_intel" 40*ed9652daSAlexander Motin.Ed 41*ed9652daSAlexander Motin.Pp 42*ed9652daSAlexander MotinOr, to load the driver as a module at boot, place the following line in 43*ed9652daSAlexander Motin.Xr loader.conf 5 : 44*ed9652daSAlexander Motin.Bd -literal -offset indent 45*ed9652daSAlexander Motinntb_hw_intel_load="YES" 46*ed9652daSAlexander Motin.Ed 47*ed9652daSAlexander Motin.Sh DESCRIPTION 48*ed9652daSAlexander MotinThe 49*ed9652daSAlexander Motin.Nm ntb_hw_intel 50*ed9652daSAlexander Motindriver provides support for the Non-Transparent Bridge (NTB) hardware in 51*ed9652daSAlexander MotinIntel Xeon E3/E5 and S1200 processor families, which allow one of their PCIe 52*ed9652daSAlexander Motinports to be switched from transparent to non-transparent bridge mode. 53*ed9652daSAlexander MotinIn this mode bridge looks not as a PCI bridge, but as PCI endpoint device. 54*ed9652daSAlexander MotinThe driver hides hardware details, exposing memory windows, scratchpads 55*ed9652daSAlexander Motinand doorbells of the other side via hardware independent KPI to 56*ed9652daSAlexander Motin.Xr ntb 4 57*ed9652daSAlexander Motinsubsystem. 58*ed9652daSAlexander Motin.Pp 59*ed9652daSAlexander MotinThe hardware provides 2 or 3 memory windows to the other system's memory, 60*ed9652daSAlexander Motin16 scratchpad registers and 14 or 34 doorbells to interrupt the other system. 61*ed9652daSAlexander MotinOn Xeon processors one of memory windows is typically consumed by the driver 62*ed9652daSAlexander Motinitself to workaround multiple hardware erratas. 63*ed9652daSAlexander Motin.Sh CONFIGURATION 64*ed9652daSAlexander MotinThe NTB configuration should be set by BIOS. 65*ed9652daSAlexander MotinIt includes enabling NTB, choosing between NTB-to-NTB (back-to-back) or 66*ed9652daSAlexander MotinNTB-to-Root Port mode, 67*ed9652daSAlexander Motinenabling split BAR mode (one of two 64-bit BARs can be split into two 32-bit 68*ed9652daSAlexander Motinones) and configuring BAR sizes in bits (from 12 to 29/39) for both NTB sides. 69*ed9652daSAlexander Motin.Pp 70*ed9652daSAlexander MotinThe recommended configuration is NTB-to-NTB mode, split bar is enabled and 71*ed9652daSAlexander Motinall BAR sizes are set to 20 (1 MiB). 72*ed9652daSAlexander MotinThis needs to be done on both systems. 73*ed9652daSAlexander Motin.Sh SEE ALSO 74*ed9652daSAlexander Motin.Xr if_ntb 4 , 75*ed9652daSAlexander Motin.Xr ntb_transport 4 , 76*ed9652daSAlexander Motin.Xr ntb 4 , 77*ed9652daSAlexander Motin.Sh AUTHORS 78*ed9652daSAlexander Motin.An -nosplit 79*ed9652daSAlexander MotinThe 80*ed9652daSAlexander Motin.Nm 81*ed9652daSAlexander Motindriver was developed by Intel and originally written by 82*ed9652daSAlexander Motin.An Carl Delsey Aq Mt carl@FreeBSD.org . 83*ed9652daSAlexander MotinLater improvements were done by 84*ed9652daSAlexander Motin.An Conrad E. Meyer Aq Mt cem@FreeBSD.org 85*ed9652daSAlexander Motinand 86*ed9652daSAlexander Motin.An Alexander Motin Aq Mt mav@FreeBSD.org . 87*ed9652daSAlexander Motin.Sh BUGS 88*ed9652daSAlexander MotinNTB-to-Root Port mode is not yet supported, but it doesn't look very useful. 89*ed9652daSAlexander Motin.Pp 90*ed9652daSAlexander MotinOn Xeon v2/v3/v4 processors split BAR mode should be enabled to allow 91*ed9652daSAlexander MotinSB01BASE_LOCKUP errata workaround to be applied by the driver. 92*ed9652daSAlexander Motin.Pp 93*ed9652daSAlexander MotinThere is no way to protect your system from malicious behavior on the other 94*ed9652daSAlexander Motinsystem once the link is brought up. 95*ed9652daSAlexander MotinAnyone with root or kernel access on the other system can read or write to 96*ed9652daSAlexander Motinany location on your system. 97*ed9652daSAlexander MotinIn other words, only connect two systems that completely trust each other. 98