1.\" 2.\" Copyright (c) 2016-2017 Alexander Motin <mav@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.\" $FreeBSD$ 27.\" 28.Dd October 11, 2020 29.Dt NTB_HW_INTEL 4 30.Os 31.Sh NAME 32.Nm ntb_hw_intel 33.Nd Intel(R) Non-Transparent Bridge driver 34.Sh SYNOPSIS 35To compile this driver into your kernel, 36place the following lines in your kernel configuration file: 37.Bd -ragged -offset indent 38.Cd "device ntb" 39.Cd "device ntb_hw_intel" 40.Ed 41.Pp 42Or, to load the driver as a module at boot, place the following line in 43.Xr loader.conf 5 : 44.Bd -literal -offset indent 45ntb_hw_intel_load="YES" 46.Ed 47.Sh DESCRIPTION 48The 49.Nm ntb_hw_intel 50driver provides support for the Non-Transparent Bridge (NTB) hardware in 51Intel Xeon E3/E5 and S1200 processor families, which allow one of their PCIe 52ports to be switched from transparent to non-transparent bridge mode. 53In this mode the bridge looks not like a PCI bridge, but like a PCI endpoint 54device. 55The driver hides hardware details, exposing memory windows, scratchpads 56and doorbells of the other side via a hardware independent KPI to the 57.Xr ntb 4 58subsystem. 59.Pp 60The hardware provides 2 or 3 memory windows to the other system's memory, 6116 scratchpad registers and 14, 31 or 34 doorbells to interrupt the other 62system, depending on the platform. 63On Xeon processors one of the memory windows is typically consumed by the driver 64itself to work around multiple hardware errata. 65.Sh CONFIGURATION 66The NTB configuration should be set by BIOS. 67It includes enabling NTB, choosing between NTB-to-NTB (back-to-back) or 68NTB-to-Root Port mode, 69enabling split BAR mode (one of two 64-bit BARs can be split into two 32-bit 70ones) and configuring BAR sizes in bits (from 12 to 29/39) for both NTB sides. 71.Pp 72The recommended configuration is NTB-to-NTB mode, split bar enabled and 73all BAR sizes set to 20 (1 MiB). 74This needs to be done on both systems. 75Note, on Xeon SkyLake and newer platforms, split bar mode is not available. 76.Sh SEE ALSO 77.Xr if_ntb 4 , 78.Xr ntb 4 , 79.Xr ntb_transport 4 80.Sh AUTHORS 81.An -nosplit 82The 83.Nm 84driver was developed by Intel and originally written by 85.An Carl Delsey Aq Mt carl@FreeBSD.org . 86Later improvements were done by 87.An Conrad E. Meyer Aq Mt cem@FreeBSD.org 88and 89.An Alexander Motin Aq Mt mav@FreeBSD.org . 90.Sh BUGS 91NTB-to-Root Port mode is not yet supported, but it doesn't look very useful. 92.Pp 93On Xeon v2/v3/v4 processors split BAR mode should be enabled to allow 94SB01BASE_LOCKUP errata workaround to be applied by the driver. 95.Pp 96There is no way to protect your system from malicious behavior on the other 97system once the link is brought up. 98Anyone with root or kernel access on the other system can read or write to 99any location on your system. 100In other words, only connect two systems that completely trust each other. 101