1733b9277SNavdeep Parhar# Chelsio T4 Factory Default configuration file. 2733b9277SNavdeep Parhar# 3*7c0cad38SNavdeep Parhar# Copyright (C) 2010-2017 Chelsio Communications. All rights reserved. 4733b9277SNavdeep Parhar# 52a5f6b0eSNavdeep Parhar# DO NOT MODIFY THIS FILE UNDER ANY CIRCUMSTANCES. MODIFICATION OF 62a5f6b0eSNavdeep Parhar# THIS FILE WILL RESULT IN A NON-FUNCTIONAL T4 ADAPTER AND MAY RESULT 72a5f6b0eSNavdeep Parhar# IN PHYSICAL DAMAGE TO T4 ADAPTERS. 8733b9277SNavdeep Parhar 9733b9277SNavdeep Parhar# This file provides the default, power-on configuration for 4-port T4-based 10733b9277SNavdeep Parhar# adapters shipped from the factory. These defaults are designed to address 11733b9277SNavdeep Parhar# the needs of the vast majority of T4 customers. The basic idea is to have 12733b9277SNavdeep Parhar# a default configuration which allows a customer to plug a T4 adapter in and 13733b9277SNavdeep Parhar# have it work regardless of OS, driver or application except in the most 14733b9277SNavdeep Parhar# unusual and/or demanding customer applications. 15733b9277SNavdeep Parhar# 16733b9277SNavdeep Parhar# Many of the T4 resources which are described by this configuration are 17733b9277SNavdeep Parhar# finite. This requires balancing the configuration/operation needs of 18733b9277SNavdeep Parhar# device drivers across OSes and a large number of customer application. 19733b9277SNavdeep Parhar# 20612226d7SPedro F. Giffuni# Some of the more important resources to allocate and their constaints are: 21733b9277SNavdeep Parhar# 1. Virtual Interfaces: 128. 22733b9277SNavdeep Parhar# 2. Ingress Queues with Free Lists: 1024. PCI-E SR-IOV Virtual Functions 23733b9277SNavdeep Parhar# must use a power of 2 Ingress Queues. 24733b9277SNavdeep Parhar# 3. Egress Queues: 128K. PCI-E SR-IOV Virtual Functions must use a 25733b9277SNavdeep Parhar# power of 2 Egress Queues. 26733b9277SNavdeep Parhar# 4. MSI-X Vectors: 1088. A complication here is that the PCI-E SR-IOV 27733b9277SNavdeep Parhar# Virtual Functions based off of a Physical Function all get the 28733b9277SNavdeep Parhar# same umber of MSI-X Vectors as the base Physical Function. 29733b9277SNavdeep Parhar# Additionally, regardless of whether Virtual Functions are enabled or 30733b9277SNavdeep Parhar# not, their MSI-X "needs" are counted by the PCI-E implementation. 31733b9277SNavdeep Parhar# And finally, all Physical Funcations capable of supporting Virtual 32733b9277SNavdeep Parhar# Functions (PF0-3) must have the same number of configured TotalVFs in 33733b9277SNavdeep Parhar# their SR-IOV Capabilities. 34733b9277SNavdeep Parhar# 5. Multi-Port Support (MPS) TCAM: 336 entries to support MAC destination 35733b9277SNavdeep Parhar# address matching on Ingress Packets. 36733b9277SNavdeep Parhar# 37733b9277SNavdeep Parhar# Some of the important OS/Driver resource needs are: 38733b9277SNavdeep Parhar# 6. Some OS Drivers will manage all resources through a single Physical 39733b9277SNavdeep Parhar# Function (currently PF0 but it could be any Physical Function). Thus, 40733b9277SNavdeep Parhar# this "Unified PF" will need to have enough resources allocated to it 41733b9277SNavdeep Parhar# to allow for this. And because of the MSI-X resource allocation 42733b9277SNavdeep Parhar# constraints mentioned above, this probably means we'll either have to 43733b9277SNavdeep Parhar# severely limit the TotalVFs if we continue to use PF0 as the Unified PF 44733b9277SNavdeep Parhar# or we'll need to move the Unified PF into the PF4-7 range since those 45733b9277SNavdeep Parhar# Physical Functions don't have any Virtual Functions associated with 46733b9277SNavdeep Parhar# them. 47733b9277SNavdeep Parhar# 7. Some OS Drivers will manage different ports and functions (NIC, 48733b9277SNavdeep Parhar# storage, etc.) on different Physical Functions. For example, NIC 49733b9277SNavdeep Parhar# functions for ports 0-3 on PF0-3, FCoE on PF4, iSCSI on PF5, etc. 50733b9277SNavdeep Parhar# 51733b9277SNavdeep Parhar# Some of the customer application needs which need to be accommodated: 52733b9277SNavdeep Parhar# 8. Some customers will want to support large CPU count systems with 53733b9277SNavdeep Parhar# good scaling. Thus, we'll need to accommodate a number of 54733b9277SNavdeep Parhar# Ingress Queues and MSI-X Vectors to allow up to some number of CPUs 55733b9277SNavdeep Parhar# to be involved per port and per application function. For example, 56733b9277SNavdeep Parhar# in the case where all ports and application functions will be 57733b9277SNavdeep Parhar# managed via a single Unified PF and we want to accommodate scaling up 58733b9277SNavdeep Parhar# to 8 CPUs, we would want: 59733b9277SNavdeep Parhar# 60733b9277SNavdeep Parhar# 4 ports * 61733b9277SNavdeep Parhar# 3 application functions (NIC, FCoE, iSCSI) per port * 62733b9277SNavdeep Parhar# 8 Ingress Queue/MSI-X Vectors per application function 63733b9277SNavdeep Parhar# 64733b9277SNavdeep Parhar# for a total of 96 Ingress Queues and MSI-X Vectors on the Unified PF. 65733b9277SNavdeep Parhar# (Plus a few for Firmware Event Queues, etc.) 66733b9277SNavdeep Parhar# 67733b9277SNavdeep Parhar# 9. Some customers will want to use T4's PCI-E SR-IOV Capability to allow 68733b9277SNavdeep Parhar# Virtual Machines to directly access T4 functionality via SR-IOV 69733b9277SNavdeep Parhar# Virtual Functions and "PCI Device Passthrough" -- this is especially 70733b9277SNavdeep Parhar# true for the NIC application functionality. (Note that there is 71733b9277SNavdeep Parhar# currently no ability to use the TOE, FCoE, iSCSI, etc. via Virtual 72733b9277SNavdeep Parhar# Functions so this is in fact solely limited to NIC.) 73733b9277SNavdeep Parhar# 74733b9277SNavdeep Parhar 75733b9277SNavdeep Parhar 76733b9277SNavdeep Parhar# Global configuration settings. 77733b9277SNavdeep Parhar# 78733b9277SNavdeep Parhar[global] 79733b9277SNavdeep Parhar rss_glb_config_mode = basicvirtual 80733b9277SNavdeep Parhar rss_glb_config_options = tnlmapen,hashtoeplitz,tnlalllkp 81733b9277SNavdeep Parhar 82733b9277SNavdeep Parhar # The following Scatter Gather Engine (SGE) settings assume a 4KB Host 83733b9277SNavdeep Parhar # Page Size and a 64B L1 Cache Line Size. It programs the 84733b9277SNavdeep Parhar # EgrStatusPageSize and IngPadBoundary to 64B and the PktShift to 2. 85733b9277SNavdeep Parhar # If a Master PF Driver finds itself on a machine with different 86733b9277SNavdeep Parhar # parameters, then the Master PF Driver is responsible for initializing 87733b9277SNavdeep Parhar # these parameters to appropriate values. 88733b9277SNavdeep Parhar # 89733b9277SNavdeep Parhar # Notes: 90733b9277SNavdeep Parhar # 1. The Free List Buffer Sizes below are raw and the firmware will 91733b9277SNavdeep Parhar # round them up to the Ingress Padding Boundary. 92733b9277SNavdeep Parhar # 2. The SGE Timer Values below are expressed below in microseconds. 93733b9277SNavdeep Parhar # The firmware will convert these values to Core Clock Ticks when 94733b9277SNavdeep Parhar # it processes the configuration parameters. 95733b9277SNavdeep Parhar # 96733b9277SNavdeep Parhar reg[0x1008] = 0x40810/0x21c70 # SGE_CONTROL 97733b9277SNavdeep Parhar reg[0x100c] = 0x22222222 # SGE_HOST_PAGE_SIZE 98733b9277SNavdeep Parhar reg[0x10a0] = 0x01040810 # SGE_INGRESS_RX_THRESHOLD 99733b9277SNavdeep Parhar reg[0x1044] = 4096 # SGE_FL_BUFFER_SIZE0 100733b9277SNavdeep Parhar reg[0x1048] = 65536 # SGE_FL_BUFFER_SIZE1 101733b9277SNavdeep Parhar reg[0x104c] = 1536 # SGE_FL_BUFFER_SIZE2 102733b9277SNavdeep Parhar reg[0x1050] = 9024 # SGE_FL_BUFFER_SIZE3 103733b9277SNavdeep Parhar reg[0x1054] = 9216 # SGE_FL_BUFFER_SIZE4 104733b9277SNavdeep Parhar reg[0x1058] = 2048 # SGE_FL_BUFFER_SIZE5 105733b9277SNavdeep Parhar reg[0x105c] = 128 # SGE_FL_BUFFER_SIZE6 106733b9277SNavdeep Parhar reg[0x1060] = 8192 # SGE_FL_BUFFER_SIZE7 107733b9277SNavdeep Parhar reg[0x1064] = 16384 # SGE_FL_BUFFER_SIZE8 108733b9277SNavdeep Parhar reg[0x10a4] = 0xa000a000/0xf000f000 # SGE_DBFIFO_STATUS 109733b9277SNavdeep Parhar reg[0x10a8] = 0x2000/0x2000 # SGE_DOORBELL_CONTROL 110733b9277SNavdeep Parhar sge_timer_value = 5, 10, 20, 50, 100, 200 # SGE_TIMER_VALUE* in usecs 111733b9277SNavdeep Parhar 11248d05478SNavdeep Parhar # enable TP_OUT_CONFIG.IPIDSPLITMODE 11348d05478SNavdeep Parhar reg[0x7d04] = 0x00010000/0x00010000 11448d05478SNavdeep Parhar 115dd991bd5SNavdeep Parhar # disable TP_PARA_REG3.RxFragEn 116dd991bd5SNavdeep Parhar reg[0x7d6c] = 0x00000000/0x00007000 117dd991bd5SNavdeep Parhar 118c7dbd802SNavdeep Parhar reg[0x7dc0] = 0x0e2f8849 # TP_SHIFT_CNT 119733b9277SNavdeep Parhar 12048d05478SNavdeep Parhar # TP_VLAN_PRI_MAP to select filter tuples 12148d05478SNavdeep Parhar # filter tuples : fragmentation, mpshittype, macmatch, ethertype, 12248d05478SNavdeep Parhar # protocol, tos, vlan, vnic_id, port, fcoe 12348d05478SNavdeep Parhar # valid filterModes are described the Terminator 4 Data Book 1242a5f6b0eSNavdeep Parhar filterMode = fragmentation, mpshittype, protocol, vlan, port, fcoe 125733b9277SNavdeep Parhar 12648d05478SNavdeep Parhar # filter tuples enforced in LE active region (equal to or subset of filterMode) 12748d05478SNavdeep Parhar filterMask = protocol, fcoe 12848d05478SNavdeep Parhar 129733b9277SNavdeep Parhar # Percentage of dynamic memory (in either the EDRAM or external MEM) 130733b9277SNavdeep Parhar # to use for TP RX payload 131c7dbd802SNavdeep Parhar tp_pmrx = 34 132733b9277SNavdeep Parhar 133733b9277SNavdeep Parhar # TP RX payload page size 134733b9277SNavdeep Parhar tp_pmrx_pagesize = 64K 135733b9277SNavdeep Parhar 136f72b68a1SNavdeep Parhar # TP number of RX channels 137f72b68a1SNavdeep Parhar tp_nrxch = 0 # 0 (auto) = 1 138f72b68a1SNavdeep Parhar 139733b9277SNavdeep Parhar # Percentage of dynamic memory (in either the EDRAM or external MEM) 140733b9277SNavdeep Parhar # to use for TP TX payload 141c7dbd802SNavdeep Parhar tp_pmtx = 32 142733b9277SNavdeep Parhar 143733b9277SNavdeep Parhar # TP TX payload page size 144733b9277SNavdeep Parhar tp_pmtx_pagesize = 64K 145733b9277SNavdeep Parhar 146f72b68a1SNavdeep Parhar # TP number of TX channels 147f72b68a1SNavdeep Parhar tp_ntxch = 0 # 0 (auto) = equal number of ports 148f72b68a1SNavdeep Parhar 14948d05478SNavdeep Parhar # TP OFLD MTUs 15048d05478SNavdeep Parhar tp_mtus = 88, 256, 512, 576, 808, 1024, 1280, 1488, 1500, 2002, 2048, 4096, 4352, 8192, 9000, 9600 15148d05478SNavdeep Parhar 152c7dbd802SNavdeep Parhar # ULPRX iSCSI Page Sizes 153c7dbd802SNavdeep Parhar reg[0x19168] = 0x04020100 # 64K, 16K, 8K and 4K 154c7dbd802SNavdeep Parhar 155733b9277SNavdeep Parhar# Some "definitions" to make the rest of this a bit more readable. We support 156733b9277SNavdeep Parhar# 4 ports, 3 functions (NIC, FCoE and iSCSI), scaling up to 8 "CPU Queue Sets" 157733b9277SNavdeep Parhar# per function per port ... 158733b9277SNavdeep Parhar# 159733b9277SNavdeep Parhar# NMSIX = 1088 # available MSI-X Vectors 160733b9277SNavdeep Parhar# NVI = 128 # available Virtual Interfaces 161733b9277SNavdeep Parhar# NMPSTCAM = 336 # MPS TCAM entries 162733b9277SNavdeep Parhar# 163733b9277SNavdeep Parhar# NPORTS = 4 # ports 164733b9277SNavdeep Parhar# NCPUS = 8 # CPUs we want to support scalably 165733b9277SNavdeep Parhar# NFUNCS = 3 # functions per port (NIC, FCoE, iSCSI) 166733b9277SNavdeep Parhar 167733b9277SNavdeep Parhar# Breakdown of Virtual Interface/Queue/Interrupt resources for the "Unified 168733b9277SNavdeep Parhar# PF" which many OS Drivers will use to manage most or all functions. 169733b9277SNavdeep Parhar# 170733b9277SNavdeep Parhar# Each Ingress Queue can use one MSI-X interrupt but some Ingress Queues can 171733b9277SNavdeep Parhar# use Forwarded Interrupt Ingress Queues. For these latter, an Ingress Queue 172733b9277SNavdeep Parhar# would be created and the Queue ID of a Forwarded Interrupt Ingress Queue 173733b9277SNavdeep Parhar# will be specified as the "Ingress Queue Asynchronous Destination Index." 174733b9277SNavdeep Parhar# Thus, the number of MSI-X Vectors assigned to the Unified PF will be less 175733b9277SNavdeep Parhar# than or equal to the number of Ingress Queues ... 176733b9277SNavdeep Parhar# 177733b9277SNavdeep Parhar# NVI_NIC = 4 # NIC access to NPORTS 178733b9277SNavdeep Parhar# NFLIQ_NIC = 32 # NIC Ingress Queues with Free Lists 179733b9277SNavdeep Parhar# NETHCTRL_NIC = 32 # NIC Ethernet Control/TX Queues 180733b9277SNavdeep Parhar# NEQ_NIC = 64 # NIC Egress Queues (FL, ETHCTRL/TX) 181733b9277SNavdeep Parhar# NMPSTCAM_NIC = 16 # NIC MPS TCAM Entries (NPORTS*4) 182733b9277SNavdeep Parhar# NMSIX_NIC = 32 # NIC MSI-X Interrupt Vectors (FLIQ) 183733b9277SNavdeep Parhar# 184733b9277SNavdeep Parhar# NVI_OFLD = 0 # Offload uses NIC function to access ports 185733b9277SNavdeep Parhar# NFLIQ_OFLD = 16 # Offload Ingress Queues with Free Lists 186733b9277SNavdeep Parhar# NETHCTRL_OFLD = 0 # Offload Ethernet Control/TX Queues 187733b9277SNavdeep Parhar# NEQ_OFLD = 16 # Offload Egress Queues (FL) 188733b9277SNavdeep Parhar# NMPSTCAM_OFLD = 0 # Offload MPS TCAM Entries (uses NIC's) 189733b9277SNavdeep Parhar# NMSIX_OFLD = 16 # Offload MSI-X Interrupt Vectors (FLIQ) 190733b9277SNavdeep Parhar# 191733b9277SNavdeep Parhar# NVI_RDMA = 0 # RDMA uses NIC function to access ports 192733b9277SNavdeep Parhar# NFLIQ_RDMA = 4 # RDMA Ingress Queues with Free Lists 193733b9277SNavdeep Parhar# NETHCTRL_RDMA = 0 # RDMA Ethernet Control/TX Queues 194733b9277SNavdeep Parhar# NEQ_RDMA = 4 # RDMA Egress Queues (FL) 195733b9277SNavdeep Parhar# NMPSTCAM_RDMA = 0 # RDMA MPS TCAM Entries (uses NIC's) 196733b9277SNavdeep Parhar# NMSIX_RDMA = 4 # RDMA MSI-X Interrupt Vectors (FLIQ) 197733b9277SNavdeep Parhar# 198733b9277SNavdeep Parhar# NEQ_WD = 128 # Wire Direct TX Queues and FLs 199733b9277SNavdeep Parhar# NETHCTRL_WD = 64 # Wire Direct TX Queues 200733b9277SNavdeep Parhar# NFLIQ_WD = 64 ` # Wire Direct Ingress Queues with Free Lists 201733b9277SNavdeep Parhar# 202733b9277SNavdeep Parhar# NVI_ISCSI = 4 # ISCSI access to NPORTS 203733b9277SNavdeep Parhar# NFLIQ_ISCSI = 4 # ISCSI Ingress Queues with Free Lists 204733b9277SNavdeep Parhar# NETHCTRL_ISCSI = 0 # ISCSI Ethernet Control/TX Queues 205733b9277SNavdeep Parhar# NEQ_ISCSI = 4 # ISCSI Egress Queues (FL) 206733b9277SNavdeep Parhar# NMPSTCAM_ISCSI = 4 # ISCSI MPS TCAM Entries (NPORTS) 207733b9277SNavdeep Parhar# NMSIX_ISCSI = 4 # ISCSI MSI-X Interrupt Vectors (FLIQ) 208733b9277SNavdeep Parhar# 209733b9277SNavdeep Parhar# NVI_FCOE = 4 # FCOE access to NPORTS 210733b9277SNavdeep Parhar# NFLIQ_FCOE = 34 # FCOE Ingress Queues with Free Lists 211733b9277SNavdeep Parhar# NETHCTRL_FCOE = 32 # FCOE Ethernet Control/TX Queues 212733b9277SNavdeep Parhar# NEQ_FCOE = 66 # FCOE Egress Queues (FL) 213733b9277SNavdeep Parhar# NMPSTCAM_FCOE = 32 # FCOE MPS TCAM Entries (NPORTS) 214733b9277SNavdeep Parhar# NMSIX_FCOE = 34 # FCOE MSI-X Interrupt Vectors (FLIQ) 215733b9277SNavdeep Parhar 216733b9277SNavdeep Parhar# Two extra Ingress Queues per function for Firmware Events and Forwarded 217733b9277SNavdeep Parhar# Interrupts, and two extra interrupts per function for Firmware Events (or a 218733b9277SNavdeep Parhar# Forwarded Interrupt Queue) and General Interrupts per function. 219733b9277SNavdeep Parhar# 220733b9277SNavdeep Parhar# NFLIQ_EXTRA = 6 # "extra" Ingress Queues 2*NFUNCS (Firmware and 221733b9277SNavdeep Parhar# # Forwarded Interrupts 222733b9277SNavdeep Parhar# NMSIX_EXTRA = 6 # extra interrupts 2*NFUNCS (Firmware and 223733b9277SNavdeep Parhar# # General Interrupts 224733b9277SNavdeep Parhar 225733b9277SNavdeep Parhar# Microsoft HyperV resources. The HyperV Virtual Ingress Queues will have 226733b9277SNavdeep Parhar# their interrupts forwarded to another set of Forwarded Interrupt Queues. 227733b9277SNavdeep Parhar# 228733b9277SNavdeep Parhar# NVI_HYPERV = 16 # VMs we want to support 229733b9277SNavdeep Parhar# NVIIQ_HYPERV = 2 # Virtual Ingress Queues with Free Lists per VM 230733b9277SNavdeep Parhar# NFLIQ_HYPERV = 40 # VIQs + NCPUS Forwarded Interrupt Queues 231733b9277SNavdeep Parhar# NEQ_HYPERV = 32 # VIQs Free Lists 232733b9277SNavdeep Parhar# NMPSTCAM_HYPERV = 16 # MPS TCAM Entries (NVI_HYPERV) 233733b9277SNavdeep Parhar# NMSIX_HYPERV = 8 # NCPUS Forwarded Interrupt Queues 234733b9277SNavdeep Parhar 235733b9277SNavdeep Parhar# Adding all of the above Unified PF resource needs together: (NIC + OFLD + 236733b9277SNavdeep Parhar# RDMA + ISCSI + FCOE + EXTRA + HYPERV) 237733b9277SNavdeep Parhar# 238733b9277SNavdeep Parhar# NVI_UNIFIED = 28 239733b9277SNavdeep Parhar# NFLIQ_UNIFIED = 106 240733b9277SNavdeep Parhar# NETHCTRL_UNIFIED = 32 241733b9277SNavdeep Parhar# NEQ_UNIFIED = 124 242733b9277SNavdeep Parhar# NMPSTCAM_UNIFIED = 40 243733b9277SNavdeep Parhar# 244733b9277SNavdeep Parhar# The sum of all the MSI-X resources above is 74 MSI-X Vectors but we'll round 245733b9277SNavdeep Parhar# that up to 128 to make sure the Unified PF doesn't run out of resources. 246733b9277SNavdeep Parhar# 247733b9277SNavdeep Parhar# NMSIX_UNIFIED = 128 248733b9277SNavdeep Parhar# 249733b9277SNavdeep Parhar# The Storage PFs could need up to NPORTS*NCPUS + NMSIX_EXTRA MSI-X Vectors 250733b9277SNavdeep Parhar# which is 34 but they're probably safe with 32. 251733b9277SNavdeep Parhar# 252733b9277SNavdeep Parhar# NMSIX_STORAGE = 32 253733b9277SNavdeep Parhar 254733b9277SNavdeep Parhar# Note: The UnifiedPF is PF4 which doesn't have any Virtual Functions 255733b9277SNavdeep Parhar# associated with it. Thus, the MSI-X Vector allocations we give to the 256733b9277SNavdeep Parhar# UnifiedPF aren't inherited by any Virtual Functions. As a result we can 257733b9277SNavdeep Parhar# provision many more Virtual Functions than we can if the UnifiedPF were 258733b9277SNavdeep Parhar# one of PF0-3. 259733b9277SNavdeep Parhar# 260733b9277SNavdeep Parhar 261733b9277SNavdeep Parhar# All of the below PCI-E parameters are actually stored in various *_init.txt 262733b9277SNavdeep Parhar# files. We include them below essentially as comments. 263733b9277SNavdeep Parhar# 264733b9277SNavdeep Parhar# For PF0-3 we assign 8 vectors each for NIC Ingress Queues of the associated 265733b9277SNavdeep Parhar# ports 0-3. 266733b9277SNavdeep Parhar# 267733b9277SNavdeep Parhar# For PF4, the Unified PF, we give it an MSI-X Table Size as outlined above. 268733b9277SNavdeep Parhar# 269733b9277SNavdeep Parhar# For PF5-6 we assign enough MSI-X Vectors to support FCoE and iSCSI 270733b9277SNavdeep Parhar# storage applications across all four possible ports. 271733b9277SNavdeep Parhar# 272733b9277SNavdeep Parhar# Additionally, since the UnifiedPF isn't one of the per-port Physical 273733b9277SNavdeep Parhar# Functions, we give the UnifiedPF and the PF0-3 Physical Functions 274733b9277SNavdeep Parhar# different PCI Device IDs which will allow Unified and Per-Port Drivers 275733b9277SNavdeep Parhar# to directly select the type of Physical Function to which they wish to be 276733b9277SNavdeep Parhar# attached. 277733b9277SNavdeep Parhar# 278612226d7SPedro F. Giffuni# Note that the actual values used for the PCI-E Intelectual Property will be 279733b9277SNavdeep Parhar# 1 less than those below since that's the way it "counts" things. For 280733b9277SNavdeep Parhar# readability, we use the number we actually mean ... 281733b9277SNavdeep Parhar# 282733b9277SNavdeep Parhar# PF0_INT = 8 # NCPUS 283733b9277SNavdeep Parhar# PF1_INT = 8 # NCPUS 284733b9277SNavdeep Parhar# PF2_INT = 8 # NCPUS 285733b9277SNavdeep Parhar# PF3_INT = 8 # NCPUS 286733b9277SNavdeep Parhar# PF0_3_INT = 32 # PF0_INT + PF1_INT + PF2_INT + PF3_INT 287733b9277SNavdeep Parhar# 288733b9277SNavdeep Parhar# PF4_INT = 128 # NMSIX_UNIFIED 289733b9277SNavdeep Parhar# PF5_INT = 32 # NMSIX_STORAGE 290733b9277SNavdeep Parhar# PF6_INT = 32 # NMSIX_STORAGE 291733b9277SNavdeep Parhar# PF7_INT = 0 # Nothing Assigned 292733b9277SNavdeep Parhar# PF4_7_INT = 192 # PF4_INT + PF5_INT + PF6_INT + PF7_INT 293733b9277SNavdeep Parhar# 294733b9277SNavdeep Parhar# PF0_7_INT = 224 # PF0_3_INT + PF4_7_INT 295733b9277SNavdeep Parhar# 296733b9277SNavdeep Parhar# With the above we can get 17 VFs/PF0-3 (limited by 336 MPS TCAM entries) 297733b9277SNavdeep Parhar# but we'll lower that to 16 to make our total 64 and a nice power of 2 ... 298733b9277SNavdeep Parhar# 299733b9277SNavdeep Parhar# NVF = 16 300733b9277SNavdeep Parhar 301733b9277SNavdeep Parhar# For those OSes which manage different ports on different PFs, we need 302733b9277SNavdeep Parhar# only enough resources to support a single port's NIC application functions 303733b9277SNavdeep Parhar# on PF0-3. The below assumes that we're only doing NIC with NCPUS "Queue 304733b9277SNavdeep Parhar# Sets" for ports 0-3. The FCoE and iSCSI functions for such OSes will be 305733b9277SNavdeep Parhar# managed on the "storage PFs" (see below). 306733b9277SNavdeep Parhar# 307733b9277SNavdeep Parhar[function "0"] 308733b9277SNavdeep Parhar nvf = 16 # NVF on this function 309733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 310733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 311733b9277SNavdeep Parhar nvi = 1 # 1 port 312733b9277SNavdeep Parhar niqflint = 8 # NCPUS "Queue Sets" 313733b9277SNavdeep Parhar nethctrl = 8 # NCPUS "Queue Sets" 314733b9277SNavdeep Parhar neq = 16 # niqflint + nethctrl Egress Queues 315733b9277SNavdeep Parhar nexactf = 8 # number of exact MPSTCAM MAC filters 316733b9277SNavdeep Parhar cmask = all # access to all channels 317733b9277SNavdeep Parhar pmask = 0x1 # access to only one port 318733b9277SNavdeep Parhar 319733b9277SNavdeep Parhar[function "1"] 320733b9277SNavdeep Parhar nvf = 16 # NVF on this function 321733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 322733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 323733b9277SNavdeep Parhar nvi = 1 # 1 port 324733b9277SNavdeep Parhar niqflint = 8 # NCPUS "Queue Sets" 325733b9277SNavdeep Parhar nethctrl = 8 # NCPUS "Queue Sets" 326733b9277SNavdeep Parhar neq = 16 # niqflint + nethctrl Egress Queues 327733b9277SNavdeep Parhar nexactf = 8 # number of exact MPSTCAM MAC filters 328733b9277SNavdeep Parhar cmask = all # access to all channels 329733b9277SNavdeep Parhar pmask = 0x2 # access to only one port 330733b9277SNavdeep Parhar 331733b9277SNavdeep Parhar[function "2"] 332733b9277SNavdeep Parhar nvf = 16 # NVF on this function 333733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 334733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 335733b9277SNavdeep Parhar nvi = 1 # 1 port 336733b9277SNavdeep Parhar niqflint = 8 # NCPUS "Queue Sets" 337733b9277SNavdeep Parhar nethctrl = 8 # NCPUS "Queue Sets" 338733b9277SNavdeep Parhar neq = 16 # niqflint + nethctrl Egress Queues 339733b9277SNavdeep Parhar nexactf = 8 # number of exact MPSTCAM MAC filters 340733b9277SNavdeep Parhar cmask = all # access to all channels 341733b9277SNavdeep Parhar pmask = 0x4 # access to only one port 342733b9277SNavdeep Parhar 343733b9277SNavdeep Parhar[function "3"] 344733b9277SNavdeep Parhar nvf = 16 # NVF on this function 345733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 346733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 347733b9277SNavdeep Parhar nvi = 1 # 1 port 348733b9277SNavdeep Parhar niqflint = 8 # NCPUS "Queue Sets" 349733b9277SNavdeep Parhar nethctrl = 8 # NCPUS "Queue Sets" 350733b9277SNavdeep Parhar neq = 16 # niqflint + nethctrl Egress Queues 351733b9277SNavdeep Parhar nexactf = 8 # number of exact MPSTCAM MAC filters 352733b9277SNavdeep Parhar cmask = all # access to all channels 353733b9277SNavdeep Parhar pmask = 0x8 # access to only one port 354733b9277SNavdeep Parhar 355733b9277SNavdeep Parhar# Some OS Drivers manage all application functions for all ports via PF4. 356733b9277SNavdeep Parhar# Thus we need to provide a large number of resources here. For Egress 357733b9277SNavdeep Parhar# Queues we need to account for both TX Queues as well as Free List Queues 358733b9277SNavdeep Parhar# (because the host is responsible for producing Free List Buffers for the 359733b9277SNavdeep Parhar# hardware to consume). 360733b9277SNavdeep Parhar# 361733b9277SNavdeep Parhar[function "4"] 362733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 363733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 364733b9277SNavdeep Parhar nvi = 28 # NVI_UNIFIED 365733b9277SNavdeep Parhar niqflint = 170 # NFLIQ_UNIFIED + NLFIQ_WD 3662a5f6b0eSNavdeep Parhar nethctrl = 100 # NETHCTRL_UNIFIED + NETHCTRL_WD 3672a5f6b0eSNavdeep Parhar neq = 256 # NEQ_UNIFIED + NEQ_WD 368733b9277SNavdeep Parhar nexactf = 40 # NMPSTCAM_UNIFIED 369733b9277SNavdeep Parhar cmask = all # access to all channels 370733b9277SNavdeep Parhar pmask = all # access to all four ports ... 3712a5f6b0eSNavdeep Parhar nethofld = 1024 # number of user mode ethernet flow contexts 372733b9277SNavdeep Parhar nroute = 32 # number of routing region entries 373733b9277SNavdeep Parhar nclip = 32 # number of clip region entries 3742a5f6b0eSNavdeep Parhar nfilter = 496 # number of filter region entries 3752a5f6b0eSNavdeep Parhar nserver = 496 # number of server region entries 3762a5f6b0eSNavdeep Parhar nhash = 12288 # number of hash region entries 377733b9277SNavdeep Parhar protocol = nic_vm, ofld, rddp, rdmac, iscsi_initiator_pdu, iscsi_target_pdu 3782a5f6b0eSNavdeep Parhar tp_l2t = 3072 37948d05478SNavdeep Parhar tp_ddp = 3 380733b9277SNavdeep Parhar tp_ddp_iscsi = 2 38148d05478SNavdeep Parhar tp_stag = 3 38248d05478SNavdeep Parhar tp_pbl = 10 38348d05478SNavdeep Parhar tp_rq = 13 384733b9277SNavdeep Parhar 385733b9277SNavdeep Parhar# We have FCoE and iSCSI storage functions on PF5 and PF6 each of which may 386733b9277SNavdeep Parhar# need to have Virtual Interfaces on each of the four ports with up to NCPUS 387733b9277SNavdeep Parhar# "Queue Sets" each. 388733b9277SNavdeep Parhar# 389733b9277SNavdeep Parhar[function "5"] 390733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 391733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 392733b9277SNavdeep Parhar nvi = 4 # NPORTS 393733b9277SNavdeep Parhar niqflint = 34 # NPORTS*NCPUS + NMSIX_EXTRA 394733b9277SNavdeep Parhar nethctrl = 32 # NPORTS*NCPUS 395733b9277SNavdeep Parhar neq = 64 # NPORTS*NCPUS * 2 (FL, ETHCTRL/TX) 396733b9277SNavdeep Parhar nexactf = 4 # NPORTS 397733b9277SNavdeep Parhar cmask = all # access to all channels 398733b9277SNavdeep Parhar pmask = all # access to all four ports ... 3992a5f6b0eSNavdeep Parhar nserver = 16 4002a5f6b0eSNavdeep Parhar nhash = 2048 401c7dbd802SNavdeep Parhar tp_l2t = 1020 4022a5f6b0eSNavdeep Parhar protocol = iscsi_initiator_fofld 4032a5f6b0eSNavdeep Parhar tp_ddp_iscsi = 2 4042a5f6b0eSNavdeep Parhar iscsi_ntask = 2048 4052a5f6b0eSNavdeep Parhar iscsi_nsess = 2048 4062a5f6b0eSNavdeep Parhar iscsi_nconn_per_session = 1 4072a5f6b0eSNavdeep Parhar iscsi_ninitiator_instance = 64 408733b9277SNavdeep Parhar 409733b9277SNavdeep Parhar[function "6"] 410733b9277SNavdeep Parhar wx_caps = all # write/execute permissions for all commands 411733b9277SNavdeep Parhar r_caps = all # read permissions for all commands 412733b9277SNavdeep Parhar nvi = 4 # NPORTS 413733b9277SNavdeep Parhar niqflint = 34 # NPORTS*NCPUS + NMSIX_EXTRA 414733b9277SNavdeep Parhar nethctrl = 32 # NPORTS*NCPUS 415733b9277SNavdeep Parhar neq = 66 # NPORTS*NCPUS * 2 (FL, ETHCTRL/TX) + 2 (EXTRA) 416733b9277SNavdeep Parhar nexactf = 32 # NPORTS + adding 28 exact entries for FCoE 417733b9277SNavdeep Parhar # which is OK since < MIN(SUM PF0..3, PF4) 418733b9277SNavdeep Parhar # and we never load PF0..3 and PF4 concurrently 419733b9277SNavdeep Parhar cmask = all # access to all channels 420733b9277SNavdeep Parhar pmask = all # access to all four ports ... 4212a5f6b0eSNavdeep Parhar nhash = 2048 422c7dbd802SNavdeep Parhar tp_l2t = 4 423733b9277SNavdeep Parhar protocol = fcoe_initiator 42448d05478SNavdeep Parhar tp_ddp = 1 425733b9277SNavdeep Parhar fcoe_nfcf = 16 426733b9277SNavdeep Parhar fcoe_nvnp = 32 427733b9277SNavdeep Parhar fcoe_nssn = 1024 428733b9277SNavdeep Parhar 4292a5f6b0eSNavdeep Parhar# The following function, 1023, is not an actual PCIE function but is used to 4302a5f6b0eSNavdeep Parhar# configure and reserve firmware internal resources that come from the global 4312a5f6b0eSNavdeep Parhar# resource pool. 4322a5f6b0eSNavdeep Parhar# 4332a5f6b0eSNavdeep Parhar[function "1023"] 4342a5f6b0eSNavdeep Parhar wx_caps = all # write/execute permissions for all commands 4352a5f6b0eSNavdeep Parhar r_caps = all # read permissions for all commands 4362a5f6b0eSNavdeep Parhar nvi = 4 # NVI_UNIFIED 4372a5f6b0eSNavdeep Parhar cmask = all # access to all channels 4382a5f6b0eSNavdeep Parhar pmask = all # access to all four ports ... 4392a5f6b0eSNavdeep Parhar nexactf = 8 # NPORTS + DCBX + 4402a5f6b0eSNavdeep Parhar nfilter = 16 # number of filter region entries 4412a5f6b0eSNavdeep Parhar 442733b9277SNavdeep Parhar# For Virtual functions, we only allow NIC functionality and we only allow 443733b9277SNavdeep Parhar# access to one port (1 << PF). Note that because of limitations in the 444733b9277SNavdeep Parhar# Scatter Gather Engine (SGE) hardware which checks writes to VF KDOORBELL 445733b9277SNavdeep Parhar# and GTS registers, the number of Ingress and Egress Queues must be a power 446733b9277SNavdeep Parhar# of 2. 447733b9277SNavdeep Parhar# 448733b9277SNavdeep Parhar[function "0/*"] # NVF 449733b9277SNavdeep Parhar wx_caps = 0x82 # DMAQ | VF 450733b9277SNavdeep Parhar r_caps = 0x86 # DMAQ | VF | PORT 451733b9277SNavdeep Parhar nvi = 1 # 1 port 452733b9277SNavdeep Parhar niqflint = 4 # 2 "Queue Sets" + NXIQ 453733b9277SNavdeep Parhar nethctrl = 2 # 2 "Queue Sets" 454733b9277SNavdeep Parhar neq = 4 # 2 "Queue Sets" * 2 455733b9277SNavdeep Parhar nexactf = 4 456733b9277SNavdeep Parhar cmask = all # access to all channels 457733b9277SNavdeep Parhar pmask = 0x1 # access to only one port ... 458733b9277SNavdeep Parhar 459733b9277SNavdeep Parhar[function "1/*"] # NVF 460733b9277SNavdeep Parhar wx_caps = 0x82 # DMAQ | VF 461733b9277SNavdeep Parhar r_caps = 0x86 # DMAQ | VF | PORT 462733b9277SNavdeep Parhar nvi = 1 # 1 port 463733b9277SNavdeep Parhar niqflint = 4 # 2 "Queue Sets" + NXIQ 464733b9277SNavdeep Parhar nethctrl = 2 # 2 "Queue Sets" 465733b9277SNavdeep Parhar neq = 4 # 2 "Queue Sets" * 2 466733b9277SNavdeep Parhar nexactf = 4 467733b9277SNavdeep Parhar cmask = all # access to all channels 468733b9277SNavdeep Parhar pmask = 0x2 # access to only one port ... 469733b9277SNavdeep Parhar 470733b9277SNavdeep Parhar[function "2/*"] # NVF 471733b9277SNavdeep Parhar wx_caps = 0x82 # DMAQ | VF 472733b9277SNavdeep Parhar r_caps = 0x86 # DMAQ | VF | PORT 473733b9277SNavdeep Parhar nvi = 1 # 1 port 474733b9277SNavdeep Parhar niqflint = 4 # 2 "Queue Sets" + NXIQ 475733b9277SNavdeep Parhar nethctrl = 2 # 2 "Queue Sets" 476733b9277SNavdeep Parhar neq = 4 # 2 "Queue Sets" * 2 477733b9277SNavdeep Parhar nexactf = 4 478733b9277SNavdeep Parhar cmask = all # access to all channels 479733b9277SNavdeep Parhar pmask = 0x4 # access to only one port ... 480733b9277SNavdeep Parhar 481733b9277SNavdeep Parhar[function "3/*"] # NVF 482733b9277SNavdeep Parhar wx_caps = 0x82 # DMAQ | VF 483733b9277SNavdeep Parhar r_caps = 0x86 # DMAQ | VF | PORT 484733b9277SNavdeep Parhar nvi = 1 # 1 port 485733b9277SNavdeep Parhar niqflint = 4 # 2 "Queue Sets" + NXIQ 486733b9277SNavdeep Parhar nethctrl = 2 # 2 "Queue Sets" 487733b9277SNavdeep Parhar neq = 4 # 2 "Queue Sets" * 2 488733b9277SNavdeep Parhar nexactf = 4 489733b9277SNavdeep Parhar cmask = all # access to all channels 490733b9277SNavdeep Parhar pmask = 0x8 # access to only one port ... 491733b9277SNavdeep Parhar 492733b9277SNavdeep Parhar# MPS features a 196608 bytes ingress buffer that is used for ingress buffering 493733b9277SNavdeep Parhar# for packets from the wire as well as the loopback path of the L2 switch. The 494733b9277SNavdeep Parhar# folling params control how the buffer memory is distributed and the L2 flow 495733b9277SNavdeep Parhar# control settings: 496733b9277SNavdeep Parhar# 497733b9277SNavdeep Parhar# bg_mem: %-age of mem to use for port/buffer group 498733b9277SNavdeep Parhar# lpbk_mem: %-age of port/bg mem to use for loopback 499733b9277SNavdeep Parhar# hwm: high watermark; bytes available when starting to send pause 500733b9277SNavdeep Parhar# frames (in units of 0.1 MTU) 501733b9277SNavdeep Parhar# lwm: low watermark; bytes remaining when sending 'unpause' frame 502733b9277SNavdeep Parhar# (in inuits of 0.1 MTU) 503733b9277SNavdeep Parhar# dwm: minimum delta between high and low watermark (in units of 100 504733b9277SNavdeep Parhar# Bytes) 505733b9277SNavdeep Parhar# 506327235b3SNavdeep Parhar# 507327235b3SNavdeep Parhar 508733b9277SNavdeep Parhar[port "0"] 509733b9277SNavdeep Parhar dcb = ppp, dcbx # configure for DCB PPP and enable DCBX offload 510733b9277SNavdeep Parhar bg_mem = 25 511733b9277SNavdeep Parhar lpbk_mem = 25 512733b9277SNavdeep Parhar hwm = 30 513733b9277SNavdeep Parhar lwm = 15 514733b9277SNavdeep Parhar dwm = 30 515327235b3SNavdeep Parhar dcb_app_tlv[0] = 0x8906, ethertype, 3 516327235b3SNavdeep Parhar dcb_app_tlv[1] = 0x8914, ethertype, 3 517327235b3SNavdeep Parhar dcb_app_tlv[2] = 3260, socketnum, 5 518733b9277SNavdeep Parhar 519733b9277SNavdeep Parhar[port "1"] 520733b9277SNavdeep Parhar dcb = ppp, dcbx 521733b9277SNavdeep Parhar bg_mem = 25 522733b9277SNavdeep Parhar lpbk_mem = 25 523733b9277SNavdeep Parhar hwm = 30 524733b9277SNavdeep Parhar lwm = 15 525733b9277SNavdeep Parhar dwm = 30 526327235b3SNavdeep Parhar dcb_app_tlv[0] = 0x8906, ethertype, 3 527327235b3SNavdeep Parhar dcb_app_tlv[1] = 0x8914, ethertype, 3 528327235b3SNavdeep Parhar dcb_app_tlv[2] = 3260, socketnum, 5 529733b9277SNavdeep Parhar 530733b9277SNavdeep Parhar[port "2"] 531733b9277SNavdeep Parhar dcb = ppp, dcbx 532733b9277SNavdeep Parhar bg_mem = 25 533733b9277SNavdeep Parhar lpbk_mem = 25 534733b9277SNavdeep Parhar hwm = 30 535733b9277SNavdeep Parhar lwm = 15 536733b9277SNavdeep Parhar dwm = 30 537327235b3SNavdeep Parhar dcb_app_tlv[0] = 0x8906, ethertype, 3 538327235b3SNavdeep Parhar dcb_app_tlv[1] = 0x8914, ethertype, 3 539327235b3SNavdeep Parhar dcb_app_tlv[2] = 3260, socketnum, 5 540733b9277SNavdeep Parhar 541733b9277SNavdeep Parhar[port "3"] 542733b9277SNavdeep Parhar dcb = ppp, dcbx 543733b9277SNavdeep Parhar bg_mem = 25 544733b9277SNavdeep Parhar lpbk_mem = 25 545733b9277SNavdeep Parhar hwm = 30 546733b9277SNavdeep Parhar lwm = 15 547733b9277SNavdeep Parhar dwm = 30 548327235b3SNavdeep Parhar dcb_app_tlv[0] = 0x8906, ethertype, 3 549327235b3SNavdeep Parhar dcb_app_tlv[1] = 0x8914, ethertype, 3 550327235b3SNavdeep Parhar dcb_app_tlv[2] = 3260, socketnum, 5 551733b9277SNavdeep Parhar 552733b9277SNavdeep Parhar[fini] 553*7c0cad38SNavdeep Parhar version = 0x01000028 554*7c0cad38SNavdeep Parhar checksum = 0x5ceab421 555733b9277SNavdeep Parhar 556733b9277SNavdeep Parhar# Total resources used by above allocations: 557733b9277SNavdeep Parhar# Virtual Interfaces: 104 558733b9277SNavdeep Parhar# Ingress Queues/w Free Lists and Interrupts: 526 559733b9277SNavdeep Parhar# Egress Queues: 702 560733b9277SNavdeep Parhar# MPS TCAM Entries: 336 561733b9277SNavdeep Parhar# MSI-X Vectors: 736 562733b9277SNavdeep Parhar# Virtual Functions: 64 563733b9277SNavdeep Parhar# 564733b9277SNavdeep Parhar# 565