xref: /freebsd/sys/dev/cxgbe/firmware/t4fw_cfg_uwire.txt (revision 031beb4e239bfce798af17f5fe8dba8bcaf13d99)
1733b9277SNavdeep Parhar# Chelsio T4 Factory Default configuration file.
2733b9277SNavdeep Parhar#
3*7c0cad38SNavdeep Parhar# Copyright (C) 2010-2017 Chelsio Communications.  All rights reserved.
4733b9277SNavdeep Parhar#
52a5f6b0eSNavdeep Parhar#   DO NOT MODIFY THIS FILE UNDER ANY CIRCUMSTANCES.  MODIFICATION OF
62a5f6b0eSNavdeep Parhar#   THIS FILE WILL RESULT IN A NON-FUNCTIONAL T4 ADAPTER AND MAY RESULT
72a5f6b0eSNavdeep Parhar#   IN PHYSICAL DAMAGE TO T4 ADAPTERS.
8733b9277SNavdeep Parhar
9733b9277SNavdeep Parhar# This file provides the default, power-on configuration for 4-port T4-based
10733b9277SNavdeep Parhar# adapters shipped from the factory.  These defaults are designed to address
11733b9277SNavdeep Parhar# the needs of the vast majority of T4 customers.  The basic idea is to have
12733b9277SNavdeep Parhar# a default configuration which allows a customer to plug a T4 adapter in and
13733b9277SNavdeep Parhar# have it work regardless of OS, driver or application except in the most
14733b9277SNavdeep Parhar# unusual and/or demanding customer applications.
15733b9277SNavdeep Parhar#
16733b9277SNavdeep Parhar# Many of the T4 resources which are described by this configuration are
17733b9277SNavdeep Parhar# finite.  This requires balancing the configuration/operation needs of
18733b9277SNavdeep Parhar# device drivers across OSes and a large number of customer application.
19733b9277SNavdeep Parhar#
20612226d7SPedro F. Giffuni# Some of the more important resources to allocate and their constaints are:
21733b9277SNavdeep Parhar#  1. Virtual Interfaces: 128.
22733b9277SNavdeep Parhar#  2. Ingress Queues with Free Lists: 1024.  PCI-E SR-IOV Virtual Functions
23733b9277SNavdeep Parhar#     must use a power of 2 Ingress Queues.
24733b9277SNavdeep Parhar#  3. Egress Queues: 128K.  PCI-E SR-IOV Virtual Functions must use a
25733b9277SNavdeep Parhar#     power of 2 Egress Queues.
26733b9277SNavdeep Parhar#  4. MSI-X Vectors: 1088.  A complication here is that the PCI-E SR-IOV
27733b9277SNavdeep Parhar#     Virtual Functions based off of a Physical Function all get the
28733b9277SNavdeep Parhar#     same umber of MSI-X Vectors as the base Physical Function.
29733b9277SNavdeep Parhar#     Additionally, regardless of whether Virtual Functions are enabled or
30733b9277SNavdeep Parhar#     not, their MSI-X "needs" are counted by the PCI-E implementation.
31733b9277SNavdeep Parhar#     And finally, all Physical Funcations capable of supporting Virtual
32733b9277SNavdeep Parhar#     Functions (PF0-3) must have the same number of configured TotalVFs in
33733b9277SNavdeep Parhar#     their SR-IOV Capabilities.
34733b9277SNavdeep Parhar#  5. Multi-Port Support (MPS) TCAM: 336 entries to support MAC destination
35733b9277SNavdeep Parhar#     address matching on Ingress Packets.
36733b9277SNavdeep Parhar#
37733b9277SNavdeep Parhar# Some of the important OS/Driver resource needs are:
38733b9277SNavdeep Parhar#  6. Some OS Drivers will manage all resources through a single Physical
39733b9277SNavdeep Parhar#     Function (currently PF0 but it could be any Physical Function).  Thus,
40733b9277SNavdeep Parhar#     this "Unified PF"  will need to have enough resources allocated to it
41733b9277SNavdeep Parhar#     to allow for this.  And because of the MSI-X resource allocation
42733b9277SNavdeep Parhar#     constraints mentioned above, this probably means we'll either have to
43733b9277SNavdeep Parhar#     severely limit the TotalVFs if we continue to use PF0 as the Unified PF
44733b9277SNavdeep Parhar#     or we'll need to move the Unified PF into the PF4-7 range since those
45733b9277SNavdeep Parhar#     Physical Functions don't have any Virtual Functions associated with
46733b9277SNavdeep Parhar#     them.
47733b9277SNavdeep Parhar#  7. Some OS Drivers will manage different ports and functions (NIC,
48733b9277SNavdeep Parhar#     storage, etc.) on different Physical Functions.  For example, NIC
49733b9277SNavdeep Parhar#     functions for ports 0-3 on PF0-3, FCoE on PF4, iSCSI on PF5, etc.
50733b9277SNavdeep Parhar#
51733b9277SNavdeep Parhar# Some of the customer application needs which need to be accommodated:
52733b9277SNavdeep Parhar#  8. Some customers will want to support large CPU count systems with
53733b9277SNavdeep Parhar#     good scaling.  Thus, we'll need to accommodate a number of
54733b9277SNavdeep Parhar#     Ingress Queues and MSI-X Vectors to allow up to some number of CPUs
55733b9277SNavdeep Parhar#     to be involved per port and per application function.  For example,
56733b9277SNavdeep Parhar#     in the case where all ports and application functions will be
57733b9277SNavdeep Parhar#     managed via a single Unified PF and we want to accommodate scaling up
58733b9277SNavdeep Parhar#     to 8 CPUs, we would want:
59733b9277SNavdeep Parhar#
60733b9277SNavdeep Parhar#         4 ports *
61733b9277SNavdeep Parhar#         3 application functions (NIC, FCoE, iSCSI) per port *
62733b9277SNavdeep Parhar#         8 Ingress Queue/MSI-X Vectors per application function
63733b9277SNavdeep Parhar#
64733b9277SNavdeep Parhar#     for a total of 96 Ingress Queues and MSI-X Vectors on the Unified PF.
65733b9277SNavdeep Parhar#     (Plus a few for Firmware Event Queues, etc.)
66733b9277SNavdeep Parhar#
67733b9277SNavdeep Parhar#  9. Some customers will want to use T4's PCI-E SR-IOV Capability to allow
68733b9277SNavdeep Parhar#     Virtual Machines to directly access T4 functionality via SR-IOV
69733b9277SNavdeep Parhar#     Virtual Functions and "PCI Device Passthrough" -- this is especially
70733b9277SNavdeep Parhar#     true for the NIC application functionality.  (Note that there is
71733b9277SNavdeep Parhar#     currently no ability to use the TOE, FCoE, iSCSI, etc. via Virtual
72733b9277SNavdeep Parhar#     Functions so this is in fact solely limited to NIC.)
73733b9277SNavdeep Parhar#
74733b9277SNavdeep Parhar
75733b9277SNavdeep Parhar
76733b9277SNavdeep Parhar# Global configuration settings.
77733b9277SNavdeep Parhar#
78733b9277SNavdeep Parhar[global]
79733b9277SNavdeep Parhar	rss_glb_config_mode = basicvirtual
80733b9277SNavdeep Parhar	rss_glb_config_options = tnlmapen,hashtoeplitz,tnlalllkp
81733b9277SNavdeep Parhar
82733b9277SNavdeep Parhar	# The following Scatter Gather Engine (SGE) settings assume a 4KB Host
83733b9277SNavdeep Parhar	# Page Size and a 64B L1 Cache Line Size. It programs the
84733b9277SNavdeep Parhar	# EgrStatusPageSize and IngPadBoundary to 64B and the PktShift to 2.
85733b9277SNavdeep Parhar	# If a Master PF Driver finds itself on a machine with different
86733b9277SNavdeep Parhar	# parameters, then the Master PF Driver is responsible for initializing
87733b9277SNavdeep Parhar	# these parameters to appropriate values.
88733b9277SNavdeep Parhar	#
89733b9277SNavdeep Parhar	# Notes:
90733b9277SNavdeep Parhar	#  1. The Free List Buffer Sizes below are raw and the firmware will
91733b9277SNavdeep Parhar	#     round them up to the Ingress Padding Boundary.
92733b9277SNavdeep Parhar	#  2. The SGE Timer Values below are expressed below in microseconds.
93733b9277SNavdeep Parhar	#     The firmware will convert these values to Core Clock Ticks when
94733b9277SNavdeep Parhar	#     it processes the configuration parameters.
95733b9277SNavdeep Parhar	#
96733b9277SNavdeep Parhar	reg[0x1008] = 0x40810/0x21c70	# SGE_CONTROL
97733b9277SNavdeep Parhar	reg[0x100c] = 0x22222222	# SGE_HOST_PAGE_SIZE
98733b9277SNavdeep Parhar	reg[0x10a0] = 0x01040810	# SGE_INGRESS_RX_THRESHOLD
99733b9277SNavdeep Parhar	reg[0x1044] = 4096		# SGE_FL_BUFFER_SIZE0
100733b9277SNavdeep Parhar	reg[0x1048] = 65536		# SGE_FL_BUFFER_SIZE1
101733b9277SNavdeep Parhar	reg[0x104c] = 1536		# SGE_FL_BUFFER_SIZE2
102733b9277SNavdeep Parhar	reg[0x1050] = 9024		# SGE_FL_BUFFER_SIZE3
103733b9277SNavdeep Parhar	reg[0x1054] = 9216		# SGE_FL_BUFFER_SIZE4
104733b9277SNavdeep Parhar	reg[0x1058] = 2048		# SGE_FL_BUFFER_SIZE5
105733b9277SNavdeep Parhar	reg[0x105c] = 128		# SGE_FL_BUFFER_SIZE6
106733b9277SNavdeep Parhar	reg[0x1060] = 8192		# SGE_FL_BUFFER_SIZE7
107733b9277SNavdeep Parhar	reg[0x1064] = 16384		# SGE_FL_BUFFER_SIZE8
108733b9277SNavdeep Parhar	reg[0x10a4] = 0xa000a000/0xf000f000 # SGE_DBFIFO_STATUS
109733b9277SNavdeep Parhar	reg[0x10a8] = 0x2000/0x2000	# SGE_DOORBELL_CONTROL
110733b9277SNavdeep Parhar	sge_timer_value = 5, 10, 20, 50, 100, 200 # SGE_TIMER_VALUE* in usecs
111733b9277SNavdeep Parhar
11248d05478SNavdeep Parhar	# enable TP_OUT_CONFIG.IPIDSPLITMODE
11348d05478SNavdeep Parhar	reg[0x7d04] = 0x00010000/0x00010000
11448d05478SNavdeep Parhar
115dd991bd5SNavdeep Parhar	# disable TP_PARA_REG3.RxFragEn
116dd991bd5SNavdeep Parhar	reg[0x7d6c] = 0x00000000/0x00007000
117dd991bd5SNavdeep Parhar
118c7dbd802SNavdeep Parhar	reg[0x7dc0] = 0x0e2f8849		# TP_SHIFT_CNT
119733b9277SNavdeep Parhar
12048d05478SNavdeep Parhar	# TP_VLAN_PRI_MAP to select filter tuples
12148d05478SNavdeep Parhar	# filter tuples : fragmentation, mpshittype, macmatch, ethertype,
12248d05478SNavdeep Parhar	#		  protocol, tos, vlan, vnic_id, port, fcoe
12348d05478SNavdeep Parhar	# valid filterModes are described the Terminator 4 Data Book
1242a5f6b0eSNavdeep Parhar	filterMode = fragmentation, mpshittype, protocol, vlan, port, fcoe
125733b9277SNavdeep Parhar
12648d05478SNavdeep Parhar	# filter tuples enforced in LE active region (equal to or subset of filterMode)
12748d05478SNavdeep Parhar	filterMask = protocol, fcoe
12848d05478SNavdeep Parhar
129733b9277SNavdeep Parhar	# Percentage of dynamic memory (in either the EDRAM or external MEM)
130733b9277SNavdeep Parhar	# to use for TP RX payload
131c7dbd802SNavdeep Parhar	tp_pmrx = 34
132733b9277SNavdeep Parhar
133733b9277SNavdeep Parhar	# TP RX payload page size
134733b9277SNavdeep Parhar	tp_pmrx_pagesize = 64K
135733b9277SNavdeep Parhar
136f72b68a1SNavdeep Parhar	# TP number of RX channels
137f72b68a1SNavdeep Parhar	tp_nrxch = 0		# 0 (auto) = 1
138f72b68a1SNavdeep Parhar
139733b9277SNavdeep Parhar	# Percentage of dynamic memory (in either the EDRAM or external MEM)
140733b9277SNavdeep Parhar	# to use for TP TX payload
141c7dbd802SNavdeep Parhar	tp_pmtx = 32
142733b9277SNavdeep Parhar
143733b9277SNavdeep Parhar	# TP TX payload page size
144733b9277SNavdeep Parhar	tp_pmtx_pagesize = 64K
145733b9277SNavdeep Parhar
146f72b68a1SNavdeep Parhar	# TP number of TX channels
147f72b68a1SNavdeep Parhar	tp_ntxch = 0		# 0 (auto) = equal number of ports
148f72b68a1SNavdeep Parhar
14948d05478SNavdeep Parhar	# TP OFLD MTUs
15048d05478SNavdeep Parhar	tp_mtus = 88, 256, 512, 576, 808, 1024, 1280, 1488, 1500, 2002, 2048, 4096, 4352, 8192, 9000, 9600
15148d05478SNavdeep Parhar
152c7dbd802SNavdeep Parhar	# ULPRX iSCSI Page Sizes
153c7dbd802SNavdeep Parhar	reg[0x19168] = 0x04020100 # 64K, 16K, 8K and 4K
154c7dbd802SNavdeep Parhar
155733b9277SNavdeep Parhar# Some "definitions" to make the rest of this a bit more readable.  We support
156733b9277SNavdeep Parhar# 4 ports, 3 functions (NIC, FCoE and iSCSI), scaling up to 8 "CPU Queue Sets"
157733b9277SNavdeep Parhar# per function per port ...
158733b9277SNavdeep Parhar#
159733b9277SNavdeep Parhar# NMSIX = 1088			# available MSI-X Vectors
160733b9277SNavdeep Parhar# NVI = 128			# available Virtual Interfaces
161733b9277SNavdeep Parhar# NMPSTCAM = 336		# MPS TCAM entries
162733b9277SNavdeep Parhar#
163733b9277SNavdeep Parhar# NPORTS = 4			# ports
164733b9277SNavdeep Parhar# NCPUS = 8			# CPUs we want to support scalably
165733b9277SNavdeep Parhar# NFUNCS = 3			# functions per port (NIC, FCoE, iSCSI)
166733b9277SNavdeep Parhar
167733b9277SNavdeep Parhar# Breakdown of Virtual Interface/Queue/Interrupt resources for the "Unified
168733b9277SNavdeep Parhar# PF" which many OS Drivers will use to manage most or all functions.
169733b9277SNavdeep Parhar#
170733b9277SNavdeep Parhar# Each Ingress Queue can use one MSI-X interrupt but some Ingress Queues can
171733b9277SNavdeep Parhar# use Forwarded Interrupt Ingress Queues.  For these latter, an Ingress Queue
172733b9277SNavdeep Parhar# would be created and the Queue ID of a Forwarded Interrupt Ingress Queue
173733b9277SNavdeep Parhar# will be specified as the "Ingress Queue Asynchronous Destination Index."
174733b9277SNavdeep Parhar# Thus, the number of MSI-X Vectors assigned to the Unified PF will be less
175733b9277SNavdeep Parhar# than or equal to the number of Ingress Queues ...
176733b9277SNavdeep Parhar#
177733b9277SNavdeep Parhar# NVI_NIC = 4			# NIC access to NPORTS
178733b9277SNavdeep Parhar# NFLIQ_NIC = 32		# NIC Ingress Queues with Free Lists
179733b9277SNavdeep Parhar# NETHCTRL_NIC = 32		# NIC Ethernet Control/TX Queues
180733b9277SNavdeep Parhar# NEQ_NIC = 64			# NIC Egress Queues (FL, ETHCTRL/TX)
181733b9277SNavdeep Parhar# NMPSTCAM_NIC = 16		# NIC MPS TCAM Entries (NPORTS*4)
182733b9277SNavdeep Parhar# NMSIX_NIC = 32		# NIC MSI-X Interrupt Vectors (FLIQ)
183733b9277SNavdeep Parhar#
184733b9277SNavdeep Parhar# NVI_OFLD = 0			# Offload uses NIC function to access ports
185733b9277SNavdeep Parhar# NFLIQ_OFLD = 16		# Offload Ingress Queues with Free Lists
186733b9277SNavdeep Parhar# NETHCTRL_OFLD = 0		# Offload Ethernet Control/TX Queues
187733b9277SNavdeep Parhar# NEQ_OFLD = 16			# Offload Egress Queues (FL)
188733b9277SNavdeep Parhar# NMPSTCAM_OFLD = 0		# Offload MPS TCAM Entries (uses NIC's)
189733b9277SNavdeep Parhar# NMSIX_OFLD = 16		# Offload MSI-X Interrupt Vectors (FLIQ)
190733b9277SNavdeep Parhar#
191733b9277SNavdeep Parhar# NVI_RDMA = 0			# RDMA uses NIC function to access ports
192733b9277SNavdeep Parhar# NFLIQ_RDMA = 4		# RDMA Ingress Queues with Free Lists
193733b9277SNavdeep Parhar# NETHCTRL_RDMA = 0		# RDMA Ethernet Control/TX Queues
194733b9277SNavdeep Parhar# NEQ_RDMA = 4			# RDMA Egress Queues (FL)
195733b9277SNavdeep Parhar# NMPSTCAM_RDMA = 0		# RDMA MPS TCAM Entries (uses NIC's)
196733b9277SNavdeep Parhar# NMSIX_RDMA = 4		# RDMA MSI-X Interrupt Vectors (FLIQ)
197733b9277SNavdeep Parhar#
198733b9277SNavdeep Parhar# NEQ_WD = 128			# Wire Direct TX Queues and FLs
199733b9277SNavdeep Parhar# NETHCTRL_WD = 64		# Wire Direct TX Queues
200733b9277SNavdeep Parhar# NFLIQ_WD = 64	`		# Wire Direct Ingress Queues with Free Lists
201733b9277SNavdeep Parhar#
202733b9277SNavdeep Parhar# NVI_ISCSI = 4			# ISCSI access to NPORTS
203733b9277SNavdeep Parhar# NFLIQ_ISCSI = 4		# ISCSI Ingress Queues with Free Lists
204733b9277SNavdeep Parhar# NETHCTRL_ISCSI = 0		# ISCSI Ethernet Control/TX Queues
205733b9277SNavdeep Parhar# NEQ_ISCSI = 4			# ISCSI Egress Queues (FL)
206733b9277SNavdeep Parhar# NMPSTCAM_ISCSI = 4		# ISCSI MPS TCAM Entries (NPORTS)
207733b9277SNavdeep Parhar# NMSIX_ISCSI = 4		# ISCSI MSI-X Interrupt Vectors (FLIQ)
208733b9277SNavdeep Parhar#
209733b9277SNavdeep Parhar# NVI_FCOE = 4			# FCOE access to NPORTS
210733b9277SNavdeep Parhar# NFLIQ_FCOE = 34		# FCOE Ingress Queues with Free Lists
211733b9277SNavdeep Parhar# NETHCTRL_FCOE = 32		# FCOE Ethernet Control/TX Queues
212733b9277SNavdeep Parhar# NEQ_FCOE = 66			# FCOE Egress Queues (FL)
213733b9277SNavdeep Parhar# NMPSTCAM_FCOE = 32 		# FCOE MPS TCAM Entries (NPORTS)
214733b9277SNavdeep Parhar# NMSIX_FCOE = 34		# FCOE MSI-X Interrupt Vectors (FLIQ)
215733b9277SNavdeep Parhar
216733b9277SNavdeep Parhar# Two extra Ingress Queues per function for Firmware Events and Forwarded
217733b9277SNavdeep Parhar# Interrupts, and two extra interrupts per function for Firmware Events (or a
218733b9277SNavdeep Parhar# Forwarded Interrupt Queue) and General Interrupts per function.
219733b9277SNavdeep Parhar#
220733b9277SNavdeep Parhar# NFLIQ_EXTRA = 6		# "extra" Ingress Queues 2*NFUNCS (Firmware and
221733b9277SNavdeep Parhar# 				#   Forwarded Interrupts
222733b9277SNavdeep Parhar# NMSIX_EXTRA = 6		# extra interrupts 2*NFUNCS (Firmware and
223733b9277SNavdeep Parhar# 				#   General Interrupts
224733b9277SNavdeep Parhar
225733b9277SNavdeep Parhar# Microsoft HyperV resources.  The HyperV Virtual Ingress Queues will have
226733b9277SNavdeep Parhar# their interrupts forwarded to another set of Forwarded Interrupt Queues.
227733b9277SNavdeep Parhar#
228733b9277SNavdeep Parhar# NVI_HYPERV = 16		# VMs we want to support
229733b9277SNavdeep Parhar# NVIIQ_HYPERV = 2		# Virtual Ingress Queues with Free Lists per VM
230733b9277SNavdeep Parhar# NFLIQ_HYPERV = 40		# VIQs + NCPUS Forwarded Interrupt Queues
231733b9277SNavdeep Parhar# NEQ_HYPERV = 32		# VIQs Free Lists
232733b9277SNavdeep Parhar# NMPSTCAM_HYPERV = 16		# MPS TCAM Entries (NVI_HYPERV)
233733b9277SNavdeep Parhar# NMSIX_HYPERV = 8		# NCPUS Forwarded Interrupt Queues
234733b9277SNavdeep Parhar
235733b9277SNavdeep Parhar# Adding all of the above Unified PF resource needs together: (NIC + OFLD +
236733b9277SNavdeep Parhar# RDMA + ISCSI + FCOE + EXTRA + HYPERV)
237733b9277SNavdeep Parhar#
238733b9277SNavdeep Parhar# NVI_UNIFIED = 28
239733b9277SNavdeep Parhar# NFLIQ_UNIFIED = 106
240733b9277SNavdeep Parhar# NETHCTRL_UNIFIED = 32
241733b9277SNavdeep Parhar# NEQ_UNIFIED = 124
242733b9277SNavdeep Parhar# NMPSTCAM_UNIFIED = 40
243733b9277SNavdeep Parhar#
244733b9277SNavdeep Parhar# The sum of all the MSI-X resources above is 74 MSI-X Vectors but we'll round
245733b9277SNavdeep Parhar# that up to 128 to make sure the Unified PF doesn't run out of resources.
246733b9277SNavdeep Parhar#
247733b9277SNavdeep Parhar# NMSIX_UNIFIED = 128
248733b9277SNavdeep Parhar#
249733b9277SNavdeep Parhar# The Storage PFs could need up to NPORTS*NCPUS + NMSIX_EXTRA MSI-X Vectors
250733b9277SNavdeep Parhar# which is 34 but they're probably safe with 32.
251733b9277SNavdeep Parhar#
252733b9277SNavdeep Parhar# NMSIX_STORAGE = 32
253733b9277SNavdeep Parhar
254733b9277SNavdeep Parhar# Note: The UnifiedPF is PF4 which doesn't have any Virtual Functions
255733b9277SNavdeep Parhar# associated with it.  Thus, the MSI-X Vector allocations we give to the
256733b9277SNavdeep Parhar# UnifiedPF aren't inherited by any Virtual Functions.  As a result we can
257733b9277SNavdeep Parhar# provision many more Virtual Functions than we can if the UnifiedPF were
258733b9277SNavdeep Parhar# one of PF0-3.
259733b9277SNavdeep Parhar#
260733b9277SNavdeep Parhar
261733b9277SNavdeep Parhar# All of the below PCI-E parameters are actually stored in various *_init.txt
262733b9277SNavdeep Parhar# files.  We include them below essentially as comments.
263733b9277SNavdeep Parhar#
264733b9277SNavdeep Parhar# For PF0-3 we assign 8 vectors each for NIC Ingress Queues of the associated
265733b9277SNavdeep Parhar# ports 0-3.
266733b9277SNavdeep Parhar#
267733b9277SNavdeep Parhar# For PF4, the Unified PF, we give it an MSI-X Table Size as outlined above.
268733b9277SNavdeep Parhar#
269733b9277SNavdeep Parhar# For PF5-6 we assign enough MSI-X Vectors to support FCoE and iSCSI
270733b9277SNavdeep Parhar# storage applications across all four possible ports.
271733b9277SNavdeep Parhar#
272733b9277SNavdeep Parhar# Additionally, since the UnifiedPF isn't one of the per-port Physical
273733b9277SNavdeep Parhar# Functions, we give the UnifiedPF and the PF0-3 Physical Functions
274733b9277SNavdeep Parhar# different PCI Device IDs which will allow Unified and Per-Port Drivers
275733b9277SNavdeep Parhar# to directly select the type of Physical Function to which they wish to be
276733b9277SNavdeep Parhar# attached.
277733b9277SNavdeep Parhar#
278612226d7SPedro F. Giffuni# Note that the actual values used for the PCI-E Intelectual Property will be
279733b9277SNavdeep Parhar# 1 less than those below since that's the way it "counts" things.  For
280733b9277SNavdeep Parhar# readability, we use the number we actually mean ...
281733b9277SNavdeep Parhar#
282733b9277SNavdeep Parhar# PF0_INT = 8			# NCPUS
283733b9277SNavdeep Parhar# PF1_INT = 8			# NCPUS
284733b9277SNavdeep Parhar# PF2_INT = 8			# NCPUS
285733b9277SNavdeep Parhar# PF3_INT = 8			# NCPUS
286733b9277SNavdeep Parhar# PF0_3_INT = 32		# PF0_INT + PF1_INT + PF2_INT + PF3_INT
287733b9277SNavdeep Parhar#
288733b9277SNavdeep Parhar# PF4_INT = 128			# NMSIX_UNIFIED
289733b9277SNavdeep Parhar# PF5_INT = 32			# NMSIX_STORAGE
290733b9277SNavdeep Parhar# PF6_INT = 32			# NMSIX_STORAGE
291733b9277SNavdeep Parhar# PF7_INT = 0			# Nothing Assigned
292733b9277SNavdeep Parhar# PF4_7_INT = 192		# PF4_INT + PF5_INT + PF6_INT + PF7_INT
293733b9277SNavdeep Parhar#
294733b9277SNavdeep Parhar# PF0_7_INT = 224		# PF0_3_INT + PF4_7_INT
295733b9277SNavdeep Parhar#
296733b9277SNavdeep Parhar# With the above we can get 17 VFs/PF0-3 (limited by 336 MPS TCAM entries)
297733b9277SNavdeep Parhar# but we'll lower that to 16 to make our total 64 and a nice power of 2 ...
298733b9277SNavdeep Parhar#
299733b9277SNavdeep Parhar# NVF = 16
300733b9277SNavdeep Parhar
301733b9277SNavdeep Parhar# For those OSes which manage different ports on different PFs, we need
302733b9277SNavdeep Parhar# only enough resources to support a single port's NIC application functions
303733b9277SNavdeep Parhar# on PF0-3.  The below assumes that we're only doing NIC with NCPUS "Queue
304733b9277SNavdeep Parhar# Sets" for ports 0-3.  The FCoE and iSCSI functions for such OSes will be
305733b9277SNavdeep Parhar# managed on the "storage PFs" (see below).
306733b9277SNavdeep Parhar#
307733b9277SNavdeep Parhar[function "0"]
308733b9277SNavdeep Parhar	nvf = 16		# NVF on this function
309733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
310733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
311733b9277SNavdeep Parhar	nvi = 1			# 1 port
312733b9277SNavdeep Parhar	niqflint = 8		# NCPUS "Queue Sets"
313733b9277SNavdeep Parhar	nethctrl = 8		# NCPUS "Queue Sets"
314733b9277SNavdeep Parhar	neq = 16		# niqflint + nethctrl Egress Queues
315733b9277SNavdeep Parhar	nexactf = 8		# number of exact MPSTCAM MAC filters
316733b9277SNavdeep Parhar	cmask = all		# access to all channels
317733b9277SNavdeep Parhar	pmask = 0x1		# access to only one port
318733b9277SNavdeep Parhar
319733b9277SNavdeep Parhar[function "1"]
320733b9277SNavdeep Parhar	nvf = 16		# NVF on this function
321733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
322733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
323733b9277SNavdeep Parhar	nvi = 1			# 1 port
324733b9277SNavdeep Parhar	niqflint = 8		# NCPUS "Queue Sets"
325733b9277SNavdeep Parhar	nethctrl = 8		# NCPUS "Queue Sets"
326733b9277SNavdeep Parhar	neq = 16		# niqflint + nethctrl Egress Queues
327733b9277SNavdeep Parhar	nexactf = 8		# number of exact MPSTCAM MAC filters
328733b9277SNavdeep Parhar	cmask = all		# access to all channels
329733b9277SNavdeep Parhar	pmask = 0x2		# access to only one port
330733b9277SNavdeep Parhar
331733b9277SNavdeep Parhar[function "2"]
332733b9277SNavdeep Parhar	nvf = 16		# NVF on this function
333733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
334733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
335733b9277SNavdeep Parhar	nvi = 1			# 1 port
336733b9277SNavdeep Parhar	niqflint = 8		# NCPUS "Queue Sets"
337733b9277SNavdeep Parhar	nethctrl = 8		# NCPUS "Queue Sets"
338733b9277SNavdeep Parhar	neq = 16		# niqflint + nethctrl Egress Queues
339733b9277SNavdeep Parhar	nexactf = 8		# number of exact MPSTCAM MAC filters
340733b9277SNavdeep Parhar	cmask = all		# access to all channels
341733b9277SNavdeep Parhar	pmask = 0x4		# access to only one port
342733b9277SNavdeep Parhar
343733b9277SNavdeep Parhar[function "3"]
344733b9277SNavdeep Parhar	nvf = 16		# NVF on this function
345733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
346733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
347733b9277SNavdeep Parhar	nvi = 1			# 1 port
348733b9277SNavdeep Parhar	niqflint = 8		# NCPUS "Queue Sets"
349733b9277SNavdeep Parhar	nethctrl = 8		# NCPUS "Queue Sets"
350733b9277SNavdeep Parhar	neq = 16		# niqflint + nethctrl Egress Queues
351733b9277SNavdeep Parhar	nexactf = 8		# number of exact MPSTCAM MAC filters
352733b9277SNavdeep Parhar	cmask = all		# access to all channels
353733b9277SNavdeep Parhar	pmask = 0x8		# access to only one port
354733b9277SNavdeep Parhar
355733b9277SNavdeep Parhar# Some OS Drivers manage all application functions for all ports via PF4.
356733b9277SNavdeep Parhar# Thus we need to provide a large number of resources here.  For Egress
357733b9277SNavdeep Parhar# Queues we need to account for both TX Queues as well as Free List Queues
358733b9277SNavdeep Parhar# (because the host is responsible for producing Free List Buffers for the
359733b9277SNavdeep Parhar# hardware to consume).
360733b9277SNavdeep Parhar#
361733b9277SNavdeep Parhar[function "4"]
362733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
363733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
364733b9277SNavdeep Parhar	nvi = 28		# NVI_UNIFIED
365733b9277SNavdeep Parhar	niqflint = 170		# NFLIQ_UNIFIED + NLFIQ_WD
3662a5f6b0eSNavdeep Parhar	nethctrl = 100		# NETHCTRL_UNIFIED + NETHCTRL_WD
3672a5f6b0eSNavdeep Parhar	neq = 256		# NEQ_UNIFIED + NEQ_WD
368733b9277SNavdeep Parhar	nexactf = 40		# NMPSTCAM_UNIFIED
369733b9277SNavdeep Parhar	cmask = all		# access to all channels
370733b9277SNavdeep Parhar	pmask = all		# access to all four ports ...
3712a5f6b0eSNavdeep Parhar	nethofld = 1024		# number of user mode ethernet flow contexts
372733b9277SNavdeep Parhar	nroute = 32		# number of routing region entries
373733b9277SNavdeep Parhar	nclip = 32		# number of clip region entries
3742a5f6b0eSNavdeep Parhar	nfilter = 496		# number of filter region entries
3752a5f6b0eSNavdeep Parhar	nserver = 496		# number of server region entries
3762a5f6b0eSNavdeep Parhar	nhash = 12288		# number of hash region entries
377733b9277SNavdeep Parhar	protocol = nic_vm, ofld, rddp, rdmac, iscsi_initiator_pdu, iscsi_target_pdu
3782a5f6b0eSNavdeep Parhar	tp_l2t = 3072
37948d05478SNavdeep Parhar	tp_ddp = 3
380733b9277SNavdeep Parhar	tp_ddp_iscsi = 2
38148d05478SNavdeep Parhar	tp_stag = 3
38248d05478SNavdeep Parhar	tp_pbl = 10
38348d05478SNavdeep Parhar	tp_rq = 13
384733b9277SNavdeep Parhar
385733b9277SNavdeep Parhar# We have FCoE and iSCSI storage functions on PF5 and PF6 each of which may
386733b9277SNavdeep Parhar# need to have Virtual Interfaces on each of the four ports with up to NCPUS
387733b9277SNavdeep Parhar# "Queue Sets" each.
388733b9277SNavdeep Parhar#
389733b9277SNavdeep Parhar[function "5"]
390733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
391733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
392733b9277SNavdeep Parhar	nvi = 4			# NPORTS
393733b9277SNavdeep Parhar	niqflint = 34		# NPORTS*NCPUS + NMSIX_EXTRA
394733b9277SNavdeep Parhar	nethctrl = 32		# NPORTS*NCPUS
395733b9277SNavdeep Parhar	neq = 64		# NPORTS*NCPUS * 2 (FL, ETHCTRL/TX)
396733b9277SNavdeep Parhar	nexactf = 4		# NPORTS
397733b9277SNavdeep Parhar	cmask = all		# access to all channels
398733b9277SNavdeep Parhar	pmask = all		# access to all four ports ...
3992a5f6b0eSNavdeep Parhar	nserver = 16
4002a5f6b0eSNavdeep Parhar	nhash = 2048
401c7dbd802SNavdeep Parhar	tp_l2t = 1020
4022a5f6b0eSNavdeep Parhar	protocol = iscsi_initiator_fofld
4032a5f6b0eSNavdeep Parhar	tp_ddp_iscsi = 2
4042a5f6b0eSNavdeep Parhar	iscsi_ntask = 2048
4052a5f6b0eSNavdeep Parhar	iscsi_nsess = 2048
4062a5f6b0eSNavdeep Parhar	iscsi_nconn_per_session = 1
4072a5f6b0eSNavdeep Parhar	iscsi_ninitiator_instance = 64
408733b9277SNavdeep Parhar
409733b9277SNavdeep Parhar[function "6"]
410733b9277SNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
411733b9277SNavdeep Parhar	r_caps = all		# read permissions for all commands
412733b9277SNavdeep Parhar	nvi = 4			# NPORTS
413733b9277SNavdeep Parhar	niqflint = 34		# NPORTS*NCPUS + NMSIX_EXTRA
414733b9277SNavdeep Parhar	nethctrl = 32		# NPORTS*NCPUS
415733b9277SNavdeep Parhar	neq = 66		# NPORTS*NCPUS * 2 (FL, ETHCTRL/TX) + 2 (EXTRA)
416733b9277SNavdeep Parhar	nexactf = 32		# NPORTS + adding 28 exact entries for FCoE
417733b9277SNavdeep Parhar				# which is OK since < MIN(SUM PF0..3, PF4)
418733b9277SNavdeep Parhar				# and we never load PF0..3 and PF4 concurrently
419733b9277SNavdeep Parhar	cmask = all		# access to all channels
420733b9277SNavdeep Parhar	pmask = all		# access to all four ports ...
4212a5f6b0eSNavdeep Parhar	nhash = 2048
422c7dbd802SNavdeep Parhar	tp_l2t = 4
423733b9277SNavdeep Parhar	protocol = fcoe_initiator
42448d05478SNavdeep Parhar	tp_ddp = 1
425733b9277SNavdeep Parhar	fcoe_nfcf = 16
426733b9277SNavdeep Parhar	fcoe_nvnp = 32
427733b9277SNavdeep Parhar	fcoe_nssn = 1024
428733b9277SNavdeep Parhar
4292a5f6b0eSNavdeep Parhar# The following function, 1023, is not an actual PCIE function but is used to
4302a5f6b0eSNavdeep Parhar# configure and reserve firmware internal resources that come from the global
4312a5f6b0eSNavdeep Parhar# resource pool.
4322a5f6b0eSNavdeep Parhar#
4332a5f6b0eSNavdeep Parhar[function "1023"]
4342a5f6b0eSNavdeep Parhar	wx_caps = all		# write/execute permissions for all commands
4352a5f6b0eSNavdeep Parhar	r_caps = all		# read permissions for all commands
4362a5f6b0eSNavdeep Parhar	nvi = 4			# NVI_UNIFIED
4372a5f6b0eSNavdeep Parhar	cmask = all		# access to all channels
4382a5f6b0eSNavdeep Parhar	pmask = all		# access to all four ports ...
4392a5f6b0eSNavdeep Parhar	nexactf = 8		# NPORTS + DCBX +
4402a5f6b0eSNavdeep Parhar	nfilter = 16		# number of filter region entries
4412a5f6b0eSNavdeep Parhar
442733b9277SNavdeep Parhar# For Virtual functions, we only allow NIC functionality and we only allow
443733b9277SNavdeep Parhar# access to one port (1 << PF).  Note that because of limitations in the
444733b9277SNavdeep Parhar# Scatter Gather Engine (SGE) hardware which checks writes to VF KDOORBELL
445733b9277SNavdeep Parhar# and GTS registers, the number of Ingress and Egress Queues must be a power
446733b9277SNavdeep Parhar# of 2.
447733b9277SNavdeep Parhar#
448733b9277SNavdeep Parhar[function "0/*"]		# NVF
449733b9277SNavdeep Parhar	wx_caps = 0x82		# DMAQ | VF
450733b9277SNavdeep Parhar	r_caps = 0x86		# DMAQ | VF | PORT
451733b9277SNavdeep Parhar	nvi = 1			# 1 port
452733b9277SNavdeep Parhar	niqflint = 4		# 2 "Queue Sets" + NXIQ
453733b9277SNavdeep Parhar	nethctrl = 2		# 2 "Queue Sets"
454733b9277SNavdeep Parhar	neq = 4			# 2 "Queue Sets" * 2
455733b9277SNavdeep Parhar	nexactf = 4
456733b9277SNavdeep Parhar	cmask = all		# access to all channels
457733b9277SNavdeep Parhar	pmask = 0x1		# access to only one port ...
458733b9277SNavdeep Parhar
459733b9277SNavdeep Parhar[function "1/*"]		# NVF
460733b9277SNavdeep Parhar	wx_caps = 0x82		# DMAQ | VF
461733b9277SNavdeep Parhar	r_caps = 0x86		# DMAQ | VF | PORT
462733b9277SNavdeep Parhar	nvi = 1			# 1 port
463733b9277SNavdeep Parhar	niqflint = 4		# 2 "Queue Sets" + NXIQ
464733b9277SNavdeep Parhar	nethctrl = 2		# 2 "Queue Sets"
465733b9277SNavdeep Parhar	neq = 4			# 2 "Queue Sets" * 2
466733b9277SNavdeep Parhar	nexactf = 4
467733b9277SNavdeep Parhar	cmask = all		# access to all channels
468733b9277SNavdeep Parhar	pmask = 0x2		# access to only one port ...
469733b9277SNavdeep Parhar
470733b9277SNavdeep Parhar[function "2/*"]		# NVF
471733b9277SNavdeep Parhar	wx_caps = 0x82		# DMAQ | VF
472733b9277SNavdeep Parhar	r_caps = 0x86		# DMAQ | VF | PORT
473733b9277SNavdeep Parhar	nvi = 1			# 1 port
474733b9277SNavdeep Parhar	niqflint = 4		# 2 "Queue Sets" + NXIQ
475733b9277SNavdeep Parhar	nethctrl = 2		# 2 "Queue Sets"
476733b9277SNavdeep Parhar	neq = 4			# 2 "Queue Sets" * 2
477733b9277SNavdeep Parhar	nexactf = 4
478733b9277SNavdeep Parhar	cmask = all		# access to all channels
479733b9277SNavdeep Parhar	pmask = 0x4		# access to only one port ...
480733b9277SNavdeep Parhar
481733b9277SNavdeep Parhar[function "3/*"]		# NVF
482733b9277SNavdeep Parhar	wx_caps = 0x82		# DMAQ | VF
483733b9277SNavdeep Parhar	r_caps = 0x86		# DMAQ | VF | PORT
484733b9277SNavdeep Parhar	nvi = 1			# 1 port
485733b9277SNavdeep Parhar	niqflint = 4		# 2 "Queue Sets" + NXIQ
486733b9277SNavdeep Parhar	nethctrl = 2		# 2 "Queue Sets"
487733b9277SNavdeep Parhar	neq = 4			# 2 "Queue Sets" * 2
488733b9277SNavdeep Parhar	nexactf = 4
489733b9277SNavdeep Parhar	cmask = all		# access to all channels
490733b9277SNavdeep Parhar	pmask = 0x8		# access to only one port ...
491733b9277SNavdeep Parhar
492733b9277SNavdeep Parhar# MPS features a 196608 bytes ingress buffer that is used for ingress buffering
493733b9277SNavdeep Parhar# for packets from the wire as well as the loopback path of the L2 switch. The
494733b9277SNavdeep Parhar# folling params control how the buffer memory is distributed and the L2 flow
495733b9277SNavdeep Parhar# control settings:
496733b9277SNavdeep Parhar#
497733b9277SNavdeep Parhar# bg_mem:	%-age of mem to use for port/buffer group
498733b9277SNavdeep Parhar# lpbk_mem:	%-age of port/bg mem to use for loopback
499733b9277SNavdeep Parhar# hwm:		high watermark; bytes available when starting to send pause
500733b9277SNavdeep Parhar#		frames (in units of 0.1 MTU)
501733b9277SNavdeep Parhar# lwm:		low watermark; bytes remaining when sending 'unpause' frame
502733b9277SNavdeep Parhar#		(in inuits of 0.1 MTU)
503733b9277SNavdeep Parhar# dwm:		minimum delta between high and low watermark (in units of 100
504733b9277SNavdeep Parhar#		Bytes)
505733b9277SNavdeep Parhar#
506327235b3SNavdeep Parhar#
507327235b3SNavdeep Parhar
508733b9277SNavdeep Parhar[port "0"]
509733b9277SNavdeep Parhar	dcb = ppp, dcbx		# configure for DCB PPP and enable DCBX offload
510733b9277SNavdeep Parhar	bg_mem = 25
511733b9277SNavdeep Parhar	lpbk_mem = 25
512733b9277SNavdeep Parhar	hwm = 30
513733b9277SNavdeep Parhar	lwm = 15
514733b9277SNavdeep Parhar	dwm = 30
515327235b3SNavdeep Parhar	dcb_app_tlv[0] = 0x8906, ethertype, 3
516327235b3SNavdeep Parhar	dcb_app_tlv[1] = 0x8914, ethertype, 3
517327235b3SNavdeep Parhar	dcb_app_tlv[2] = 3260, socketnum, 5
518733b9277SNavdeep Parhar
519733b9277SNavdeep Parhar[port "1"]
520733b9277SNavdeep Parhar	dcb = ppp, dcbx
521733b9277SNavdeep Parhar	bg_mem = 25
522733b9277SNavdeep Parhar	lpbk_mem = 25
523733b9277SNavdeep Parhar	hwm = 30
524733b9277SNavdeep Parhar	lwm = 15
525733b9277SNavdeep Parhar	dwm = 30
526327235b3SNavdeep Parhar	dcb_app_tlv[0] = 0x8906, ethertype, 3
527327235b3SNavdeep Parhar	dcb_app_tlv[1] = 0x8914, ethertype, 3
528327235b3SNavdeep Parhar	dcb_app_tlv[2] = 3260, socketnum, 5
529733b9277SNavdeep Parhar
530733b9277SNavdeep Parhar[port "2"]
531733b9277SNavdeep Parhar	dcb = ppp, dcbx
532733b9277SNavdeep Parhar	bg_mem = 25
533733b9277SNavdeep Parhar	lpbk_mem = 25
534733b9277SNavdeep Parhar	hwm = 30
535733b9277SNavdeep Parhar	lwm = 15
536733b9277SNavdeep Parhar	dwm = 30
537327235b3SNavdeep Parhar	dcb_app_tlv[0] = 0x8906, ethertype, 3
538327235b3SNavdeep Parhar	dcb_app_tlv[1] = 0x8914, ethertype, 3
539327235b3SNavdeep Parhar	dcb_app_tlv[2] = 3260, socketnum, 5
540733b9277SNavdeep Parhar
541733b9277SNavdeep Parhar[port "3"]
542733b9277SNavdeep Parhar	dcb = ppp, dcbx
543733b9277SNavdeep Parhar	bg_mem = 25
544733b9277SNavdeep Parhar	lpbk_mem = 25
545733b9277SNavdeep Parhar	hwm = 30
546733b9277SNavdeep Parhar	lwm = 15
547733b9277SNavdeep Parhar	dwm = 30
548327235b3SNavdeep Parhar	dcb_app_tlv[0] = 0x8906, ethertype, 3
549327235b3SNavdeep Parhar	dcb_app_tlv[1] = 0x8914, ethertype, 3
550327235b3SNavdeep Parhar	dcb_app_tlv[2] = 3260, socketnum, 5
551733b9277SNavdeep Parhar
552733b9277SNavdeep Parhar[fini]
553*7c0cad38SNavdeep Parhar	version = 0x01000028
554*7c0cad38SNavdeep Parhar	checksum = 0x5ceab421
555733b9277SNavdeep Parhar
556733b9277SNavdeep Parhar# Total resources used by above allocations:
557733b9277SNavdeep Parhar#   Virtual Interfaces: 104
558733b9277SNavdeep Parhar#   Ingress Queues/w Free Lists and Interrupts: 526
559733b9277SNavdeep Parhar#   Egress Queues: 702
560733b9277SNavdeep Parhar#   MPS TCAM Entries: 336
561733b9277SNavdeep Parhar#   MSI-X Vectors: 736
562733b9277SNavdeep Parhar#   Virtual Functions: 64
563733b9277SNavdeep Parhar#
564733b9277SNavdeep Parhar#
565