1.. SPDX-License-Identifier: GPL-2.0 2 3=================================================== 4FSP (Foundation Security Processor) and Secure Boot 5=================================================== 6This document describes the role of the FSP in the GPU boot sequence on 7Hopper and Blackwell GPUs, and how it differs from the earlier Ampere boot 8flow. It also provides a brief overview of the PRC (Product Reconfiguration 9Control) protocol used to query device configuration through FSP. As with 10other documents in this directory, the information is subject to change and 11is intended to help developers understand the corresponding kernel code. 12 13What is FSP? 14============ 15The Foundation Security Processor (FSP) is the GPU's Internal Root of Trust 16(IROT). It is a dedicated security processor that boots from immutable ROM 17(Boot ROM) inside the GPU and is responsible for establishing the Chain of 18Trust before any other firmware is allowed to run. 19 20FSP runs independently of the host CPU and starts executing as soon as the 21GPU is powered on. By the time the nova-core driver is loaded, FSP has 22already completed its own secure boot and is ready to accept commands from 23the driver. 24 25Simplified boot flow (Hopper/Blackwell) 26======================================= 27Starting with Hopper, the boot flow is significantly simplified compared to 28earlier GPU generations like Ampere. 29 30On an **Ampere** GPU, the boot verification chain involves multiple Falcon 31engines and multiple ucode stages (see falcon.rst for details):: 32 33 Hardware BROM (SEC2) 34 -> HS Booter (SEC2) 35 -> LS GSP-RM (GSP) 36 37The driver must extract ucode from VBIOS, manage SEC2 and GSP, and 38orchestrate the Booter to load GSP-RM. This involves FWSEC-FRTS, devinit, 39and the Booter stages. 40 41On **Hopper/Blackwell** GPUs, FSP replaces this multi-stage process with a 42single message-driven interface:: 43 44 FSP (hardware root of trust, boots from ROM) 45 -> FMC (Falcon Microcontroller, verified by FSP) 46 -> GSP-RM (verified and loaded by FMC) 47 48The driver only needs to: 49 501. Wait for FSP to complete its own secure boot (polling a scratch register). 512. Send a Chain of Trust (COT) message to FSP with the FMC firmware location, 52 cryptographic signatures, and GSP boot parameters. 533. FSP authenticates the FMC firmware and boots it, FMC in turn loads GSP-RM. 54 55There is no SEC2 involvement, no Booter ucode, and no FWSEC-FRTS stage. The 56entire secure boot is driven by a single FSP message exchange. 57 58Chain of Trust (COT) protocol 59============================= 60The Chain of Trust establishes a cryptographically enforced boot sequence, 61ensuring the GPU reaches a known, trusted state. 62 63The driver communicates with FSP using a message queue (Falcon MSGQ 64interface). Each message consists of an MCTP (Management Component Transport 65Protocol) transport header and an NVDM (NVIDIA Vendor Defined Message) header, 66followed by a protocol-specific payload. 67 68For Chain of Trust, the payload includes: 69 70- The system memory address of the FMC firmware image. 71- Cryptographic material: a SHA-384 hash, RSA-3K public key, and RSA-3K 72 signature extracted from the FMC ELF firmware. 73- FRTS (Firmware Runtime Services) region information (vidmem offset and size). 74- The system memory address of the GSP boot arguments structure. 75 76FSP verifies the signature against the provided public key and hash, and if 77verification succeeds, boots the FMC. The FMC then authenticates and launches 78GSP-RM. 79 80The message flow is:: 81 82 nova-core FSP 83 | | 84 | 1. Poll scratch register | 85 | (wait for FSP boot complete) | 86 | | 87 | 2. COT message ------------> | 88 | (FMC addr, signatures, | 89 | boot params) | 90 | | 91 | |--- Verify FMC signature 92 | |--- Boot FMC 93 | |--- FMC loads GSP-RM 94 | | 95 | 3. COT response <------------ | 96 | (success/error) | 97 | | 98 99FSP message format 100================== 101All FSP messages share a common header format consisting of two 32-bit words: 102 103**MCTP header** (Management Component Transport Protocol): 104 105- Bit 31: SOM (Start of Message) 106- Bit 30: EOM (End of Message) 107- Bits 29:28: Packet sequence number 108- Bits 23:16: Source Endpoint ID 109 110**NVDM header** (NVIDIA Vendor Defined Message): 111 112- Bits 6:0: MCTP message type (0x7e = vendor-defined PCI) 113- Bits 23:8: PCI vendor ID (0x10de = NVIDIA) 114- Bits 31:24: NVDM type (0x14 = COT, 0x13 = PRC, 0x15 = FSP response) 115 116PRC (Product Reconfiguration Control) protocol 117=============================================== 118PRC is an API system exposed through FSP's Management Partition that allows 119querying and modifying device configuration without firmware updates. 120 121Configuration parameters are called "knobs". Each knob has a unique object 122ID and controls a specific device behavior. Examples include vGPU mode, ECC 123enable, confidential computing mode, and NVLINK configuration. 124 125Each knob has two values: 126 127- **Active**: the currently effective value for this boot cycle. 128- **Persistent**: the value stored in InfoROM, applied on subsequent boots. 129 130The nova-core driver uses PRC to read the vGPU mode knob (object ID 0x29) 131during early boot, before firmware loading, to determine whether the GPU 132should operate in vGPU mode. 133 134The PRC message format follows the same MCTP/NVDM header structure as COT, 135with NVDM type 0x13. The payload contains: 136 137- A sub-command (e.g., 0x0c for read). 138- Flags indicating which value to read (bit 0 = persistent, bit 1 = active). 139- The knob object ID. 140 141The response includes the common FSP response header (with error status) 142followed by the knob's 16-bit state value. 143