1.. SPDX-License-Identifier: (GPL-2.0+ OR MIT) 2 3========================= 4FWSEC (Firmware Security) 5========================= 6This document briefly/conceptually describes the FWSEC (Firmware Security) image 7and its role in the GPU boot sequence. As such, this information is subject to 8change in the future and is only current as of the Ampere GPU family. However, 9hopefully the concepts described will be useful for understanding the kernel code 10that deals with it. All the information is derived from publicly available 11sources such as public drivers and documentation. 12 13The role of FWSEC is to provide a secure boot process. It runs in 14'Heavy-secure' mode, and performs firmware verification after a GPU reset 15before loading various ucode images onto other microcontrollers on the GPU, 16such as the PMU and GSP. 17 18FWSEC itself is an application stored in the VBIOS ROM in the FWSEC partition of 19ROM (see vbios.rst for more details). It contains different commands like FRTS 20(Firmware Runtime Services) and SB (Secure Booting other microcontrollers after 21reset and loading them with other non-FWSEC ucode). The kernel driver only needs 22to perform FRTS, since Secure Boot (SB) has already completed by the time the driver 23is loaded. 24 25The FRTS command carves out the WPR2 region (Write protected region) which contains 26data required for power management. Once setup, only HS mode ucode can access it 27(see falcon.rst for privilege levels). 28 29The FWSEC image is located in the VBIOS ROM in the partition of the ROM that contains 30various ucode images (also known as applications) -- one of them being FWSEC. For how 31it is extracted, see vbios.rst and the vbios.rs source code. 32 33The Falcon data for each ucode images (including the FWSEC image) is a combination 34of headers, data sections (DMEM) and instruction code sections (IMEM). All these 35ucode images are stored in the same ROM partition and the PMU table is used to look 36up the application to load it based on its application ID (see vbios.rs). 37 38For the nova-core driver, the FWSEC contains an 'application interface' called 39DMEMMAPPER. This interface is used to execute the 'FWSEC-FRTS' command, among others. 40For Ampere, FWSEC is running on the GSP in Heavy-secure mode and runs FRTS. 41 42FWSEC Memory Layout 43------------------- 44The memory layout of the FWSEC image is as follows:: 45 46 +---------------------------------------------------------------+ 47 | FWSEC ROM image (type 0xE0) | 48 | | 49 | +---------------------------------+ | 50 | | PMU Falcon Ucode Table | | 51 | | (PmuLookupTable) | | 52 | | +-------------------------+ | | 53 | | | Table Header | | | 54 | | | - version: 0x01 | | | 55 | | | - header_size: 6 | | | 56 | | | - entry_size: 6 | | | 57 | | | - entry_count: N | | | 58 | | | - desc_version:3(unused)| | | 59 | | +-------------------------+ | | 60 | | ... | | 61 | | +-------------------------+ | | 62 | | | Entry for FWSEC (0x85) | | | 63 | | | (PmuLookupTableEntry) | | | 64 | | | - app_id: 0x85 (FWSEC) |----|----+ | 65 | | | - target_id: 0x01 (PMU) | | | | 66 | | | - data: offset ---------|----|----|---+ look up FWSEC | 67 | | +-------------------------+ | | | | 68 | +---------------------------------+ | | | 69 | | | | 70 | | | | 71 | +---------------------------------+ | | | 72 | | FWSEC Ucode Component |<---+ | | 73 | | (aka Falcon data) | | | 74 | | +-------------------------+ | | | 75 | | | FalconUCodeDescV3 |<---|--------+ | 76 | | | - hdr | | | 77 | | | - stored_size | | | 78 | | | - pkc_data_offset | | | 79 | | | - interface_offset -----|----|----------------+ | 80 | | | - imem_phys_base | | | | 81 | | | - imem_load_size | | | | 82 | | | - imem_virt_base | | | | 83 | | | - dmem_phys_base | | | | 84 | | | - dmem_load_size | | | | 85 | | | - engine_id_mask | | | | 86 | | | - ucode_id | | | | 87 | | | - signature_count | | look up sig | | 88 | | | - signature_versions --------------+ | | 89 | | +-------------------------+ | | | | 90 | | (no gap) | | | | 91 | | +-------------------------+ | | | | 92 | | | Signatures Section |<---|-----+ | | 93 | | | (384 bytes per sig) | | | | 94 | | | - RSA-3K Signature 1 | | | | 95 | | | - RSA-3K Signature 2 | | | | 96 | | | ... | | | | 97 | | +-------------------------+ | | | 98 | | | | | 99 | | +-------------------------+ | | | 100 | | | IMEM Section (Code) | | | | 101 | | | | | | | 102 | | | Contains instruction | | | | 103 | | | code etc. | | | | 104 | | +-------------------------+ | | | 105 | | | | | 106 | | +-------------------------+ | | | 107 | | | DMEM Section (Data) | | | | 108 | | | | | | | 109 | | | +---------------------+ | | | | 110 | | | | Application | |<---|----------------+ | 111 | | | | Interface Table | | | | 112 | | | | (FalconAppifHdrV1) | | | | 113 | | | | Header: | | | | 114 | | | | - version: 0x01 | | | | 115 | | | | - header_size: 4 | | | | 116 | | | | - entry_size: 8 | | | | 117 | | | | - entry_count: N | | | | 118 | | | | | | | | 119 | | | | Entries: | | | | 120 | | | | +-----------------+ | | | | 121 | | | | | DEVINIT (ID 1) | | | | | 122 | | | | | - id: 0x01 | | | | | 123 | | | | | - dmemOffset X -|-|-|----+ | 124 | | | | +-----------------+ | | | | 125 | | | | +-----------------+ | | | | 126 | | | | | DMEMMAPPER(ID 4)| | | | | 127 | | | | | - id: 0x04 | | | | Used only for DevInit | 128 | | | | | (NVFW_FALCON_ | | | | application (not FWSEC) | 129 | | | | | APPIF_ID_DMEMMAPPER) | | 130 | | | | | - dmemOffset Y -|-|-|----|-----+ | 131 | | | | +-----------------+ | | | | | 132 | | | +---------------------+ | | | | 133 | | | | | | | 134 | | | +---------------------+ | | | | 135 | | | | DEVINIT Engine |<|----+ | Used by FWSEC | 136 | | | | Interface | | | | app. | 137 | | | +---------------------+ | | | | 138 | | | | | | | 139 | | | +---------------------+ | | | | 140 | | | | DMEM Mapper (ID 4) |<|----+-----+ | 141 | | | | (FalconAppifDmemmapperV3) | | 142 | | | | - signature: "DMAP" | | | | 143 | | | | - version: 0x0003 | | | | 144 | | | | - Size: 64 bytes | | | | 145 | | | | - cmd_in_buffer_off | |----|------------+ | 146 | | | | - cmd_in_buffer_size| | | | | 147 | | | | - cmd_out_buffer_off| |----|------------|-----+ | 148 | | | | - cmd_out_buffer_sz | | | | | | 149 | | | | - init_cmd | | | | | | 150 | | | | - features | | | | | | 151 | | | | - cmd_mask0/1 | | | | | | 152 | | | +---------------------+ | | | | | 153 | | | | | | | | 154 | | | +---------------------+ | | | | | 155 | | | | Command Input Buffer|<|----|------------+ | | 156 | | | | - Command data | | | | | 157 | | | | - Arguments | | | | | 158 | | | +---------------------+ | | | | 159 | | | | | | | 160 | | | +---------------------+ | | | | 161 | | | | Command Output |<|----|------------------+ | 162 | | | | Buffer | | | | 163 | | | | - Results | | | | 164 | | | | - Status | | | | 165 | | | +---------------------+ | | | 166 | | +-------------------------+ | | 167 | +---------------------------------+ | 168 | | 169 +---------------------------------------------------------------+ 170 171.. note:: 172 This is using an GA-102 Ampere GPU as an example and could vary for future GPUs. 173 174.. note:: 175 The FWSEC image also plays a role in memory scrubbing (ECC initialization) and VPR 176 (Video Protected Region) initialization as well. Before the nova-core driver is even 177 loaded, the FWSEC image is running on the GSP in heavy-secure mode. After the devinit 178 sequence completes, it does VRAM memory scrubbing (ECC initialization). On consumer 179 GPUs, it scrubs only part of memory and then initiates 'async scrubbing'. Before this 180 async scrubbing completes, the unscrubbed VRAM cannot be used for allocation (thus DRM 181 memory allocators need to wait for this scrubbing to complete). 182