11f597b1aSCarlos Bilbao====================================================== 21f597b1aSCarlos BilbaoConfidential Computing in Linux for x86 virtualization 31f597b1aSCarlos Bilbao====================================================== 41f597b1aSCarlos Bilbao 51f597b1aSCarlos Bilbao.. contents:: :local: 61f597b1aSCarlos Bilbao 7*5bc23521SBilbao, CarlosBy: Elena Reshetova <elena.reshetova@intel.com> and Carlos Bilbao <carlos.bilbao.osdev@gmail.com> 81f597b1aSCarlos Bilbao 91f597b1aSCarlos BilbaoMotivation 101f597b1aSCarlos Bilbao========== 111f597b1aSCarlos Bilbao 121f597b1aSCarlos BilbaoKernel developers working on confidential computing for virtualized 131f597b1aSCarlos Bilbaoenvironments in x86 operate under a set of assumptions regarding the Linux 141f597b1aSCarlos Bilbaokernel threat model that differ from the traditional view. Historically, 151f597b1aSCarlos Bilbaothe Linux threat model acknowledges attackers residing in userspace, as 161f597b1aSCarlos Bilbaowell as a limited set of external attackers that are able to interact with 171f597b1aSCarlos Bilbaothe kernel through various networking or limited HW-specific exposed 181f597b1aSCarlos Bilbaointerfaces (USB, thunderbolt). The goal of this document is to explain 191f597b1aSCarlos Bilbaoadditional attack vectors that arise in the confidential computing space 201f597b1aSCarlos Bilbaoand discuss the proposed protection mechanisms for the Linux kernel. 211f597b1aSCarlos Bilbao 221f597b1aSCarlos BilbaoOverview and terminology 231f597b1aSCarlos Bilbao======================== 241f597b1aSCarlos Bilbao 251f597b1aSCarlos BilbaoConfidential Computing (CoCo) is a broad term covering a wide range of 261f597b1aSCarlos Bilbaosecurity technologies that aim to protect the confidentiality and integrity 271f597b1aSCarlos Bilbaoof data in use (vs. data at rest or data in transit). At its core, CoCo 281f597b1aSCarlos Bilbaosolutions provide a Trusted Execution Environment (TEE), where secure data 291f597b1aSCarlos Bilbaoprocessing can be performed and, as a result, they are typically further 301f597b1aSCarlos Bilbaoclassified into different subtypes depending on the SW that is intended 311f597b1aSCarlos Bilbaoto be run in TEE. This document focuses on a subclass of CoCo technologies 321f597b1aSCarlos Bilbaothat are targeting virtualized environments and allow running Virtual 331f597b1aSCarlos BilbaoMachines (VM) inside TEE. From now on in this document will be referring 341f597b1aSCarlos Bilbaoto this subclass of CoCo as 'Confidential Computing (CoCo) for the 351f597b1aSCarlos Bilbaovirtualized environments (VE)'. 361f597b1aSCarlos Bilbao 371f597b1aSCarlos BilbaoCoCo, in the virtualization context, refers to a set of HW and/or SW 381f597b1aSCarlos Bilbaotechnologies that allow for stronger security guarantees for the SW running 391f597b1aSCarlos Bilbaoinside a CoCo VM. Namely, confidential computing allows its users to 401f597b1aSCarlos Bilbaoconfirm the trustworthiness of all SW pieces to include in its reduced 411f597b1aSCarlos BilbaoTrusted Computing Base (TCB) given its ability to attest the state of these 421f597b1aSCarlos Bilbaotrusted components. 431f597b1aSCarlos Bilbao 441f597b1aSCarlos BilbaoWhile the concrete implementation details differ between technologies, all 451f597b1aSCarlos Bilbaoavailable mechanisms aim to provide increased confidentiality and 461f597b1aSCarlos Bilbaointegrity for the VM's guest memory and execution state (vCPU registers), 471f597b1aSCarlos Bilbaomore tightly controlled guest interrupt injection, as well as some 481f597b1aSCarlos Bilbaoadditional mechanisms to control guest-host page mapping. More details on 491f597b1aSCarlos Bilbaothe x86-specific solutions can be found in 501f597b1aSCarlos Bilbao:doc:`Intel Trust Domain Extensions (TDX) </arch/x86/tdx>` and 511f597b1aSCarlos Bilbao`AMD Memory Encryption <https://www.amd.com/system/files/techdocs/sev-snp-strengthening-vm-isolation-with-integrity-protection-and-more.pdf>`_. 521f597b1aSCarlos Bilbao 531f597b1aSCarlos BilbaoThe basic CoCo guest layout includes the host, guest, the interfaces that 541f597b1aSCarlos Bilbaocommunicate guest and host, a platform capable of supporting CoCo VMs, and 551f597b1aSCarlos Bilbaoa trusted intermediary between the guest VM and the underlying platform 561f597b1aSCarlos Bilbaothat acts as a security manager. The host-side virtual machine monitor 571f597b1aSCarlos Bilbao(VMM) typically consists of a subset of traditional VMM features and 581f597b1aSCarlos Bilbaois still in charge of the guest lifecycle, i.e. create or destroy a CoCo 591f597b1aSCarlos BilbaoVM, manage its access to system resources, etc. However, since it 601f597b1aSCarlos Bilbaotypically stays out of CoCo VM TCB, its access is limited to preserve the 611f597b1aSCarlos Bilbaosecurity objectives. 621f597b1aSCarlos Bilbao 631f597b1aSCarlos BilbaoIn the following diagram, the "<--->" lines represent bi-directional 641f597b1aSCarlos Bilbaocommunication channels or interfaces between the CoCo security manager and 651f597b1aSCarlos Bilbaothe rest of the components (data flow for guest, host, hardware) :: 661f597b1aSCarlos Bilbao 671f597b1aSCarlos Bilbao +-------------------+ +-----------------------+ 681f597b1aSCarlos Bilbao | CoCo guest VM |<---->| | 691f597b1aSCarlos Bilbao +-------------------+ | | 701f597b1aSCarlos Bilbao | Interfaces | | CoCo security manager | 711f597b1aSCarlos Bilbao +-------------------+ | | 721f597b1aSCarlos Bilbao | Host VMM |<---->| | 731f597b1aSCarlos Bilbao +-------------------+ | | 741f597b1aSCarlos Bilbao | | 751f597b1aSCarlos Bilbao +--------------------+ | | 761f597b1aSCarlos Bilbao | CoCo platform |<--->| | 771f597b1aSCarlos Bilbao +--------------------+ +-----------------------+ 781f597b1aSCarlos Bilbao 791f597b1aSCarlos BilbaoThe specific details of the CoCo security manager vastly diverge between 801f597b1aSCarlos Bilbaotechnologies. For example, in some cases, it will be implemented in HW 811f597b1aSCarlos Bilbaowhile in others it may be pure SW. 821f597b1aSCarlos Bilbao 831f597b1aSCarlos BilbaoExisting Linux kernel threat model 841f597b1aSCarlos Bilbao================================== 851f597b1aSCarlos Bilbao 861f597b1aSCarlos BilbaoThe overall components of the current Linux kernel threat model are:: 871f597b1aSCarlos Bilbao 881f597b1aSCarlos Bilbao +-----------------------+ +-------------------+ 891f597b1aSCarlos Bilbao | |<---->| Userspace | 901f597b1aSCarlos Bilbao | | +-------------------+ 911f597b1aSCarlos Bilbao | External attack | | Interfaces | 921f597b1aSCarlos Bilbao | vectors | +-------------------+ 931f597b1aSCarlos Bilbao | |<---->| Linux Kernel | 941f597b1aSCarlos Bilbao | | +-------------------+ 951f597b1aSCarlos Bilbao +-----------------------+ +-------------------+ 961f597b1aSCarlos Bilbao | Bootloader/BIOS | 971f597b1aSCarlos Bilbao +-------------------+ 981f597b1aSCarlos Bilbao +-------------------+ 991f597b1aSCarlos Bilbao | HW platform | 1001f597b1aSCarlos Bilbao +-------------------+ 1011f597b1aSCarlos Bilbao 1021f597b1aSCarlos BilbaoThere is also communication between the bootloader and the kernel during 1031f597b1aSCarlos Bilbaothe boot process, but this diagram does not represent it explicitly. The 1041f597b1aSCarlos Bilbao"Interfaces" box represents the various interfaces that allow 1051f597b1aSCarlos Bilbaocommunication between kernel and userspace. This includes system calls, 1061f597b1aSCarlos Bilbaokernel APIs, device drivers, etc. 1071f597b1aSCarlos Bilbao 1081f597b1aSCarlos BilbaoThe existing Linux kernel threat model typically assumes execution on a 1091f597b1aSCarlos Bilbaotrusted HW platform with all of the firmware and bootloaders included on 1101f597b1aSCarlos Bilbaoits TCB. The primary attacker resides in the userspace, and all of the data 1111f597b1aSCarlos Bilbaocoming from there is generally considered untrusted, unless userspace is 1121f597b1aSCarlos Bilbaoprivileged enough to perform trusted actions. In addition, external 1131f597b1aSCarlos Bilbaoattackers are typically considered, including those with access to enabled 1141f597b1aSCarlos Bilbaoexternal networks (e.g. Ethernet, Wireless, Bluetooth), exposed hardware 1151f597b1aSCarlos Bilbaointerfaces (e.g. USB, Thunderbolt), and the ability to modify the contents 1161f597b1aSCarlos Bilbaoof disks offline. 1171f597b1aSCarlos Bilbao 1181f597b1aSCarlos BilbaoRegarding external attack vectors, it is interesting to note that in most 1191f597b1aSCarlos Bilbaocases external attackers will try to exploit vulnerabilities in userspace 1201f597b1aSCarlos Bilbaofirst, but that it is possible for an attacker to directly target the 1211f597b1aSCarlos Bilbaokernel; particularly if the host has physical access. Examples of direct 1221f597b1aSCarlos Bilbaokernel attacks include the vulnerabilities CVE-2019-19524, CVE-2022-0435 1231f597b1aSCarlos Bilbaoand CVE-2020-24490. 1241f597b1aSCarlos Bilbao 1251f597b1aSCarlos BilbaoConfidential Computing threat model and its security objectives 1261f597b1aSCarlos Bilbao=============================================================== 1271f597b1aSCarlos Bilbao 1281f597b1aSCarlos BilbaoConfidential Computing adds a new type of attacker to the above list: a 1291f597b1aSCarlos Bilbaopotentially misbehaving host (which can also include some part of a 1301f597b1aSCarlos Bilbaotraditional VMM or all of it), which is typically placed outside of the 1311f597b1aSCarlos BilbaoCoCo VM TCB due to its large SW attack surface. It is important to note 1321f597b1aSCarlos Bilbaothat this doesn’t imply that the host or VMM are intentionally 1331f597b1aSCarlos Bilbaomalicious, but that there exists a security value in having a small CoCo 1341f597b1aSCarlos BilbaoVM TCB. This new type of adversary may be viewed as a more powerful type 1351f597b1aSCarlos Bilbaoof external attacker, as it resides locally on the same physical machine 1361f597b1aSCarlos Bilbao(in contrast to a remote network attacker) and has control over the guest 1371f597b1aSCarlos Bilbaokernel communication with most of the HW:: 1381f597b1aSCarlos Bilbao 1391f597b1aSCarlos Bilbao +------------------------+ 1401f597b1aSCarlos Bilbao | CoCo guest VM | 1411f597b1aSCarlos Bilbao +-----------------------+ | +-------------------+ | 1421f597b1aSCarlos Bilbao | |<--->| | Userspace | | 1431f597b1aSCarlos Bilbao | | | +-------------------+ | 1441f597b1aSCarlos Bilbao | External attack | | | Interfaces | | 1451f597b1aSCarlos Bilbao | vectors | | +-------------------+ | 1461f597b1aSCarlos Bilbao | |<--->| | Linux Kernel | | 1471f597b1aSCarlos Bilbao | | | +-------------------+ | 1481f597b1aSCarlos Bilbao +-----------------------+ | +-------------------+ | 1491f597b1aSCarlos Bilbao | | Bootloader/BIOS | | 1501f597b1aSCarlos Bilbao +-----------------------+ | +-------------------+ | 1511f597b1aSCarlos Bilbao | |<--->+------------------------+ 1521f597b1aSCarlos Bilbao | | | Interfaces | 1531f597b1aSCarlos Bilbao | | +------------------------+ 1541f597b1aSCarlos Bilbao | CoCo security |<--->| Host/Host-side VMM | 1551f597b1aSCarlos Bilbao | manager | +------------------------+ 1561f597b1aSCarlos Bilbao | | +------------------------+ 1571f597b1aSCarlos Bilbao | |<--->| CoCo platform | 1581f597b1aSCarlos Bilbao +-----------------------+ +------------------------+ 1591f597b1aSCarlos Bilbao 1601f597b1aSCarlos BilbaoWhile traditionally the host has unlimited access to guest data and can 1611f597b1aSCarlos Bilbaoleverage this access to attack the guest, the CoCo systems mitigate such 1621f597b1aSCarlos Bilbaoattacks by adding security features like guest data confidentiality and 1631f597b1aSCarlos Bilbaointegrity protection. This threat model assumes that those features are 1641f597b1aSCarlos Bilbaoavailable and intact. 1651f597b1aSCarlos Bilbao 1661f597b1aSCarlos BilbaoThe **Linux kernel CoCo VM security objectives** can be summarized as follows: 1671f597b1aSCarlos Bilbao 1681f597b1aSCarlos Bilbao1. Preserve the confidentiality and integrity of CoCo guest's private 1691f597b1aSCarlos Bilbaomemory and registers. 1701f597b1aSCarlos Bilbao 1711f597b1aSCarlos Bilbao2. Prevent privileged escalation from a host into a CoCo guest Linux kernel. 1721f597b1aSCarlos BilbaoWhile it is true that the host (and host-side VMM) requires some level of 1731f597b1aSCarlos Bilbaoprivilege to create, destroy, or pause the guest, part of the goal of 1741f597b1aSCarlos Bilbaopreventing privileged escalation is to ensure that these operations do not 1751f597b1aSCarlos Bilbaoprovide a pathway for attackers to gain access to the guest's kernel. 1761f597b1aSCarlos Bilbao 1771f597b1aSCarlos BilbaoThe above security objectives result in two primary **Linux kernel CoCo 1781f597b1aSCarlos BilbaoVM assets**: 1791f597b1aSCarlos Bilbao 1801f597b1aSCarlos Bilbao1. Guest kernel execution context. 1811f597b1aSCarlos Bilbao2. Guest kernel private memory. 1821f597b1aSCarlos Bilbao 1831f597b1aSCarlos BilbaoThe host retains full control over the CoCo guest resources, and can deny 1841f597b1aSCarlos Bilbaoaccess to them at any time. Examples of resources include CPU time, memory 1851f597b1aSCarlos Bilbaothat the guest can consume, network bandwidth, etc. Because of this, the 1861f597b1aSCarlos Bilbaohost Denial of Service (DoS) attacks against CoCo guests are beyond the 1871f597b1aSCarlos Bilbaoscope of this threat model. 1881f597b1aSCarlos Bilbao 1891f597b1aSCarlos BilbaoThe **Linux CoCo VM attack surface** is any interface exposed from a CoCo 1901f597b1aSCarlos Bilbaoguest Linux kernel towards an untrusted host that is not covered by the 1911f597b1aSCarlos BilbaoCoCo technology SW/HW protection. This includes any possible 1921f597b1aSCarlos Bilbaoside-channels, as well as transient execution side channels. Examples of 1931f597b1aSCarlos Bilbaoexplicit (not side-channel) interfaces include accesses to port I/O, MMIO 1941f597b1aSCarlos Bilbaoand DMA interfaces, access to PCI configuration space, VMM-specific 1951f597b1aSCarlos Bilbaohypercalls (towards Host-side VMM), access to shared memory pages, 1961f597b1aSCarlos Bilbaointerrupts allowed to be injected into the guest kernel by the host, as 1971f597b1aSCarlos Bilbaowell as CoCo technology-specific hypercalls, if present. Additionally, the 1981f597b1aSCarlos Bilbaohost in a CoCo system typically controls the process of creating a CoCo 1991f597b1aSCarlos Bilbaoguest: it has a method to load into a guest the firmware and bootloader 2001f597b1aSCarlos Bilbaoimages, the kernel image together with the kernel command line. All of this 2011f597b1aSCarlos Bilbaodata should also be considered untrusted until its integrity and 2021f597b1aSCarlos Bilbaoauthenticity is established via attestation. 2031f597b1aSCarlos Bilbao 2041f597b1aSCarlos BilbaoThe table below shows a threat matrix for the CoCo guest Linux kernel but 2051f597b1aSCarlos Bilbaodoes not discuss potential mitigation strategies. The matrix refers to 2061f597b1aSCarlos BilbaoCoCo-specific versions of the guest, host and platform. 2071f597b1aSCarlos Bilbao 2081f597b1aSCarlos Bilbao.. list-table:: CoCo Linux guest kernel threat matrix 2091f597b1aSCarlos Bilbao :widths: auto 2101f597b1aSCarlos Bilbao :align: center 2111f597b1aSCarlos Bilbao :header-rows: 1 2121f597b1aSCarlos Bilbao 2131f597b1aSCarlos Bilbao * - Threat name 2141f597b1aSCarlos Bilbao - Threat description 2151f597b1aSCarlos Bilbao 2161f597b1aSCarlos Bilbao * - Guest malicious configuration 2171f597b1aSCarlos Bilbao - A misbehaving host modifies one of the following guest's 2181f597b1aSCarlos Bilbao configuration: 2191f597b1aSCarlos Bilbao 2201f597b1aSCarlos Bilbao 1. Guest firmware or bootloader 2211f597b1aSCarlos Bilbao 2221f597b1aSCarlos Bilbao 2. Guest kernel or module binaries 2231f597b1aSCarlos Bilbao 2241f597b1aSCarlos Bilbao 3. Guest command line parameters 2251f597b1aSCarlos Bilbao 2261f597b1aSCarlos Bilbao This allows the host to break the integrity of the code running 2271f597b1aSCarlos Bilbao inside a CoCo guest, and violates the CoCo security objectives. 2281f597b1aSCarlos Bilbao 2291f597b1aSCarlos Bilbao * - CoCo guest data attacks 2301f597b1aSCarlos Bilbao - A misbehaving host retains full control of the CoCo guest's data 2311f597b1aSCarlos Bilbao in-transit between the guest and the host-managed physical or 2321f597b1aSCarlos Bilbao virtual devices. This allows any attack against confidentiality, 2331f597b1aSCarlos Bilbao integrity or freshness of such data. 2341f597b1aSCarlos Bilbao 2351f597b1aSCarlos Bilbao * - Malformed runtime input 2361f597b1aSCarlos Bilbao - A misbehaving host injects malformed input via any communication 2371f597b1aSCarlos Bilbao interface used by the guest's kernel code. If the code is not 2381f597b1aSCarlos Bilbao prepared to handle this input correctly, this can result in a host 2391f597b1aSCarlos Bilbao --> guest kernel privilege escalation. This includes traditional 2401f597b1aSCarlos Bilbao side-channel and/or transient execution attack vectors. 2411f597b1aSCarlos Bilbao 2421f597b1aSCarlos Bilbao * - Malicious runtime input 2431f597b1aSCarlos Bilbao - A misbehaving host injects a specific input value via any 2441f597b1aSCarlos Bilbao communication interface used by the guest's kernel code. The 2451f597b1aSCarlos Bilbao difference with the previous attack vector (malformed runtime input) 2461f597b1aSCarlos Bilbao is that this input is not malformed, but its value is crafted to 2471f597b1aSCarlos Bilbao impact the guest's kernel security. Examples of such inputs include 2481f597b1aSCarlos Bilbao providing a malicious time to the guest or the entropy to the guest 2491f597b1aSCarlos Bilbao random number generator. Additionally, the timing of such events can 2501f597b1aSCarlos Bilbao be an attack vector on its own, if it results in a particular guest 2511f597b1aSCarlos Bilbao kernel action (i.e. processing of a host-injected interrupt). 2521f597b1aSCarlos Bilbao resistant to supplied host input. 2531f597b1aSCarlos Bilbao 254