1.. SPDX-License-Identifier: GPL-2.0 2.. _ultravisor: 3 4============================ 5Protected Execution Facility 6============================ 7 8.. contents:: 9 :depth: 3 10 11Introduction 12############ 13 14 Protected Execution Facility (PEF) is an architectural change for 15 POWER 9 that enables Secure Virtual Machines (SVMs). DD2.3 chips 16 (PVR=0x004e1203) or greater will be PEF-capable. A new ISA release 17 will include the PEF RFC02487 changes. 18 19 When enabled, PEF adds a new higher privileged mode, called Ultravisor 20 mode, to POWER architecture. Along with the new mode there is new 21 firmware called the Protected Execution Ultravisor (or Ultravisor 22 for short). Ultravisor mode is the highest privileged mode in POWER 23 architecture. 24 25 +------------------+ 26 | Privilege States | 27 +==================+ 28 | Problem | 29 +------------------+ 30 | Supervisor | 31 +------------------+ 32 | Hypervisor | 33 +------------------+ 34 | Ultravisor | 35 +------------------+ 36 37 PEF protects SVMs from the hypervisor, privileged users, and other 38 VMs in the system. SVMs are protected while at rest and can only be 39 executed by an authorized machine. All virtual machines utilize 40 hypervisor services. The Ultravisor filters calls between the SVMs 41 and the hypervisor to assure that information does not accidentally 42 leak. All hypercalls except H_RANDOM are reflected to the hypervisor. 43 H_RANDOM is not reflected to prevent the hypervisor from influencing 44 random values in the SVM. 45 46 To support this there is a refactoring of the ownership of resources 47 in the CPU. Some of the resources which were previously hypervisor 48 privileged are now ultravisor privileged. 49 50Hardware 51======== 52 53 The hardware changes include the following: 54 55 * There is a new bit in the MSR that determines whether the current 56 process is running in secure mode, MSR(S) bit 41. MSR(S)=1, process 57 is in secure mode, MSR(s)=0 process is in normal mode. 58 59 * The MSR(S) bit can only be set by the Ultravisor. 60 61 * HRFID cannot be used to set the MSR(S) bit. If the hypervisor needs 62 to return to a SVM it must use an ultracall. It can determine if 63 the VM it is returning to is secure. 64 65 * There is a new Ultravisor privileged register, SMFCTRL, which has an 66 enable/disable bit SMFCTRL(E). 67 68 * The privilege of a process is now determined by three MSR bits, 69 MSR(S, HV, PR). In each of the tables below the modes are listed 70 from least privilege to highest privilege. The higher privilege 71 modes can access all the resources of the lower privilege modes. 72 73 **Secure Mode MSR Settings** 74 75 +---+---+---+---------------+ 76 | S | HV| PR|Privilege | 77 +===+===+===+===============+ 78 | 1 | 0 | 1 | Problem | 79 +---+---+---+---------------+ 80 | 1 | 0 | 0 | Privileged(OS)| 81 +---+---+---+---------------+ 82 | 1 | 1 | 0 | Ultravisor | 83 +---+---+---+---------------+ 84 | 1 | 1 | 1 | Reserved | 85 +---+---+---+---------------+ 86 87 **Normal Mode MSR Settings** 88 89 +---+---+---+---------------+ 90 | S | HV| PR|Privilege | 91 +===+===+===+===============+ 92 | 0 | 0 | 1 | Problem | 93 +---+---+---+---------------+ 94 | 0 | 0 | 0 | Privileged(OS)| 95 +---+---+---+---------------+ 96 | 0 | 1 | 0 | Hypervisor | 97 +---+---+---+---------------+ 98 | 0 | 1 | 1 | Problem (Host)| 99 +---+---+---+---------------+ 100 101 * Memory is partitioned into secure and normal memory. Only processes 102 that are running in secure mode can access secure memory. 103 104 * The hardware does not allow anything that is not running secure to 105 access secure memory. This means that the Hypervisor cannot access 106 the memory of the SVM without using an ultracall (asking the 107 Ultravisor). The Ultravisor will only allow the hypervisor to see 108 the SVM memory encrypted. 109 110 * I/O systems are not allowed to directly address secure memory. This 111 limits the SVMs to virtual I/O only. 112 113 * The architecture allows the SVM to share pages of memory with the 114 hypervisor that are not protected with encryption. However, this 115 sharing must be initiated by the SVM. 116 117 * When a process is running in secure mode all hypercalls 118 (syscall lev=1) go to the Ultravisor. 119 120 * When a process is in secure mode all interrupts go to the 121 Ultravisor. 122 123 * The following resources have become Ultravisor privileged and 124 require an Ultravisor interface to manipulate: 125 126 * Processor configurations registers (SCOMs). 127 128 * Stop state information. 129 130 * The debug registers CIABR, DAWR, and DAWRX when SMFCTRL(D) is set. 131 If SMFCTRL(D) is not set they do not work in secure mode. When set, 132 reading and writing requires an Ultravisor call, otherwise that 133 will cause a Hypervisor Emulation Assistance interrupt. 134 135 * PTCR and partition table entries (partition table is in secure 136 memory). An attempt to write to PTCR will cause a Hypervisor 137 Emulation Assitance interrupt. 138 139 * LDBAR (LD Base Address Register) and IMC (In-Memory Collection) 140 non-architected registers. An attempt to write to them will cause a 141 Hypervisor Emulation Assistance interrupt. 142 143 * Paging for an SVM, sharing of memory with Hypervisor for an SVM. 144 (Including Virtual Processor Area (VPA) and virtual I/O). 145 146 147Software/Microcode 148================== 149 150 The software changes include: 151 152 * SVMs are created from normal VM using (open source) tooling supplied 153 by IBM. 154 155 * All SVMs start as normal VMs and utilize an ultracall, UV_ESM 156 (Enter Secure Mode), to make the transition. 157 158 * When the UV_ESM ultracall is made the Ultravisor copies the VM into 159 secure memory, decrypts the verification information, and checks the 160 integrity of the SVM. If the integrity check passes the Ultravisor 161 passes control in secure mode. 162 163 * The verification information includes the pass phrase for the 164 encrypted disk associated with the SVM. This pass phrase is given 165 to the SVM when requested. 166 167 * The Ultravisor is not involved in protecting the encrypted disk of 168 the SVM while at rest. 169 170 * For external interrupts the Ultravisor saves the state of the SVM, 171 and reflects the interrupt to the hypervisor for processing. 172 For hypercalls, the Ultravisor inserts neutral state into all 173 registers not needed for the hypercall then reflects the call to 174 the hypervisor for processing. The H_RANDOM hypercall is performed 175 by the Ultravisor and not reflected. 176 177 * For virtual I/O to work bounce buffering must be done. 178 179 * The Ultravisor uses AES (IAPM) for protection of SVM memory. IAPM 180 is a mode of AES that provides integrity and secrecy concurrently. 181 182 * The movement of data between normal and secure pages is coordinated 183 with the Ultravisor by a new HMM plug-in in the Hypervisor. 184 185 The Ultravisor offers new services to the hypervisor and SVMs. These 186 are accessed through ultracalls. 187 188Terminology 189=========== 190 191 * Hypercalls: special system calls used to request services from 192 Hypervisor. 193 194 * Normal memory: Memory that is accessible to Hypervisor. 195 196 * Normal page: Page backed by normal memory and available to 197 Hypervisor. 198 199 * Shared page: A page backed by normal memory and available to both 200 the Hypervisor/QEMU and the SVM (i.e page has mappings in SVM and 201 Hypervisor/QEMU). 202 203 * Secure memory: Memory that is accessible only to Ultravisor and 204 SVMs. 205 206 * Secure page: Page backed by secure memory and only available to 207 Ultravisor and SVM. 208 209 * SVM: Secure Virtual Machine. 210 211 * Ultracalls: special system calls used to request services from 212 Ultravisor. 213 214 215Ultravisor calls API 216#################### 217 218 This section describes Ultravisor calls (ultracalls) needed to 219 support Secure Virtual Machines (SVM)s and Paravirtualized KVM. The 220 ultracalls allow the SVMs and Hypervisor to request services from the 221 Ultravisor such as accessing a register or memory region that can only 222 be accessed when running in Ultravisor-privileged mode. 223 224 The specific service needed from an ultracall is specified in register 225 R3 (the first parameter to the ultracall). Other parameters to the 226 ultracall, if any, are specified in registers R4 through R12. 227 228 Return value of all ultracalls is in register R3. Other output values 229 from the ultracall, if any, are returned in registers R4 through R12. 230 The only exception to this register usage is the ``UV_RETURN`` 231 ultracall described below. 232 233 Each ultracall returns specific error codes, applicable in the context 234 of the ultracall. However, like with the PowerPC Architecture Platform 235 Reference (PAPR), if no specific error code is defined for a 236 particular situation, then the ultracall will fallback to an erroneous 237 parameter-position based code. i.e U_PARAMETER, U_P2, U_P3 etc 238 depending on the ultracall parameter that may have caused the error. 239 240 Some ultracalls involve transferring a page of data between Ultravisor 241 and Hypervisor. Secure pages that are transferred from secure memory 242 to normal memory may be encrypted using dynamically generated keys. 243 When the secure pages are transferred back to secure memory, they may 244 be decrypted using the same dynamically generated keys. Generation and 245 management of these keys will be covered in a separate document. 246 247 For now this only covers ultracalls currently implemented and being 248 used by Hypervisor and SVMs but others can be added here when it 249 makes sense. 250 251 The full specification for all hypercalls/ultracalls will eventually 252 be made available in the public/OpenPower version of the PAPR 253 specification. 254 255 .. note:: 256 257 If PEF is not enabled, the ultracalls will be redirected to the 258 Hypervisor which must handle/fail the calls. 259 260Ultracalls used by Hypervisor 261============================= 262 263 This section describes the virtual memory management ultracalls used 264 by the Hypervisor to manage SVMs. 265 266UV_PAGE_OUT 267----------- 268 269 Encrypt and move the contents of a page from secure memory to normal 270 memory. 271 272Syntax 273~~~~~~ 274 275.. code-block:: c 276 277 uint64_t ultracall(const uint64_t UV_PAGE_OUT, 278 uint16_t lpid, /* LPAR ID */ 279 uint64_t dest_ra, /* real address of destination page */ 280 uint64_t src_gpa, /* source guest-physical-address */ 281 uint8_t flags, /* flags */ 282 uint64_t order) /* page size order */ 283 284Return values 285~~~~~~~~~~~~~ 286 287 One of the following values: 288 289 * U_SUCCESS on success. 290 * U_PARAMETER if ``lpid`` is invalid. 291 * U_P2 if ``dest_ra`` is invalid. 292 * U_P3 if the ``src_gpa`` address is invalid. 293 * U_P4 if any bit in the ``flags`` is unrecognized 294 * U_P5 if the ``order`` parameter is unsupported. 295 * U_FUNCTION if functionality is not supported. 296 * U_BUSY if page cannot be currently paged-out. 297 298Description 299~~~~~~~~~~~ 300 301 Encrypt the contents of a secure-page and make it available to 302 Hypervisor in a normal page. 303 304 By default, the source page is unmapped from the SVM's partition- 305 scoped page table. But the Hypervisor can provide a hint to the 306 Ultravisor to retain the page mapping by setting the ``UV_SNAPSHOT`` 307 flag in ``flags`` parameter. 308 309 If the source page is already a shared page the call returns 310 U_SUCCESS, without doing anything. 311 312Use cases 313~~~~~~~~~ 314 315 #. QEMU attempts to access an address belonging to the SVM but the 316 page frame for that address is not mapped into QEMU's address 317 space. In this case, the Hypervisor will allocate a page frame, 318 map it into QEMU's address space and issue the ``UV_PAGE_OUT`` 319 call to retrieve the encrypted contents of the page. 320 321 #. When Ultravisor runs low on secure memory and it needs to page-out 322 an LRU page. In this case, Ultravisor will issue the 323 ``H_SVM_PAGE_OUT`` hypercall to the Hypervisor. The Hypervisor will 324 then allocate a normal page and issue the ``UV_PAGE_OUT`` ultracall 325 and the Ultravisor will encrypt and move the contents of the secure 326 page into the normal page. 327 328 #. When Hypervisor accesses SVM data, the Hypervisor requests the 329 Ultravisor to transfer the corresponding page into a insecure page, 330 which the Hypervisor can access. The data in the normal page will 331 be encrypted though. 332 333UV_PAGE_IN 334---------- 335 336 Move the contents of a page from normal memory to secure memory. 337 338Syntax 339~~~~~~ 340 341.. code-block:: c 342 343 uint64_t ultracall(const uint64_t UV_PAGE_IN, 344 uint16_t lpid, /* the LPAR ID */ 345 uint64_t src_ra, /* source real address of page */ 346 uint64_t dest_gpa, /* destination guest physical address */ 347 uint64_t flags, /* flags */ 348 uint64_t order) /* page size order */ 349 350Return values 351~~~~~~~~~~~~~ 352 353 One of the following values: 354 355 * U_SUCCESS on success. 356 * U_BUSY if page cannot be currently paged-in. 357 * U_FUNCTION if functionality is not supported 358 * U_PARAMETER if ``lpid`` is invalid. 359 * U_P2 if ``src_ra`` is invalid. 360 * U_P3 if the ``dest_gpa`` address is invalid. 361 * U_P4 if any bit in the ``flags`` is unrecognized 362 * U_P5 if the ``order`` parameter is unsupported. 363 364Description 365~~~~~~~~~~~ 366 367 Move the contents of the page identified by ``src_ra`` from normal 368 memory to secure memory and map it to the guest physical address 369 ``dest_gpa``. 370 371 If `dest_gpa` refers to a shared address, map the page into the 372 partition-scoped page-table of the SVM. If `dest_gpa` is not shared, 373 copy the contents of the page into the corresponding secure page. 374 Depending on the context, decrypt the page before being copied. 375 376 The caller provides the attributes of the page through the ``flags`` 377 parameter. Valid values for ``flags`` are: 378 379 * CACHE_INHIBITED 380 * CACHE_ENABLED 381 * WRITE_PROTECTION 382 383 The Hypervisor must pin the page in memory before making 384 ``UV_PAGE_IN`` ultracall. 385 386Use cases 387~~~~~~~~~ 388 389 #. When a normal VM switches to secure mode, all its pages residing 390 in normal memory, are moved into secure memory. 391 392 #. When an SVM requests to share a page with Hypervisor the Hypervisor 393 allocates a page and informs the Ultravisor. 394 395 #. When an SVM accesses a secure page that has been paged-out, 396 Ultravisor invokes the Hypervisor to locate the page. After 397 locating the page, the Hypervisor uses UV_PAGE_IN to make the 398 page available to Ultravisor. 399 400UV_PAGE_INVAL 401------------- 402 403 Invalidate the Ultravisor mapping of a page. 404 405Syntax 406~~~~~~ 407 408.. code-block:: c 409 410 uint64_t ultracall(const uint64_t UV_PAGE_INVAL, 411 uint16_t lpid, /* the LPAR ID */ 412 uint64_t guest_pa, /* destination guest-physical-address */ 413 uint64_t order) /* page size order */ 414 415Return values 416~~~~~~~~~~~~~ 417 418 One of the following values: 419 420 * U_SUCCESS on success. 421 * U_PARAMETER if ``lpid`` is invalid. 422 * U_P2 if ``guest_pa`` is invalid (or corresponds to a secure 423 page mapping). 424 * U_P3 if the ``order`` is invalid. 425 * U_FUNCTION if functionality is not supported. 426 * U_BUSY if page cannot be currently invalidated. 427 428Description 429~~~~~~~~~~~ 430 431 This ultracall informs Ultravisor that the page mapping in Hypervisor 432 corresponding to the given guest physical address has been invalidated 433 and that the Ultravisor should not access the page. If the specified 434 ``guest_pa`` corresponds to a secure page, Ultravisor will ignore the 435 attempt to invalidate the page and return U_P2. 436 437Use cases 438~~~~~~~~~ 439 440 #. When a shared page is unmapped from the QEMU's page table, possibly 441 because it is paged-out to disk, Ultravisor needs to know that the 442 page should not be accessed from its side too. 443 444 445UV_WRITE_PATE 446------------- 447 448 Validate and write the partition table entry (PATE) for a given 449 partition. 450 451Syntax 452~~~~~~ 453 454.. code-block:: c 455 456 uint64_t ultracall(const uint64_t UV_WRITE_PATE, 457 uint32_t lpid, /* the LPAR ID */ 458 uint64_t dw0 /* the first double word to write */ 459 uint64_t dw1) /* the second double word to write */ 460 461Return values 462~~~~~~~~~~~~~ 463 464 One of the following values: 465 466 * U_SUCCESS on success. 467 * U_BUSY if PATE cannot be currently written to. 468 * U_FUNCTION if functionality is not supported. 469 * U_PARAMETER if ``lpid`` is invalid. 470 * U_P2 if ``dw0`` is invalid. 471 * U_P3 if the ``dw1`` address is invalid. 472 * U_PERMISSION if the Hypervisor is attempting to change the PATE 473 of a secure virtual machine or if called from a 474 context other than Hypervisor. 475 476Description 477~~~~~~~~~~~ 478 479 Validate and write a LPID and its partition-table-entry for the given 480 LPID. If the LPID is already allocated and initialized, this call 481 results in changing the partition table entry. 482 483Use cases 484~~~~~~~~~ 485 486 #. The Partition table resides in Secure memory and its entries, 487 called PATE (Partition Table Entries), point to the partition- 488 scoped page tables for the Hypervisor as well as each of the 489 virtual machines (both secure and normal). The Hypervisor 490 operates in partition 0 and its partition-scoped page tables 491 reside in normal memory. 492 493 #. This ultracall allows the Hypervisor to register the partition- 494 scoped and process-scoped page table entries for the Hypervisor 495 and other partitions (virtual machines) with the Ultravisor. 496 497 #. If the value of the PATE for an existing partition (VM) changes, 498 the TLB cache for the partition is flushed. 499 500 #. The Hypervisor is responsible for allocating LPID. The LPID and 501 its PATE entry are registered together. The Hypervisor manages 502 the PATE entries for a normal VM and can change the PATE entry 503 anytime. Ultravisor manages the PATE entries for an SVM and 504 Hypervisor is not allowed to modify them. 505 506UV_RETURN 507--------- 508 509 Return control from the Hypervisor back to the Ultravisor after 510 processing an hypercall or interrupt that was forwarded (aka 511 *reflected*) to the Hypervisor. 512 513Syntax 514~~~~~~ 515 516.. code-block:: c 517 518 uint64_t ultracall(const uint64_t UV_RETURN) 519 520Return values 521~~~~~~~~~~~~~ 522 523 This call never returns to Hypervisor on success. It returns 524 U_INVALID if ultracall is not made from a Hypervisor context. 525 526Description 527~~~~~~~~~~~ 528 529 When an SVM makes an hypercall or incurs some other exception, the 530 Ultravisor usually forwards (aka *reflects*) the exceptions to the 531 Hypervisor. After processing the exception, Hypervisor uses the 532 ``UV_RETURN`` ultracall to return control back to the SVM. 533 534 The expected register state on entry to this ultracall is: 535 536 * Non-volatile registers are restored to their original values. 537 * If returning from an hypercall, register R0 contains the return 538 value (**unlike other ultracalls**) and, registers R4 through R12 539 contain any output values of the hypercall. 540 * R3 contains the ultracall number, i.e UV_RETURN. 541 * If returning with a synthesized interrupt, R2 contains the 542 synthesized interrupt number. 543 544Use cases 545~~~~~~~~~ 546 547 #. Ultravisor relies on the Hypervisor to provide several services to 548 the SVM such as processing hypercall and other exceptions. After 549 processing the exception, Hypervisor uses UV_RETURN to return 550 control back to the Ultravisor. 551 552 #. Hypervisor has to use this ultracall to return control to the SVM. 553 554 555UV_REGISTER_MEM_SLOT 556-------------------- 557 558 Register an SVM address-range with specified properties. 559 560Syntax 561~~~~~~ 562 563.. code-block:: c 564 565 uint64_t ultracall(const uint64_t UV_REGISTER_MEM_SLOT, 566 uint64_t lpid, /* LPAR ID of the SVM */ 567 uint64_t start_gpa, /* start guest physical address */ 568 uint64_t size, /* size of address range in bytes */ 569 uint64_t flags /* reserved for future expansion */ 570 uint16_t slotid) /* slot identifier */ 571 572Return values 573~~~~~~~~~~~~~ 574 575 One of the following values: 576 577 * U_SUCCESS on success. 578 * U_PARAMETER if ``lpid`` is invalid. 579 * U_P2 if ``start_gpa`` is invalid. 580 * U_P3 if ``size`` is invalid. 581 * U_P4 if any bit in the ``flags`` is unrecognized. 582 * U_P5 if the ``slotid`` parameter is unsupported. 583 * U_PERMISSION if called from context other than Hypervisor. 584 * U_FUNCTION if functionality is not supported. 585 586 587Description 588~~~~~~~~~~~ 589 590 Register a memory range for an SVM. The memory range starts at the 591 guest physical address ``start_gpa`` and is ``size`` bytes long. 592 593Use cases 594~~~~~~~~~ 595 596 597 #. When a virtual machine goes secure, all the memory slots managed by 598 the Hypervisor move into secure memory. The Hypervisor iterates 599 through each of memory slots, and registers the slot with 600 Ultravisor. Hypervisor may discard some slots such as those used 601 for firmware (SLOF). 602 603 #. When new memory is hot-plugged, a new memory slot gets registered. 604 605 606UV_UNREGISTER_MEM_SLOT 607---------------------- 608 609 Unregister an SVM address-range that was previously registered using 610 UV_REGISTER_MEM_SLOT. 611 612Syntax 613~~~~~~ 614 615.. code-block:: c 616 617 uint64_t ultracall(const uint64_t UV_UNREGISTER_MEM_SLOT, 618 uint64_t lpid, /* LPAR ID of the SVM */ 619 uint64_t slotid) /* reservation slotid */ 620 621Return values 622~~~~~~~~~~~~~ 623 624 One of the following values: 625 626 * U_SUCCESS on success. 627 * U_FUNCTION if functionality is not supported. 628 * U_PARAMETER if ``lpid`` is invalid. 629 * U_P2 if ``slotid`` is invalid. 630 * U_PERMISSION if called from context other than Hypervisor. 631 632Description 633~~~~~~~~~~~ 634 635 Release the memory slot identified by ``slotid`` and free any 636 resources allocated towards the reservation. 637 638Use cases 639~~~~~~~~~ 640 641 #. Memory hot-remove. 642 643 644UV_SVM_TERMINATE 645---------------- 646 647 Terminate an SVM and release its resources. 648 649Syntax 650~~~~~~ 651 652.. code-block:: c 653 654 uint64_t ultracall(const uint64_t UV_SVM_TERMINATE, 655 uint64_t lpid, /* LPAR ID of the SVM */) 656 657Return values 658~~~~~~~~~~~~~ 659 660 One of the following values: 661 662 * U_SUCCESS on success. 663 * U_FUNCTION if functionality is not supported. 664 * U_PARAMETER if ``lpid`` is invalid. 665 * U_INVALID if VM is not secure. 666 * U_PERMISSION if not called from a Hypervisor context. 667 668Description 669~~~~~~~~~~~ 670 671 Terminate an SVM and release all its resources. 672 673Use cases 674~~~~~~~~~ 675 676 #. Called by Hypervisor when terminating an SVM. 677 678 679Ultracalls used by SVM 680====================== 681 682UV_SHARE_PAGE 683------------- 684 685 Share a set of guest physical pages with the Hypervisor. 686 687Syntax 688~~~~~~ 689 690.. code-block:: c 691 692 uint64_t ultracall(const uint64_t UV_SHARE_PAGE, 693 uint64_t gfn, /* guest page frame number */ 694 uint64_t num) /* number of pages of size PAGE_SIZE */ 695 696Return values 697~~~~~~~~~~~~~ 698 699 One of the following values: 700 701 * U_SUCCESS on success. 702 * U_FUNCTION if functionality is not supported. 703 * U_INVALID if the VM is not secure. 704 * U_PARAMETER if ``gfn`` is invalid. 705 * U_P2 if ``num`` is invalid. 706 707Description 708~~~~~~~~~~~ 709 710 Share the ``num`` pages starting at guest physical frame number ``gfn`` 711 with the Hypervisor. Assume page size is PAGE_SIZE bytes. Zero the 712 pages before returning. 713 714 If the address is already backed by a secure page, unmap the page and 715 back it with an insecure page, with the help of the Hypervisor. If it 716 is not backed by any page yet, mark the PTE as insecure and back it 717 with an insecure page when the address is accessed. If it is already 718 backed by an insecure page, zero the page and return. 719 720Use cases 721~~~~~~~~~ 722 723 #. The Hypervisor cannot access the SVM pages since they are backed by 724 secure pages. Hence an SVM must explicitly request Ultravisor for 725 pages it can share with Hypervisor. 726 727 #. Shared pages are needed to support virtio and Virtual Processor Area 728 (VPA) in SVMs. 729 730 731UV_UNSHARE_PAGE 732--------------- 733 734 Restore a shared SVM page to its initial state. 735 736Syntax 737~~~~~~ 738 739.. code-block:: c 740 741 uint64_t ultracall(const uint64_t UV_UNSHARE_PAGE, 742 uint64_t gfn, /* guest page frame number */ 743 uint73 num) /* number of pages of size PAGE_SIZE*/ 744 745Return values 746~~~~~~~~~~~~~ 747 748 One of the following values: 749 750 * U_SUCCESS on success. 751 * U_FUNCTION if functionality is not supported. 752 * U_INVALID if VM is not secure. 753 * U_PARAMETER if ``gfn`` is invalid. 754 * U_P2 if ``num`` is invalid. 755 756Description 757~~~~~~~~~~~ 758 759 Stop sharing ``num`` pages starting at ``gfn`` with the Hypervisor. 760 Assume that the page size is PAGE_SIZE. Zero the pages before 761 returning. 762 763 If the address is already backed by an insecure page, unmap the page 764 and back it with a secure page. Inform the Hypervisor to release 765 reference to its shared page. If the address is not backed by a page 766 yet, mark the PTE as secure and back it with a secure page when that 767 address is accessed. If it is already backed by an secure page zero 768 the page and return. 769 770Use cases 771~~~~~~~~~ 772 773 #. The SVM may decide to unshare a page from the Hypervisor. 774 775 776UV_UNSHARE_ALL_PAGES 777-------------------- 778 779 Unshare all pages the SVM has shared with Hypervisor. 780 781Syntax 782~~~~~~ 783 784.. code-block:: c 785 786 uint64_t ultracall(const uint64_t UV_UNSHARE_ALL_PAGES) 787 788Return values 789~~~~~~~~~~~~~ 790 791 One of the following values: 792 793 * U_SUCCESS on success. 794 * U_FUNCTION if functionality is not supported. 795 * U_INVAL if VM is not secure. 796 797Description 798~~~~~~~~~~~ 799 800 Unshare all shared pages from the Hypervisor. All unshared pages are 801 zeroed on return. Only pages explicitly shared by the SVM with the 802 Hypervisor (using UV_SHARE_PAGE ultracall) are unshared. Ultravisor 803 may internally share some pages with the Hypervisor without explicit 804 request from the SVM. These pages will not be unshared by this 805 ultracall. 806 807Use cases 808~~~~~~~~~ 809 810 #. This call is needed when ``kexec`` is used to boot a different 811 kernel. It may also be needed during SVM reset. 812 813UV_ESM 814------ 815 816 Secure the virtual machine (*enter secure mode*). 817 818Syntax 819~~~~~~ 820 821.. code-block:: c 822 823 uint64_t ultracall(const uint64_t UV_ESM, 824 uint64_t esm_blob_addr, /* location of the ESM blob */ 825 unint64_t fdt) /* Flattened device tree */ 826 827Return values 828~~~~~~~~~~~~~ 829 830 One of the following values: 831 832 * U_SUCCESS on success (including if VM is already secure). 833 * U_FUNCTION if functionality is not supported. 834 * U_INVALID if VM is not secure. 835 * U_PARAMETER if ``esm_blob_addr`` is invalid. 836 * U_P2 if ``fdt`` is invalid. 837 * U_PERMISSION if any integrity checks fail. 838 * U_RETRY insufficient memory to create SVM. 839 * U_NO_KEY symmetric key unavailable. 840 841Description 842~~~~~~~~~~~ 843 844 Secure the virtual machine. On successful completion, return 845 control to the virtual machine at the address specified in the 846 ESM blob. 847 848Use cases 849~~~~~~~~~ 850 851 #. A normal virtual machine can choose to switch to a secure mode. 852 853Hypervisor Calls API 854#################### 855 856 This document describes the Hypervisor calls (hypercalls) that are 857 needed to support the Ultravisor. Hypercalls are services provided by 858 the Hypervisor to virtual machines and Ultravisor. 859 860 Register usage for these hypercalls is identical to that of the other 861 hypercalls defined in the Power Architecture Platform Reference (PAPR) 862 document. i.e on input, register R3 identifies the specific service 863 that is being requested and registers R4 through R11 contain 864 additional parameters to the hypercall, if any. On output, register 865 R3 contains the return value and registers R4 through R9 contain any 866 other output values from the hypercall. 867 868 This document only covers hypercalls currently implemented/planned 869 for Ultravisor usage but others can be added here when it makes sense. 870 871 The full specification for all hypercalls/ultracalls will eventually 872 be made available in the public/OpenPower version of the PAPR 873 specification. 874 875Hypervisor calls to support Ultravisor 876====================================== 877 878 Following are the set of hypercalls needed to support Ultravisor. 879 880H_SVM_INIT_START 881---------------- 882 883 Begin the process of converting a normal virtual machine into an SVM. 884 885Syntax 886~~~~~~ 887 888.. code-block:: c 889 890 uint64_t hypercall(const uint64_t H_SVM_INIT_START) 891 892Return values 893~~~~~~~~~~~~~ 894 895 One of the following values: 896 897 * H_SUCCESS on success. 898 * H_STATE if the VM is not in a position to switch to secure. 899 900Description 901~~~~~~~~~~~ 902 903 Initiate the process of securing a virtual machine. This involves 904 coordinating with the Ultravisor, using ultracalls, to allocate 905 resources in the Ultravisor for the new SVM, transferring the VM's 906 pages from normal to secure memory etc. When the process is 907 completed, Ultravisor issues the H_SVM_INIT_DONE hypercall. 908 909Use cases 910~~~~~~~~~ 911 912 #. Ultravisor uses this hypercall to inform Hypervisor that a VM 913 has initiated the process of switching to secure mode. 914 915 916H_SVM_INIT_DONE 917--------------- 918 919 Complete the process of securing an SVM. 920 921Syntax 922~~~~~~ 923 924.. code-block:: c 925 926 uint64_t hypercall(const uint64_t H_SVM_INIT_DONE) 927 928Return values 929~~~~~~~~~~~~~ 930 931 One of the following values: 932 933 * H_SUCCESS on success. 934 * H_UNSUPPORTED if called from the wrong context (e.g. 935 from an SVM or before an H_SVM_INIT_START 936 hypercall). 937 * H_STATE if the hypervisor could not successfully 938 transition the VM to Secure VM. 939 940Description 941~~~~~~~~~~~ 942 943 Complete the process of securing a virtual machine. This call must 944 be made after a prior call to ``H_SVM_INIT_START`` hypercall. 945 946Use cases 947~~~~~~~~~ 948 949 On successfully securing a virtual machine, the Ultravisor informs 950 Hypervisor about it. Hypervisor can use this call to finish setting 951 up its internal state for this virtual machine. 952 953 954H_SVM_INIT_ABORT 955---------------- 956 957 Abort the process of securing an SVM. 958 959Syntax 960~~~~~~ 961 962.. code-block:: c 963 964 uint64_t hypercall(const uint64_t H_SVM_INIT_ABORT) 965 966Return values 967~~~~~~~~~~~~~ 968 969 One of the following values: 970 971 * H_PARAMETER on successfully cleaning up the state, 972 Hypervisor will return this value to the 973 **guest**, to indicate that the underlying 974 UV_ESM ultracall failed. 975 976 * H_STATE if called after a VM has gone secure (i.e 977 H_SVM_INIT_DONE hypercall was successful). 978 979 * H_UNSUPPORTED if called from a wrong context (e.g. from a 980 normal VM). 981 982Description 983~~~~~~~~~~~ 984 985 Abort the process of securing a virtual machine. This call must 986 be made after a prior call to ``H_SVM_INIT_START`` hypercall and 987 before a call to ``H_SVM_INIT_DONE``. 988 989 On entry into this hypercall the non-volatile GPRs and FPRs are 990 expected to contain the values they had at the time the VM issued 991 the UV_ESM ultracall. Further ``SRR0`` is expected to contain the 992 address of the instruction after the ``UV_ESM`` ultracall and ``SRR1`` 993 the MSR value with which to return to the VM. 994 995 This hypercall will cleanup any partial state that was established for 996 the VM since the prior ``H_SVM_INIT_START`` hypercall, including paging 997 out pages that were paged-into secure memory, and issue the 998 ``UV_SVM_TERMINATE`` ultracall to terminate the VM. 999 1000 After the partial state is cleaned up, control returns to the VM 1001 (**not Ultravisor**), at the address specified in ``SRR0`` with the 1002 MSR values set to the value in ``SRR1``. 1003 1004Use cases 1005~~~~~~~~~ 1006 1007 If after a successful call to ``H_SVM_INIT_START``, the Ultravisor 1008 encounters an error while securing a virtual machine, either due 1009 to lack of resources or because the VM's security information could 1010 not be validated, Ultravisor informs the Hypervisor about it. 1011 Hypervisor should use this call to clean up any internal state for 1012 this virtual machine and return to the VM. 1013 1014H_SVM_PAGE_IN 1015------------- 1016 1017 Move the contents of a page from normal memory to secure memory. 1018 1019Syntax 1020~~~~~~ 1021 1022.. code-block:: c 1023 1024 uint64_t hypercall(const uint64_t H_SVM_PAGE_IN, 1025 uint64_t guest_pa, /* guest-physical-address */ 1026 uint64_t flags, /* flags */ 1027 uint64_t order) /* page size order */ 1028 1029Return values 1030~~~~~~~~~~~~~ 1031 1032 One of the following values: 1033 1034 * H_SUCCESS on success. 1035 * H_PARAMETER if ``guest_pa`` is invalid. 1036 * H_P2 if ``flags`` is invalid. 1037 * H_P3 if ``order`` of page is invalid. 1038 1039Description 1040~~~~~~~~~~~ 1041 1042 Retrieve the content of the page, belonging to the VM at the specified 1043 guest physical address. 1044 1045 Only valid value(s) in ``flags`` are: 1046 1047 * H_PAGE_IN_SHARED which indicates that the page is to be shared 1048 with the Ultravisor. 1049 1050 * H_PAGE_IN_NONSHARED indicates that the UV is not anymore 1051 interested in the page. Applicable if the page is a shared page. 1052 1053 The ``order`` parameter must correspond to the configured page size. 1054 1055Use cases 1056~~~~~~~~~ 1057 1058 #. When a normal VM becomes a secure VM (using the UV_ESM ultracall), 1059 the Ultravisor uses this hypercall to move contents of each page of 1060 the VM from normal memory to secure memory. 1061 1062 #. Ultravisor uses this hypercall to ask Hypervisor to provide a page 1063 in normal memory that can be shared between the SVM and Hypervisor. 1064 1065 #. Ultravisor uses this hypercall to page-in a paged-out page. This 1066 can happen when the SVM touches a paged-out page. 1067 1068 #. If SVM wants to disable sharing of pages with Hypervisor, it can 1069 inform Ultravisor to do so. Ultravisor will then use this hypercall 1070 and inform Hypervisor that it has released access to the normal 1071 page. 1072 1073H_SVM_PAGE_OUT 1074--------------- 1075 1076 Move the contents of the page to normal memory. 1077 1078Syntax 1079~~~~~~ 1080 1081.. code-block:: c 1082 1083 uint64_t hypercall(const uint64_t H_SVM_PAGE_OUT, 1084 uint64_t guest_pa, /* guest-physical-address */ 1085 uint64_t flags, /* flags (currently none) */ 1086 uint64_t order) /* page size order */ 1087 1088Return values 1089~~~~~~~~~~~~~ 1090 1091 One of the following values: 1092 1093 * H_SUCCESS on success. 1094 * H_PARAMETER if ``guest_pa`` is invalid. 1095 * H_P2 if ``flags`` is invalid. 1096 * H_P3 if ``order`` is invalid. 1097 1098Description 1099~~~~~~~~~~~ 1100 1101 Move the contents of the page identified by ``guest_pa`` to normal 1102 memory. 1103 1104 Currently ``flags`` is unused and must be set to 0. The ``order`` 1105 parameter must correspond to the configured page size. 1106 1107Use cases 1108~~~~~~~~~ 1109 1110 #. If Ultravisor is running low on secure pages, it can move the 1111 contents of some secure pages, into normal pages using this 1112 hypercall. The content will be encrypted. 1113 1114References 1115########## 1116 1117- `Supporting Protected Computing on IBM Power Architecture <https://developer.ibm.com/articles/l-support-protected-computing/>`_ 1118