xref: /linux/Documentation/gpu/nova/core/interrupts.rst (revision cb8fef4b07f71da01843457b66e579ad248a625e)
1.. SPDX-License-Identifier: GPL-2.0
2.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
3
4=============================================
5GPU interrupt handling: GIN and the GSP event
6=============================================
7
8This document describes how nova-core receives interrupts from the GPU on Turing
9and later chipsets. It covers the GPU Interrupt and Notification unit (GIN),
10which is the GPU's interrupt controller, and the GSP event, the interrupt that
11nova-core services in normal operation.
12
13Throughout, *CPU* means the CPU and the nova-core driver running on it. The GPU
14also has on-chip processors that run their own firmware and receive their own
15interrupts. The GSP (GPU System Processor) is one of them.
16
17Register names are the names from the GPU hardware reference headers. The
18pre-Hopper headers call the controller ``NV_CTRL`` and the Hopper-plus headers
19call it ``NV_GIN``. This document calls it GIN throughout, because the tree that
20nova-core services is the same on every supported chipset. OpenRM, NVIDIA's
21open-source GPU kernel driver, is cited wherever nova-core follows it.
22
23Terminology
24===========
25
26The three levels of the controller, innermost first:
27
28leaf
29    One ``LEAF`` register. Each of its 32 bits is the pending bit of one
30    interrupt source. A Turing, Ampere, or Ada tree has 8 leaves. A Hopper or
31    Blackwell tree has 16.
32
33subtree
34    Two consecutive leaves, summarized by one bit of ``TOP``. A subtree is the
35    unit of enabling at ``TOP``, and under MSI-X it is the unit of delivery:
36    every interrupt from one subtree arrives on one MSI-X entry.
37
38tree
39    One ``TOP`` register and the leaves under it. Every PCIe function has its
40    own tree, and nova-core services the CPU tree of one function.
41
42The hardware headers, OpenRM, and the Linux PCI API all use the word "vector",
43each for a different number. This document gives each one its own name, and a
44bare "vector" always means a GIN vector.
45
46GIN vector
47    The GPU-internal interrupt source number. It addresses one bit of one leaf.
48    A 16-leaf tree holds vectors 0 through 511, and an 8-leaf tree holds 0
49    through 255. The CPU doorbell is vector 129 and the GSP event is vector
50    155.
51
52MSI-X entry
53    An index into the device's MSI-X table. One entry serves one subtree.
54
55PCI vector
56    One of the interrupts that ``pci_alloc_irq_vectors()`` allocates: an MSI-X
57    entry, or the single MSI message.
58
59Linux IRQ number
60    What ``request_irq()`` takes, obtained from ``pci_irq_vector()`` for a PCI
61    vector. Linux's ``struct msix_entry`` calls this number ``.vector`` as
62    well.
63
64The remaining terms, each named for the register or the specification that
65defines it:
66
67enable, disable a vector
68    Writes to ``LEAF_EN_SET`` and ``LEAF_EN_CLEAR``.
69
70enable, disable a subtree
71    Writes to ``TOP_EN_SET`` and ``TOP_EN_CLEAR``.
72
73serviced subtree
74    A subtree that nova-core enables and has a handler for.
75
76rearm
77    Restoring PCI interrupt delivery after servicing an interrupt. See
78    "Rearming PCI interrupt delivery".
79
80mask
81    Reserved for the two places where hardware and the PCI specification use
82    the word: the MSI-X per-entry Vector Control mask bit, which Linux
83    controls, and the falcon interrupt masks. It never names a GIN enable.
84
85latched, pending
86    Two names for one state, a set ``LEAF`` bit. The vector's source sets the
87    bit whether or not the vector is enabled.
88
89clear a vector
90    Write a 1 to the vector's bit in ``LEAF``. OpenRM calls the same operation
91    ``intrClearLeafVector_HAL``.
92
93pending bits
94    The plain 32-bit value read from a ``LEAF`` register.
95
96notification
97    An interrupt whose only content is that something happened, such as a
98    posted message. Servicing a notification means reading what it announces.
99    The unit that raised it needs no attention. The GSP event is one.
100
101unit
102    Any block that raises an interrupt. "Engine" is reserved for the blocks
103    that do user work: GR, CE, NVDEC, and the like.
104
105falcon
106    One of the GPU's microcontrollers (see
107    Documentation/gpu/nova/core/falcon.rst). The GSP runs on the RISC-V core
108    inside its falcon. A falcon latches each of its interrupt causes and routes
109    it either to the host, meaning the CPU, or to its own core.
110
111The GIN controller
112==================
113
114A GPU has many interrupt sources: the GSP, the copy engines, the graphics
115engine, video decode and encode, the MMU fault path, timers, and others. GIN
116records which of them are pending and raises the PCI interrupt to the CPU.
117
118Trees
119-----
120
121GIN keeps one tree for each destination it can deliver an interrupt to. The CPU
122has one tree per PCIe function, so the physical function and each virtual
123function have their own. The GSP has a tree, and so do the other on-chip
124processors that receive interrupts. Every tree has the same two-level layout,
125and a function reaches its own tree through the per-function register aperture.
126
127nova-core services the CPU tree of one function. A virtual function's tree
128belongs to that function's driver, and a processor's tree belongs to the
129firmware running on that processor.
130
131The two-level tree
132------------------
133
134A tree is a set of ``LEAF`` registers and one ``TOP`` register.
135
136* ``LEAF(i)`` is a 32-bit register that holds the pending bits of vectors
137  ``32i`` through ``32i + 31``. A set bit is a pending vector.
138* ``TOP`` is a 32-bit read-only register. Bit ``N`` summarizes subtree ``N``,
139  which is ``LEAF(2N)`` and ``LEAF(2N + 1)``. The bit is set when an enabled
140  vector is pending in either leaf.
141
142A tree with L leaves has L / 2 subtrees and uses TOP bits 0 through L / 2 - 1.
143The other TOP bits read 0. The leaves and subtrees that a chipset has are its
144implemented leaves and subtrees, and "Per-architecture differences" gives the
145counts. For an 8-leaf tree::
146
147    TOP bit 0  ->  subtree 0  ->  LEAF(0), LEAF(1)   vectors   0..63
148    TOP bit 1  ->  subtree 1  ->  LEAF(2), LEAF(3)   vectors  64..127
149    TOP bit 2  ->  subtree 2  ->  LEAF(4), LEAF(5)   vectors 128..191
150    TOP bit 3  ->  subtree 3  ->  LEAF(6), LEAF(7)   vectors 192..255
151
152    LEAF(4), one bit per vector, holds vectors 128..159:
153
154      bit 1  = vector 129  (CPU doorbell)
155      bit 27 = vector 155  (GSP event)
156
157A vector's number fixes its place in the tree::
158
159    leaf    = vector / 32
160    bit     = vector % 32
161    subtree = leaf / 2
162
163Registers
164---------
165
166nova-core defines the tree's registers in the ``irq`` module's ``regs.rs``. The
167leaf registers are arrays indexed by leaf number.
168
169* ``LEAF(i)`` reads as the pending bits of leaf ``i``. Writing a 1 to a bit
170  clears that vector, and a 0 leaves the bit as it was.
171* ``LEAF_EN_SET(i)`` and ``LEAF_EN_CLEAR(i)`` enable and disable the vectors
172  of leaf ``i``, one bit per vector.
173* ``TOP_EN_SET`` and ``TOP_EN_CLEAR`` enable and disable subtrees, one bit per
174  subtree.
175* ``LEAF_TRIGGER`` takes a vector number and latches that vector, exactly as
176  the vector's own source would. It is write-only. The self-test uses it.
177
178nova-core does not read ``TOP``. "Servicing the tree" says why.
179
180Every set and clear register acts per bit: a 1 performs the action for that
181bit, and a 0 leaves the bit alone. No register needs a read-modify-write.
182
183GIN delivers a vector to the CPU only when its leaf enable bit and its
184subtree's TOP enable bit are both set. The enables do not affect the latch. The
185source of a disabled vector still sets its ``LEAF`` bit. ``TOP`` does not show
186that bit, so reading the leaf is the only way to see it.
187
188How a unit interrupt reaches the CPU
189------------------------------------
190
191A unit does not write a ``LEAF`` register. Each unit has an interrupt control
192register that GSP firmware programs. The control register holds the unit's
193vector, the GFID that identifies the PCIe function whose tree receives the
194interrupt, and one enable bit per destination: the CPU, the GSP, and the other
195on-chip processors. When the unit has an event::
196
197    1. The unit sends GIN an interrupt message carrying the vector, the GFID,
198       and the destination enables from its control register.
199    2. In the tree of each destination that the message selects, GIN sets bit
200       (vector % 32) of LEAF(vector / 32).
201    3. If the vector and its subtree are enabled in the CPU tree, GIN raises
202       the PCI interrupt.
203
204Because firmware assigns the vectors, nova-core does not hardcode which vector
205belongs to which unit, with two exceptions. The hardware headers of every
206supported chipset define the GSP event as vector 155, and GSP firmware's own
207interrupt table uses that definition. The CPU doorbell is vector 129. Pre-Hopper
208hardware fixes that number, and GSP firmware keeps it on Hopper and later.
209nova-core names both by number. A driver can fetch the full unit-to-vector
210table from the GSP by RPC, and nouveau does. nova-core does not, because a
211fixed vector needs no lookup.
212
213Edge-triggered delivery
214-----------------------
215
216A ``LEAF`` bit is a latch. Its source sets it on a rising edge, and it stays set
217until the CPU clears it. A source that stays high does not set the bit again.
218
219GIN raises the PCI interrupt for a subtree when the subtree's enabled pending
220state goes from low to high::
221
222    Per vector, in leaf i at bit b:
223        LEAF(i)[b] AND LEAF_EN(i)[b]
224
225    Per subtree N, across leaves 2N and 2N + 1:
226        OR of every enabled pending bit  ->  TOP[N]
227
228    Delivery for subtree N:
229        TOP[N] AND TOP_EN[N]  ->  rising edge  ->  PCI interrupt
230
231``TOP_EN`` applies after the summary, so disabling a subtree stops delivery
232without changing what ``TOP`` reports.
233
234Three consequences:
235
236* Code that must find every pending vector reads the leaves. A vector that
237  latched while disabled is not in ``TOP``.
238* Writing ``TOP_EN_SET`` for a subtree with an enabled pending bit produces a
239  new edge. GIN delivers an interrupt for a pending bit left uncleared as soon
240  as its subtree is enabled again.
241* A source that holds its signal high produces no new edge after the CPU
242  clears the leaf bit. Such a source has to re-emit its interrupt. The falcons
243  do that through ``INTR_RETRIGGER`` (see "Retriggering a falcon").
244
245Delivery over PCI
246=================
247
248GIN delivers the tree's interrupts to the CPU as MSI or MSI-X, whichever Linux
249grants. nova-core requests MSI-X first and falls back to MSI, and never uses
250INTx.
251
252MSI has a single message, and every subtree raises that one message, so one
253Linux IRQ serves the whole tree.
254
255MSI-X gives each subtree its own table entry, at the index equal to the subtree
256number. Linux masks every entry until a driver requests its Linux IRQ number,
257and a masked entry sends no message: the GPU records the interrupt in the MSI-X
258pending bit array, where it stays until Linux unmasks the entry. An entry that
259the driver never requests is never unmasked. A driver that enables a subtree
260without requesting that subtree's entry loses every interrupt from that
261subtree, with nothing reported: the leaf and TOP registers show the vector
262pending and enabled while no handler runs.
263
264The serviced-subtree invariant
265------------------------------
266
267Every subtree enabled at ``TOP`` has an allocated PCI vector with a registered
268handler.
269
270MSI satisfies this with its single message. MSI-X needs one allocated entry per
271serviced subtree, and a PCI allocation cannot be sparse, so nova-core requests
272entries 0 through the highest serviced subtree::
273
274    MSI-X, with subtree 2 serviced:
275
276      subtree 0  ->  entry 0   allocated, no handler, stays masked
277      subtree 1  ->  entry 1   allocated, no handler, stays masked
278      subtree 2  ->  entry 2   handler here, and its rearm covers subtree 2
279
280    MSI, with any serviced set:
281
282      every serviced subtree  ->  the one allocated PCI vector, whose
283                                  handler's rearm covers the whole serviced set
284
285An allocated entry whose subtree nova-core does not service costs nothing. The
286entry stays masked, and a disabled subtree raises no interrupt.
287
288nova-core services one subtree. The GSP event, vector 155, is in leaf 4, which
289is in subtree 2. OpenRM's headers place its UVM_SHARED interrupt category in
290subtree 2 on every chipset that nova-core supports. The self-test doorbell,
291vector 129, is in the same leaf, and the test allocates its own vectors for it.
292
293Rearming PCI interrupt delivery
294-------------------------------
295
296A message-signaled interrupt is delivered once per edge, and the PCI side
297delivers no further interrupt until the CPU rearms it. The rearm operation
298depends on the GPU family and on the interrupt type that Linux granted:
299
300==================  =====  ===========================================
301Architecture        Type   Rearm operation
302==================  =====  ===========================================
303Turing through Ada  MSI    write the configuration-mirror EOI register
304Hopper and later    MSI    clear then set the serviced TOP enables
305Any                 MSI-X  clear then set the handler's own TOP enable
306==================  =====  ===========================================
307
308The end-of-interrupt register is ``NV_XVE_CYA_2`` in the BAR0 mirror of PCI
309configuration space, and the value written does not matter. The ``TOP_EN``
310cycle produces a new delivery edge. The MSI forms cover every serviced subtree,
311because one message serves all of them. The MSI-X form covers one subtree,
312because each serviced subtree has its own entry and its own handler.
313
314A handler rearms once per delivered interrupt, on every path, including the
315path where it finds its vector not pending. A handler that skips the rearm
316receives no further interrupts.
317
318OpenRM makes the same split. It writes the configuration-space EOI for MSI on
319pre-Hopper chipsets, and cycles the TOP enables of the subtrees that it
320services for Hopper-plus MSI and for MSI-X.
321
322Servicing the tree
323==================
324
325Servicing a leaf has a required order: read its pending bits, then clear them.
326Clearing a leaf before reading it discards every vector latched in it, and no
327register reports the loss. In nova-core, reading a leaf produces the handle that
328clears it, so that the wrong order does not compile. The handle clears exactly
329the bits that it read, so a vector that latched after the read stays pending.
330
331A handler clears its bit before it services the vector. Clearing afterwards
332would discard an interrupt that the source raised while the handler ran.
333
334nova-core services the tree in two ways.
335
336The notification path services one vector. It reads the vector's leaf, clears
337only the vector's bit, and rearms. The subtree stays enabled, and a vector
338pending beside it in the same leaf keeps its bit set for the code that services
339that vector. The GSP event handler takes this path, and so does the self-test
340handler.
341
342The startup drain walks the whole tree, because it must clear whatever is
343pending across every subtree rather than one known vector. It disables the
344serviced subtrees at ``TOP``, reads and clears every implemented leaf, and
345leaves the subtrees disabled for its caller to enable once the caller is ready
346for deliveries. The drain reads every leaf rather than descending from ``TOP``,
347because sources latch vectors during boot while those vectors are disabled, and
348``TOP`` does not show them. OpenRM's stall-interrupt path reads every leaf for
349the same reason.
350
351The two paths as register operations::
352
353    Startup drain, run once during probe:
354        write TOP_EN_CLEAR = serviced        stop new deliveries
355        for each implemented leaf i:
356            pending = read LEAF(i)
357            write LEAF(i) = pending          clear what was read
358        (returns with TOP_EN still clear)
359
360    Notification, the subtree stays enabled:
361        pending = read LEAF(leaf)            is the handler's bit set?
362        write LEAF(leaf) = bit               clear that one bit
363        rearm PCI interrupt delivery
364
365The drain clears every pending bit, including bits that nova-core never
366services. An uncleared bit holds its subtree in the pending state, and enabling
367that subtree again would deliver an interrupt for a vector that no handler
368services.
369
370The drain's ``TOP_EN_CLEAR`` is not a rearm, and pre-Hopper MSI rearms through
371the configuration mirror, which the drain never writes. An interrupt delivered
372before probe had no handler to rearm it, so the tree reset rearms explicitly
373after the drain. A TOP-enable rearm leaves the serviced subtrees enabled, so
374the reset then disables them again.
375
376nova-core does not serialize access to the tree. The GSP event handler
377touches only its own leaf, and the drain runs during probe, before that handler
378is registered.
379
380Per-architecture differences
381============================
382
383The tree is the same on every supported GPU except for its size, which changes
384at Hopper:
385
386===================  ======  ========  ====================
387GPUs                 Leaves  Subtrees  Implemented subtrees
388===================  ======  ========  ====================
389Turing, Ampere, Ada  8       4         ``0x0f``
390Hopper, Blackwell    16      8         ``0xff``
391===================  ======  ========  ====================
392
393The interrupt HAL provides the leaf count, and the subtree count and the
394implemented-subtree set derive from it. A subtree that the chipset does not
395implement has no TOP bit, so building a tree that services one fails with
396``EINVAL``. Vectors 129 and 155 are in the 8-leaf tree, so every supported
397chipset has them.
398
399OpenRM's headers assign every interrupt category of a 16-leaf tree to leaves 0
400through 11. The drain reads all 16, because a vector can be latched in any
401implemented leaf.
402
403The HAL performs the rearm as well (see "Rearming PCI interrupt delivery"). Two
404falcon properties also differ by family, and the falcon HAL carries them:
405Turing falcons have no ``INTR_RETRIGGER``, and the RISC-V routing registers
406moved at GA102 (see "Retriggering a falcon").
407
408The GSP event
409=============
410
411When the GSP has output for the CPU, it writes messages into the GSP-to-CPU
412queue in shared memory and raises SWGEN0, one of the software-generated
413interrupt causes of the GSP falcon. SWGEN0 is routed to the host, at vector
414155, leaf 4 bit 27, in subtree 2.
415
416The queue carries notifications (log records, error records, lifecycle events)
417and command replies. A thread waiting for a reply reads the queue itself, so
418the interrupt is only the trigger to drain the queue (see "Draining the
419GSP-to-CPU queue").
420
421The falcon latches every cause that it raises, SWGEN0 among them, in its
422``IRQSTAT`` register. The handler services the host-routed causes and clears
423their latches, and then it writes ``INTR_RETRIGGER`` so that the falcon re-emits
424any cause that latched in the meantime. "Retriggering a falcon" has the details.
425
426Draining the queue takes the command-queue mutex and walks shared memory, so it
427cannot run in hard interrupt context. nova-core registers a threaded handler,
428under the name ``nova-core`` in ``/proc/interrupts``. The top half runs in hard
429interrupt context and reads and writes only registers, and it wakes the IRQ
430thread to drain the queue::
431
432    GSP writes messages into the GSP-to-CPU queue
433    GSP raises SWGEN0
434    GIN sets bit 27 of LEAF(4), and subtree 2 becomes pending
435    PCI interrupt -> Linux IRQ -> top half, in hard interrupt context:
436        read LEAF(4), and if bit 27 is clear, rearm and return
437        clear bit 27 (the subtree stays enabled)
438        read the falcon causes routed to the host, clearing SWGEN0 if set
439        for every other host cause: log it, clear its latch, and read the
440            host causes back
441        if the clear ended all of them: retrigger the falcon
442        otherwise: disable vector 155 at its leaf and skip the retrigger
443        rearm PCI interrupt delivery
444        wake the IRQ thread if SWGEN0 was set
445    IRQ thread, which may sleep:
446        take the command-queue mutex and drain the GSP-to-CPU queue
447
448A halt and a posted message can be pending together, so the top half services
449every cause that the status reports.
450
451A drain fails when a message's framing or checksum is bad (see "Draining the
452GSP-to-CPU queue"). The message stays at the queue head, so every later drain
453would fail the same way, and the IRQ thread disables vector 155 and logs the
454failure, which leaves the queue unserviced until the device is reset.
455
456The handler, the self-test, and the rest of the driver read BAR0 through one
457shared mapping. nova-core unregisters an interrupt handler when the device
458unbinds, so a handler runs only while the mapping exists.
459
460Retriggering a falcon
461---------------------
462
463A falcon signals the tree when its set of host-routed causes goes from empty to
464non-empty. A cause left latched keeps the set non-empty, so no later cause
465signals the tree, and the vector is lost. For a cause that stays latched, the
466handler can clear the tree leaf first or the falcon latch first, and the loss
467is the same.
468
469``IRQSTAT`` latches every cause in the falcon, including the causes routed to
470the falcon's own RISC-V core and owned by the firmware running on it. A host
471handler owns only the causes that ``PRISCV_RISCV_IRQMASK`` and
472``PRISCV_RISCV_IRQDEST`` both select, so it intersects ``IRQSTAT`` with both
473before it reads or clears a cause. OpenRM computes the same intersection in
474``kflcnRiscvReadIntrStatus``. GA100 keeps the Turing offsets of the two routing
475registers and GA102 moves them, so the offsets change at GA102 rather than at
476the Ampere boundary. The handler masks no cause: ``PRISCV_RISCV_IRQMASK`` is
477read-only to the host, and ``FALCON_IRQMASK`` has no effect on host routing on
478a RISC-V falcon.
479
480``INTR_RETRIGGER`` makes the falcon re-emit its host-routed causes into the
481tree, which supplies the transition that clearing the leaf lost. The handler
482writes ``INTR_RETRIGGER`` only on a path where it ended every cause that it
483read, because a re-emitted cause that stays set arrives again at once and
484on every pass after that.
485
486``IRQSCLR`` ends a latch and does not end the source behind it, so a cause
487driven from outside the falcon stays set after the write. On Blackwell the
488fault-containment and ECC causes are driven that way: they appear in
489``IRQSTAT`` but come from ``PRISCV_RISCV_FAULT_CONTAINMENT_SRCSTAT`` and
490``PGSP_ECC_INTR_STATUS``, and only a device reset ends them. So the handler
491clears the latch of every host cause other than SWGEN0, reads the host causes
492back, and retriggers only when the read-back is empty. When a cause is still
493set, the handler disables vector 155 instead and reports that the device needs
494a reset. Disabling loses no notification: the cause that is still set holds the
495host-routed set non-empty, so the falcon would signal nothing further either
496way. OpenRM makes the same choice, and ``kgspService_TU102`` skips
497``kflcnIntrRetrigger`` once it has recorded a fatal error.
498
499The handler cannot distinguish a fault cause that arrives after the clear from
500one that the clear failed to end, so it disables the vector in that case too.
501Both mean that the GSP has faulted.
502
503Turing falcons have no ``INTR_RETRIGGER``, so a Turing handler cannot re-create
504a transition that it has lost. It must leave no host cause latched: it reads the
505host-routed status once and takes every cause that the status reports, rather
506than stopping at the first one that it recognizes. One window stays open. A
507cause that arrives after the handler has read the status is not in the value
508that the handler clears, so it stays latched after the leaf has been cleared,
509and no later cause from that falcon signals the tree. OpenRM has the same window
510on Turing, where ``kflcnIntrRetrigger`` does nothing.
511
512Enabling the GSP event
513----------------------
514
515SWGEN0 is a latch, and the GSP drives no new edge into the tree while it stays
516set. nova-core's GSP boot code consumes the GSP's notifications by polling the
517queue, which leaves the latch set and leaves pending bits in the tree. The
518handoff from polling to interrupts has a required order::
519
520    disable every implemented vector    drop enables left by boot, or by a
521                                        driver that ran before this one
522    drain the tree                      clear stale pending bits
523    rearm PCI interrupt delivery        required under pre-Hopper MSI, which
524                                        no later step rearms
525    disable subtree 2 at TOP            a TOP-enable rearm enabled it again
526    clear the SWGEN0 latch              so the next message makes an edge
527    register the threaded handler       no delivery can reach it yet
528    enable vector 155 at LEAF(4)        the subtree is still disabled
529    enable subtree 2 at TOP             deliveries become possible here
530    drain the GSP-to-CPU queue          messages posted before the clear
531
532The first four steps are the tree reset. nova-core runs them, and clears the
533latch, before it registers the handler. Registering unmasks the PCI interrupt,
534and GIN would then deliver a vector that boot left enabled to a handler that
535services one vector and has no way to service any other. OpenRM clears every
536leaf enable at the same point for the same reason.
537
538nova-core clears the latch after the drain. If nova-core cleared the latch
539first, a message posted before the drain could set it again, along with bit 27
540of LEAF(4). The drain would then clear the leaf bit while the latch stays set,
541and no later message would signal the tree. Clearing after the drain can
542instead leave the leaf bit pending with the latch already clear. Enabling the
543subtree then delivers one interrupt whose ``IRQSTAT`` reads zero. The top half
544clears the leaf bit, rearms, and does not wake the IRQ thread, and the queue
545drain that follows reads the message.
546
547Clearing the latch makes the first interrupt possible. A message that the GSP
548posted before that clear produces no interrupt, so the sequence ends by
549draining the queue.
550
551The subtree is enabled at ``TOP`` only once the handler is registered. A
552TOP-enable rearm sets the enables that it cycles, so the reset disables the
553serviced subtrees again after its rearm, and the tree reaches the registration
554with subtree 2 disabled under every rearm method. On teardown nova-core disables
555the vector at its leaf, so that no vector in the subtree can be delivered, then
556calls ``free_irq()``, and disables the subtree last. Disabling the subtree
557earlier would let a handler still in flight enable it again through the
558``TOP_EN`` cycle of its rearm, which would leave the subtree enabled with no
559handler registered. The driver's ``Gpu`` object, which owns the tree, holds the
560subtree enable, and a handler's registration holds only the enable of its own
561vector, so that tearing down one handler does not disable a subtree that another
562handler shares. nova-core tears down the registration before it frees the queue
563that the handler drains and the falcon that the handler reads, and before it
564unloads the GSP.
565
566Draining the GSP-to-CPU queue
567-----------------------------
568
569The queue carries command replies and unsolicited events, and a message's
570function code says which it is.
571
572* A function code that matches the awaited reply: the message is decoded and
573  returned to the caller that sent the command.
574* Anything else is an event. An OS error record and a robust-channel record
575  are logged at error level, and an unrecognized function code at warning
576  level. The other known events (GSP logs, libos prints, assertion records,
577  lifecycle notices) need no action and get no line of their own, because the
578  receive trace at debug level already records every message's arrival with
579  its sequence number, function code, and length.
580
581The sequence number takes no part in the match, because the GSP does not echo
582the command's sequence number on every reply. On r570 the reply to
583``UnloadingGuestDriver`` carries sequence 0.
584
585The read pointer advances past every message that passes framing and checksum
586validation, whether it matched or was an event. A matched message that is too
587short to decode is the exception, and it stays at the queue head.
588
589A message's length is inside the region that the checksum covers, so once the
590framing or the checksum fails there is no trustworthy length with which to skip
591the message. Such a message also stays at the queue head, and every later
592receive fails on it with ``EIO`` until the device is reset.
593
594The polling path and the IRQ thread both read the queue under the command-queue
595mutex. Replies and events share one queue and one read pointer, so one lock is
596held across the whole drain. A thread waiting for a reply logs each event that
597arrives before the reply and keeps waiting. One deadline of 5 seconds applies to
598the whole wait, rather than a fresh timeout after each message, and the thread
599holds the mutex for the whole wait, so that no other caller consumes the message
600that it waits for.
601
602With one lock, a drain waits for an in-flight command's receive to finish or
603time out. For log and error records that delay does not matter.
604