Lines Matching refs:rcu_node
28 of ``rcu_node`` structures. Each leaf node of the ``rcu_node`` tree has up
34 which results in a three-level ``rcu_node`` tree.
36 at boot time, resulting in an ``rcu_node`` tree with only a single node.
43 and other events are recorded by the leaf-level ``rcu_node``
46 grace periods are completed at the tree's root ``rcu_node``
66 | ``rcu_node`` structures than further up the tree. Therefore, if the |
67 | leaf ``rcu_node`` structures have fanout of 64, the contention on |
70 | for the leaves of the ``rcu_node`` tree. |
74 | ``rcu_node`` structures must also be reduced. Such reduction can be |
77 | on the non-leaf ``rcu_node`` structures, you may use the |
83 | the ``rcu_node`` structures align with hardware boundaries. |
90 65,536 CPUs, RCU would configure the ``rcu_node`` tree as follows:
104 the last CPU to report a quiescent state into a given ``rcu_node``
105 structure need advance to the ``rcu_node`` structure at the next level
106 up the tree. This means that at the leaf-level ``rcu_node`` structure,
108 internal ``rcu_node`` structures, the situation is even more extreme:
113 period will progress all the way to the root ``rcu_node`` structure,
114 thus ensuring that the lock contention on that root ``rcu_node``
138 are protected by the corresponding root ``rcu_node`` structure's lock.
139 #. Each ``rcu_node`` structure has a spinlock.
155 ``rcu_node`` and ``rcu_data`` structures, tracks grace periods,
160 #. ``rcu_node``: This structure forms the combining tree that propagates
169 ``CONFIG_RCU_BOOST``, it manages the per-\ ``rcu_node``
175 to the corresponding leaf ``rcu_node`` structure to allow
176 more-efficient propagation of quiescent states up the ``rcu_node``
177 combining tree. Like the ``rcu_node`` structure, it provides a local
189 following sections give more details on the ``rcu_state``, ``rcu_node``
197 between the ``rcu_node`` and ``rcu_data`` structures, tracks grace
206 Relationship to rcu_node and rcu_data Structures
213 1 struct rcu_node node[NUM_RCU_NODES];
214 2 struct rcu_node *level[NUM_RCU_LVLS + 1];
220 | Wait a minute! You said that the ``rcu_node`` structures formed a |
232 The ``rcu_node`` tree is embedded into the ``->node[]`` array as shown
243 Each entry of the ``->level`` array references the first ``rcu_node``
250 ``rcu_node`` structure, the first element references the first child of
251 the root ``rcu_node``, and finally the second element references the
252 first leaf ``rcu_node`` structure.
280 the root ``rcu_node`` structure's ``->lock`` field.
282 There are ``->gp_seq`` fields in the ``rcu_node`` and ``rcu_data``
286 distributed fashion. The values flow from ``rcu_state`` to ``rcu_node``
301 in jiffies. It is protected by the root ``rcu_node``'s ``->lock``.
307 The ``rcu_node`` Structure
310 The ``rcu_node`` structures form the combining tree that propagates
319 manage the per-\ ``rcu_node`` priority-boosting kernel threads
323 The ``rcu_node`` structure's fields are discussed, singly and in groups,
329 This portion of the ``rcu_node`` structure is declared as follows:
333 1 struct rcu_node *parent;
340 The ``->parent`` pointer references the ``rcu_node`` one level up in the
341 tree, and is ``NULL`` for the root ``rcu_node``. The RCU implementation
351 ``rcu_node`` structure in its parent's bitmasks, which are described
353 lowest and highest numbered CPU served by this ``rcu_node`` structure,
362 This field of the ``rcu_node`` structure is declared as follows:
379 This portion of the ``rcu_node`` structure is declared as follows:
386 The ``rcu_node`` structures' ``->gp_seq`` fields are the counterparts of
389 two bits of a given ``rcu_node`` structure's ``->gp_seq`` field is zero,
390 then this ``rcu_node`` structure believes that RCU is idle.
392 The ``>gp_seq`` field of each ``rcu_node`` structure is updated at the
396 period request seen by the corresponding ``rcu_node`` structure. The
403 | Suppose that this ``rcu_node`` structure doesn't see a request for a |
421 This portion of the ``rcu_node`` structure has fields as follows:
430 The ``->qsmask`` field tracks which of this ``rcu_node`` structure's
433 corresponding bit. Note that the leaf ``rcu_node`` structures should be
435 Similarly, the ``->expmask`` field tracks which of this ``rcu_node``
443 ``rcu_node`` structure's children cover for at least one online CPU.
470 | its quiescent state up the ``rcu_node`` tree. |
489 know that the ``rcu_node`` structure tracks them.
502 ``->blkd_tasks`` list for the leaf ``rcu_node`` structure corresponding
523 critical section, then the state of the last leaf ``rcu_node``
538 Sizing the ``rcu_node`` Array
541 The ``rcu_node`` array is sized via a series of C-preprocessor
613 The maximum number of levels in the ``rcu_node`` structure is currently
623 ``rcu_node`` boundaries). In addition, building kernels with a full four
624 levels of ``rcu_node`` tree permits better testing of RCU's
628 each non-leaf level of the ``rcu_node`` tree. If the
633 each leaf ``rcu_node`` structure. Experience has shown that allowing a
634 given leaf ``rcu_node`` structure to handle 64 CPUs, as permitted by the
636 excessive contention for the leaf ``rcu_node`` structures' ``->lock``
637 fields. The number of CPUs per leaf ``rcu_node`` structure is therefore
644 single-level (which contains a single ``rcu_node`` structure),
645 two-level, three-level, and four-level ``rcu_node`` tree, respectively,
652 spanning lines 26-66 that computes the number of ``rcu_node`` structures
656 ``rcu_node`` structures for the topmost level of the tree is always
659 (if any) of the ``rcu_node`` tree are computed by dividing the maximum
786 its relationship to the corresponding leaf ``rcu_node`` structure to
787 allow more-efficient propagation of quiescent states up the ``rcu_node``
788 combining tree. Like the ``rcu_node`` structure, it provides a local
805 2 struct rcu_node *mynode;
810 ``->mynode`` field references the corresponding ``rcu_node`` structure.
836 in the ``rcu_state`` and ``rcu_node`` structures. The
838 name in the rcu_node structure. They may each lag up to one behind their
839 ``rcu_node`` counterparts, but in ``CONFIG_NO_HZ_IDLE`` and
900 its leaf ``rcu_node`` structure. Recall that each ``rcu_node``
1064 5 struct rcu_node *rcu_blocked_node;
1079 ``->rcu_blocked_node`` field references the ``rcu_node`` structure whose
1101 1 static struct rcu_node *rcu_get_root(struct rcu_state *rsp)
1116 root ``rcu_node`` structure.
1119 advantage of the layout of the ``rcu_node`` structures in the
1123 array, thus traversing only the leaf ``rcu_node`` structures.
1128 | What does ``rcu_for_each_leaf_node()`` do if the ``rcu_node`` tree |
1141 contains a combining tree of ``rcu_node`` and ``rcu_data`` structures.