1.\"- 2.\" Copyright (c) 2013 Gleb Smirnoff <glebius@FreeBSD.org> 3.\" All rights reserved. 4.\" 5.\" Redistribution and use in source and binary forms, with or without 6.\" modification, are permitted provided that the following conditions 7.\" are met: 8.\" 1. Redistributions of source code must retain the above copyright 9.\" notice, this list of conditions and the following disclaimer. 10.\" 2. Redistributions in binary form must reproduce the above copyright 11.\" notice, this list of conditions and the following disclaimer in the 12.\" documentation and/or other materials provided with the distribution. 13.\" 14.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 18.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24.\" SUCH DAMAGE. 25.\" 26.Dd March 11, 2021 27.Dt COUNTER 9 28.Os 29.Sh NAME 30.Nm counter 31.Nd "SMP-friendly kernel counter implementation" 32.Sh SYNOPSIS 33.In sys/types.h 34.In sys/systm.h 35.In sys/counter.h 36.Ft counter_u64_t 37.Fn counter_u64_alloc "int wait" 38.Ft void 39.Fn counter_u64_free "counter_u64_t c" 40.Ft void 41.Fn counter_u64_add "counter_u64_t c" "int64_t v" 42.Ft void 43.Fn counter_enter 44.Ft void 45.Fn counter_exit 46.Ft void 47.Fn counter_u64_add_protected "counter_u64_t c" "int64_t v" 48.Ft uint64_t 49.Fn counter_u64_fetch "counter_u64_t c" 50.Ft void 51.Fn counter_u64_zero "counter_u64_t c" 52.Ft int64_t 53.Fn counter_ratecheck "struct counter_rate *cr" "int64_t limit" 54.Fn COUNTER_U64_SYSINIT "counter_u64_t c" 55.Fn COUNTER_U64_DEFINE_EARLY "counter_u64_t c" 56.In sys/sysctl.h 57.Fn SYSCTL_COUNTER_U64 parent nbr name access ptr descr 58.Fn SYSCTL_ADD_COUNTER_U64 ctx parent nbr name access ptr descr 59.Fn SYSCTL_COUNTER_U64_ARRAY parent nbr name access ptr len descr 60.Fn SYSCTL_ADD_COUNTER_U64_ARRAY ctx parent nbr name access ptr len descr 61.Sh DESCRIPTION 62.Nm 63is a generic facility to create counters 64that can be utilized for any purpose (such as collecting statistical 65data). 66A 67.Nm 68is guaranteed to be lossless when several kernel threads do simultaneous 69updates. 70However, 71.Nm 72does not block the calling thread, 73also no 74.Xr atomic 9 75operations are used for the update, therefore the counters 76can be used in any non-interrupt context. 77Moreover, 78.Nm 79has special optimisations for SMP environments, making 80.Nm 81update faster than simple arithmetic on the global variable. 82Thus 83.Nm 84is considered suitable for accounting in the performance-critical 85code paths. 86.Bl -tag -width indent 87.It Fn counter_u64_alloc wait 88Allocate a new 64-bit unsigned counter. 89The 90.Fa wait 91argument is the 92.Xr malloc 9 93wait flag, should be either 94.Va M_NOWAIT 95or 96.Va M_WAITOK . 97If 98.Va M_NOWAIT 99is specified the operation may fail and return 100.Dv NULL . 101.It Fn counter_u64_free c 102Free the previously allocated counter 103.Fa c . 104It is safe to pass 105.Dv NULL . 106.It Fn counter_u64_add c v 107Add 108.Fa v 109to 110.Fa c . 111The KPI does not guarantee any protection from wraparound. 112.It Fn counter_enter 113Enter mode that would allow the safe update of several counters via 114.Fn counter_u64_add_protected . 115On some machines this expands to 116.Xr critical 9 117section, while on other is a nop. 118See 119.Sx IMPLEMENTATION DETAILS . 120.It Fn counter_exit 121Exit mode for updating several counters. 122.It Fn counter_u64_add_protected c v 123Same as 124.Fn counter_u64_add , 125but should be preceded by 126.Fn counter_enter . 127.It Fn counter_u64_fetch c 128Take a snapshot of counter 129.Fa c . 130The data obtained is not guaranteed to reflect the real cumulative 131value for any moment. 132.It Fn counter_u64_zero c 133Clear the counter 134.Fa c 135and set it to zero. 136.It Fn counter_ratecheck cr limit 137The function is a multiprocessor-friendly version of 138.Fn ppsratecheck 139which uses 140.Nm 141internally. 142Returns non-negative value if the rate is not yet reached during the current 143second, and a negative value otherwise. 144If the limit was reached on previous second, but was just reset back to zero, 145then 146.Fn counter_ratecheck 147returns number of events since previous reset. 148.It Fn COUNTER_U64_SYSINIT c 149Define a 150.Xr SYSINIT 9 151initializer for the global counter 152.Fa c . 153.It Fn COUNTER_U64_DEFINE_EARLY c 154Define and initialize a global counter 155.Fa c . 156It is always safe to increment 157.Fa c , 158though updates prior to the 159.Dv SI_SUB_COUNTER 160.Xr SYSINIT 9 161event are lost. 162.It Fn SYSCTL_COUNTER_U64 parent nbr name access ptr descr 163Declare a static 164.Xr sysctl 9 165oid that would represent a 166.Nm . 167The 168.Fa ptr 169argument should be a pointer to allocated 170.Vt counter_u64_t . 171A read of the oid returns value obtained through 172.Fn counter_u64_fetch . 173Any write to the oid zeroes it. 174.It Fn SYSCTL_ADD_COUNTER_U64 ctx parent nbr name access ptr descr 175Create a 176.Xr sysctl 9 177oid that would represent a 178.Nm . 179The 180.Fa ptr 181argument should be a pointer to allocated 182.Vt counter_u64_t . 183A read of the oid returns value obtained through 184.Fn counter_u64_fetch . 185Any write to the oid zeroes it. 186.It Fn SYSCTL_COUNTER_U64_ARRAY parent nbr name access ptr len descr 187Declare a static 188.Xr sysctl 9 189oid that would represent an array of 190.Nm . 191The 192.Fa ptr 193argument should be a pointer to allocated array of 194.Vt counter_u64_t's . 195The 196.Fa len 197argument should specify number of elements in the array. 198A read of the oid returns len-sized array of 199.Vt uint64_t 200values obtained through 201.Fn counter_u64_fetch . 202Any write to the oid zeroes all array elements. 203.It Fn SYSCTL_ADD_COUNTER_U64_ARRAY ctx parent nbr name access ptr len descr 204Create a 205.Xr sysctl 9 206oid that would represent an array of 207.Nm . 208The 209.Fa ptr 210argument should be a pointer to allocated array of 211.Vt counter_u64_t's . 212The 213.Fa len 214argument should specify number of elements in the array. 215A read of the oid returns len-sized array of 216.Vt uint64_t 217values obtained through 218.Fn counter_u64_fetch . 219Any write to the oid zeroes all array elements. 220.El 221.Sh IMPLEMENTATION DETAILS 222On all architectures 223.Nm 224is implemented using per-CPU data fields that are specially aligned 225in memory, to avoid inter-CPU bus traffic due to shared use 226of the variables between CPUs. 227These are allocated using 228.Va UMA_ZONE_PCPU 229.Xr uma 9 230zone. 231The update operation only touches the field that is private to current CPU. 232Fetch operation loops through all per-CPU fields and obtains a snapshot 233sum of all fields. 234.Pp 235On amd64 a 236.Nm counter 237update is implemented as a single instruction without lock semantics, 238operating on the private data for the current CPU, 239which is safe against preemption and interrupts. 240.Pp 241On i386 architecture, when machine supports the cmpxchg8 instruction, 242this instruction is used. 243The multi-instruction sequence provides the same guarantees as the 244amd64 single-instruction implementation. 245.Pp 246On some architectures updating a counter require a 247.Xr critical 9 248section. 249.Sh EXAMPLES 250The following example creates a static counter array exported to 251userspace through a sysctl: 252.Bd -literal -offset indent 253#define MY_SIZE 8 254static counter_u64_t array[MY_SIZE]; 255SYSCTL_COUNTER_U64_ARRAY(_debug, OID_AUTO, counter_array, CTLFLAG_RW, 256 &array[0], MY_SIZE, "Test counter array"); 257.Ed 258.Sh SEE ALSO 259.Xr atomic 9 , 260.Xr critical 9 , 261.Xr locking 9 , 262.Xr malloc 9 , 263.Xr ratecheck 9 , 264.Xr sysctl 9 , 265.Xr SYSINIT 9 , 266.Xr uma 9 267.Sh HISTORY 268The 269.Nm 270facility first appeared in 271.Fx 10.0 . 272.Sh AUTHORS 273.An -nosplit 274The 275.Nm 276facility was written by 277.An Gleb Smirnoff 278and 279.An Konstantin Belousov . 280