1.\" Copyright (c) 2003-2008 Joseph Koshy 2.\" Copyright (c) 2007,2023 The FreeBSD Foundation 3.\" 4.\" Portions of this software were developed by A. Joseph Koshy under 5.\" sponsorship from the FreeBSD Foundation and Google, Inc. 6.\" 7.\" Portions of this documentation were written by Mitchell Horne 8.\" under sponsorship from the FreeBSD Foundation. 9.\" 10.\" Redistribution and use in source and binary forms, with or without 11.\" modification, are permitted provided that the following conditions 12.\" are met: 13.\" 1. Redistributions of source code must retain the above copyright 14.\" notice, this list of conditions and the following disclaimer. 15.\" 2. Redistributions in binary form must reproduce the above copyright 16.\" notice, this list of conditions and the following disclaimer in the 17.\" documentation and/or other materials provided with the distribution. 18.\" 19.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 20.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 21.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 22.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 23.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 24.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 25.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 26.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 27.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 28.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 29.\" SUCH DAMAGE. 30.\" 31.\" $FreeBSD$ 32.\" 33.Dd July 8, 2023 34.Dt HWPMC 4 35.Os 36.Sh NAME 37.Nm hwpmc 38.Nd "Hardware Performance Monitoring Counter support" 39.Sh SYNOPSIS 40The following option must be present in the kernel configuration file: 41.Bd -ragged -offset indent 42.Cd "options HWPMC_HOOKS" 43.Ed 44.Pp 45Additionally, for i386 systems: 46.Bd -ragged -offset indent 47.Cd "device apic" 48.Ed 49.Pp 50To load the driver as a module at boot time: 51.Bd -literal -offset indent 52sysrc kld_list+=hwpmc 53.Ed 54.Pp 55Alternatively, to compile the driver into the kernel: 56.Bd -ragged -offset indent 57.Cd "device hwpmc" 58.Ed 59.Pp 60To enable debugging features 61.Po see 62.Sx DEBUGGING 63.Pc : 64.Bd -ragged -offset indent 65.Cd "options KTR" 66.Cd "options KTR_COMPILE=(KTR_SUBSYS)" 67.Cd "options KTR_MASK=(KTR_SUBSYS)" 68.Cd "options HWPMC_DEBUG" 69.Ed 70.Sh DESCRIPTION 71The 72.Nm 73driver virtualizes the hardware performance monitoring facilities in 74modern CPUs and provides support for using these facilities from 75user level processes. 76.Pp 77The driver supports multi-processor systems. 78.Pp 79PMCs are allocated using the 80.Dv PMC_OP_PMCALLOCATE 81request. 82A successful 83.Dv PMC_OP_PMCALLOCATE 84request will return a handle to the requesting process. 85Subsequent operations on the allocated PMC use this handle to denote 86the specific PMC. 87A process that has successfully allocated a PMC is termed an 88.Dq "owner process" . 89.Pp 90PMCs may be allocated with process or system scope. 91.Bl -tag -width ".Em Process-scope" 92.It Em "Process-scope" 93The PMC is active only when a thread belonging 94to a process it is attached to is scheduled on a CPU. 95.It Em "System-scope" 96The PMC operates independently of processes and 97measures hardware events for the system as a whole. 98.El 99.Pp 100PMCs may be allocated for counting or for sampling: 101.Bl -tag -width ".Em Counting" 102.It Em Counting 103In counting modes, the PMCs count hardware events. 104These counts are retrievable using the 105.Dv PMC_OP_PMCREAD 106system call on all architectures. 107Some architectures offer faster methods of reading these counts. 108.It Em Sampling 109In sampling modes, the PMCs are configured to sample the CPU 110instruction pointer (and optionally to capture the call chain leading 111up to the sampled instruction pointer) after a configurable number of 112hardware events have been observed. 113Instruction pointer samples and call chain records are usually 114directed to a log file for subsequent analysis. 115.El 116.Pp 117Scope and operational mode are orthogonal; a PMC may thus be 118configured to operate in one of the following four modes: 119.Bl -tag -width indent 120.It Process-scope, counting 121These PMCs count hardware events whenever a thread in their attached process is 122scheduled on a CPU. 123These PMCs normally count from zero, but the initial count may be 124set using the 125.Dv PMC_OP_SETCOUNT 126operation. 127Applications can read the value of the PMC anytime using the 128.Dv PMC_OP_PMCRW 129operation. 130.It Process-scope, sampling 131These PMCs sample the target processes instruction pointer after they 132have seen the configured number of hardware events. 133The PMCs only count events when a thread belonging to their attached 134process is active. 135The desired frequency of sampling is set using the 136.Dv PMC_OP_SETCOUNT 137operation prior to starting the PMC. 138Log files are configured using the 139.Dv PMC_OP_CONFIGURELOG 140operation. 141.It System-scope, counting 142These PMCs count hardware events seen by them independent of the 143processes that are executing. 144The current count on these PMCs can be read using the 145.Dv PMC_OP_PMCRW 146request. 147These PMCs normally count from zero, but the initial count may be 148set using the 149.Dv PMC_OP_SETCOUNT 150operation. 151.It System-scope, sampling 152These PMCs will periodically sample the instruction pointer of the CPU 153they are allocated on, and will write the sample to a log for further 154processing. 155The desired frequency of sampling is set using the 156.Dv PMC_OP_SETCOUNT 157operation prior to starting the PMC. 158Log files are configured using the 159.Dv PMC_OP_CONFIGURELOG 160operation. 161.Pp 162System-wide statistical sampling can only be enabled by a process with 163super-user privileges. 164.El 165.Pp 166Processes are allowed to allocate as many PMCs as the hardware and 167current operating conditions permit. 168Processes may mix allocations of system-wide and process-private 169PMCs. 170Multiple processes may be using PMCs simultaneously. 171.Pp 172Allocated PMCs are started using the 173.Dv PMC_OP_PMCSTART 174operation, and stopped using the 175.Dv PMC_OP_PMCSTOP 176operation. 177Stopping and starting a PMC is permitted at any time the owner process 178has a valid handle to the PMC. 179.Pp 180Process-private PMCs need to be attached to a target process before 181they can be used. 182Attaching a process to a PMC is done using the 183.Dv PMC_OP_PMCATTACH 184operation. 185An already attached PMC may be detached from its target process 186using the converse 187.Dv PMC_OP_PMCDETACH 188operation. 189Issuing a 190.Dv PMC_OP_PMCSTART 191operation on an as yet unattached PMC will cause it to be attached 192to its owner process. 193The following rules determine whether a given process may attach 194a PMC to another target process: 195.Bl -bullet -compact 196.It 197A non-jailed process with super-user privileges is allowed to attach 198to any other process in the system. 199.It 200Other processes are only allowed to attach to targets that they would 201be able to attach to for debugging (as determined by 202.Xr p_candebug 9 ) . 203.El 204.Pp 205PMCs are released using 206.Dv PMC_OP_PMCRELEASE . 207After a successful 208.Dv PMC_OP_PMCRELEASE 209operation the handle to the PMC will become invalid. 210.Ss Modifier Flags 211The 212.Dv PMC_OP_PMCALLOCATE 213operation supports the following flags that modify the behavior 214of an allocated PMC: 215.Bl -tag -width indent 216.It Dv PMC_F_CALLCHAIN 217This modifier informs sampling PMCs to record a callchain when 218capturing a sample. 219The maximum depth to which call chains are recorded is specified 220by the 221.Va "kern.hwpmc.callchaindepth" 222kernel tunable. 223.It Dv PMC_F_DESCENDANTS 224This modifier is valid only for a PMC being allocated in process-private 225mode. 226It signifies that the PMC will track hardware events for its 227target process and the target's current and future descendants. 228.It Dv PMC_F_LOG_PROCCSW 229This modifier is valid only for a PMC being allocated in process-private 230mode. 231When this modifier is present, at every context switch, 232.Nm 233will log a record containing the number of hardware events 234seen by the target process when it was scheduled on the CPU. 235.It Dv PMC_F_LOG_PROCEXIT 236This modifier is valid only for a PMC being allocated in process-private 237mode. 238With this modifier present, 239.Nm 240will maintain per-process counts for each target process attached to 241a PMC. 242At process exit time, a record containing the target process' PID and 243the accumulated per-process count for that process will be written to the 244configured log file. 245.El 246.Pp 247Modifiers 248.Dv PMC_F_LOG_PROCEXIT 249and 250.Dv PMC_F_LOG_PROCCSW 251may be used in combination with modifier 252.Dv PMC_F_DESCENDANTS 253to track the behavior of complex pipelines of processes. 254PMCs with modifiers 255.Dv PMC_F_LOG_PROCEXIT 256and 257.Dv PMC_F_LOG_PROCCSW 258cannot be started until their owner process has configured a log file. 259.Ss Signals 260The 261.Nm 262driver may deliver signals to processes that have allocated PMCs: 263.Bl -tag -width ".Dv SIGBUS" 264.It Dv SIGIO 265A 266.Dv PMC_OP_PMCRW 267operation was attempted on a process-private PMC that does not have 268attached target processes. 269.It Dv SIGBUS 270The 271.Nm 272driver is being unloaded from the kernel. 273.El 274.Ss PMC ROW DISPOSITIONS 275A PMC row is defined as the set of PMC resources at the same hardware 276address in the CPUs in a system. 277Since process scope PMCs need to move between CPUs following their 278target threads, allocation of a process scope PMC reserves all PMCs in 279a PMC row for use only with process scope PMCs. 280Accordingly a PMC row will be in one of the following dispositions: 281.Bl -tag -width ".Dv PMC_DISP_STANDALONE" -compact 282.It Dv PMC_DISP_FREE 283Hardware counters in this row are free and may be use to satisfy 284either of system scope or process scope allocation requests. 285.It Dv PMC_DISP_THREAD 286Hardware counters in this row are in use by process scope PMCs 287and are only available for process scope allocation requests. 288.It Dv PMC_DISP_STANDALONE 289Some hardware counters in this row have been administratively 290disabled or are in use by system scope PMCs. 291Non-disabled hardware counters in such a row may be used 292for satisfying system scope allocation requests. 293No process scope PMCs will use hardware counters in this row. 294.El 295.Sh COMPATIBILITY 296The API and ABI documented in this manual page may change in the future. 297This interface is intended to be consumed by the 298.Xr pmc 3 299library; other consumers are unsupported. 300Applications targeting PMCs should use the 301.Xr pmc 3 302library API. 303.Sh PROGRAMMING API 304The 305.Nm 306driver operates using a system call number that is dynamically 307allotted to it when it is loaded into the kernel. 308.Pp 309The 310.Nm 311driver supports the following operations: 312.Bl -tag -width indent 313.It Dv PMC_OP_CONFIGURELOG 314Configure a log file for PMCs that require a log file. 315The 316.Nm 317driver will write log data to this file asynchronously. 318If it encounters an error, logging will be stopped and the error code 319encountered will be saved for subsequent retrieval by a 320.Dv PMC_OP_FLUSHLOG 321request. 322.It Dv PMC_OP_FLUSHLOG 323Transfer buffered log data inside 324.Nm 325to a configured output file. 326This operation returns to the caller after the write operation 327has returned. 328The returned error code reflects any pending error state inside 329.Nm . 330.It Dv PMC_OP_GETCPUINFO 331Retrieve information about the highest possible CPU number for the system, 332and the number of hardware performance monitoring counters available per CPU. 333.It Dv PMC_OP_GETDRIVERSTATS 334Retrieve module statistics (for analyzing the behavior of 335.Nm 336itself). 337.It Dv PMC_OP_GETMODULEVERSION 338Retrieve the version number of API. 339.It Dv PMC_OP_GETPMCINFO 340Retrieve information about the current state of the PMCs on a 341given CPU. 342.It Dv PMC_OP_PMCADMIN 343Set the administrative state (i.e., whether enabled or disabled) for 344the hardware PMCs managed by the 345.Nm 346driver. 347The invoking process needs to possess the 348.Dv PRIV_PMC_MANAGE 349privilege. 350.It Dv PMC_OP_PMCALLOCATE 351Allocate and configure a PMC. 352On successful allocation, a handle to the PMC (a 32 bit value) 353is returned. 354.It Dv PMC_OP_PMCATTACH 355Attach a process mode PMC to a target process. 356The PMC will be active whenever a thread in the target process is 357scheduled on a CPU. 358.Pp 359If the 360.Dv PMC_F_DESCENDANTS 361flag had been specified at PMC allocation time, then the PMC is 362attached to all current and future descendants of the target process. 363.It Dv PMC_OP_PMCDETACH 364Detach a PMC from its target process. 365.It Dv PMC_OP_PMCRELEASE 366Release a PMC. 367.It Dv PMC_OP_PMCRW 368Read and write a PMC. 369This operation is valid only for PMCs configured in counting modes. 370.It Dv PMC_OP_SETCOUNT 371Set the initial count (for counting mode PMCs) or the desired sampling 372rate (for sampling mode PMCs). 373.It Dv PMC_OP_PMCSTART 374Start a PMC. 375.It Dv PMC_OP_PMCSTOP 376Stop a PMC. 377.It Dv PMC_OP_WRITELOG 378Insert a timestamped user record into the log file. 379.El 380.Ss i386 Specific API 381Some i386 family CPUs support the RDPMC instruction which allows a 382user process to read a PMC value without needing to invoke a 383.Dv PMC_OP_PMCRW 384operation. 385On such CPUs, the machine address associated with an allocated PMC is 386retrievable using the 387.Dv PMC_OP_PMCX86GETMSR 388system call. 389.Bl -tag -width indent 390.It Dv PMC_OP_PMCX86GETMSR 391Retrieve the MSR (machine specific register) number associated with 392the given PMC handle. 393.Pp 394The PMC needs to be in process-private mode and allocated without the 395.Dv PMC_F_DESCENDANTS 396modifier flag, and should be attached only to its owner process at the 397time of the call. 398.El 399.Ss amd64 Specific API 400AMD64 CPUs support the RDPMC instruction which allows a 401user process to read a PMC value without needing to invoke a 402.Dv PMC_OP_PMCRW 403operation. 404The machine address associated with an allocated PMC is 405retrievable using the 406.Dv PMC_OP_PMCX86GETMSR 407system call. 408.Bl -tag -width indent 409.It Dv PMC_OP_PMCX86GETMSR 410Retrieve the MSR (machine specific register) number associated with 411the given PMC handle. 412.Pp 413The PMC needs to be in process-private mode and allocated without the 414.Dv PMC_F_DESCENDANTS 415modifier flag, and should be attached only to its owner process at the 416time of the call. 417.El 418.Sh SYSCTL VARIABLES AND LOADER TUNABLES 419The behavior of 420.Nm 421is influenced by the following 422.Xr sysctl 8 423and 424.Xr loader 8 425tunables: 426.Bl -tag -width indent 427.It Va kern.hwpmc.callchaindepth Pq integer, read-only 428The maximum number of call chain records to capture per sample. 429The default is 8. 430.It Va kern.hwpmc.debugflags Pq string, read-write 431(Only available if the 432.Nm 433driver was compiled with 434.Fl DDEBUG . ) 435Control the verbosity of debug messages from the 436.Nm 437driver. 438.It Va kern.hwpmc.hashsize Pq integer, read-only 439The number of rows in the hash tables used to keep track of owner and 440target processes. 441The default is 16. 442.It Va kern.hwpmc.logbuffersize Pq integer, read-only 443The size in kilobytes of each log buffer used by 444.Nm Ns 's 445logging function. 446The default buffer size is 4KB. 447.It Va kern.hwpmc.mincount Pq integer, read-write 448The minimum sampling rate for sampling mode PMCs. 449The default count is 1000 events. 450.It Va kern.hwpmc.mtxpoolsize Pq integer, read-only 451The size of the spin mutex pool used by the PMC driver. 452The default is 32. 453.It Va kern.hwpmc.nbuffers_pcpu Pq integer, read-only 454The number of log buffers used by 455.Nm 456for logging. 457The default is 64. 458.It Va kern.hwpmc.nsamples Pq integer, read-only 459The number of entries in the per-CPU ring buffer used during sampling. 460The default is 512. 461.It Va security.bsd.unprivileged_syspmcs Pq boolean, read-write 462If set to non-zero, allow unprivileged processes to allocate system-wide 463PMCs. 464The default value is 0. 465.It Va security.bsd.unprivileged_proc_debug Pq boolean, read-write 466If set to 0, the 467.Nm 468driver will only allow privileged processes to attach PMCs to other 469processes. 470.El 471.Pp 472These variables may be set in the kernel environment using 473.Xr kenv 1 474before 475.Nm 476is loaded. 477.Sh IMPLEMENTATION NOTES 478.Ss SMP Symmetry 479The kernel driver requires all physical CPUs in an SMP system to have 480identical performance monitoring counter hardware. 481.Ss Sparse CPU Numbering 482On platforms that sparsely number CPUs and which support hot-plugging 483of CPUs, requests that specify non-existent or disabled CPUs will fail 484with an error. 485Applications allocating system-scope PMCs need to be aware of 486the possibility of such transient failures. 487.Ss x86 TSC Handling 488Historically, on the x86 architecture, 489.Fx 490has permitted user processes running at a processor CPL of 3 to 491read the TSC using the RDTSC instruction. 492The 493.Nm 494driver preserves this behavior. 495.Ss Intel P4/HTT Handling 496On CPUs with HTT support, Intel P4 PMCs are capable of qualifying 497only a subset of hardware events on a per-logical CPU basis. 498Consequently, if HTT is enabled on a system with Intel Pentium P4 499PMCs, then the 500.Nm 501driver will reject allocation requests for process-private PMCs that 502request counting of hardware events that cannot be counted separately 503for each logical CPU. 504.Sh DIAGNOSTICS 505.Bl -diag 506.It "hwpmc: [class/npmc/capabilities]..." 507Announce the presence of 508.Va npmc 509PMCs of class 510.Va class , 511with capabilities described by bit string 512.Va capabilities . 513.It "hwpmc: kernel version (0x%x) does not match module version (0x%x)." 514The module loading process failed because a version mismatch was detected 515between the currently executing kernel and the module being loaded. 516.It "hwpmc: this kernel has not been compiled with 'options HWPMC_HOOKS'." 517The module loading process failed because the currently executing kernel 518was not configured with the required configuration option 519.Dv HWPMC_HOOKS . 520.It "hwpmc: tunable hashsize=%d must be greater than zero." 521A negative value was supplied for tunable 522.Va kern.hwpmc.hashsize . 523.It "hwpmc: tunable logbuffersize=%d must be greater than zero." 524A negative value was supplied for tunable 525.Va kern.hwpmc.logbuffersize . 526.It "hwpmc: tunable nlogbuffers=%d must be greater than zero." 527A negative value was supplied for tunable 528.Va kern.hwpmc.nlogbuffers . 529.It "hwpmc: tunable nsamples=%d out of range." 530The value for tunable 531.Va kern.hwpmc.nsamples 532was negative or greater than 65535. 533.El 534.Sh DEBUGGING 535The 536.Nm 537module can be configured to record trace entries using the 538.Xr ktr 4 539interface. 540This is useful for debugging the driver's functionality, primarily during 541development. 542This debugging functionality is not enabled by default, and requires 543recompiling the kernel and 544.Nm 545module after adding the following to the kernel config: 546.Bd -literal -offset indent 547.Cd options KTR 548.Cd options KTR_COMPILE=(KTR_SUBSYS) 549.Cd options KTR_MASK=(KTR_SUBSYS) 550.Cd options HWPMC_DEBUG 551.Ed 552.Pp 553This alone is not enough to enable tracing; one must also configure the 554.Va kern.hwpmc.debugflags 555.Xr sysctl 8 556variable, which provides fine-grained control over which types of events are 557logged to the trace buffer. 558.Pp 559.Nm 560trace events are grouped by 'major' and 'minor' flag types. 561The major flag names are as follows: 562.Pp 563.Bl -tag -width "sampling" -compact -offset indent 564.It cpu 565CPU events 566.It csw 567Context switch events 568.It logging 569Logging events 570.It md 571Machine-dependent/class-dependent events 572.It module 573Miscellaneous events 574.It owner 575PMC owner events 576.It pmc 577PMC management events 578.It process 579Process events 580.It sampling 581Sampling events 582.El 583.Pp 584The minor flags for each major flag group can vary. 585The individual minor flag names are: 586.Bd -ragged -offset indent 587allocaterow, 588allocate, 589attach, 590bind, 591config, 592exec, 593exit, 594find, 595flush, 596fork, 597getbuf, 598hook, 599init, 600intr, 601linktarget, 602mayberemove, 603ops, 604read, 605register, 606release, 607remove, 608sample, 609scheduleio, 610select, 611signal, 612swi, 613swo, 614start, 615stop, 616syscall, 617unlinktarget, 618write 619.Ed 620.Pp 621The 622.Va kern.hwpmc.debugflags 623variable is a string with a custom format. 624The string should contain a space-separated list of event specifiers. 625Each event specifier consists of the major flag name, followed by an equal sign 626(=), followed by a comma-separated list of minor event types. 627To track all events for a major group, an asterisk (*) can be given instead of 628minor event names. 629.Pp 630For example, to trace all allocation and release events, set 631.Va debugflags 632as follows: 633.Bd -literal -offset indent 634kern.hwpmc.debugflags="pmc=allocate,release md=allocate,release" 635.Ed 636.Pp 637To trace all events in the process and context switch major flag groups: 638.Bd -literal -offset indent 639kern.hwpmc.debugflags="process=* csw=*" 640.Ed 641.Pp 642To disable all trace events, set the variable to an empty string. 643.Bd -literal -offset indent 644kern.hwpmc.debugflags="" 645.Ed 646.Pp 647Trace events are recorded by 648.Xr ktr 4 , 649and can be inspected at run-time using the 650.Xr ktrdump 8 651utility, or at the 652.Xr ddb 4 653prompt after a panic with the 'show ktr' command. 654.Sh ERRORS 655A command issued to the 656.Nm 657driver may fail with the following errors: 658.Bl -tag -width Er 659.It Bq Er EAGAIN 660Helper process creation failed for a 661.Dv PMC_OP_CONFIGURELOG 662request due to a temporary resource shortage in the kernel. 663.It Bq Er EBUSY 664A 665.Dv PMC_OP_CONFIGURELOG 666operation was requested while an existing log was active. 667.It Bq Er EBUSY 668A DISABLE operation was requested using the 669.Dv PMC_OP_PMCADMIN 670request for a set of hardware resources currently in use for 671process-private PMCs. 672.It Bq Er EBUSY 673A 674.Dv PMC_OP_PMCADMIN 675operation was requested on an active system mode PMC. 676.It Bq Er EBUSY 677A 678.Dv PMC_OP_PMCATTACH 679operation was requested for a target process that already had another 680PMC using the same hardware resources attached to it. 681.It Bq Er EBUSY 682A 683.Dv PMC_OP_PMCRW 684request writing a new value was issued on a PMC that was active. 685.It Bq Er EBUSY 686A 687.Dv PMC_OP_PMCSETCOUNT 688request was issued on a PMC that was active. 689.It Bq Er EDOOFUS 690A 691.Dv PMC_OP_PMCSTART 692operation was requested without a log file being configured for a 693PMC allocated with 694.Dv PMC_F_LOG_PROCCSW 695and 696.Dv PMC_F_LOG_PROCEXIT 697modifiers. 698.It Bq Er EDOOFUS 699A 700.Dv PMC_OP_PMCSTART 701operation was requested on a system-wide sampling PMC without a log 702file being configured. 703.It Bq Er EEXIST 704A 705.Dv PMC_OP_PMCATTACH 706request was reissued for a target process that already is the target 707of this PMC. 708.It Bq Er EFAULT 709A bad address was passed in to the driver. 710.It Bq Er EINVAL 711An invalid PMC handle was specified. 712.It Bq Er EINVAL 713An invalid CPU number was passed in for a 714.Dv PMC_OP_GETPMCINFO 715operation. 716.It Bq Er EINVAL 717The 718.Ar pm_flags 719argument to a 720.Dv PMC_OP_CONFIGURELOG 721request contained unknown flags. 722.It Bq Er EINVAL 723A 724.Dv PMC_OP_CONFIGURELOG 725request to de-configure a log file was issued without a log file 726being configured. 727.It Bq Er EINVAL 728A 729.Dv PMC_OP_FLUSHLOG 730request was issued without a log file being configured. 731.It Bq Er EINVAL 732An invalid CPU number was passed in for a 733.Dv PMC_OP_PMCADMIN 734operation. 735.It Bq Er EINVAL 736An invalid operation request was passed in for a 737.Dv PMC_OP_PMCADMIN 738operation. 739.It Bq Er EINVAL 740An invalid PMC ID was passed in for a 741.Dv PMC_OP_PMCADMIN 742operation. 743.It Bq Er EINVAL 744A suitable PMC matching the parameters passed in to a 745.Dv PMC_OP_PMCALLOCATE 746request could not be allocated. 747.It Bq Er EINVAL 748An invalid PMC mode was requested during a 749.Dv PMC_OP_PMCALLOCATE 750request. 751.It Bq Er EINVAL 752An invalid CPU number was specified during a 753.Dv PMC_OP_PMCALLOCATE 754request. 755.It Bq Er EINVAL 756A CPU other than 757.Dv PMC_CPU_ANY 758was specified in a 759.Dv PMC_OP_PMCALLOCATE 760request for a process-private PMC. 761.It Bq Er EINVAL 762A CPU number of 763.Dv PMC_CPU_ANY 764was specified in a 765.Dv PMC_OP_PMCALLOCATE 766request for a system-wide PMC. 767.It Bq Er EINVAL 768The 769.Ar pm_flags 770argument to an 771.Dv PMC_OP_PMCALLOCATE 772request contained unknown flags. 773.It Bq Er EINVAL 774(On Intel Pentium 4 CPUs with HTT support) 775A 776.Dv PMC_OP_PMCALLOCATE 777request for a process-private PMC was issued for an event that does 778not support counting on a per-logical CPU basis. 779.It Bq Er EINVAL 780A PMC allocated for system-wide operation was specified with a 781.Dv PMC_OP_PMCATTACH 782or 783.Dv PMC_OP_PMCDETACH 784request. 785.It Bq Er EINVAL 786The 787.Ar pm_pid 788argument to a 789.Dv PMC_OP_PMCATTACH 790or 791.Dv PMC_OP_PMCDETACH 792request specified an illegal process ID. 793.It Bq Er EINVAL 794A 795.Dv PMC_OP_PMCDETACH 796request was issued for a PMC not attached to the target process. 797.It Bq Er EINVAL 798Argument 799.Ar pm_flags 800to a 801.Dv PMC_OP_PMCRW 802request contained illegal flags. 803.It Bq Er EINVAL 804A 805.Dv PMC_OP_PMCX86GETMSR 806operation was requested for a PMC not in process-virtual mode, or 807for a PMC that is not solely attached to its owner process, or for 808a PMC that was allocated with flag 809.Dv PMC_F_DESCENDANTS . 810.It Bq Er EINVAL 811A 812.Dv PMC_OP_WRITELOG 813request was issued for an owner process without a log file 814configured. 815.It Bq Er ENOMEM 816The system was not able to allocate kernel memory. 817.It Bq Er ENOSYS 818(On i386 and amd64 architectures) 819A 820.Dv PMC_OP_PMCX86GETMSR 821operation was requested for hardware that does not support reading 822PMCs directly with the RDPMC instruction. 823.It Bq Er ENXIO 824A 825.Dv PMC_OP_GETPMCINFO 826operation was requested for an absent or disabled CPU. 827.It Bq Er ENXIO 828A 829.Dv PMC_OP_PMCALLOCATE 830operation specified allocation of a system-wide PMC on an absent or 831disabled CPU. 832.It Bq Er ENXIO 833A 834.Dv PMC_OP_PMCSTART 835or 836.Dv PMC_OP_PMCSTOP 837request was issued for a system-wide PMC that was allocated on a CPU 838that is currently absent or disabled. 839.It Bq Er EOPNOTSUPP 840A 841.Dv PMC_OP_PMCALLOCATE 842request was issued for PMC capabilities not supported 843by the specified PMC class. 844.It Bq Er EOPNOTSUPP 845(i386 architectures) 846A sampling mode PMC was requested on a CPU lacking an APIC. 847.It Bq Er EPERM 848A 849.Dv PMC_OP_PMCADMIN 850request was issued by a process without super-user 851privilege or by a jailed super-user process. 852.It Bq Er EPERM 853A 854.Dv PMC_OP_PMCATTACH 855operation was issued for a target process that the current process 856does not have permission to attach to. 857.It Bq Er EPERM 858(i386 and amd64 architectures) 859A 860.Dv PMC_OP_PMCATTACH 861operation was issued on a PMC whose MSR has been retrieved using 862.Dv PMC_OP_PMCX86GETMSR . 863.It Bq Er ESRCH 864A process issued a PMC operation request without having allocated any 865PMCs. 866.It Bq Er ESRCH 867A process issued a PMC operation request after the PMC was detached 868from all of its target processes. 869.It Bq Er ESRCH 870A 871.Dv PMC_OP_PMCATTACH 872or 873.Dv PMC_OP_PMCDETACH 874request specified a non-existent process ID. 875.It Bq Er ESRCH 876The target process for a 877.Dv PMC_OP_PMCDETACH 878operation is not being monitored by 879.Nm . 880.El 881.Sh SEE ALSO 882.Xr kenv 1 , 883.Xr pmc 3 , 884.Xr pmclog 3 , 885.Xr ddb 4 , 886.Xr ktr 4 , 887.Xr kldload 8 , 888.Xr ktrdump 8 , 889.Xr pmccontrol 8 , 890.Xr pmcstat 8 , 891.Xr sysctl 8 , 892.Xr kproc_create 9 , 893.Xr p_candebug 9 894.Sh HISTORY 895The 896.Nm 897driver first appeared in 898.Fx 6.0 . 899.Sh AUTHORS 900The 901.Nm 902driver was written by 903.An Joseph Koshy Aq Mt jkoshy@FreeBSD.org . 904.Sh BUGS 905The driver samples the state of the kernel's logical processor support 906at the time of initialization (i.e., at module load time). 907On CPUs supporting logical processors, the driver could misbehave if 908logical processors are subsequently enabled or disabled while the 909driver is active. 910.Pp 911On the i386 architecture, the driver requires that the local APIC on the 912CPU be enabled for sampling mode to be supported. 913Many single-processor motherboards keep the APIC disabled in BIOS; on 914such systems 915.Nm 916will not support sampling PMCs. 917.Sh SECURITY CONSIDERATIONS 918PMCs may be used to monitor the actual behavior of the system on hardware. 919In situations where this constitutes an undesirable information leak, 920the following options are available: 921.Bl -enum 922.It 923Set the 924.Xr sysctl 8 925tunable 926.Va security.bsd.unprivileged_syspmcs 927to 0. 928This ensures that unprivileged processes cannot allocate system-wide 929PMCs and thus cannot observe the hardware behavior of the system 930as a whole. 931This tunable may also be set at boot time using 932.Xr loader 8 , 933or with 934.Xr kenv 1 935prior to loading the 936.Nm 937driver into the kernel. 938.It 939Set the 940.Xr sysctl 8 941tunable 942.Va security.bsd.unprivileged_proc_debug 943to 0. 944This will ensure that an unprivileged process cannot attach a PMC 945to any process other than itself and thus cannot observe the hardware 946behavior of other processes with the same credentials. 947.El 948.Pp 949System administrators should note that on IA-32 platforms 950.Fx 951makes the content of the IA-32 TSC counter available to all processes 952via the RDTSC instruction. 953