1perf-stat(1) 2============ 3 4NAME 5---- 6perf-stat - Run a command and gather performance counter statistics 7 8SYNOPSIS 9-------- 10[verse] 11'perf stat' [-e <EVENT> | --event=EVENT] [-a] <command> 12'perf stat' [-e <EVENT> | --event=EVENT] [-a] \-- <command> [<options>] 13'perf stat' [-e <EVENT> | --event=EVENT] [-a] record [-o file] \-- <command> [<options>] 14'perf stat' report [-i file] 15 16DESCRIPTION 17----------- 18This command runs a command and gathers performance counter statistics 19from it. 20 21 22OPTIONS 23------- 24<command>...:: 25 Any command you can specify in a shell. 26 27record:: 28 See STAT RECORD. 29 30report:: 31 See STAT REPORT. 32 33-e:: 34--event=:: 35 Select the PMU event. Selection can be: 36 37 - a symbolic event name (use 'perf list' to list all events) 38 39 - a raw PMU event in the form of rN where N is a hexadecimal value 40 that represents the raw register encoding with the layout of the 41 event control registers as described by entries in 42 /sys/bus/event_source/devices/cpu/format/*. 43 44 - a symbolic or raw PMU event followed by an optional colon 45 and a list of event modifiers, e.g., cpu-cycles:p. See the 46 linkperf:perf-list[1] man page for details on event modifiers. 47 48 - a symbolically formed event like 'pmu/param1=0x3,param2/' where 49 param1 and param2 are defined as formats for the PMU in 50 /sys/bus/event_source/devices/<pmu>/format/* 51 52 'percore' is a event qualifier that sums up the event counts for both 53 hardware threads in a core. For example: 54 perf stat -A -a -e cpu/event,percore=1/,otherevent ... 55 56 - a symbolically formed event like 'pmu/config=M,config1=N,config2=K/' 57 where M, N, K are numbers (in decimal, hex, octal format). 58 Acceptable values for each of 'config', 'config1' and 'config2' 59 parameters are defined by corresponding entries in 60 /sys/bus/event_source/devices/<pmu>/format/* 61 62 Note that the last two syntaxes support prefix and glob matching in 63 the PMU name to simplify creation of events across multiple instances 64 of the same type of PMU in large systems (e.g. memory controller PMUs). 65 Multiple PMU instances are typical for uncore PMUs, so the prefix 66 'uncore_' is also ignored when performing this match. 67 68 69-i:: 70--no-inherit:: 71 child tasks do not inherit counters 72-p:: 73--pid=<pid>:: 74 stat events on existing process id (comma separated list) 75 76-t:: 77--tid=<tid>:: 78 stat events on existing thread id (comma separated list) 79 80-b:: 81--bpf-prog:: 82 stat events on existing bpf program id (comma separated list), 83 requiring root rights. bpftool-prog could be used to find program 84 id all bpf programs in the system. For example: 85 86 # bpftool prog | head -n 1 87 17247: tracepoint name sys_enter tag 192d548b9d754067 gpl 88 89 # perf stat -e cycles,instructions --bpf-prog 17247 --timeout 1000 90 91 Performance counter stats for 'BPF program(s) 17247': 92 93 85,967 cycles 94 28,982 instructions # 0.34 insn per cycle 95 96 1.102235068 seconds time elapsed 97 98--bpf-counters:: 99 Use BPF programs to aggregate readings from perf_events. This 100 allows multiple perf-stat sessions that are counting the same metric (cycles, 101 instructions, etc.) to share hardware counters. 102 To use BPF programs on common events by default, use 103 "perf config stat.bpf-counter-events=<list_of_events>". 104 105--bpf-attr-map:: 106 With option "--bpf-counters", different perf-stat sessions share 107 information about shared BPF programs and maps via a pinned hashmap. 108 Use "--bpf-attr-map" to specify the path of this pinned hashmap. 109 The default path is /sys/fs/bpf/perf_attr_map. 110 111ifdef::HAVE_LIBPFM[] 112--pfm-events events:: 113Select a PMU event using libpfm4 syntax (see http://perfmon2.sf.net) 114including support for event filters. For example '--pfm-events 115inst_retired:any_p:u:c=1:i'. More than one event can be passed to the 116option using the comma separator. Hardware events and generic hardware 117events cannot be mixed together. The latter must be used with the -e 118option. The -e option and this one can be mixed and matched. Events 119can be grouped using the {} notation. 120endif::HAVE_LIBPFM[] 121 122-a:: 123--all-cpus:: 124 system-wide collection from all CPUs (default if no target is specified) 125 126--no-scale:: 127 Don't scale/normalize counter values 128 129-d:: 130--detailed:: 131 print more detailed statistics, can be specified up to 3 times 132 133 -d: detailed events, L1 and LLC data cache 134 -d -d: more detailed events, dTLB and iTLB events 135 -d -d -d: very detailed events, adding prefetch events 136 137-r:: 138--repeat=<n>:: 139 repeat command and print average + stddev (max: 100). 0 means forever. 140 141-B:: 142--big-num:: 143 print large numbers with thousands' separators according to locale. 144 Enabled by default. Use "--no-big-num" to disable. 145 Default setting can be changed with "perf config stat.big-num=false". 146 147-C:: 148--cpu=:: 149Count only on the list of CPUs provided. Multiple CPUs can be provided as a 150comma-separated list with no space: 0,1. Ranges of CPUs are specified with -: 0-2. 151In per-thread mode, this option is ignored. The -a option is still necessary 152to activate system-wide monitoring. Default is to count on all CPUs. 153 154-A:: 155--no-aggr:: 156Do not aggregate counts across all monitored CPUs. 157 158-n:: 159--null:: 160null run - Don't start any counters. 161 162This can be useful to measure just elapsed wall-clock time - or to assess the 163raw overhead of perf stat itself, without running any counters. 164 165-v:: 166--verbose:: 167 be more verbose (show counter open errors, etc) 168 169-x SEP:: 170--field-separator SEP:: 171print counts using a CSV-style output to make it easy to import directly into 172spreadsheets. Columns are separated by the string specified in SEP. 173 174--table:: Display time for each run (-r option), in a table format, e.g.: 175 176 $ perf stat --null -r 5 --table perf bench sched pipe 177 178 Performance counter stats for 'perf bench sched pipe' (5 runs): 179 180 # Table of individual measurements: 181 5.189 (-0.293) # 182 5.189 (-0.294) # 183 5.186 (-0.296) # 184 5.663 (+0.181) ## 185 6.186 (+0.703) #### 186 187 # Final result: 188 5.483 +- 0.198 seconds time elapsed ( +- 3.62% ) 189 190-G name:: 191--cgroup name:: 192monitor only in the container (cgroup) called "name". This option is available only 193in per-cpu mode. The cgroup filesystem must be mounted. All threads belonging to 194container "name" are monitored when they run on the monitored CPUs. Multiple cgroups 195can be provided. Each cgroup is applied to the corresponding event, i.e., first cgroup 196to first event, second cgroup to second event and so on. It is possible to provide 197an empty cgroup (monitor all the time) using, e.g., -G foo,,bar. Cgroups must have 198corresponding events, i.e., they always refer to events defined earlier on the command 199line. If the user wants to track multiple events for a specific cgroup, the user can 200use '-e e1 -e e2 -G foo,foo' or just use '-e e1 -e e2 -G foo'. 201 202If wanting to monitor, say, 'cycles' for a cgroup and also for system wide, this 203command line can be used: 'perf stat -e cycles -G cgroup_name -a -e cycles'. 204 205--for-each-cgroup name:: 206Expand event list for each cgroup in "name" (allow multiple cgroups separated 207by comma). It also support regex patterns to match multiple groups. This has same 208effect that repeating -e option and -G option for each event x name. This option 209cannot be used with -G/--cgroup option. 210 211-o file:: 212--output file:: 213Print the output into the designated file. 214 215--append:: 216Append to the output file designated with the -o option. Ignored if -o is not specified. 217 218--log-fd:: 219 220Log output to fd, instead of stderr. Complementary to --output, and mutually exclusive 221with it. --append may be used here. Examples: 222 3>results perf stat --log-fd 3 \-- $cmd 223 3>>results perf stat --log-fd 3 --append \-- $cmd 224 225--control=fifo:ctl-fifo[,ack-fifo]:: 226--control=fd:ctl-fd[,ack-fd]:: 227ctl-fifo / ack-fifo are opened and used as ctl-fd / ack-fd as follows. 228Listen on ctl-fd descriptor for command to control measurement ('enable': enable events, 229'disable': disable events). Measurements can be started with events disabled using 230--delay=-1 option. Optionally send control command completion ('ack\n') to ack-fd descriptor 231to synchronize with the controlling process. Example of bash shell script to enable and 232disable events during measurements: 233 234 #!/bin/bash 235 236 ctl_dir=/tmp/ 237 238 ctl_fifo=${ctl_dir}perf_ctl.fifo 239 test -p ${ctl_fifo} && unlink ${ctl_fifo} 240 mkfifo ${ctl_fifo} 241 exec {ctl_fd}<>${ctl_fifo} 242 243 ctl_ack_fifo=${ctl_dir}perf_ctl_ack.fifo 244 test -p ${ctl_ack_fifo} && unlink ${ctl_ack_fifo} 245 mkfifo ${ctl_ack_fifo} 246 exec {ctl_fd_ack}<>${ctl_ack_fifo} 247 248 perf stat -D -1 -e cpu-cycles -a -I 1000 \ 249 --control fd:${ctl_fd},${ctl_fd_ack} \ 250 \-- sleep 30 & 251 perf_pid=$! 252 253 sleep 5 && echo 'enable' >&${ctl_fd} && read -u ${ctl_fd_ack} e1 && echo "enabled(${e1})" 254 sleep 10 && echo 'disable' >&${ctl_fd} && read -u ${ctl_fd_ack} d1 && echo "disabled(${d1})" 255 256 exec {ctl_fd_ack}>&- 257 unlink ${ctl_ack_fifo} 258 259 exec {ctl_fd}>&- 260 unlink ${ctl_fifo} 261 262 wait -n ${perf_pid} 263 exit $? 264 265 266--pre:: 267--post:: 268 Pre and post measurement hooks, e.g.: 269 270perf stat --repeat 10 --null --sync --pre 'make -s O=defconfig-build/clean' \-- make -s -j64 O=defconfig-build/ bzImage 271 272-I msecs:: 273--interval-print msecs:: 274Print count deltas every N milliseconds (minimum: 1ms) 275The overhead percentage could be high in some cases, for instance with small, sub 100ms intervals. Use with caution. 276 example: 'perf stat -I 1000 -e cycles -a sleep 5' 277 278If the metric exists, it is calculated by the counts generated in this interval and the metric is printed after #. 279 280--interval-count times:: 281Print count deltas for fixed number of times. 282This option should be used together with "-I" option. 283 example: 'perf stat -I 1000 --interval-count 2 -e cycles -a' 284 285--interval-clear:: 286Clear the screen before next interval. 287 288--timeout msecs:: 289Stop the 'perf stat' session and print count deltas after N milliseconds (minimum: 10 ms). 290This option is not supported with the "-I" option. 291 example: 'perf stat --time 2000 -e cycles -a' 292 293--metric-only:: 294Only print computed metrics. Print them in a single line. 295Don't show any raw values. Not supported with --per-thread. 296 297--per-socket:: 298Aggregate counts per processor socket for system-wide mode measurements. This 299is a useful mode to detect imbalance between sockets. To enable this mode, 300use --per-socket in addition to -a. (system-wide). The output includes the 301socket number and the number of online processors on that socket. This is 302useful to gauge the amount of aggregation. 303 304--per-die:: 305Aggregate counts per processor die for system-wide mode measurements. This 306is a useful mode to detect imbalance between dies. To enable this mode, 307use --per-die in addition to -a. (system-wide). The output includes the 308die number and the number of online processors on that die. This is 309useful to gauge the amount of aggregation. 310 311--per-cluster:: 312Aggregate counts per processor cluster for system-wide mode measurement. This 313is a useful mode to detect imbalance between clusters. To enable this mode, 314use --per-cluster in addition to -a. (system-wide). The output includes the 315cluster number and the number of online processors on that cluster. This is 316useful to gauge the amount of aggregation. The information of cluster ID and 317related CPUs can be gotten from /sys/devices/system/cpu/cpuX/topology/cluster_{id, cpus}. 318 319--per-cache:: 320Aggregate counts per cache instance for system-wide mode measurements. By 321default, the aggregation happens for the cache level at the highest index 322in the system. To specify a particular level, mention the cache level 323alongside the option in the format [Ll][1-9][0-9]*. For example: 324Using option "--per-cache=l3" or "--per-cache=L3" will aggregate the 325information at the boundary of the level 3 cache in the system. 326 327--per-core:: 328Aggregate counts per physical processor for system-wide mode measurements. This 329is a useful mode to detect imbalance between physical cores. To enable this mode, 330use --per-core in addition to -a. (system-wide). The output includes the 331core number and the number of online logical processors on that physical processor. 332 333--per-thread:: 334Aggregate counts per monitored threads, when monitoring threads (-t option) 335or processes (-p option). 336 337--per-node:: 338Aggregate counts per NUMA nodes for system-wide mode measurements. This 339is a useful mode to detect imbalance between NUMA nodes. To enable this 340mode, use --per-node in addition to -a. (system-wide). 341 342-D msecs:: 343--delay msecs:: 344After starting the program, wait msecs before measuring (-1: start with events 345disabled). This is useful to filter out the startup phase of the program, 346which is often very different. 347 348-T:: 349--transaction:: 350 351Print statistics of transactional execution if supported. 352 353--metric-no-group:: 354By default, events to compute a metric are placed in weak groups. The 355group tries to enforce scheduling all or none of the events. The 356--metric-no-group option places events outside of groups and may 357increase the chance of the event being scheduled - leading to more 358accuracy. However, as events may not be scheduled together accuracy 359for metrics like instructions per cycle can be lower - as both metrics 360may no longer be being measured at the same time. 361 362--metric-no-merge:: 363By default metric events in different weak groups can be shared if one 364group contains all the events needed by another. In such cases one 365group will be eliminated reducing event multiplexing and making it so 366that certain groups of metrics sum to 100%. A downside to sharing a 367group is that the group may require multiplexing and so accuracy for a 368small group that need not have multiplexing is lowered. This option 369forbids the event merging logic from sharing events between groups and 370may be used to increase accuracy in this case. 371 372--metric-no-threshold:: 373Metric thresholds may increase the number of events necessary to 374compute whether a metric has exceeded its threshold expression. This 375may not be desirable, for example, as the events can introduce 376multiplexing. This option disables the adding of threshold expression 377events for a metric. However, if there are sufficient events to 378compute the threshold then the threshold is still computed and used to 379color the metric's computed value. 380 381--quiet:: 382Don't print output, warnings or messages. This is useful with perf stat 383record below to only write data to the perf.data file. 384 385STAT RECORD 386----------- 387Stores stat data into perf data file. 388 389-o file:: 390--output file:: 391Output file name. 392 393STAT REPORT 394----------- 395Reads and reports stat data from perf data file. 396 397-i file:: 398--input file:: 399Input file name. 400 401--per-socket:: 402Aggregate counts per processor socket for system-wide mode measurements. 403 404--per-die:: 405Aggregate counts per processor die for system-wide mode measurements. 406 407--per-cluster:: 408Aggregate counts perf processor cluster for system-wide mode measurements. 409 410--per-cache:: 411Aggregate counts per cache instance for system-wide mode measurements. By 412default, the aggregation happens for the cache level at the highest index 413in the system. To specify a particular level, mention the cache level 414alongside the option in the format [Ll][1-9][0-9]*. For example: Using 415option "--per-cache=l3" or "--per-cache=L3" will aggregate the 416information at the boundary of the level 3 cache in the system. 417 418--per-core:: 419Aggregate counts per physical processor for system-wide mode measurements. 420 421-M:: 422--metrics:: 423Print metrics or metricgroups specified in a comma separated list. 424For a group all metrics from the group are added. 425The events from the metrics are automatically measured. 426See perf list output for the possible metrics and metricgroups. 427 428 When threshold information is available for a metric, the 429 color red is used to signify a metric has exceeded a threshold 430 while green shows it hasn't. The default color means that 431 no threshold information was available or the threshold 432 couldn't be computed. 433 434-A:: 435--no-aggr:: 436--no-merge:: 437Do not aggregate/merge counts across monitored CPUs or PMUs. 438 439When multiple events are created from a single event specification, 440stat will, by default, aggregate the event counts and show the result 441in a single row. This option disables that behavior and shows the 442individual events and counts. 443 444Multiple events are created from a single event specification when: 445 4461. PID monitoring isn't requested and the system has more than one 447 CPU. For example, a system with 8 SMT threads will have one event 448 opened on each thread and aggregation is performed across them. 449 4502. Prefix or glob wildcard matching is used for the PMU name. For 451 example, multiple memory controller PMUs may exist typically with a 452 suffix of _0, _1, etc. By default the event counts will all be 453 combined if the PMU is specified without the suffix such as 454 uncore_imc rather than uncore_imc_0. 455 4563. Aliases, which are listed immediately after the Kernel PMU events 457 by perf list, are used. 458 459--hybrid-merge:: 460Merge core event counts from all core PMUs. In hybrid or big.LITTLE 461systems by default each core PMU will report its count 462separately. This option forces core PMU counts to be combined to give 463a behavior closer to having a single CPU type in the system. 464 465--topdown:: 466Print top-down metrics supported by the CPU. This allows to determine 467bottle necks in the CPU pipeline for CPU bound workloads, by breaking 468the cycles consumed down into frontend bound, backend bound, bad 469speculation and retiring. 470 471Frontend bound means that the CPU cannot fetch and decode instructions fast 472enough. Backend bound means that computation or memory access is the bottle 473neck. Bad Speculation means that the CPU wasted cycles due to branch 474mispredictions and similar issues. Retiring means that the CPU computed without 475an apparently bottleneck. The bottleneck is only the real bottleneck 476if the workload is actually bound by the CPU and not by something else. 477 478For best results it is usually a good idea to use it with interval 479mode like -I 1000, as the bottleneck of workloads can change often. 480 481This enables --metric-only, unless overridden with --no-metric-only. 482 483The following restrictions only apply to older Intel CPUs and Atom, 484on newer CPUs (IceLake and later) TopDown can be collected for any thread: 485 486The top down metrics are collected per core instead of per 487CPU thread. Per core mode is automatically enabled 488and -a (global monitoring) is needed, requiring root rights or 489perf.perf_event_paranoid=-1. 490 491Topdown uses the full Performance Monitoring Unit, and needs 492disabling of the NMI watchdog (as root): 493echo 0 > /proc/sys/kernel/nmi_watchdog 494for best results. Otherwise the bottlenecks may be inconsistent 495on workload with changing phases. 496 497To interpret the results it is usually needed to know on which 498CPUs the workload runs on. If needed the CPUs can be forced using 499taskset. 500 501--td-level:: 502Print the top-down statistics that equal the input level. It allows 503users to print the interested top-down metrics level instead of the 504level 1 top-down metrics. 505 506As the higher levels gather more metrics and use more counters they 507will be less accurate. By convention a metric can be examined by 508appending '_group' to it and this will increase accuracy compared to 509gathering all metrics for a level. For example, level 1 analysis may 510highlight 'tma_frontend_bound'. This metric may be drilled into with 511'tma_frontend_bound_group' with 512'perf stat -M tma_frontend_bound_group...'. 513 514Error out if the input is higher than the supported max level. 515 516--smi-cost:: 517Measure SMI cost if msr/aperf/ and msr/smi/ events are supported. 518 519During the measurement, the /sys/device/cpu/freeze_on_smi will be set to 520freeze core counters on SMI. 521The aperf counter will not be effected by the setting. 522The cost of SMI can be measured by (aperf - unhalted core cycles). 523 524In practice, the percentages of SMI cycles is very useful for performance 525oriented analysis. --metric_only will be applied by default. 526The output is SMI cycles%, equals to (aperf - unhalted core cycles) / aperf 527 528Users who wants to get the actual value can apply --no-metric-only. 529 530--all-kernel:: 531Configure all used events to run in kernel space. 532 533--all-user:: 534Configure all used events to run in user space. 535 536--percore-show-thread:: 537The event modifier "percore" has supported to sum up the event counts 538for all hardware threads in a core and show the counts per core. 539 540This option with event modifier "percore" enabled also sums up the event 541counts for all hardware threads in a core but show the sum counts per 542hardware thread. This is essentially a replacement for the any bit and 543convenient for post processing. 544 545--summary:: 546Print summary for interval mode (-I). 547 548--no-csv-summary:: 549Don't print 'summary' at the first column for CVS summary output. 550This option must be used with -x and --summary. 551 552This option can be enabled in perf config by setting the variable 553'stat.no-csv-summary'. 554 555$ perf config stat.no-csv-summary=true 556 557--cputype:: 558Only enable events on applying cpu with this type for hybrid platform 559(e.g. core or atom)" 560 561EXAMPLES 562-------- 563 564$ perf stat \-- make 565 566 Performance counter stats for 'make': 567 568 83723.452481 task-clock:u (msec) # 1.004 CPUs utilized 569 0 context-switches:u # 0.000 K/sec 570 0 cpu-migrations:u # 0.000 K/sec 571 3,228,188 page-faults:u # 0.039 M/sec 572 229,570,665,834 cycles:u # 2.742 GHz 573 313,163,853,778 instructions:u # 1.36 insn per cycle 574 69,704,684,856 branches:u # 832.559 M/sec 575 2,078,861,393 branch-misses:u # 2.98% of all branches 576 577 83.409183620 seconds time elapsed 578 579 74.684747000 seconds user 580 8.739217000 seconds sys 581 582TIMINGS 583------- 584As displayed in the example above we can display 3 types of timings. 585We always display the time the counters were enabled/alive: 586 587 83.409183620 seconds time elapsed 588 589For workload sessions we also display time the workloads spent in 590user/system lands: 591 592 74.684747000 seconds user 593 8.739217000 seconds sys 594 595Those times are the very same as displayed by the 'time' tool. 596 597CSV FORMAT 598---------- 599 600With -x, perf stat is able to output a not-quite-CSV format output 601Commas in the output are not put into "". To make it easy to parse 602it is recommended to use a different character like -x \; 603 604The fields are in this order: 605 606 - optional usec time stamp in fractions of second (with -I xxx) 607 - optional CPU, core, or socket identifier 608 - optional number of logical CPUs aggregated 609 - counter value 610 - unit of the counter value or empty 611 - event name 612 - run time of counter 613 - percentage of measurement time the counter was running 614 - optional variance if multiple values are collected with -r 615 - optional metric value 616 - optional unit of metric 617 618Additional metrics may be printed with all earlier fields being empty. 619 620include::intel-hybrid.txt[] 621 622JSON FORMAT 623----------- 624 625With -j, perf stat is able to print out a JSON format output 626that can be used for parsing. 627 628- timestamp : optional usec time stamp in fractions of second (with -I) 629- optional aggregate options: 630 - core : core identifier (with --per-core) 631 - die : die identifier (with --per-die) 632 - socket : socket identifier (with --per-socket) 633 - node : node identifier (with --per-node) 634 - thread : thread identifier (with --per-thread) 635- counter-value : counter value 636- unit : unit of the counter value or empty 637- event : event name 638- variance : optional variance if multiple values are collected (with -r) 639- runtime : run time of counter 640- metric-value : optional metric value 641- metric-unit : optional unit of metric 642 643SEE ALSO 644-------- 645linkperf:perf-top[1], linkperf:perf-list[1] 646