1.\" 2.\" Mach Operating System 3.\" Copyright (c) 1991,1990 Carnegie Mellon University 4.\" Copyright (c) 2007 Robert N. M. Watson 5.\" All Rights Reserved. 6.\" 7.\" Permission to use, copy, modify and distribute this software and its 8.\" documentation is hereby granted, provided that both the copyright 9.\" notice and this permission notice appear in all copies of the 10.\" software, derivative works or modified versions, and any portions 11.\" thereof, and that both notices appear in supporting documentation. 12.\" 13.\" CARNEGIE MELLON ALLOWS FREE USE OF THIS SOFTWARE IN ITS "AS IS" 14.\" CONDITION. CARNEGIE MELLON DISCLAIMS ANY LIABILITY OF ANY KIND FOR 15.\" ANY DAMAGES WHATSOEVER RESULTING FROM THE USE OF THIS SOFTWARE. 16.\" 17.\" Carnegie Mellon requests users of this software to return to 18.\" 19.\" Software Distribution Coordinator or Software.Distribution@CS.CMU.EDU 20.\" School of Computer Science 21.\" Carnegie Mellon University 22.\" Pittsburgh PA 15213-3890 23.\" 24.\" any improvements or extensions that they make and grant Carnegie Mellon 25.\" the rights to redistribute these changes. 26.\" 27.Dd November 10, 2022 28.Dt DDB 4 29.Os 30.Sh NAME 31.Nm ddb 32.Nd interactive kernel debugger 33.Sh SYNOPSIS 34In order to enable kernel debugging facilities include: 35.Bd -ragged -offset indent 36.Cd options KDB 37.Cd options DDB 38.Ed 39.Pp 40To prevent activation of the debugger on kernel 41.Xr panic 9 : 42.Bd -ragged -offset indent 43.Cd options KDB_UNATTENDED 44.Ed 45.Pp 46In order to print a stack trace of the current thread on the console 47for a panic: 48.Bd -ragged -offset indent 49.Cd options KDB_TRACE 50.Ed 51.Pp 52To print the numerical value of symbols in addition to the symbolic 53representation, define: 54.Bd -ragged -offset indent 55.Cd options DDB_NUMSYM 56.Ed 57.Pp 58To enable the 59.Xr gdb 4 60backend, so that remote debugging with 61.Xr kgdb 1 Pq Pa ports/devel/gdb 62is possible, include: 63.Bd -ragged -offset indent 64.Cd options GDB 65.Ed 66.Sh DESCRIPTION 67The 68.Nm 69kernel debugger is an interactive debugger with a syntax inspired by 70.Xr gdb 1 Pq Pa ports/devel/gdb . 71If linked into the running kernel, 72it can be invoked locally with the 73.Ql debug 74.Xr keymap 5 75action, usually mapped to Ctrl+Alt+Esc, or by setting the 76.Va debug.kdb.enter 77sysctl to 1. 78The debugger is also invoked on kernel 79.Xr panic 9 80if the 81.Va debug.debugger_on_panic 82.Xr sysctl 8 83MIB variable is set non-zero, 84which is the default 85unless the 86.Dv KDB_UNATTENDED 87option is specified. 88Similarly, if the 89.Va debug.debugger_on_recursive_panic 90variable is set to 91.Dv 1 , 92then the debugger will be invoked on a recursive kernel panic. 93This variable has a default value of 94.Dv 0 , 95and has no effect if 96.Va debug.debugger_on_panic 97is already set non-zero. 98.Pp 99The current location is called 100.Va dot . 101The 102.Va dot 103is displayed with 104a hexadecimal format at a prompt. 105The commands 106.Ic examine 107and 108.Ic write 109update 110.Va dot 111to the address of the last line 112examined or the last location modified, and set 113.Va next 114to the address of 115the next location to be examined or changed. 116Other commands do not change 117.Va dot , 118and set 119.Va next 120to be the same as 121.Va dot . 122.Pp 123The general command syntax is: 124.Ar command Ns Op Li / Ns Ar modifier 125.Oo Ar addr Oc Ns Op , Ns Ar count 126.Pp 127A blank line repeats the previous command from the address 128.Va next 129with 130count 1 and no modifiers. 131Specifying 132.Ar addr 133sets 134.Va dot 135to the address. 136Omitting 137.Ar addr 138uses 139.Va dot . 140A missing 141.Ar count 142is taken 143to be 1 for printing commands or infinity for stack traces. 144A 145.Ar count 146of -1 is equivalent to a missing 147.Ar count . 148Options that are supplied but not supported by the given 149.Ar command 150are usually ignored. 151.Pp 152The 153.Nm 154debugger has a pager feature (like the 155.Xr more 1 156command) 157for the output. 158If an output line exceeds the number set in the 159.Va lines 160variable, it displays 161.Dq Li --More-- 162and waits for a response. 163The valid responses for it are: 164.Pp 165.Bl -tag -compact -width ".Li SPC" 166.It Li SPC 167one more page 168.It Li RET 169one more line 170.It Li q 171abort the current command, and return to the command input mode 172.El 173.Pp 174Finally, 175.Nm 176provides a small (currently 10 items) command history, and offers 177simple 178.Nm emacs Ns -style 179command line editing capabilities. 180In addition to 181the 182.Nm emacs 183control keys, the usual ANSI arrow keys may be used to browse through the 184history buffer, and move the cursor within the current line. 185.Sh COMMANDS 186.Ss COMMON DEBUGGER COMMANDS 187.Bl -tag -width indent -compact 188.It Ic help 189Print a short summary of the available commands and command 190abbreviations. 191.Pp 192.It Xo 193.Ic examine Ns Op Li / Ns Cm AISabcdghilmorsuxz ... 194.Oo Ar addr Oc Ns Op , Ns Ar count 195.Xc 196.It Xo 197.Ic x Ns Op Li / Ns Cm AISabcdghilmorsuxz ... 198.Oo Ar addr Oc Ns Op , Ns Ar count 199.Xc 200Display the addressed locations according to the formats in the modifier. 201Multiple modifier formats display multiple locations. 202If no format is specified, the last format specified for this command 203is used. 204.Pp 205The format characters are: 206.Bl -tag -compact -width indent 207.It Cm b 208look at by bytes (8 bits) 209.It Cm h 210look at by half words (16 bits) 211.It Cm l 212look at by long words (32 bits) 213.It Cm g 214look at by quad words (64 bits) 215.It Cm a 216print the location being displayed 217.It Cm A 218print the location with a line number if possible 219.It Cm x 220display in unsigned hex 221.It Cm z 222display in signed hex 223.It Cm o 224display in unsigned octal 225.It Cm d 226display in signed decimal 227.It Cm u 228display in unsigned decimal 229.It Cm r 230display in current radix, signed 231.It Cm c 232display low 8 bits as a character. 233Non-printing characters are displayed as an octal escape code (e.g., 234.Ql \e000 ) . 235.It Cm s 236display the null-terminated string at the location. 237Non-printing characters are displayed as octal escapes. 238.It Cm m 239display in unsigned hex with character dump at the end of each line. 240The location is also displayed in hex at the beginning of each line. 241.It Cm i 242display as a disassembled instruction 243.It Cm I 244display as a disassembled instruction with possible alternate formats 245depending on the machine. 246On i386, this selects the alternate format for the instruction decoding 247(16 bits in a 32-bit code segment and vice versa). 248.It Cm S 249display a symbol name for the pointer stored at the address 250.El 251.Pp 252.It Ic xf 253Examine forward: 254execute an 255.Ic examine 256command with the last specified parameters to it 257except that the next address displayed by it is used as the start address. 258.Pp 259.It Ic xb 260Examine backward: 261execute an 262.Ic examine 263command with the last specified parameters to it 264except that the last start address subtracted by the size displayed by it 265is used as the start address. 266.Pp 267.It Ic print Ns Op Li / Ns Cm acdoruxz 268.It Ic p Ns Op Li / Ns Cm acdoruxz 269Print 270.Ar addr Ns s 271according to the modifier character (as described above for 272.Cm examine ) . 273Valid formats are: 274.Cm a , x , z , o , d , u , r , 275and 276.Cm c . 277If no modifier is specified, the last one specified to it is used. 278The argument 279.Ar addr 280can be a string, in which case it is printed as it is. 281For example: 282.Bd -literal -offset indent 283print/x "eax = " $eax "\enecx = " $ecx "\en" 284.Ed 285.Pp 286will print like: 287.Bd -literal -offset indent 288eax = xxxxxx 289ecx = yyyyyy 290.Ed 291.Pp 292.It Ic pprint Ns Oo Li / Ns Cm d depth Oc Oo Ar name Oc 293Pretty-print symbol specified by 294.Ar name 295using CTF debugging data. Works for all symbols exported by the kernel and loaded kernel modules. 296.Pp 297If the 298.Cm d 299modifier has been specified, contents of structs nested up to 300.Ar depth 301levels deep will also be included in the output. 302.Ed 303.Pp 304.It Ic pprint struct Ns Oo Li / Ns Cm d depth Ic Oc Oo Ar name Oc Ns Op Ns Ar addr 305Print memory at 306.Ar addr 307as struct 308.Ar name Ns . 309Works for all structs defined by the kernel and loaded kernel modules. 310.Pp 311If the 312.Cm d 313modifier has been specified, contents of structs nested up to 314.Ar depth 315levels deep will also be included in the output. 316.Ed 317.Pp 318.It Xo 319.Ic write Ns Op Li / Ns Cm bhl 320.Ar addr expr1 Op Ar expr2 ... 321.Xc 322.It Xo 323.Ic w Ns Op Li / Ns Cm bhl 324.Ar addr expr1 Op Ar expr2 ... 325.Xc 326Write the expressions specified after 327.Ar addr 328on the command line at succeeding locations starting with 329.Ar addr . 330The write unit size can be specified in the modifier with a letter 331.Cm b 332(byte), 333.Cm h 334(half word) or 335.Cm l 336(long word) respectively. 337If omitted, 338long word is assumed. 339.Pp 340.Sy Warning : 341since there is no delimiter between expressions, strange 342things may happen. 343It is best to enclose each expression in parentheses. 344.Pp 345.It Ic set Li $ Ns Ar variable Oo Li = Oc Ar expr 346Set the named variable or register with the value of 347.Ar expr . 348Valid variable names are described below. 349.Pp 350.It Ic break Ns Oo Li / Ns Cm u Oc Oo Ar addr Oc Ns Op , Ns Ar count 351.It Ic b Ns Oo Li / Ns Cm u Oc Oo Ar addr Oc Ns Op , Ns Ar count 352Set a break point at 353.Ar addr . 354If 355.Ar count 356is supplied, the 357.Ic continue 358command will not stop at this break point on the first 359.Ar count 360\- 1 times that it is hit. 361If the break point is set, a break point number is 362printed with 363.Ql # . 364This number can be used in deleting the break point 365or adding conditions to it. 366.Pp 367If the 368.Cm u 369modifier is specified, this command sets a break point in user 370address space. 371Without the 372.Cm u 373option, the address is considered to be in the kernel 374space, and a wrong space address is rejected with an error message. 375This modifier can be used only if it is supported by machine dependent 376routines. 377.Pp 378.Sy Warning : 379If a user text is shadowed by a normal user space debugger, 380user space break points may not work correctly. 381Setting a break 382point at the low-level code paths may also cause strange behavior. 383.Pp 384.It Ic delete Op Ar addr 385.It Ic d Op Ar addr 386.It Ic delete Li # Ns Ar number 387.It Ic d Li # Ns Ar number 388Delete the specified break point. 389The break point can be specified by a 390break point number with 391.Ql # , 392or by using the same 393.Ar addr 394specified in the original 395.Ic break 396command, or by omitting 397.Ar addr 398to get the default address of 399.Va dot . 400.Pp 401.It Ic halt 402Halt the system. 403.Pp 404.It Ic watch Oo Ar addr Oc Ns Op , Ns Ar size 405Set a watchpoint for a region. 406Execution stops when an attempt to modify the region occurs. 407The 408.Ar size 409argument defaults to 4. 410If you specify a wrong space address, the request is rejected 411with an error message. 412.Pp 413.Sy Warning : 414Attempts to watch wired kernel memory 415may cause unrecoverable error in some systems such as i386. 416Watchpoints on user addresses work best. 417.Pp 418.It Ic hwatch Oo Ar addr Oc Ns Op , Ns Ar size 419Set a hardware watchpoint for a region if supported by the 420architecture. 421Execution stops when an attempt to modify the region occurs. 422The 423.Ar size 424argument defaults to 4. 425.Pp 426.Sy Warning : 427The hardware debug facilities do not have a concept of separate 428address spaces like the watch command does. 429Use 430.Ic hwatch 431for setting watchpoints on kernel address locations only, and avoid 432its use on user mode address spaces. 433.Pp 434.It Ic dhwatch Oo Ar addr Oc Ns Op , Ns Ar size 435Delete specified hardware watchpoint. 436.Pp 437.It Ic kill Ar sig pid 438Send signal 439.Ar sig 440to process 441.Ar pid . 442The signal is acted on upon returning from the debugger. 443This command can be used to kill a process causing resource contention 444in the case of a hung system. 445See 446.Xr signal 3 447for a list of signals. 448Note that the arguments are reversed relative to 449.Xr kill 2 . 450.Pp 451.It Ic step Ns Oo Li / Ns Cm p Oc Ns Op , Ns Ar count 452.It Ic s Ns Oo Li / Ns Cm p Oc Ns Op , Ns Ar count 453Single step 454.Ar count 455times. 456If the 457.Cm p 458modifier is specified, print each instruction at each step. 459Otherwise, only print the last instruction. 460.Pp 461.Sy Warning : 462depending on machine type, it may not be possible to 463single-step through some low-level code paths or user space code. 464On machines with software-emulated single-stepping (e.g., pmax), 465stepping through code executed by interrupt handlers will probably 466do the wrong thing. 467.Pp 468.It Ic continue Ns Op Li / Ns Cm c 469.It Ic c Ns Op Li / Ns Cm c 470Continue execution until a breakpoint or watchpoint. 471If the 472.Cm c 473modifier is specified, count instructions while executing. 474Some machines (e.g., pmax) also count loads and stores. 475.Pp 476.Sy Warning : 477when counting, the debugger is really silently single-stepping. 478This means that single-stepping on low-level code may cause strange 479behavior. 480.Pp 481.It Ic until Ns Op Li / Ns Cm p 482Stop at the next call or return instruction. 483If the 484.Cm p 485modifier is specified, print the call nesting depth and the 486cumulative instruction count at each call or return. 487Otherwise, 488only print when the matching return is hit. 489.Pp 490.It Ic next Ns Op Li / Ns Cm p 491.It Ic match Ns Op Li / Ns Cm p 492Stop at the matching return instruction. 493If the 494.Cm p 495modifier is specified, print the call nesting depth and the 496cumulative instruction count at each call or return. 497Otherwise, only print when the matching return is hit. 498.Pp 499.It Xo 500.Ic trace Ns Op Li / Ns Cm u 501.Op Ar pid | tid Ns 502.Op , Ns Ar count 503.Xc 504.It Xo 505.Ic t Ns Op Li / Ns Cm u 506.Op Ar pid | tid Ns 507.Op , Ns Ar count 508.Xc 509.It Xo 510.Ic where Ns Op Li / Ns Cm u 511.Op Ar pid | tid Ns 512.Op , Ns Ar count 513.Xc 514.It Xo 515.Ic bt Ns Op Li / Ns Cm u 516.Op Ar pid | tid Ns 517.Op , Ns Ar count 518.Xc 519Stack trace. 520The 521.Cm u 522option traces user space; if omitted, 523.Ic trace 524only traces 525kernel space. 526The optional argument 527.Ar count 528is the number of frames to be traced. 529If 530.Ar count 531is omitted, all frames are printed. 532.Pp 533.Sy Warning : 534User space stack trace is valid 535only if the machine dependent code supports it. 536.Pp 537.It Xo 538.Ic search Ns Op Li / Ns Cm bhl 539.Ar addr 540.Ar value 541.Op Ar mask Ns 542.Op , Ns Ar count 543.Xc 544Search memory for 545.Ar value . 546The optional 547.Ar count 548argument limits the search. 549.\" 550.Pp 551.It Xo 552.Ic Ic reboot Ns Op Li / Ns Cm s 553.Op Ar seconds 554.Xc 555.It Xo 556.Ic Ic reset Ns Op Li / Ns Cm s 557.Op Ar seconds 558.Xc 559Hard reset the system. 560If the optional argument 561.Ar seconds 562is given, the debugger will wait for this long, at most a week, 563before rebooting. 564When the 565.Cm s 566modifier is given, the command will skip running any registered shutdown 567handlers and attempt the most basic reset. 568.Pp 569.It Ic thread Ar addr | tid 570Switch the debugger to the thread with ID 571.Ar tid , 572if the argument is a decimal number, or address 573.Ar addr , 574otherwise. 575.Pp 576.It Ic watchdog Op Ar exp 577Program the 578.Xr watchdog 4 579timer to fire in 580.Pf 2^ Ar exp 581seconds. 582If no argument is provided, the watchdog timer is disabled. 583.El 584.Ss SPECIALIZED HELPER COMMANDS 585.Bl -tag -width indent -compact 586.It Xo 587.Ic findstack 588.Ar addr 589.Xc 590Prints the address of the thread whose kernel-mode stack contains 591.Ar addr , 592if any. 593.Pp 594.It Ic show Cm active trace 595.It acttrace 596Show a stack trace for every thread running on a CPU. 597.Pp 598.It Ic show Cm all procs Ns Op Li / Ns Cm a 599.It Ic ps Ns Op Li / Ns Cm a 600Display all process information. 601The process information may not be shown if it is not 602supported in the machine, or the bottom of the stack of the 603target process is not in the main memory at that time. 604The 605.Cm a 606modifier will print command line arguments for each process. 607.\" 608.Pp 609.It Ic show Cm all trace 610.It Ic alltrace 611Show a stack trace for every thread in the system. 612.Pp 613.It Ic show Cm all ttys 614Show all TTY's within the system. 615Output is similar to 616.Xr pstat 8 , 617but also includes the address of the TTY structure. 618.\" 619.Pp 620.It Ic show Cm all vnets 621Show the same output as "show vnet" does, but lists all 622virtualized network stacks within the system. 623.\" 624.Pp 625.It Ic show Cm allchains 626Show the same information like "show lockchain" does, but 627for every thread in the system. 628.\" 629.Pp 630.It Ic show Cm alllocks 631Show all locks that are currently held. 632This command is only available if 633.Xr witness 4 634is included in the kernel. 635.\" 636.Pp 637.It Ic show Cm allpcpu 638The same as "show pcpu", but for every CPU present in the system. 639.\" 640.Pp 641.It Ic show Cm allrman 642Show information related with resource management, including 643interrupt request lines, DMA request lines, I/O ports, I/O memory 644addresses, and Resource IDs. 645.\" 646.Pp 647.It Ic show Cm apic 648Dump data about APIC IDT vector mappings. 649.\" 650.Pp 651.It Ic show Cm badstacks 652Walk the 653.Xr witness 4 654graph and print any lock-order violations. 655This command is only available if 656.Xr witness 4 657is included in the kernel. 658.\" 659.Pp 660.It Ic show Cm breaks 661Show breakpoints set with the "break" command. 662.\" 663.Pp 664.It Ic show Cm bio Ar addr 665Show information about the bio structure 666.Vt struct bio 667present at 668.Ar addr . 669See the 670.Pa sys/bio.h 671header file and 672.Xr g_bio 9 673for more details on the exact meaning of the structure fields. 674.\" 675.Pp 676.It Ic show Cm buffer Ar addr 677Show information about the buf structure 678.Vt struct buf 679present at 680.Ar addr . 681See the 682.Pa sys/buf.h 683header file for more details on the exact meaning of the structure fields. 684.\" 685.Pp 686.It Ic show Cm callout Ar addr 687Show information about the callout structure 688.Vt struct callout 689present at 690.Ar addr . 691.\" 692.Pp 693.It Ic show Cm cdev Op Ar addr 694Show the internal devfs state of the cdev structure located at 695.Ar addr . 696If no argument is provided, show the list of all created cdevs, consisting of 697the devfs node name and the 698.Vt struct cdev 699address. 700.\" 701.Pp 702.It Ic show Cm conifhk 703Lists hooks currently waiting for completion in 704.Fn run_interrupt_driven_config_hooks . 705.\" 706.Pp 707.It Ic show Cm cpusets 708Print numbered root and assigned CPU affinity sets. 709See 710.Xr cpuset 2 711for more details. 712.\" 713.Pp 714.It Ic show Cm cyrixreg 715Show registers specific to the Cyrix processor. 716.\" 717.Pp 718.It Ic show Cm devmap 719Prints the contents of the static device mapping table. 720Currently only available on the 721ARM 722architecture. 723.\" 724.Pp 725.It Ic show Cm domain Ar addr 726Print protocol domain structure 727.Vt struct domain 728at address 729.Ar addr . 730See the 731.Pa sys/domain.h 732header file for more details on the exact meaning of the structure fields. 733.\" 734.Pp 735.It Ic show Cm ffs Op Ar addr 736Show brief information about ffs mount at the address 737.Ar addr , 738if argument is given. 739Otherwise, provides the summary about each ffs mount. 740.\" 741.Pp 742.It Ic show Cm file Ar addr 743Show information about the file structure 744.Vt struct file 745present at address 746.Ar addr . 747.\" 748.Pp 749.It Ic show Cm files 750Show information about every file structure in the system. 751.\" 752.Pp 753.It Ic show Cm freepages 754Show the number of physical pages in each of the free lists. 755.\" 756.Pp 757.It Ic show Cm geom Op Ar addr 758If the 759.Ar addr 760argument is not given, displays the entire GEOM topology. 761If 762.Ar addr 763is given, displays details about the given GEOM object (class, geom, 764provider or consumer). 765.\" 766.Pp 767.It Ic show Cm idt 768Show IDT layout. 769The first column specifies the IDT vector. 770The second one is the name of the interrupt/trap handler. 771Those functions are machine dependent. 772.\" 773.Pp 774.It Ic show Cm igi_list Ar addr 775Show information about the IGMP structure 776.Vt struct igmp_ifsoftc 777present at 778.Ar addr . 779.\" 780.Pp 781.It Ic show Cm iosched Ar addr 782Show information about the I/O scheduler 783.Vt struct cam_iosched_softc 784located at 785.Ar addr . 786.\" 787.Pp 788.It Ic show Cm inodedeps Op Ar addr 789Show brief information about each inodedep structure. 790If 791.Ar addr 792is given, only inodedeps belonging to the fs located at the 793supplied address are shown. 794.\" 795.Pp 796.It Ic show Cm inpcb Ar addr 797Show information on IP Control Block 798.Vt struct in_pcb 799present at 800.Ar addr . 801.\" 802.Pp 803.It Ic show Cm intr 804Dump information about interrupt handlers. 805.\" 806.Pp 807.It Ic show Cm intrcnt 808Dump the interrupt statistics. 809.\" 810.Pp 811.It Ic show Cm irqs 812Show interrupt lines and their respective kernel threads. 813.\" 814.Pp 815.It Ic show Cm ktr Ns Op Li / Ns Cm a Ns Cm v Ns Cm V 816Print the contents of the 817.Xr ktr 4 818trace buffer. 819The 820.Cm v 821modifier will request fully verbose output, causing the file, line number, and 822timestamp to be printed for each trace entry. 823The 824.Cm V 825modifier will request only the timestamps to be printed. 826The 827.Cm a 828modifier will request that the output be unpaginated. 829.\" 830.Pp 831.It Ic show Cm lapic 832Show information from the local APIC registers for this CPU. 833.\" 834.Pp 835.It Ic show Cm lock Ar addr 836Show lock structure. 837The output format is as follows: 838.Bl -tag -width "flags" 839.It Ic class : 840Class of the lock. 841Possible types include 842.Xr mutex 9 , 843.Xr rmlock 9 , 844.Xr rwlock 9 , 845.Xr sx 9 . 846.It Ic name : 847Name of the lock. 848.It Ic flags : 849Flags passed to the lock initialization function. 850.Em flags 851values are lock class specific. 852.It Ic state : 853Current state of a lock. 854.Em state 855values are lock class specific. 856.It Ic owner : 857Lock owner. 858.El 859.\" 860.Pp 861.It Ic show Cm lockchain Ar addr 862Show all threads a particular thread at address 863.Ar addr 864is waiting on based on non-spin locks. 865.\" 866.Pp 867.It Ic show Cm lockedbufs 868Show the same information as "show buf", but for every locked 869.Vt struct buf 870object. 871.\" 872.Pp 873.It Ic show Cm lockedvnods 874List all locked vnodes in the system. 875.\" 876.Pp 877.It Ic show Cm locks 878Prints all locks that are currently acquired. 879This command is only available if 880.Xr witness 4 881is included in the kernel. 882.\" 883.Pp 884.It Ic show Cm locktree 885.\" 886.Pp 887.It Ic show Cm malloc Ns Op Li / Ns Cm i 888Prints 889.Xr malloc 9 890memory allocator statistics. 891If the 892.Cm i 893modifier is specified, format output as machine-parseable comma-separated 894values ("CSV"). 895The output columns are as follows: 896.Pp 897.Bl -tag -compact -offset indent -width "Requests" 898.It Ic Type 899Specifies a type of memory. 900It is the same as a description string used while defining the 901given memory type with 902.Xr MALLOC_DECLARE 9 . 903.It Ic InUse 904Number of memory allocations of the given type, for which 905.Xr free 9 906has not been called yet. 907.It Ic MemUse 908Total memory consumed by the given allocation type. 909.It Ic Requests 910Number of memory allocation requests for the given 911memory type. 912.El 913.Pp 914The same information can be gathered in userspace with 915.Dq Nm vmstat Fl m . 916.\" 917.Pp 918.It Ic show Cm map Ns Oo Li / Ns Cm f Oc Ar addr 919Prints the VM map at 920.Ar addr . 921If the 922.Cm f 923modifier is specified the 924complete map is printed. 925.\" 926.Pp 927.It Ic show Cm msgbuf 928Print the system's message buffer. 929It is the same output as in the 930.Dq Nm dmesg 931case. 932It is useful if you got a kernel panic, attached a serial cable 933to the machine and want to get the boot messages from before the 934system hang. 935.\" 936.Pp 937.It Ic show Cm mount Op Ar addr 938Displays details about the mount point located at 939.Ar addr . 940If no 941.Ar addr 942is specified, 943displays short info about all currently mounted file systems. 944.\" 945.Pp 946.It Ic show Cm object Ns Oo Li / Ns Cm f Oc Ar addr 947Prints the VM object at 948.Ar addr . 949If the 950.Cm f 951option is specified the 952complete object is printed. 953.\" 954.Pp 955.It Ic show Cm panic 956Print the panic message if set. 957.\" 958.Pp 959.It Ic show Cm page 960Show statistics on VM pages. 961.\" 962.Pp 963.It Ic show Cm pageq 964Show statistics on VM page queues. 965.\" 966.Pp 967.It Ic show Cm pciregs 968Print PCI bus registers. 969The same information can be gathered in userspace by running 970.Dq Nm pciconf Fl lv . 971.\" 972.Pp 973.It Ic show Cm pcpu 974Print current processor state. 975The output format is as follows: 976.Pp 977.Bl -tag -compact -offset indent -width "spin locks held:" 978.It Ic cpuid 979Processor identifier. 980.It Ic curthread 981Thread pointer, process identifier and the name of the process. 982.It Ic curpcb 983Control block pointer. 984.It Ic fpcurthread 985FPU thread pointer. 986.It Ic idlethread 987Idle thread pointer. 988.It Ic APIC ID 989CPU identifier coming from APIC. 990.It Ic currentldt 991LDT pointer. 992.It Ic spin locks held 993Names of spin locks held. 994.El 995.\" 996.Pp 997.It Ic show Cm pgrpdump 998Dump process groups present within the system. 999.\" 1000.Pp 1001.It Ic show Cm prison Op Ar addr 1002Show the prison structure located at 1003.Ar addr . 1004If no 1005.Ar addr 1006argument is specified, show information about all prisons in the system. 1007.\" 1008.Pp 1009.It Ic show Cm proc Op Ar addr 1010Show information about the process structure located at address 1011.Ar addr , 1012or the current process if no argument is specified. 1013.\" 1014.Pp 1015.It Ic show Cm procvm Op Ar addr 1016Show process virtual memory layout for the process located at 1017.Ar addr , 1018or the current process if no argument is specified. 1019.\" 1020.Pp 1021.It Ic show Cm protosw Ar addr 1022Print protocol switch structure 1023.Vt struct protosw 1024at address 1025.Ar addr . 1026.\" 1027.Pp 1028.It Ic show Cm registers Ns Op Li / Ns Cm u 1029Display the register set. 1030If the 1031.Cm u 1032modifier is specified, the register contents of the thread's previous 1033trapframe are displayed instead. 1034Usually, this corresponds to the saved state from userspace. 1035.\" 1036.Pp 1037.It Ic show Cm rman Ar addr 1038Show resource manager object 1039.Vt struct rman 1040at address 1041.Ar addr . 1042Addresses of particular pointers can be gathered with "show allrman" 1043command. 1044.\" 1045.Pp 1046.It Ic show Cm route Ar addr 1047Show route table result for destination 1048.Ar addr . 1049At this time, INET and INET6 formatted addresses are supported. 1050.\" 1051.Pp 1052.It Ic show Cm routetable Oo Ar af Oc 1053Show full route table or tables. 1054If 1055.Ar af 1056is specified, show only routes for the given numeric address family. 1057If no argument is specified, dump the route table for all address families. 1058.\" 1059.Pp 1060.It Ic show Cm rtc 1061Show real time clock value. 1062Useful for long debugging sessions. 1063.\" 1064.Pp 1065.It Ic show Cm sleepchain 1066Deprecated. 1067Now an alias for 1068.Ic show Cm lockchain . 1069.\" 1070.Pp 1071.It Ic show Cm sleepq Ar addr 1072.It Ic show Cm sleepqueue Ar addr 1073Show the 1074.Xr sleepqueue 9 1075structure located at 1076.Ar addr . 1077.\" 1078.Pp 1079.It Ic show Cm sockbuf Ar addr 1080Show the socket buffer 1081.Va struct sockbuf 1082located at 1083.Ar addr . 1084.\" 1085.Pp 1086.It Ic show Cm socket Ar addr 1087Show the socket object 1088.Vt struct socket 1089located at 1090.Ar addr . 1091.\" 1092.Pp 1093.It Ic show Cm sysregs 1094Show system registers (e.g., 1095.Li cr0-4 1096on i386.) 1097Not present on some platforms. 1098.\" 1099.Pp 1100.It Ic show Cm tcpcb Ar addr 1101Print TCP control block 1102.Vt struct tcpcb 1103lying at address 1104.Ar addr . 1105For exact interpretation of output, visit 1106.Pa netinet/tcp.h 1107header file. 1108.\" 1109.Pp 1110.It Ic show Cm thread Op Ar addr | tid 1111If no 1112.Ar addr 1113or 1114.Ar tid 1115is specified, show detailed information about current thread. 1116Otherwise, print information about the thread with ID 1117.Ar tid 1118or kernel address 1119.Ar addr . 1120(If the argument is a decimal number, it is assumed to be a tid.) 1121.\" 1122.Pp 1123.It Ic show Cm threads 1124Show all threads within the system. 1125Output format is as follows: 1126.Pp 1127.Bl -tag -compact -offset indent -width "Second column" 1128.It Ic First column 1129Thread identifier (TID) 1130.It Ic Second column 1131Thread structure address 1132.It Ic Third column 1133Backtrace. 1134.El 1135.\" 1136.Pp 1137.It Ic show Cm tty Ar addr 1138Display the contents of a TTY structure in a readable form. 1139.\" 1140.Pp 1141.It Ic show Cm turnstile Ar addr 1142Show turnstile 1143.Vt struct turnstile 1144structure at address 1145.Ar addr . 1146Turnstiles are structures used within the 1147.Fx 1148kernel to implement 1149synchronization primitives which, while holding a specific type of lock, cannot 1150sleep or context switch to another thread. 1151Currently, those are: 1152.Xr mutex 9 , 1153.Xr rwlock 9 , 1154.Xr rmlock 9 . 1155.\" 1156.Pp 1157.It Ic show Cm uma Ns Op Li / Ns Cm i 1158Show UMA allocator statistics. 1159If the 1160.Cm i 1161modifier is specified, format output as machine-parseable comma-separated 1162values ("CSV"). 1163The output contains the following columns: 1164.Pp 1165.Bl -tag -compact -offset indent -width "Total Mem" 1166.It Cm "Zone" 1167Name of the UMA zone. 1168The same string that was passed to 1169.Xr uma_zcreate 9 1170as a first argument. 1171.It Cm "Size" 1172Size of a given memory object (slab). 1173.It Cm "Used" 1174Number of slabs being currently used. 1175.It Cm "Free" 1176Number of free slabs within the UMA zone. 1177.It Cm "Requests" 1178Number of allocations requests to the given zone. 1179.It Cm "Total Mem" 1180Total memory in use (either allocated or free) by a zone, in bytes. 1181.It Cm "XFree" 1182Number of free slabs within the UMA zone that were freed on a different NUMA 1183domain than allocated. 1184(The count in the 1185.Cm "Free" 1186column is inclusive of 1187.Cm "XFree" . ) 1188.El 1189.Pp 1190The same information might be gathered in the userspace 1191with the help of 1192.Dq Nm vmstat Fl z . 1193.\" 1194.Pp 1195.It Ic show Cm unpcb Ar addr 1196Shows UNIX domain socket private control block 1197.Vt struct unpcb 1198present at the address 1199.Ar addr . 1200.\" 1201.Pp 1202.It Ic show Cm vmochk 1203Prints, whether the internal VM objects are in a map somewhere 1204and none have zero ref counts. 1205.\" 1206.Pp 1207.It Ic show Cm vmopag 1208Walk the list of VM objects in the system, printing the indices and physical 1209addresses of the VM pages belonging to each object. 1210.\" 1211.Pp 1212.It Ic show Cm vnet Ar addr 1213Prints virtualized network stack 1214.Vt struct vnet 1215structure present at the address 1216.Ar addr . 1217.\" 1218.Pp 1219.It Ic show Cm vnode Ar addr 1220Prints vnode 1221.Vt struct vnode 1222structure lying at 1223.Ar addr . 1224For the exact interpretation of the output, look at the 1225.Pa sys/vnode.h 1226header file. 1227.\" 1228.Pp 1229.It Ic show Cm vnodebufs Ar addr 1230Shows clean/dirty buffer lists of the vnode located at 1231.Ar addr . 1232.\" 1233.Pp 1234.It Ic show Cm vpath Ar addr 1235Walk the namecache to lookup the pathname of the vnode located at 1236.Ar addr . 1237.\" 1238.Pp 1239.It Ic show Cm watches 1240Displays all watchpoints. 1241Shows watchpoints set with "watch" command. 1242.\" 1243.Pp 1244.It Ic show Cm witness 1245Shows information about lock acquisition coming from the 1246.Xr witness 4 1247subsystem. 1248.El 1249.Ss OFFLINE DEBUGGING COMMANDS 1250.Bl -tag -width indent -compact 1251.It Ic dump 1252Initiate a kernel core dump to the device(s) configured by 1253.Xr dumpon 8 . 1254.Pp 1255.It Ic gdb 1256Switches to remote GDB mode. 1257In remote GDB mode, another machine is required that runs 1258.Xr gdb 1 Pq Pa ports/devel/gdb 1259using the remote debug feature, with a connection to the serial 1260console port on the target machine. 1261.Pp 1262.It Ic netdump Fl s Ar server Oo Fl g Ar gateway Fl c Ar client Fl i Ar iface Oc 1263Configure 1264.Xr netdump 4 1265with the provided parameters, and immediately perform a netdump. 1266.Pp 1267There are some known limitations. 1268Principally, 1269.Xr netdump 4 1270only supports IPv4 at this time. 1271The address arguments to the 1272.Ic netdump 1273command must be dotted decimal IPv4 addresses. 1274(Hostnames are not supported.) 1275At present, the command only works if the machine is in a panic state. 1276Finally, the 1277.Nm 1278.Ic netdump 1279command does not provide any way to configure compression or encryption. 1280.Pp 1281.It Ic netgdb Fl s Ar server Oo Fl g Ar gateway Fl c Ar client Fl i Ar iface Oc 1282Initiate a 1283.Xr netgdb 4 1284session with the provided parameters. 1285.Pp 1286.Ic netgdb 1287has identical limitations to 1288.Ic netdump . 1289.Pp 1290.It Ic capture on 1291.It Ic capture off 1292.It Ic capture reset 1293.It Ic capture status 1294.Nm 1295supports a basic output capture facility, which can be used to retrieve the 1296results of debugging commands from userspace using 1297.Xr sysctl 3 . 1298.Ic capture on 1299enables output capture; 1300.Ic capture off 1301disables capture. 1302.Ic capture reset 1303will clear the capture buffer and disable capture. 1304.Ic capture status 1305will report current buffer use, buffer size, and disposition of output 1306capture. 1307.Pp 1308Userspace processes may inspect and manage 1309.Nm 1310capture state using 1311.Xr sysctl 8 : 1312.Pp 1313.Va debug.ddb.capture.bufsize 1314may be used to query or set the current capture buffer size. 1315.Pp 1316.Va debug.ddb.capture.maxbufsize 1317may be used to query the compile-time limit on the capture buffer size. 1318.Pp 1319.Va debug.ddb.capture.bytes 1320may be used to query the number of bytes of output currently in the capture 1321buffer. 1322.Pp 1323.Va debug.ddb.capture.data 1324returns the contents of the buffer as a string to an appropriately privileged 1325process. 1326.Pp 1327This facility is particularly useful in concert with the scripting and 1328.Xr textdump 4 1329facilities, allowing scripted debugging output to be captured and 1330committed to disk as part of a textdump for later analysis. 1331The contents of the capture buffer may also be inspected in a kernel core dump 1332using 1333.Xr kgdb 1 Pq Pa ports/devel/gdb . 1334.Pp 1335.It Ic run 1336.It Ic script 1337.It Ic scripts 1338.It Ic unscript 1339Run, define, list, and delete scripts. 1340See the 1341.Sx SCRIPTING 1342section for more information on the scripting facility. 1343.Pp 1344.It Ic textdump dump 1345.It Ic textdump set 1346.It Ic textdump status 1347.It Ic textdump unset 1348Use the 1349.Ic textdump dump 1350command to immediately perform a textdump. 1351More information may be found in 1352.Xr textdump 4 . 1353The 1354.Ic textdump set 1355command may be used to force the next kernel core dump to be a textdump 1356rather than a traditional memory dump or minidump. 1357.Ic textdump status 1358reports whether a textdump has been scheduled. 1359.Ic textdump unset 1360cancels a request to perform a textdump as the next kernel core dump. 1361.El 1362.Sh VARIABLES 1363The debugger accesses registers and variables as 1364.Li $ Ns Ar name . 1365Register names are as in the 1366.Dq Ic show Cm registers 1367command. 1368Some variables are suffixed with numbers, and may have some modifier 1369following a colon immediately after the variable name. 1370For example, register variables can have a 1371.Cm u 1372modifier to indicate user register (e.g., 1373.Dq Li $eax:u ) . 1374.Pp 1375Built-in variables currently supported are: 1376.Pp 1377.Bl -tag -width ".Va tabstops" -compact 1378.It Va radix 1379Input and output radix. 1380.It Va maxoff 1381Addresses are printed as 1382.Dq Ar symbol Ns Li + Ns Ar offset 1383unless 1384.Ar offset 1385is greater than 1386.Va maxoff . 1387.It Va maxwidth 1388The width of the displayed line. 1389.It Va lines 1390The number of lines. 1391It is used by the built-in pager. 1392Setting it to 0 disables paging. 1393.It Va tabstops 1394Tab stop width. 1395.It Va work Ns Ar xx 1396Work variable; 1397.Ar xx 1398can take values from 0 to 31. 1399.El 1400.Sh EXPRESSIONS 1401Most expression operators in C are supported except 1402.Ql ~ , 1403.Ql ^ , 1404and unary 1405.Ql & . 1406Special rules in 1407.Nm 1408are: 1409.Bl -tag -width ".No Identifiers" 1410.It Identifiers 1411The name of a symbol is translated to the value of the symbol, which 1412is the address of the corresponding object. 1413.Ql \&. 1414and 1415.Ql \&: 1416can be used in the identifier. 1417If supported by an object format dependent routine, 1418.Sm off 1419.Oo Ar filename : Oc Ar func : lineno , 1420.Sm on 1421.Oo Ar filename : Oc Ns Ar variable , 1422and 1423.Oo Ar filename : Oc Ns Ar lineno 1424can be accepted as a symbol. 1425.It Numbers 1426Radix is determined by the first two letters: 1427.Ql 0x : 1428hex, 1429.Ql 0o : 1430octal, 1431.Ql 0t : 1432decimal; otherwise, follow current radix. 1433.It Li \&. 1434.Va dot 1435.It Li + 1436.Va next 1437.It Li .. 1438address of the start of the last line examined. 1439Unlike 1440.Va dot 1441or 1442.Va next , 1443this is only changed by 1444.Ic examine 1445or 1446.Ic write 1447command. 1448.It Li ' 1449last address explicitly specified. 1450.It Li $ Ns Ar variable 1451Translated to the value of the specified variable. 1452It may be followed by a 1453.Ql \&: 1454and modifiers as described above. 1455.It Ar a Ns Li # Ns Ar b 1456A binary operator which rounds up the left hand side to the next 1457multiple of right hand side. 1458.It Li * Ns Ar expr 1459Indirection. 1460It may be followed by a 1461.Ql \&: 1462and modifiers as described above. 1463.El 1464.Sh SCRIPTING 1465.Nm 1466supports a basic scripting facility to allow automating tasks or responses to 1467specific events. 1468Each script consists of a list of DDB commands to be executed sequentially, 1469and is assigned a unique name. 1470Certain script names have special meaning, and will be automatically run on 1471various 1472.Nm 1473events if scripts by those names have been defined. 1474.Pp 1475The 1476.Ic script 1477command may be used to define a script by name. 1478Scripts consist of a series of 1479.Nm 1480commands separated with the 1481.Ql \&; 1482character. 1483For example: 1484.Bd -literal -offset indent 1485script kdb.enter.panic=bt; show pcpu 1486script lockinfo=show alllocks; show lockedvnods 1487.Ed 1488.Pp 1489The 1490.Ic scripts 1491command lists currently defined scripts. 1492.Pp 1493The 1494.Ic run 1495command execute a script by name. 1496For example: 1497.Bd -literal -offset indent 1498run lockinfo 1499.Ed 1500.Pp 1501The 1502.Ic unscript 1503command may be used to delete a script by name. 1504For example: 1505.Bd -literal -offset indent 1506unscript kdb.enter.panic 1507.Ed 1508.Pp 1509These functions may also be performed from userspace using the 1510.Xr ddb 8 1511command. 1512.Pp 1513Certain scripts are run automatically, if defined, for specific 1514.Nm 1515events. 1516The follow scripts are run when various events occur: 1517.Bl -tag -width kdb.enter.powerfail 1518.It Va kdb.enter.acpi 1519The kernel debugger was entered as a result of an 1520.Xr acpi 4 1521event. 1522.It Va kdb.enter.bootflags 1523The kernel debugger was entered at boot as a result of the debugger boot 1524flag being set. 1525.It Va kdb.enter.break 1526The kernel debugger was entered as a result of a serial or console break. 1527.It Va kdb.enter.cam 1528The kernel debugger was entered as a result of a 1529.Xr CAM 4 1530event. 1531.It Va kdb.enter.mac 1532The kernel debugger was entered as a result of an assertion failure in the 1533.Xr mac_test 4 1534module of the 1535TrustedBSD MAC Framework. 1536.It Va kdb.enter.netgraph 1537The kernel debugger was entered as a result of a 1538.Xr netgraph 4 1539event. 1540.It Va kdb.enter.panic 1541.Xr panic 9 1542was called. 1543.It Va kdb.enter.powerpc 1544The kernel debugger was entered as a result of an unimplemented interrupt 1545type on the powerpc platform. 1546.It Va kdb.enter.sysctl 1547The kernel debugger was entered as a result of the 1548.Va debug.kdb.enter 1549sysctl being set. 1550.It Va kdb.enter.unionfs 1551The kernel debugger was entered as a result of an assertion failure in the 1552union file system. 1553.It Va kdb.enter.unknown 1554The kernel debugger was entered, but no reason has been set. 1555.It Va kdb.enter.vfslock 1556The kernel debugger was entered as a result of a VFS lock violation. 1557.It Va kdb.enter.watchdog 1558The kernel debugger was entered as a result of a watchdog firing. 1559.It Va kdb.enter.witness 1560The kernel debugger was entered as a result of a 1561.Xr witness 4 1562violation. 1563.El 1564.Pp 1565In the event that none of these scripts is found, 1566.Nm 1567will attempt to execute a default script: 1568.Bl -tag -width kdb.enter.powerfail 1569.It Va kdb.enter.default 1570The kernel debugger was entered, but a script exactly matching the reason for 1571entering was not defined. 1572This can be used as a catch-all to handle cases not specifically of interest; 1573for example, 1574.Va kdb.enter.witness 1575might be defined to have special handling, and 1576.Va kdb.enter.default 1577might be defined to simply panic and reboot. 1578.El 1579.Sh HINTS 1580On machines with an ISA expansion bus, a simple NMI generation card can be 1581constructed by connecting a push button between the A01 and B01 (CHCHK# and 1582GND) card fingers. 1583Momentarily shorting these two fingers together may cause the bridge chipset to 1584generate an NMI, which causes the kernel to pass control to 1585.Nm . 1586Some bridge chipsets do not generate a NMI on CHCHK#, so your mileage may vary. 1587The NMI allows one to break into the debugger on a wedged machine to 1588diagnose problems. 1589Other bus' bridge chipsets may be able to generate NMI using bus specific 1590methods. 1591There are many PCI and PCIe add-in cards which can generate NMI for 1592debugging. 1593Modern server systems typically use IPMI to generate signals to enter the 1594debugger. 1595The 1596.Va devel/ipmitool 1597port can be used to send the 1598.Cd chassis power diag 1599command which delivers an NMI to the processor. 1600Embedded systems often use JTAG for debugging, but rarely use it in 1601combination with 1602.Nm . 1603.Pp 1604Serial consoles can break to the debugger by sending a BREAK 1605condition on the serial line. 1606This requires a kernel built with 1607.Cd options BREAK_TO_DEBUGGER 1608is specified in the kernel. 1609Most terminal emulation programs can send a break sequence with a 1610special key sequence or menu selection. 1611Sending the break can be difficult or even happen spuriously in some setups. 1612An alternative method is to build a kernel with 1613.Cd options ALT_BREAK_TO_DEBUGGER 1614then the sequence of CR TILDE CTRL-B enters the debugger; 1615CR TILDE CTRL-P causes a panic; and 1616CR TILDE CTRL-R causes an immediate reboot. 1617In all these sequences, CR represents Carriage Return and is usually 1618sent by pressing the Enter or Return key. 1619TILDE is the ASCII tilde character (~). 1620CTRL-x is Control x, sent by pressing the Control key, then x, then releasing 1621both. 1622.Pp 1623The break-to-debugger behavior can be enabled by setting 1624.Xr sysctl 8 1625.Va debug.kdb.break_to_debugger 1626to 1. 1627The alt-break-to-debugger behavior can be enabled by setting 1628.Xr sysctl 8 1629.Va debug.kdb.alt_break_to_debugger 1630to 1. 1631The debugger can be entered by setting 1632.Xr sysctl 8 1633.Va debug.kdb.enter 1634to 1. 1635.Pp 1636Output can be interrupted, paused, and resumed with the control 1637characters CTRL-C, CTRL-S, and CTRL-Q. 1638Because these control characters are received as in-band data from the 1639console, there is an input buffer, and once that buffer fills 1640.Nm 1641must either stop responding to control characters or drop additional 1642input while continuing to search for control characters. 1643This behavior is controlled by the tunable 1644.Xr sysctl 8 1645.Va debug.ddb.prioritize_control_input , 1646which defaults to 1. 1647The input buffer size is 512 bytes. 1648.Sh FILES 1649Header files mentioned in this manual page can be found below 1650.Pa /usr/include 1651directory. 1652.Pp 1653.Bl -dash -compact 1654.It 1655.Pa sys/buf.h 1656.It 1657.Pa sys/domain.h 1658.It 1659.Pa netinet/in_pcb.h 1660.It 1661.Pa sys/socket.h 1662.It 1663.Pa sys/vnode.h 1664.El 1665.Sh SEE ALSO 1666.Xr gdb 1 Pq Pa ports/devel/gdb , 1667.Xr kgdb 1 Pq Pa ports/devel/gdb , 1668.Xr acpi 4 , 1669.Xr CAM 4 , 1670.Xr gdb 4 , 1671.Xr mac_ddb 4 , 1672.Xr mac_test 4 , 1673.Xr netgraph 4 , 1674.Xr textdump 4 , 1675.Xr witness 4 , 1676.Xr ddb 8 , 1677.Xr sysctl 8 , 1678.Xr panic 9 1679.Sh HISTORY 1680The 1681.Nm 1682debugger was developed for Mach, and ported to 1683.Bx 386 0.1 . 1684This manual page translated from 1685.Xr man 7 1686macros by 1687.An Garrett Wollman . 1688.Pp 1689.An Robert N. M. Watson 1690added support for 1691.Nm 1692output capture, 1693.Xr textdump 4 1694and scripting in 1695.Fx 7.1 . 1696