1.\" $FreeBSD$ 2.Dd December 30, 2003 3.Dt GDB 4 4.Os 5.Sh NAME 6.Nm gdb 7.Nd external kernel debugger 8.Sh SYNOPSIS 9.Cd makeoptions DEBUG=-g 10.Cd options DDB 11.Cd options GDB_REMOTE_CHAT 12.Pp 13To prevent activation of the debugger on kernel 14.Xr panic 9 : 15.Cd options DDB_UNATTENDED 16.Sh DESCRIPTION 17The 18.Nm 19kernel debugger is a variation of 20.Xr gdb 1 21which understands some aspects of the 22.Fx 23kernel environment. It can be used in a number of ways: 24.Pp 25.Bl -bullet -offset indent -compact 26.It 27It can be used to debug another system interactively via a serial or firewire 28link. In this mode, the processor can be stopped and single stepped. 29.It 30It can be used to examine the memory of the processor on which it runs. 31.It 32It can be used to analyse a processor dump after a panic. 33.El 34.Pp 35When used for remote debugging, 36.Nm 37requires the presence of the 38.Xr ddb 4 39kernel debugger. 40Commands exist to switch between 41.Nm 42and 43.Xr ddb 4 . 44.Sh PREPARING FOR DEBUGGING 45When debugging kernels, it is practically essential to have built a kernel with 46debugging symbols 47.Cd ( makeoptions DEBUG=-g ). 48It's easiest to perform operations from the kernel build directory, by default 49.Pa /usr/obj/sys/GENERIC . 50.Pp 51First, ensure you have a copy of the debug macros in the directory: 52.Bd -literal -offset 5m 53# \f(CBmake gdbinit\fP 54.Ed 55.Pp 56This command performs some transformations on the macros installed in 57.Pa /usr/src/tools/debugscripts 58to adapt them to the local environment. 59.Ss Debugging a local machine 60To look at and change the contents of the memory of the system you're running 61on, 62.Bd -literal -offset 5m 63# \f(CBgdb -k -wcore kernel.debug /dev/mem\fP 64.Ed 65.Pp 66In this mode, you need the 67.Fl k 68flag to indicate to 69.Nm gdb 70that the ``dump file'' 71.Pa /dev/mem 72is a kernel data file. 73You can look at live data, and if you include the 74.Fl wcore 75option, you can change it at your peril. 76The system does not stop (obviously), so a number of things will not work. 77You can set breakpoints, but you can't ``continue'' execution, so they won't 78work. 79.Ss Debugging a crash dump 80By default, crash dumps are stored in the directory 81.Pa /var/crash . 82Investigate them from the kernel build directory with: 83.Bd -literal -offset 5m 84# \f(CBgdb -k kernel.debug /var/crash/vmcore.29\fP 85.Ed 86.Pp 87In this mode, the system is obviously stopped, so you can only look at it. 88.Ss Debugging a live system with a remote link 89To debug a live system with a remote link, the kernel must be compiled with the 90options 91.Cd options DDB 92and 93.Cd options GDB_REMOTE_CHAT . 94The option 95.Cd options BREAK_TO_DEBUGGER 96enables the debugging machine stop the debugged machine once a connection has 97been established by pressing 98.Li ^C. 99.Ss Debugging a live system with a remote serial link 100When using a serial port for the remote link on the i386 platform the serial 101port must be identified by setting the flag bit 102.Li 0x80 103for the specified interface. 104Generally this port will also be used as a serial console (flag bit 105.Li 0x10a), 106so the entry in 107.Pa /boot/device.hints 108should be: 109.Bd -literal -offset 5m 110hint.sio.0.flags="0x90" 111.Ed 112.Ss Debugging a live system with a remote firewire link 113As with serial debugging, to debug a live system with a firewire link, the 114kernel must be compiled with the options 115.Cd options DDB 116and 117.Cd options GDB_REMOTE_CHAT . 118.Pp 119A number of steps must be performed to set up a firewire link: 120.Pp 121.Bl -bullet -offset indent -compact 122.It 123First, ensure that the kernels of both systems include firewire support. 124If it isn't compiled into the kernel, load the klds: 125.Bd -literal -offset 5m 126# \f(CBkldload dcons\fP 127# \f(CBkldload dcons_crom\fP 128.Ed 129.Pp 130You should see something like this in the 131.Nm dmesg 132output: 133.Pp 134.Bd -literal -offset 5m 135fwohci0: BUS reset 136fwohci0: node_id=0xc000ffc1, gen=3, CYCLEMASTER mode 137firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 (me) 138firewire0: bus manager 1 (me) 139firewire0: New S400 device ID:000199000003622b 140dcons_crom0: <dcons configuration ROM> on firewire0 141dcons_crom0: bus_addr 0xf93000 142.Ed 143.Pp 144The corresponding output on the other machine looks like this: 145.Pp 146.Bd -literal -offset 5m 147fwohci0: BUS reset 148fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode 149firewire0: 2 nodes, maxhop <= 1, cable IRM = 1 150firewire0: bus manager 1 151firewire0: New S400 device ID:00c04f3226e88061 152dcons_crom0: <dcons configuration ROM> on firewire0 153dcons_crom0: bus_addr 0x22a000 154.Ed 155.Pp 156It's a good idea to load these modules at boot time with the following entry in 157.Pa /boot/loader.conf : 158.Pp 159.Bd -literal -offset 5m 160dcons_enable=YES 161.Ed 162.Pp 163.It 164Next, use 165.Nm fwcontrol 166to find the firewire node corresponding to the remote machine: 167.Pp 168.Bd -literal -offset 5m 169# \f(CBfwcontrol\fP 1702 devices (info_len=2) 171node EUI64 status 172 1 0x00c04f3226e88061 0 173 0 0x000199000003622b 1 174.Ed 175.Pp 176The first node is always the local system, so in this case, node 0 is the remote 177system. 178If there are more than two systems, check from the other end to find which node 179corresponds to the remote system. 180On the remote machine, it looks like this: 181.Pp 182.Bd -literal -offset 5m 183# \f(CBfwcontrol\fP 1842 devices (info_len=2) 185node EUI64 status 186 0 0x000199000003622b 0 187 1 0x00c04f3226e88061 1 188.Ed 189.Pp 190.It 191Next, establish a firewire connection with 192.Nm fwchat : 193.Pp 194.Bd -literal -offset 5m 195# \f(CBfwchat -b -t \f[CBI]target-address\fR 196.Ed 197.Pp 198.Ar target-address 199is the EUI64 address of the remote node, in this case 200.Li 0x000199000003622b . 201When started in this manner, 202.Nm fwchat 203establishes two local tunnel connections: 204.Ar localhost:5555 205is a connection to a 206.Nm getty 207process, which is not of interest here, and 208.Ar localhost:5556 209is a connection to the debugger. 210The port numbers can be changed with the 211.Fl p 212flag to 213.Nm fwcontrol . 214.Pp 215.Nm fwcontrol 216does not return control to the user. 217You can start it in the background, but sometimes it will produce error 218messages, so it's a good idea to start it in its own window. 219.Pp 220.It 221Finally, establish connection: 222.Bd -literal -offset 5m 223# \f(CBgdb kernel.debug\fP 224GNU gdb 5.2.1 (FreeBSD) 225\&\fI(politcal statements omitted)\fP\/ 226Ready to go. Enter 'tr' to connect to the remote target 227with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port 228or 'trf portno' to connect to the remote target with the firewire 229interface. portno defaults to 5556. 230 231Type 'getsyms' after connection to load kld symbols. 232 233If you're debugging a local system, you can use 'kldsyms' instead 234to load the kld symbols. That's a less obnoxious interface. 235(gdb) \f(CBtrf\fP 2360xc21bd378 in ?? () 237.Ed 238.Pp 239.It 240It's currently possible for the two ends of the firewire link to get out of sync 241after a reboot. 242On the 243.Nm fwchat 244screen you see: 245.Bd -literal -offset 5m 246fwchat: get crom faild 247.Ed 248.Pp 249In this case, stop 250.Nm fwchat 251and perform a firewire bus reset: 252.Pp 253.Bd -literal -offset 5m 254# \f(CBfwcontrol -r\fP 255# \f(CBfwchat -b -t 0x000199000003622b\fP 256.El 257.Sh COMMANDS 258The user interface to 259.Nm 260is via 261.Xr gdb 1 , 262so 263.Xr gdb 1 264commands also work. 265This section discusses only the extensions for kernel debugging that get 266installed in the kernel build directory. 267.Ss "Debugging Environment" 268The following macros manipulate the debugging environment: 269.Bl -ohang -offset 3m 270.It Cm ddb 271Switch back to 272.Nm ddb . 273This command is only meaningful when performing remote debugging. 274.It Cm getsyms 275Display 276.Nm kldstat 277information for the target machine and invite user to paste it back in. 278This is required because 279.Nm gdb 280does not allow data to be passed to shell scripts. 281It's necessary for remote debugging and crash dumps; for local memory debugging 282use 283.Nm kldsyms 284instead. 285.It Cm kldsyms 286Read in the symbol tables for the debugging machine. This doesn't work for 287remote debugging and crash dumps; use 288.Nm getsyms 289instead. 290.It Cm tr Ar interface 291Debug a remote system via the specified serial or firewire interface. 292.It Cm tr0 293Debug a remote system via serial interface 294.Pa /dev/cuaa0 . 295.It Cm tr1 296Debug a remote system via serial interface 297.Pa /dev/cuaa1 . 298.It Cm trf 299Debug a remote system via firewire interface at default port 5556. 300.El 301.Pp 302The commands 303.Nm tr0 , 304.Nm tr1 305and 306.Nm trf 307are convenience commands which invoke 308.Nm tr . 309.Ss "The current process environment" 310The following macros are convenience functions intended to make things easier 311than the standard 312.Nm gdb 313commands. 314.Bl -ohang -offset 3m 315.It Cm f0 316Select stack frame 0 and show assembler-level details. 317.It Cm f1 318Select stack frame 1 and show assembler-level details. 319.It Cm f2 320Select stack frame 2 and show assembler-level details. 321.It Cm f3 322Select stack frame 3 and show assembler-level details. 323.It Cm f4 324Select stack frame 4 and show assembler-level details. 325.It Cm f5 326Select stack frame 5 and show assembler-level details. 327.It Cm xb 328Show 12 words in hex, starting at current 329.Va ebp 330value. 331.It Cm xi 332List the next 10 instructions from the current 333.Va eip 334value. 335.It Cm xp 336Show the register contents and the first four parameters of the current stack 337frame. 338.It Cm xp0 339Show the first parameter of current stack frame in various formats. 340.It Cm xp1 341Show the second parameter of current stack frame in various formats. 342.It Cm xp2 343Show the third parameter of current stack frame in various formats. 344.It Cm xp3 345Show the fourth parameter of current stack frame in various formats. 346.It Cm xp4 347Show the fifth parameter of current stack frame in various formats. 348.It Cm xs 349Show the last 12 words on stack in hexadecimal. 350.It Cm xxp 351Show the register contents and the first ten parameters. 352.It Cm z 353Single step 1 instruction (over calls) and show next instruction. 354.It Cm zs 355Single step 1 instruction (through calls) and show next instruction. 356.El 357.Ss "Examining other processes" 358The following macros access other processes. 359.Nm gdb 360does not understand the concept of multiple processes, so they effectively 361bypass the entire 362.Nm gdb 363environment. 364.Bl -ohang -offset 3m 365.It Cm btp Ar pid 366Show a backtrace for the process 367.Va pid . 368.It Cm btpa 369Show backtraces for all processes in the system. 370.It Cm btpp 371Show a backtrace for the process previously selected with 372.Nm defproc . 373.It Cm btr Ar ebp 374Show a backtrace from the 375.Va ebp 376address specified 377.It Cm defproc Ar pid 378Specify the PID of the process for some other commands in this section. 379.It Cm fr Ar frame 380Show frame 381.Va frame 382of the stack of the process previously selected with 383.Nm defproc . 384.It Cm pcb Ar proc 385Show some pcb contents of process 386.Ar proc . 387.El 388.Ss "Examining data structures" 389You can use standard 390.Nm gdb 391commands to look at most data structures. The macros in this section are 392convenience functions which typically display the data in a more readable 393format, or which omit less interesting parts of the structure. 394.Bl -ohang -offset 3m 395.It Cm bp 396Show information about the buffer header pointed to by the variable 397.Va bp 398in the current frame. 399.It Cm bpd 400Show the contents 401.Vt (char*) 402of 403.Va bp->data 404in the current frame. 405.It Cm bpl 406Show detailed information about the buffer header 407.Vt (struct bp) 408pointed at by the local variable 409.Va bp . 410.It Cm bpp bp 411Show summary information about the buffer header 412.Vt (struct bp) 413pointed at by the parameter 414.Va bp . 415.It Cm bx 416Print a number of fields from the buffer header pointed at in by the pointer 417.Va bp 418in the current environment. 419.It Cm vdev 420Show some information of the vnode pointed to by the local variable 421.Va vp . 422.El 423.Ss "Miscellaneous macros" 424.Bl -ohang -offset 3m 425.It Cm checkmem 426Check unallocated memory for modifications. 427This assumes that the kernel has been compiled with 428.Cd options DIAGNOSTIC 429This causes the contents of free memory to be set to 430.Li 0xdeadc0de . 431.It Cm dmesg 432Print the system message buffer. This corresponds to the 433.Nm dmesg 434command. 435It can take a very long time over a serial line, and it's even slow via firewire 436or local memory due to inefficiencies in 437.Nm gdb . 438This macro used to be called 439.Nm msgbuf . 440.It Cm kldstat 441Equivalent of the kldstat(8) command without options 442.It Cm pname 443Print the command name of the current process. 444.It Cm ps 445Show process status. 446This corresponds in concept, but not in appearance, to the 447.Nm ps 448command. 449.It Cm y 450Kludge for writing macros. When writing macros, it's convenient to paste them 451back into the 452.Nm gdb 453window. Unfortunately, if the macro is already defined, 454.Nm gdb 455insists on asking 456.Bd -literal -offset 5m 457Redefine foo? 458.Ed 459.Pp 460It won't give up until you answer 461.Li y . 462This command is that answer. It does nothing else except to print a warning 463message to remind you to remove it again. 464.El 465.Sh AUTHORS 466This man page was written by 467.An "Greg Lehey" Aq grog@FreeBSD.org 468.Sh SEE ALSO 469.Xr ddb 4 , 470.Xr fwchat 8 , 471.Xr fwcontrol 8 , 472.Xr gdb 1 , 473.Xr vinumdebug 4 . 474.\" .Sh HISTORY 475.Sh BUGS 476.Bl -bullet -offset indent -compact 477.It 478.Nm 479was never designed to debug kernels, and it's not a very good match. 480Many problems exist. 481.It 482The debugging macros ``just growed''. 483In general, the person who wrote them did so while looking for a specific 484problem, so they may not be general enough, and they may behave badly when used 485in ways for which they were not intended, even if those ways make sense. 486.It 487Serial debugging is very slow, and race conditions can make it difficult to run 488the link at more than 9600 bps. Firewire connections do not have this problem. 489.It 490Many of these commands only work on the ia32 architecture. 491.El 492