1.. SPDX-License-Identifier: GPL-2.0 2 3========== 4Netconsole 5========== 6 7 8started by Ingo Molnar <mingo@redhat.com>, 2001.09.17 9 102.6 port and netpoll api by Matt Mackall <mpm@selenic.com>, Sep 9 2003 11 12IPv6 support by Cong Wang <xiyou.wangcong@gmail.com>, Jan 1 2013 13 14Extended console support by Tejun Heo <tj@kernel.org>, May 1 2015 15 16Release prepend support by Breno Leitao <leitao@debian.org>, Jul 7 2023 17 18Userdata append support by Matthew Wood <thepacketgeek@gmail.com>, Jan 22 2024 19 20Please send bug reports to Matt Mackall <mpm@selenic.com> 21Satyam Sharma <satyam.sharma@gmail.com>, and Cong Wang <xiyou.wangcong@gmail.com> 22 23Introduction: 24============= 25 26This module logs kernel printk messages over UDP allowing debugging of 27problem where disk logging fails and serial consoles are impractical. 28 29It can be used either built-in or as a module. As a built-in, 30netconsole initializes immediately after NIC cards and will bring up 31the specified interface as soon as possible. While this doesn't allow 32capture of early kernel panics, it does capture most of the boot 33process. 34 35Sender and receiver configuration: 36================================== 37 38It takes a string configuration parameter "netconsole" in the 39following format:: 40 41 netconsole=[+][r][src-port]@[src-ip]/[<dev>],[tgt-port]@<tgt-ip>/[tgt-macaddr] 42 43 where 44 + if present, enable extended console support 45 r if present, prepend kernel version (release) to the message 46 src-port source for UDP packets (defaults to 6665) 47 src-ip source IP to use (interface address) 48 dev network interface (eth0) 49 tgt-port port for logging agent (6666) 50 tgt-ip IP address for logging agent 51 tgt-macaddr ethernet MAC address for logging agent (broadcast) 52 53Examples:: 54 55 linux netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc 56 57or:: 58 59 insmod netconsole netconsole=@/,@10.0.0.2/ 60 61or using IPv6:: 62 63 insmod netconsole netconsole=@/,@fd00:1:2:3::1/ 64 65It also supports logging to multiple remote agents by specifying 66parameters for the multiple agents separated by semicolons and the 67complete string enclosed in "quotes", thusly:: 68 69 modprobe netconsole netconsole="@/,@10.0.0.2/;@/eth1,6892@10.0.0.3/" 70 71Built-in netconsole starts immediately after the TCP stack is 72initialized and attempts to bring up the supplied dev at the supplied 73address. 74 75The remote host has several options to receive the kernel messages, 76for example: 77 781) syslogd 79 802) netcat 81 82 On distributions using a BSD-based netcat version (e.g. Fedora, 83 openSUSE and Ubuntu) the listening port must be specified without 84 the -p switch:: 85 86 nc -u -l -p <port>' / 'nc -u -l <port> 87 88 or:: 89 90 netcat -u -l -p <port>' / 'netcat -u -l <port> 91 923) socat 93 94:: 95 96 socat udp-recv:<port> - 97 98Dynamic reconfiguration: 99======================== 100 101Dynamic reconfigurability is a useful addition to netconsole that enables 102remote logging targets to be dynamically added, removed, or have their 103parameters reconfigured at runtime from a configfs-based userspace interface. 104 105To include this feature, select CONFIG_NETCONSOLE_DYNAMIC when building the 106netconsole module (or kernel, if netconsole is built-in). 107 108Some examples follow (where configfs is mounted at the /sys/kernel/config 109mountpoint). 110 111To add a remote logging target (target names can be arbitrary):: 112 113 cd /sys/kernel/config/netconsole/ 114 mkdir target1 115 116Note that newly created targets have default parameter values (as mentioned 117above) and are disabled by default -- they must first be enabled by writing 118"1" to the "enabled" attribute (usually after setting parameters accordingly) 119as described below. 120 121To remove a target:: 122 123 rmdir /sys/kernel/config/netconsole/othertarget/ 124 125The interface exposes these parameters of a netconsole target to userspace: 126 127 =============== ================================= ============ 128 enabled Is this target currently enabled? (read-write) 129 extended Extended mode enabled (read-write) 130 release Prepend kernel release to message (read-write) 131 dev_name Local network interface name (read-write) 132 local_port Source UDP port to use (read-write) 133 remote_port Remote agent's UDP port (read-write) 134 local_ip Source IP address to use (read-write) 135 remote_ip Remote agent's IP address (read-write) 136 local_mac Local interface's MAC address (read-only) 137 remote_mac Remote agent's MAC address (read-write) 138 transmit_errors Number of packet send errors (read-only) 139 =============== ================================= ============ 140 141The "enabled" attribute is also used to control whether the parameters of 142a target can be updated or not -- you can modify the parameters of only 143disabled targets (i.e. if "enabled" is 0). 144 145To update a target's parameters:: 146 147 cat enabled # check if enabled is 1 148 echo 0 > enabled # disable the target (if required) 149 echo eth2 > dev_name # set local interface 150 echo 10.0.0.4 > remote_ip # update some parameter 151 echo cb:a9:87:65:43:21 > remote_mac # update more parameters 152 echo 1 > enabled # enable target again 153 154You can also update the local interface dynamically. This is especially 155useful if you want to use interfaces that have newly come up (and may not 156have existed when netconsole was loaded / initialized). 157 158Netconsole targets defined at boot time (or module load time) with the 159`netconsole=` param are assigned the name `cmdline<index>`. For example, the 160first target in the parameter is named `cmdline0`. You can control and modify 161these targets by creating configfs directories with the matching name. 162 163Let's suppose you have two netconsole targets defined at boot time:: 164 165 netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc;4444@10.0.0.1/eth1,9353@10.0.0.3/12:34:56:78:9a:bc 166 167You can modify these targets in runtime by creating the following targets:: 168 169 mkdir cmdline0 170 cat cmdline0/remote_ip 171 10.0.0.2 172 173 mkdir cmdline1 174 cat cmdline1/remote_ip 175 10.0.0.3 176 177Append User Data 178---------------- 179 180Custom user data can be appended to the end of messages with netconsole 181dynamic configuration enabled. User data entries can be modified without 182changing the "enabled" attribute of a target. 183 184Directories (keys) under `userdata` are limited to 53 character length, and 185data in `userdata/<key>/value` are limited to 200 bytes:: 186 187 cd /sys/kernel/config/netconsole && mkdir cmdline0 188 cd cmdline0 189 mkdir userdata/foo 190 echo bar > userdata/foo/value 191 mkdir userdata/qux 192 echo baz > userdata/qux/value 193 194Messages will now include this additional user data:: 195 196 echo "This is a message" > /dev/kmsg 197 198Sends:: 199 200 12,607,22085407756,-;This is a message 201 foo=bar 202 qux=baz 203 204Preview the userdata that will be appended with:: 205 206 cd /sys/kernel/config/netconsole/cmdline0/userdata 207 for f in `ls userdata`; do echo $f=$(cat userdata/$f/value); done 208 209If a `userdata` entry is created but no data is written to the `value` file, 210the entry will be omitted from netconsole messages:: 211 212 cd /sys/kernel/config/netconsole && mkdir cmdline0 213 cd cmdline0 214 mkdir userdata/foo 215 echo bar > userdata/foo/value 216 mkdir userdata/qux 217 218The `qux` key is omitted since it has no value:: 219 220 echo "This is a message" > /dev/kmsg 221 12,607,22085407756,-;This is a message 222 foo=bar 223 224Delete `userdata` entries with `rmdir`:: 225 226 rmdir /sys/kernel/config/netconsole/cmdline0/userdata/qux 227 228.. warning:: 229 When writing strings to user data values, input is broken up per line in 230 configfs store calls and this can cause confusing behavior:: 231 232 mkdir userdata/testing 233 printf "val1\nval2" > userdata/testing/value 234 # userdata store value is called twice, first with "val1\n" then "val2" 235 # so "val2" is stored, being the last value stored 236 cat userdata/testing/value 237 val2 238 239 It is recommended to not write user data values with newlines. 240 241Extended console: 242================= 243 244If '+' is prefixed to the configuration line or "extended" config file 245is set to 1, extended console support is enabled. An example boot 246param follows:: 247 248 linux netconsole=+4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc 249 250Log messages are transmitted with extended metadata header in the 251following format which is the same as /dev/kmsg:: 252 253 <level>,<sequnum>,<timestamp>,<contflag>;<message text> 254 255If 'r' (release) feature is enabled, the kernel release version is 256prepended to the start of the message. Example:: 257 258 6.4.0,6,444,501151268,-;netconsole: network logging started 259 260Non printable characters in <message text> are escaped using "\xff" 261notation. If the message contains optional dictionary, verbatim 262newline is used as the delimiter. 263 264If a message doesn't fit in certain number of bytes (currently 1000), 265the message is split into multiple fragments by netconsole. These 266fragments are transmitted with "ncfrag" header field added:: 267 268 ncfrag=<byte-offset>/<total-bytes> 269 270For example, assuming a lot smaller chunk size, a message "the first 271chunk, the 2nd chunk." may be split as follows:: 272 273 6,416,1758426,-,ncfrag=0/31;the first chunk, 274 6,416,1758426,-,ncfrag=16/31; the 2nd chunk. 275 276Miscellaneous notes: 277==================== 278 279.. Warning:: 280 281 the default target ethernet setting uses the broadcast 282 ethernet address to send packets, which can cause increased load on 283 other systems on the same ethernet segment. 284 285.. Tip:: 286 287 some LAN switches may be configured to suppress ethernet broadcasts 288 so it is advised to explicitly specify the remote agents' MAC addresses 289 from the config parameters passed to netconsole. 290 291.. Tip:: 292 293 to find out the MAC address of, say, 10.0.0.2, you may try using:: 294 295 ping -c 1 10.0.0.2 ; /sbin/arp -n | grep 10.0.0.2 296 297.. Tip:: 298 299 in case the remote logging agent is on a separate LAN subnet than 300 the sender, it is suggested to try specifying the MAC address of the 301 default gateway (you may use /sbin/route -n to find it out) as the 302 remote MAC address instead. 303 304.. note:: 305 306 the network device (eth1 in the above case) can run any kind 307 of other network traffic, netconsole is not intrusive. Netconsole 308 might cause slight delays in other traffic if the volume of kernel 309 messages is high, but should have no other impact. 310 311.. note:: 312 313 if you find that the remote logging agent is not receiving or 314 printing all messages from the sender, it is likely that you have set 315 the "console_loglevel" parameter (on the sender) to only send high 316 priority messages to the console. You can change this at runtime using:: 317 318 dmesg -n 8 319 320 or by specifying "debug" on the kernel command line at boot, to send 321 all kernel messages to the console. A specific value for this parameter 322 can also be set using the "loglevel" kernel boot option. See the 323 dmesg(8) man page and Documentation/admin-guide/kernel-parameters.rst 324 for details. 325 326Netconsole was designed to be as instantaneous as possible, to 327enable the logging of even the most critical kernel bugs. It works 328from IRQ contexts as well, and does not enable interrupts while 329sending packets. Due to these unique needs, configuration cannot 330be more automatic, and some fundamental limitations will remain: 331only IP networks, UDP packets and ethernet devices are supported. 332