1Kernel Support for miscellaneous Binary Formats (binfmt_misc) 2============================================================= 3 4This Kernel feature allows you to invoke almost (for restrictions see below) 5every program by simply typing its name in the shell. 6This includes for example compiled Java(TM), Python or Emacs programs. 7 8To achieve this you must tell binfmt_misc which interpreter has to be invoked 9with which binary. Binfmt_misc recognises the binary-type by matching some bytes 10at the beginning of the file with a magic byte sequence (masking out specified 11bits) you have supplied. Binfmt_misc can also recognise a filename extension 12aka ``.com`` or ``.exe``. 13 14First you must mount binfmt_misc:: 15 16 mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc 17 18To actually register a new binary type, you have to set up a string looking like 19``:name:type:offset:magic:mask:interpreter:flags`` (where you can choose the 20``:`` upon your needs) and echo it to ``/proc/sys/fs/binfmt_misc/register``. 21 22Here is what the fields mean: 23 24- ``name`` 25 is an identifier string. A new /proc file will be created with this 26 name below ``/proc/sys/fs/binfmt_misc``; cannot contain slashes ``/`` for 27 obvious reasons. 28- ``type`` 29 is the type of recognition. Give ``M`` for magic, ``E`` for extension and 30 ``B`` for a bpf-backed handler (see below). 31- ``offset`` 32 is the offset of the magic/mask in the file, counted in bytes. This 33 defaults to 0 if you omit it (i.e. you write ``:name:type::magic...``). 34 Ignored when using filename extension matching. 35- ``magic`` 36 is the byte sequence binfmt_misc is matching for. The magic string 37 may contain hex-encoded characters like ``\x0a`` or ``\xA4``. Note that you 38 must escape any NUL bytes; parsing halts at the first one. In a shell 39 environment you might have to write ``\\x0a`` to prevent the shell from 40 eating your ``\``. 41 If you chose filename extension matching, this is the extension to be 42 recognised (without the ``.``, the ``\x0a`` specials are not allowed). 43 Extension matching is case sensitive, and slashes ``/`` are not allowed! 44- ``mask`` 45 is an (optional, defaults to all 0xff) mask. You can mask out some 46 bits from matching by supplying a string like magic and as long as magic. 47 The mask is anded with the byte sequence of the file. Note that you must 48 escape any NUL bytes; parsing halts at the first one. Ignored when using 49 filename extension matching. 50- ``interpreter`` 51 is the program that should be invoked with the binary as first 52 argument (specify the full path). For ``B`` entries this field 53 carries the name of the bpf handler instead (see below). 54- ``flags`` 55 is an optional field that controls several aspects of the invocation 56 of the interpreter. It is a string of capital letters, each controls a 57 certain aspect. The following flags are supported: 58 59 ``P`` - preserve-argv[0] 60 Legacy behavior of binfmt_misc is to overwrite 61 the original argv[0] with the full path to the binary. When this 62 flag is included, binfmt_misc will add an argument to the argument 63 vector for this purpose, thus preserving the original ``argv[0]``. 64 e.g. If your interp is set to ``/bin/foo`` and you run ``blah`` 65 (which is in ``/usr/local/bin``), then the kernel will execute 66 ``/bin/foo`` with ``argv[]`` set to ``["/bin/foo", "/usr/local/bin/blah", "blah"]``. The interp has to be aware of this so it can 67 execute ``/usr/local/bin/blah`` 68 with ``argv[]`` set to ``["blah"]``. 69 ``O`` - open-binary 70 Legacy behavior of binfmt_misc is to pass the full path 71 of the binary to the interpreter as an argument. When this flag is 72 included, binfmt_misc will open the file for reading and pass its 73 descriptor into the auxilary vector with the key "AT_EXECFD", thus 74 allowing the interpreter to execute non-readable binaries. This 75 feature should be used with care - the interpreter has to be trusted 76 not to emit the contents of the non-readable binary. 77 ``C`` - credentials 78 Currently, the behavior of binfmt_misc is to calculate 79 the credentials and security token of the new process according to 80 the interpreter. When this flag is included, these attributes are 81 calculated according to the binary. It also implies the ``O`` flag. 82 This feature should be used with care as the interpreter 83 will run with root permissions when a setuid binary owned by root 84 is run with binfmt_misc. 85 ``F`` - fix binary 86 The usual behaviour of binfmt_misc is to spawn the 87 binary lazily when the misc format file is invoked. However, 88 this doesn't work very well in the face of mount namespaces and 89 changeroots, so the ``F`` mode opens the binary as soon as the 90 emulation is installed and uses the opened image to spawn the 91 emulator, meaning it is always available once installed, 92 regardless of how the environment changes. 93 ``T`` - transparent 94 Run the interpreter transparently. The binary is handed to 95 the interpreter through ``AT_EXECFD`` (``T`` implies ``O``), 96 the argument vector is left exactly as the caller built it 97 and the kernel labels ``/proc/pid/exe`` with the binary 98 instead of the interpreter. The interpreter has to load the 99 binary from ``AT_EXECFD`` and follow the 100 ``AT_FLAGS_TRANSPARENT_INTERP`` contract. Combining ``T`` 101 with ``P`` is rejected: transparency preserves the whole 102 argument vector, argv[0] included. 103 ``L`` - loader substitution 104 Do not run the interpreter on the binary at all: load the 105 binary itself as a fully native exec and substitute the 106 interpreter for the loader named in the binary's 107 ``PT_INTERP``. See the "Loader substitution" section 108 below. ``L`` rejects ``T``, ``P``, ``O`` and ``C``; 109 ``F`` composes. 110 ``D`` - registered disabled 111 The entry is created disabled instead of being matchable at 112 once, and has to be enabled by writing ``1`` to its file 113 before it dispatches anything. This splits a registration 114 into creating the entry and activating it, leaving room to 115 configure it in between - which is what a ``B`` entry that 116 binds interpreters needs; see the bpf section below. The flag 117 is spent on the registration and is not read back: what an 118 entry file reports afterwards is whether it is enabled. 119 120 121There are some restrictions: 122 123 - the whole register string may not exceed 1920 characters 124 - the magic must reside in the first 128 bytes of the file, i.e. 125 offset+size(magic) has to be less than 128 126 - the interpreter string may not exceed 127 characters 127 - an interpreter used with ``C`` or ``L`` but without ``F`` has to be 128 named by an absolute path. It is opened when the binary is executed, so 129 a relative one would be resolved against the working directory of 130 whoever runs the binary 131 - the amount of pre-opened interpreters by ``F``, or bound to a ``B`` entry 132 is limited by the ``/proc/sys/user/max_binfmt_misc_interpreters`` sysctl. A 133 registration past the limit is refused with ``-ENOSPC``. This limits an 134 unprivileged namespace pinning files. A nested namespace can raise only its 135 own limit and every ancestor is charged too 136 137 138To use binfmt_misc you have to mount it first. You can mount it with 139``mount -t binfmt_misc none /proc/sys/fs/binfmt_misc`` command, or you can add 140a line ``none /proc/sys/fs/binfmt_misc binfmt_misc defaults 0 0`` to your 141``/etc/fstab`` so it auto mounts on boot. 142 143You may want to add the binary formats in one of your ``/etc/rc`` scripts during 144boot-up. Read the manual of your init program to figure out how to do this 145right. 146 147Think about the order of adding entries! Later added entries are matched first! 148 149 150A few examples (assumed you are in ``/proc/sys/fs/binfmt_misc``): 151 152- enable support for em86 (like binfmt_em86, for Alpha AXP only):: 153 154 echo ':i386:M::\x7fELF\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03:\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfb\xff\xff:/bin/em86:' > register 155 echo ':i486:M::\x7fELF\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x06:\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfb\xff\xff:/bin/em86:' > register 156 157- enable support for packed DOS applications (pre-configured dosemu hdimages):: 158 159 echo ':DEXE:M::\x0eDEX::/usr/bin/dosexec:' > register 160 161- enable support for Windows executables using wine:: 162 163 echo ':DOSWin:M::MZ::/usr/local/bin/wine:' > register 164 165For java support see Documentation/admin-guide/java.rst 166 167 168You can enable/disable binfmt_misc or one binary type by echoing 0 (to disable) 169or 1 (to enable) to ``/proc/sys/fs/binfmt_misc/status`` or 170``/proc/.../the_name``. 171Catting the file tells you the current status of ``binfmt_misc/the_entry``. 172 173You can remove one entry or all entries by echoing -1 to ``/proc/.../the_name`` 174or ``/proc/sys/fs/binfmt_misc/status``. A single entry can also be removed 175by simply unlinking (``rm``) ``/proc/.../the_name``. 176 177 178bpf-backed handlers 179------------------- 180 181With ``CONFIG_BINFMT_MISC_BPF`` both the matching and the interpreter 182selection can be delegated to bpf programs. A handler is an instance of the 183``binfmt_misc_ops`` struct_ops with a ``match`` and a ``load`` program and a 184``name``. Once the struct_ops map is registered the handler can be activated 185with a ``B`` entry that references it by name in the ``interpreter`` field 186and carries neither offset, magic, nor mask:: 187 188 echo ':qemu:B::::my_handler:' > register 189 190Both programs receive the ``linux_binprm`` of the binary and both can 191sleep. The ``match`` program decides whether the handler applies: it is 192consulted during the entry walk exactly like magic and extension matching, 193in the same registration order with the same first-match-wins semantics. 194Unlike static matching it is not limited to the prefetched first bytes of 195the file in ``bprm->buf``: it can read the file, e.g. to parse ELF program 196headers whose data sits at arbitrary offsets. It only decides, though: the 197selection kfuncs below are rejected in it. The ``load`` program of the 198matched handler then selects the interpreter: it can equally read the file 199and derive the interpreter from the binary's location. It selects the 200interpreter by calling the ``bpf_binprm_set_interp()`` kfunc with an 201absolute path and returning ``0``. A match is committed: a failing 202``load`` fails the exec with its error instead of falling through to later 203entries; ``-ENOEXEC`` lets the remaining binary formats have a go. A path 204selected this way is opened with the credentials of the task doing the 205exec, exactly as a statically registered interpreter without ``F`` would 206be. 207 208An entry can instead bind the interpreters its handler may use, so that no 209path is resolved at exec time at all. An entry registered with ``D`` is not 210matchable yet, which is what leaves it open to being given them, one 211``+name path`` write at a time:: 212 213 echo ':qemu:B::::my_handler:D' > register 214 echo '+aarch64 /usr/bin/qemu-aarch64' > qemu 215 echo '+arm /usr/bin/qemu-arm' > qemu 216 echo 1 > qemu 217 218Each path is opened during its write, in the writing process's context and 219with the credentials the entry file was opened with, exactly the way ``F`` 220pre-opens a static entry's interpreter; the paths must be absolute. The 221path is everything past the first space, so there is nothing it cannot 222express, and no interpreter has to fit in a register string. An entry 223binds at most 100 interpreters, and each one is charged against 224``max_binfmt_misc_interpreters`` like any other binding. A write past either 225limit is refused with ``-ENOSPC``. 226 227The ``load`` program then selects one per exec by name with the 228``bpf_binprm_select_interp()`` kfunc, and every exec runs a clone of the 229file that was opened. The path decides which file is bound and nothing 230else: it is not resolved again, in any namespace, so what it holds later - 231or what it holds in the namespace of whoever runs the binary - no longer 232decides anything. 233 234Enabling the entry ends this. Its interpreters are read at exec time with 235nothing but a reference held on the entry, so an entry that has ever been 236matchable can never have its set changed again: the first ``1`` seals it, 237from then on ``+`` is refused with ``-EBUSY``, and an entry registered 238without ``D`` is sealed from the start. Binding a name twice is refused 239with ``-EEXIST``. 240 241Selection is by name so that the configuration and the program need not 242agree on an order, and so that a handler is not tied to where a distribution 243puts its interpreters. A name is a single word of printable ASCII, at most 24432 characters; a name the entry did not bind gives the program ``-ENOENT``, 245which it can act on or return. The interpreter runs under the path it was 246registered under, and the entry reports what it bound:: 247 248 $ cat /proc/sys/fs/binfmt_misc/qemu 249 enabled 250 bpf my_handler 251 bpf-interpreter aarch64 /usr/bin/qemu-aarch64 252 bpf-interpreter arm /usr/bin/qemu-arm 253 flags: 254 255The path reported is the one the interpreter was bound under, which named 256the file at that moment; it is not re-resolved, so it is a record of what 257was bound rather than a promise about what that path holds now. 258 259The ``load`` program can also pass a single argument to the interpreter with 260the ``bpf_binprm_set_interp_arg()`` kfunc. It is inserted between the 261interpreter and the binary, exactly like the optional argument of a ``#!`` 262interpreter line, e.g. for a handler that resolves ``$ORIGIN`` in a script's 263``#!`` path and needs to preserve the argument that followed it. 264 265The invocation flags a static entry fixes at registration - ``P``, ``C``, 266``O``, ``T`` and ``L`` - are per-exec choices for a bpf handler, made by the 267``load`` program with the ``bpf_binprm_set_flags()`` kfunc, so a single 268handler can decide them differently for each binary it handles: 269 270- ``BPF_BINPRM_PRESERVE_ARGV0`` keeps the caller's ``argv[0]`` (the ``P`` 271 flag). 272- ``BPF_BINPRM_CREDENTIALS`` computes credentials from the binary (the ``C`` 273 flag), bounded to user namespaces that map the binary's owner just like 274 any other setuid exec. 275- ``BPF_BINPRM_EXECFD`` opens the binary on the interpreter's behalf and 276 passes it through the ``AT_EXECFD`` aux vector entry (the ``O`` flag), so 277 the interpreter can run binaries it could not open by path. 278- ``BPF_BINPRM_TRANSPARENT`` runs the interpreter transparently (the ``T`` 279 flag): the binary is handed over through ``AT_EXECFD`` as 280 with ``BPF_BINPRM_EXECFD``, but the argument vector is also left as the 281 caller passed it. An interpreter that loads the binary from ``AT_EXECFD`` 282 then appears in ``argv[0]`` and ``/proc/pid/cmdline`` as a direct 283 execution of the binary. ``BPF_BINPRM_PRESERVE_ARGV0`` and a staged 284 interpreter argument are rejected in combination with it, just as ``P`` 285 is with ``T``. It also lets a handler 286 run a binary passed as an inaccessible ``O_CLOEXEC`` file descriptor to 287 ``execveat()``, which a path-splicing dispatch cannot: the interpreter 288 has no path by which to open it. 289- ``BPF_BINPRM_LOADER`` substitutes the interpreter for the binary's 290 ``PT_INTERP`` and runs the binary as a fully native exec (the ``L`` 291 flag). It excludes the other flags and a staged interpreter argument. 292 293Because these are program choices, a ``B`` entry carries no invocation 294flags in the register string; ``F`` has none to spell for it either, since 295the interpreters it binds already pre-open what ``F`` would. The 296registration directive ``D`` is the exception: it decides how the entry 297starts out, not how the interpreter is invoked. 298 299A handler is looked up only in the user namespace the struct_ops map was 300registered in. Handlers are not inherited, so an entry can only reference a 301handler registered in the same user namespace as its binfmt_misc instance. 302The entry keeps the handler alive; deleting the struct_ops map only prevents 303new activations. 304 305 306Transparent interpreters 307------------------------ 308 309With the ``T`` flag or ``BPF_BINPRM_TRANSPARENT`` the dispatch is invisible 310to the resulting process. The argument vector is left exactly as the caller 311built it. The binary is passed through ``AT_EXECFD``. The kernel also labels 312``/proc/pid/exe`` correctly. The binary's file is write-denied while the 313process runs and the interpreter's is not, exactly as if the binary had been 314executed directly. A transparent entry does not change how credentials are 315derived. As 316with any other entry, set*id bits of the binary are only honored with ``C`` (or 317``BPF_BINPRM_CREDENTIALS``). 318 319The interpreter has to be built for this contract. The kernel announces it 320with ``AT_FLAGS_TRANSPARENT_INTERP`` in the ``AT_FLAGS`` aux vector entry 321next to ``AT_EXECFD``. The argument vector belongs entirely to the program, 322nothing was spliced in, so the interpreter doesn't consume arguments and 323simply loads the program from the descriptor. The bit is also the loader's 324license to finish the identity. After mapping the program it may retarget the 325``AT_PHDR``/``AT_ENTRY``/``AT_BASE`` entries of ``/proc/pid/auxv`` and the 326code/data statistics markers via one ``PR_SET_MM_MAP`` which completes 327what attaching debuggers observe. What remains visibly different from a 328direct execution is the address space layout. The interpreter occupies 329the main-image position and the program lives in the mmap region. 330 331 332Loader substitution 333------------------- 334 335The ``L`` flag turns the execution model around. Instead of running the 336registered interpreter with the binary as its payload the kernel loads 337the matched binary itself as the main image and substitutes the registered 338interpreter for the loader named in the binary's ``PT_INTERP``. 339 340Because the exec is native, there is no dispatch identity to 341reconstruct and no contract the substitute has to implement. A stock 342dynamic loader works unchanged. The argument vector is untouched, 343credentials and ``AT_SECURE`` derive from the binary, there is no 344``AT_EXECFD`` and no marker in the aux vector, the binary sits in the 345main-image slot with the native brk placement so ``/proc/pid/maps``, 346core dumps and perf mmap records have the native shape, and the 347identity is already complete when ``PTRACE_EVENT_EXEC`` stops the 348tracee. So launching under a debugger works, not just attaching. ``L`` 349entries are for ELF binaries of a native architecture. Foreign-arch 350emulation and non-ELF payloads remain the domain of the classic and 351transparent modes. 352 353The override applies when the format that finally claims the file is 354ELF with a ``PT_INTERP``. A matched binary without one or an 355interpreter-less ``ET_DYN`` drops the override and runs natively. A file 356claimed by another format - a ``#!`` script, say - is handled by that 357format as if the entry had not matched. ``L`` is therefore not an 358enforcement mechanism: it decides how a binary that asks for a loader is 359run, it does not guarantee that everything matching the entry runs under 360the substitute. A format that cannot consume the override at all instead 361refuses the exec with ``ENOEXEC`` before the point of no return. 362 363A wrong-architecture ELF fails the whole exec with ``ENOEXEC`` exactly 364as if no entry had matched. A substitute that is not ELF of the right 365architecture fails with ``ELIBBAD``. The usual ``PT_INTERP`` sanity 366checks on the binary still apply. But the segment's content is otherwise 367irrelevant. 368 369``L`` rejects the classic-dispatch flags ``T``, ``P``, ``O`` and ``C`` 370at registration. ``F`` composes and is valuable: with it the substitute 371is opened at registration time, so later mount namespace or path changes 372cannot redirect it. Without it the substitute is opened when the binary 373is executed, and the path is resolved in the mount namespace and root of 374whoever runs the binary, which is why it has to be absolute. As with 375``C``, register only trusted interpreters. The substituted loader runs 376with credentials derived from the binary. 377 378 379Hints 380----- 381 382If you want to pass special arguments to your interpreter, you can 383write a wrapper script for it. 384See :doc:`Documentation/admin-guide/java.rst <./java>` for an example. 385 386Your interpreter should NOT look in the PATH for the filename; the kernel 387passes it the full filename (or the file descriptor) to use. Using ``$PATH`` can 388cause unexpected behaviour and can be a security hazard. 389 390 391Richard Günther <rguenth@tat.physik.uni-tuebingen.de> 392