1Kernel Support for miscellaneous Binary Formats (binfmt_misc) 2============================================================= 3 4This Kernel feature allows you to invoke almost (for restrictions see below) 5every program by simply typing its name in the shell. 6This includes for example compiled Java(TM), Python or Emacs programs. 7 8To achieve this you must tell binfmt_misc which interpreter has to be invoked 9with which binary. Binfmt_misc recognises the binary-type by matching some bytes 10at the beginning of the file with a magic byte sequence (masking out specified 11bits) you have supplied. Binfmt_misc can also recognise a filename extension 12aka ``.com`` or ``.exe``. 13 14First you must mount binfmt_misc:: 15 16 mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc 17 18To actually register a new binary type, you have to set up a string looking like 19``:name:type:offset:magic:mask:interpreter:flags`` (where you can choose the 20``:`` upon your needs) and echo it to ``/proc/sys/fs/binfmt_misc/register``. 21 22The first character of the string is its field delimiter and can be any 23ASCII punctuation character other than the backslash ``\``. 24 25Here is what the fields mean: 26 27- ``name`` 28 is an identifier string. A new /proc file will be created with this 29 name below ``/proc/sys/fs/binfmt_misc``; cannot contain slashes ``/`` for 30 obvious reasons. 31- ``type`` 32 is the type of recognition. Give ``M`` for magic, ``E`` for extension and 33 ``B`` for a bpf-backed handler (see below). 34- ``offset`` 35 is the offset of the magic/mask in the file, counted in bytes. This 36 defaults to 0 if you omit it (i.e. you write ``:name:type::magic...``). 37 Ignored when using filename extension matching. 38- ``magic`` 39 is the byte sequence binfmt_misc is matching for. The magic string 40 may contain hex-encoded characters like ``\x0a`` or ``\xA4``. Note that you 41 must escape any NUL bytes; parsing halts at the first one. In a shell 42 environment you might have to write ``\\x0a`` to prevent the shell from 43 eating your ``\``. 44 If you chose filename extension matching, this is the extension to be 45 recognised (without the ``.``, the ``\x0a`` specials are not allowed). 46 Extension matching is case sensitive, and slashes ``/`` are not allowed! 47- ``mask`` 48 is an (optional, defaults to all 0xff) mask. You can mask out some 49 bits from matching by supplying a string like magic and as long as magic. 50 The mask is anded with the byte sequence of the file. Note that you must 51 escape any NUL bytes; parsing halts at the first one. Ignored when using 52 filename extension matching. 53- ``interpreter`` 54 is the program that should be invoked with the binary as first 55 argument (specify the full path). For ``B`` entries this field 56 carries the name of the bpf handler instead (see below). 57- ``flags`` 58 is an optional field that controls several aspects of the invocation 59 of the interpreter. It is a string of capital letters, each controls a 60 certain aspect. The following flags are supported: 61 62 ``P`` - preserve-argv[0] 63 Legacy behavior of binfmt_misc is to overwrite 64 the original argv[0] with the full path to the binary. When this 65 flag is included, binfmt_misc will add an argument to the argument 66 vector for this purpose, thus preserving the original ``argv[0]``. 67 e.g. If your interp is set to ``/bin/foo`` and you run ``blah`` 68 (which is in ``/usr/local/bin``), then the kernel will execute 69 ``/bin/foo`` with ``argv[]`` set to ``["/bin/foo", "/usr/local/bin/blah", "blah"]``. The interp has to be aware of this so it can 70 execute ``/usr/local/bin/blah`` 71 with ``argv[]`` set to ``["blah"]``. 72 ``O`` - open-binary 73 Legacy behavior of binfmt_misc is to pass the full path 74 of the binary to the interpreter as an argument. When this flag is 75 included, binfmt_misc will open the file for reading and pass its 76 descriptor into the auxilary vector with the key "AT_EXECFD", thus 77 allowing the interpreter to execute non-readable binaries. This 78 feature should be used with care - the interpreter has to be trusted 79 not to emit the contents of the non-readable binary. 80 ``C`` - credentials 81 Currently, the behavior of binfmt_misc is to calculate 82 the credentials and security token of the new process according to 83 the interpreter. When this flag is included, these attributes are 84 calculated according to the binary. It also implies the ``O`` flag. 85 This feature should be used with care as the interpreter 86 will run with root permissions when a setuid binary owned by root 87 is run with binfmt_misc. 88 ``F`` - fix binary 89 The usual behaviour of binfmt_misc is to spawn the 90 binary lazily when the misc format file is invoked. However, 91 this doesn't work very well in the face of mount namespaces and 92 changeroots, so the ``F`` mode opens the binary as soon as the 93 emulation is installed and uses the opened image to spawn the 94 emulator, meaning it is always available once installed, 95 regardless of how the environment changes. 96 ``T`` - transparent 97 Run the interpreter transparently. The binary is handed to 98 the interpreter through ``AT_EXECFD`` (``T`` implies ``O``), 99 the argument vector is left exactly as the caller built it 100 and the kernel labels ``/proc/pid/exe`` with the binary 101 instead of the interpreter. The interpreter has to load the 102 binary from ``AT_EXECFD`` and follow the 103 ``AT_FLAGS_TRANSPARENT_INTERP`` contract. Combining ``T`` 104 with ``P`` is rejected: transparency preserves the whole 105 argument vector, argv[0] included. 106 ``L`` - loader substitution 107 Do not run the interpreter on the binary at all: load the 108 binary itself as a fully native exec and substitute the 109 interpreter for the loader named in the binary's 110 ``PT_INTERP``. See the "Loader substitution" section 111 below. ``L`` rejects ``T``, ``P``, ``O`` and ``C``; 112 ``F`` composes. 113 ``D`` - registered disabled 114 The entry is created disabled instead of being matchable at 115 once, and has to be enabled by writing ``1`` to its file 116 before it dispatches anything. This splits a registration 117 into creating the entry and activating it, leaving room to 118 configure it in between - which is what a ``B`` entry that 119 binds interpreters needs; see the bpf section below. The flag 120 is spent on the registration and is not read back: what an 121 entry file reports afterwards is whether it is enabled. 122 123 124There are some restrictions: 125 126 - the whole register string may not exceed 1920 characters 127 - the magic must reside in the first 128 bytes of the file, i.e. 128 offset+size(magic) has to be less than 128 129 - the interpreter string may not exceed 127 characters 130 - an interpreter used with ``C`` or ``L`` but without ``F`` has to be 131 named by an absolute path. It is opened when the binary is executed, so 132 a relative one would be resolved against the working directory of 133 whoever runs the binary 134 - the amount of pre-opened interpreters by ``F``, or bound to a ``B`` entry 135 is limited by the ``/proc/sys/user/max_binfmt_misc_interpreters`` sysctl. A 136 registration past the limit is refused with ``-ENOSPC``. This limits an 137 unprivileged namespace pinning files. A nested namespace can raise only its 138 own limit and every ancestor is charged too 139 140 141To use binfmt_misc you have to mount it first. You can mount it with 142``mount -t binfmt_misc none /proc/sys/fs/binfmt_misc`` command, or you can add 143a line ``none /proc/sys/fs/binfmt_misc binfmt_misc defaults 0 0`` to your 144``/etc/fstab`` so it auto mounts on boot. 145 146You may want to add the binary formats in one of your ``/etc/rc`` scripts during 147boot-up. Read the manual of your init program to figure out how to do this 148right. 149 150Think about the order of adding entries! Later added entries are matched first! 151 152 153A few examples (assumed you are in ``/proc/sys/fs/binfmt_misc``): 154 155- enable support for em86 (like binfmt_em86, for Alpha AXP only):: 156 157 echo ':i386:M::\x7fELF\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03:\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfb\xff\xff:/bin/em86:' > register 158 echo ':i486:M::\x7fELF\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x06:\xff\xff\xff\xff\xff\xfe\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfb\xff\xff:/bin/em86:' > register 159 160- enable support for packed DOS applications (pre-configured dosemu hdimages):: 161 162 echo ':DEXE:M::\x0eDEX::/usr/bin/dosexec:' > register 163 164- enable support for Windows executables using wine:: 165 166 echo ':DOSWin:M::MZ::/usr/local/bin/wine:' > register 167 168For java support see Documentation/admin-guide/java.rst 169 170 171You can enable/disable binfmt_misc or one binary type by echoing 0 (to disable) 172or 1 (to enable) to ``/proc/sys/fs/binfmt_misc/status`` or 173``/proc/.../the_name``. 174Catting the file tells you the current status of ``binfmt_misc/the_entry``. 175 176You can remove one entry or all entries by echoing -1 to ``/proc/.../the_name`` 177or ``/proc/sys/fs/binfmt_misc/status``. A single entry can also be removed 178by simply unlinking (``rm``) ``/proc/.../the_name``. 179 180 181bpf-backed handlers 182------------------- 183 184With ``CONFIG_BINFMT_MISC_BPF`` both the matching and the interpreter 185selection can be delegated to bpf programs. A handler is an instance of the 186``binfmt_misc_ops`` struct_ops with a ``match`` and a ``load`` program and a 187``name``. Once the struct_ops map is registered the handler can be activated 188with a ``B`` entry that references it by name in the ``interpreter`` field 189and carries neither offset, magic, nor mask:: 190 191 echo ':qemu:B::::my_handler:' > register 192 193Both programs receive the ``linux_binprm`` of the binary and both can 194sleep. The ``match`` program decides whether the handler applies: it is 195consulted during the entry walk exactly like magic and extension matching, 196in the same registration order with the same first-match-wins semantics. 197Unlike static matching it is not limited to the prefetched first bytes of 198the file in ``bprm->buf``: it can read the file, e.g. to parse ELF program 199headers whose data sits at arbitrary offsets. It only decides, though: the 200selection kfuncs below are rejected in it. The ``load`` program of the 201matched handler then selects the interpreter: it can equally read the file 202and derive the interpreter from the binary's location. It selects the 203interpreter by calling the ``bpf_binprm_set_interp()`` kfunc with an 204absolute path and returning ``0``. A match is committed: a failing 205``load`` fails the exec with its error instead of falling through to later 206entries; ``-ENOEXEC`` lets the remaining binary formats have a go. A path 207selected this way is opened with the credentials of the task doing the 208exec, exactly as a statically registered interpreter without ``F`` would 209be. 210 211An entry can instead bind the interpreters its handler may use, so that no 212path is resolved at exec time at all. An entry registered with ``D`` is not 213matchable yet, which is what leaves it open to being given them, one 214``+name path`` write at a time:: 215 216 echo ':qemu:B::::my_handler:D' > register 217 echo '+aarch64 /usr/bin/qemu-aarch64' > qemu 218 echo '+arm /usr/bin/qemu-arm' > qemu 219 echo 1 > qemu 220 221Each path is opened during its write, in the writing process's context and 222with the credentials the entry file was opened with, exactly the way ``F`` 223pre-opens a static entry's interpreter; the paths must be absolute. The 224path is everything past the first space, so there is nothing it cannot 225express, and no interpreter has to fit in a register string. An entry 226binds at most 100 interpreters, and each one is charged against 227``max_binfmt_misc_interpreters`` like any other binding. A write past either 228limit is refused with ``-ENOSPC``. 229 230The ``load`` program then selects one per exec by name with the 231``bpf_binprm_select_interp()`` kfunc, and every exec runs a clone of the 232file that was opened. The path decides which file is bound and nothing 233else: it is not resolved again, in any namespace, so what it holds later - 234or what it holds in the namespace of whoever runs the binary - no longer 235decides anything. 236 237Enabling the entry ends this. Its interpreters are read at exec time with 238nothing but a reference held on the entry, so an entry that has ever been 239matchable can never have its set changed again: the first ``1`` seals it, 240from then on ``+`` is refused with ``-EBUSY``, and an entry registered 241without ``D`` is sealed from the start. Binding a name twice is refused 242with ``-EEXIST``. 243 244Selection is by name so that the configuration and the program need not 245agree on an order, and so that a handler is not tied to where a distribution 246puts its interpreters. A name is a single word of printable ASCII, at most 24732 characters; a name the entry did not bind gives the program ``-ENOENT``, 248which it can act on or return. The interpreter runs under the path it was 249registered under, and the entry reports what it bound:: 250 251 $ cat /proc/sys/fs/binfmt_misc/qemu 252 enabled 253 bpf my_handler 254 bpf-interpreter aarch64 /usr/bin/qemu-aarch64 255 bpf-interpreter arm /usr/bin/qemu-arm 256 flags: 257 258The path reported is the one the interpreter was bound under, which named 259the file at that moment; it is not re-resolved, so it is a record of what 260was bound rather than a promise about what that path holds now. 261 262The ``load`` program can also pass a single argument to the interpreter with 263the ``bpf_binprm_set_interp_arg()`` kfunc. It is inserted between the 264interpreter and the binary, exactly like the optional argument of a ``#!`` 265interpreter line, e.g. for a handler that resolves ``$ORIGIN`` in a script's 266``#!`` path and needs to preserve the argument that followed it. 267 268The invocation flags a static entry fixes at registration - ``P``, ``C``, 269``O``, ``T`` and ``L`` - are per-exec choices for a bpf handler, made by the 270``load`` program with the ``bpf_binprm_set_flags()`` kfunc, so a single 271handler can decide them differently for each binary it handles: 272 273- ``BPF_BINPRM_PRESERVE_ARGV0`` keeps the caller's ``argv[0]`` (the ``P`` 274 flag). 275- ``BPF_BINPRM_CREDENTIALS`` computes credentials from the binary (the ``C`` 276 flag), bounded to user namespaces that map the binary's owner just like 277 any other setuid exec. 278- ``BPF_BINPRM_EXECFD`` opens the binary on the interpreter's behalf and 279 passes it through the ``AT_EXECFD`` aux vector entry (the ``O`` flag), so 280 the interpreter can run binaries it could not open by path. 281- ``BPF_BINPRM_TRANSPARENT`` runs the interpreter transparently (the ``T`` 282 flag): the binary is handed over through ``AT_EXECFD`` as 283 with ``BPF_BINPRM_EXECFD``, but the argument vector is also left as the 284 caller passed it. An interpreter that loads the binary from ``AT_EXECFD`` 285 then appears in ``argv[0]`` and ``/proc/pid/cmdline`` as a direct 286 execution of the binary. ``BPF_BINPRM_PRESERVE_ARGV0`` and a staged 287 interpreter argument are rejected in combination with it, just as ``P`` 288 is with ``T``. It also lets a handler 289 run a binary passed as an inaccessible ``O_CLOEXEC`` file descriptor to 290 ``execveat()``, which a path-splicing dispatch cannot: the interpreter 291 has no path by which to open it. 292- ``BPF_BINPRM_LOADER`` substitutes the interpreter for the binary's 293 ``PT_INTERP`` and runs the binary as a fully native exec (the ``L`` 294 flag). It excludes the other flags and a staged interpreter argument. 295 296Because these are program choices, a ``B`` entry carries no invocation 297flags in the register string; ``F`` has none to spell for it either, since 298the interpreters it binds already pre-open what ``F`` would. The 299registration directive ``D`` is the exception: it decides how the entry 300starts out, not how the interpreter is invoked. 301 302A handler is looked up only in the user namespace the struct_ops map was 303registered in. Handlers are not inherited, so an entry can only reference a 304handler registered in the same user namespace as its binfmt_misc instance. 305The entry keeps the handler alive; deleting the struct_ops map only prevents 306new activations. 307 308 309Transparent interpreters 310------------------------ 311 312With the ``T`` flag or ``BPF_BINPRM_TRANSPARENT`` the dispatch is invisible 313to the resulting process. The argument vector is left exactly as the caller 314built it. The binary is passed through ``AT_EXECFD``. The kernel also labels 315``/proc/pid/exe`` correctly. The binary's file is write-denied while the 316process runs and the interpreter's is not, exactly as if the binary had been 317executed directly. A transparent entry does not change how credentials are 318derived. As 319with any other entry, set*id bits of the binary are only honored with ``C`` (or 320``BPF_BINPRM_CREDENTIALS``). 321 322The interpreter has to be built for this contract. The kernel announces it 323with ``AT_FLAGS_TRANSPARENT_INTERP`` in the ``AT_FLAGS`` aux vector entry 324next to ``AT_EXECFD``. The argument vector belongs entirely to the program, 325nothing was spliced in, so the interpreter doesn't consume arguments and 326simply loads the program from the descriptor. The bit is also the loader's 327license to finish the identity. After mapping the program it may retarget the 328``AT_PHDR``/``AT_ENTRY``/``AT_BASE`` entries of ``/proc/pid/auxv`` and the 329code/data statistics markers via one ``PR_SET_MM_MAP`` which completes 330what attaching debuggers observe. What remains visibly different from a 331direct execution is the address space layout. The interpreter occupies 332the main-image position and the program lives in the mmap region. 333 334 335Loader substitution 336------------------- 337 338The ``L`` flag turns the execution model around. Instead of running the 339registered interpreter with the binary as its payload the kernel loads 340the matched binary itself as the main image and substitutes the registered 341interpreter for the loader named in the binary's ``PT_INTERP``. 342 343Because the exec is native, there is no dispatch identity to 344reconstruct and no contract the substitute has to implement. A stock 345dynamic loader works unchanged. The argument vector is untouched, 346credentials and ``AT_SECURE`` derive from the binary, there is no 347``AT_EXECFD`` and no marker in the aux vector, the binary sits in the 348main-image slot with the native brk placement so ``/proc/pid/maps``, 349core dumps and perf mmap records have the native shape, and the 350identity is already complete when ``PTRACE_EVENT_EXEC`` stops the 351tracee. So launching under a debugger works, not just attaching. ``L`` 352entries are for ELF binaries of a native architecture. Foreign-arch 353emulation and non-ELF payloads remain the domain of the classic and 354transparent modes. 355 356The override applies when the format that finally claims the file is 357ELF with a ``PT_INTERP``. A matched binary without one or an 358interpreter-less ``ET_DYN`` drops the override and runs natively. A file 359claimed by another format - a ``#!`` script, say - is handled by that 360format as if the entry had not matched. ``L`` is therefore not an 361enforcement mechanism: it decides how a binary that asks for a loader is 362run, it does not guarantee that everything matching the entry runs under 363the substitute. A format that cannot consume the override at all instead 364refuses the exec with ``ENOEXEC`` before the point of no return. 365 366A wrong-architecture ELF fails the whole exec with ``ENOEXEC`` exactly 367as if no entry had matched. A substitute that is not ELF of the right 368architecture fails with ``ELIBBAD``. The usual ``PT_INTERP`` sanity 369checks on the binary still apply. But the segment's content is otherwise 370irrelevant. 371 372``L`` rejects the classic-dispatch flags ``T``, ``P``, ``O`` and ``C`` 373at registration. ``F`` composes and is valuable: with it the substitute 374is opened at registration time, so later mount namespace or path changes 375cannot redirect it. Without it the substitute is opened when the binary 376is executed, and the path is resolved in the mount namespace and root of 377whoever runs the binary, which is why it has to be absolute. As with 378``C``, register only trusted interpreters. The substituted loader runs 379with credentials derived from the binary. 380 381 382Hints 383----- 384 385If you want to pass special arguments to your interpreter, you can 386write a wrapper script for it. 387See :doc:`Documentation/admin-guide/java.rst <./java>` for an example. 388 389Your interpreter should NOT look in the PATH for the filename; the kernel 390passes it the full filename (or the file descriptor) to use. Using ``$PATH`` can 391cause unexpected behaviour and can be a security hazard. 392 393 394Richard Günther <rguenth@tat.physik.uni-tuebingen.de> 395