1.. This file is dual-licensed: you can use it either under the terms 2.. of the GPL 2.0 or the GFDL 1.1+ license, at your option. Note that this 3.. dual licensing only applies to this file, and not this project as a 4.. whole. 5.. 6.. a) This file is free software; you can redistribute it and/or 7.. modify it under the terms of the GNU General Public License as 8.. published by the Free Software Foundation version 2 of 9.. the License. 10.. 11.. This file is distributed in the hope that it will be useful, 12.. but WITHOUT ANY WARRANTY; without even the implied warranty of 13.. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14.. GNU General Public License for more details. 15.. 16.. Or, alternatively, 17.. 18.. b) Permission is granted to copy, distribute and/or modify this 19.. document under the terms of the GNU Free Documentation License, 20.. Version 1.1 or any later version published by the Free Software 21.. Foundation, with no Invariant Sections, no Front-Cover Texts 22.. and no Back-Cover Texts. A copy of the license is included at 23.. Documentation/userspace-api/media/fdl-appendix.rst. 24.. 25.. TODO: replace it to GPL-2.0 OR GFDL-1.1-or-later WITH no-invariant-sections 26 27.. _encoder: 28 29************************************************* 30Memory-to-Memory Stateful Video Encoder Interface 31************************************************* 32 33A stateful video encoder takes raw video frames in display order and encodes 34them into a bytestream. It generates complete chunks of the bytestream, including 35all metadata, headers, etc. The resulting bytestream does not require any 36further post-processing by the client. 37 38Performing software stream processing, header generation etc. in the driver 39in order to support this interface is strongly discouraged. In case such 40operations are needed, use of the Stateless Video Encoder Interface (in 41development) is strongly advised. 42 43Conventions and Notations Used in This Document 44=============================================== 45 461. The general V4L2 API rules apply if not specified in this document 47 otherwise. 48 492. The meaning of words "must", "may", "should", etc. is as per `RFC 50 2119 <https://tools.ietf.org/html/rfc2119>`_. 51 523. All steps not marked "optional" are required. 53 544. :c:func:`VIDIOC_G_EXT_CTRLS` and :c:func:`VIDIOC_S_EXT_CTRLS` may be used 55 interchangeably with :c:func:`VIDIOC_G_CTRL` and :c:func:`VIDIOC_S_CTRL`, 56 unless specified otherwise. 57 585. Single-planar API (see :ref:`planar-apis`) and applicable structures may be 59 used interchangeably with multi-planar API, unless specified otherwise, 60 depending on encoder capabilities and following the general V4L2 guidelines. 61 626. i = [a..b]: sequence of integers from a to b, inclusive, i.e. i = 63 [0..2]: i = 0, 1, 2. 64 657. Given an ``OUTPUT`` buffer A, then A' represents a buffer on the ``CAPTURE`` 66 queue containing data that resulted from processing buffer A. 67 68Glossary 69======== 70 71Refer to :ref:`decoder-glossary`. 72 73State Machine 74============= 75 76.. kernel-render:: DOT 77 :alt: DOT digraph of encoder state machine 78 :caption: Encoder State Machine 79 80 digraph encoder_state_machine { 81 node [shape = doublecircle, label="Encoding"] Encoding; 82 83 node [shape = circle, label="Initialization"] Initialization; 84 node [shape = circle, label="Stopped"] Stopped; 85 node [shape = circle, label="Drain"] Drain; 86 node [shape = circle, label="Reset"] Reset; 87 88 node [shape = point]; qi 89 qi -> Initialization [ label = "open()" ]; 90 91 Initialization -> Encoding [ label = "Both queues streaming" ]; 92 93 Encoding -> Drain [ label = "V4L2_ENC_CMD_STOP" ]; 94 Encoding -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ]; 95 Encoding -> Stopped [ label = "VIDIOC_STREAMOFF(OUTPUT)" ]; 96 Encoding -> Encoding; 97 98 Drain -> Stopped [ label = "All CAPTURE\nbuffers dequeued\nor\nVIDIOC_STREAMOFF(OUTPUT)" ]; 99 Drain -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ]; 100 101 Reset -> Encoding [ label = "VIDIOC_STREAMON(CAPTURE)" ]; 102 Reset -> Initialization [ label = "VIDIOC_REQBUFS(OUTPUT, 0)" ]; 103 104 Stopped -> Encoding [ label = "V4L2_ENC_CMD_START\nor\nVIDIOC_STREAMON(OUTPUT)" ]; 105 Stopped -> Reset [ label = "VIDIOC_STREAMOFF(CAPTURE)" ]; 106 } 107 108Querying Capabilities 109===================== 110 1111. To enumerate the set of coded formats supported by the encoder, the 112 client may call :c:func:`VIDIOC_ENUM_FMT` on ``CAPTURE``. 113 114 * The full set of supported formats will be returned, regardless of the 115 format set on ``OUTPUT``. 116 1172. To enumerate the set of supported raw formats, the client may call 118 :c:func:`VIDIOC_ENUM_FMT` on ``OUTPUT``. 119 120 * Only the formats supported for the format currently active on ``CAPTURE`` 121 will be returned. 122 123 * In order to enumerate raw formats supported by a given coded format, 124 the client must first set that coded format on ``CAPTURE`` and then 125 enumerate the formats on ``OUTPUT``. 126 1273. The client may use :c:func:`VIDIOC_ENUM_FRAMESIZES` to detect supported 128 resolutions for a given format, passing the desired pixel format in 129 :c:type:`v4l2_frmsizeenum` ``pixel_format``. 130 131 * Values returned by :c:func:`VIDIOC_ENUM_FRAMESIZES` for a coded pixel 132 format will include all possible coded resolutions supported by the 133 encoder for the given coded pixel format. 134 135 * Values returned by :c:func:`VIDIOC_ENUM_FRAMESIZES` for a raw pixel format 136 will include all possible frame buffer resolutions supported by the 137 encoder for the given raw pixel format and coded format currently set on 138 ``CAPTURE``. 139 1404. The client may use :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` to detect supported 141 frame intervals for a given format and resolution, passing the desired pixel 142 format in :c:type:`v4l2_frmsizeenum` ``pixel_format`` and the resolution 143 in :c:type:`v4l2_frmsizeenum` ``width`` and :c:type:`v4l2_frmsizeenum` 144 ``height``. 145 146 * Values returned by :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` for a coded pixel 147 format and coded resolution will include all possible frame intervals 148 supported by the encoder for the given coded pixel format and resolution. 149 150 * Values returned by :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` for a raw pixel 151 format and resolution will include all possible frame intervals supported 152 by the encoder for the given raw pixel format and resolution and for the 153 coded format, coded resolution and coded frame interval currently set on 154 ``CAPTURE``. 155 156 * Support for :c:func:`VIDIOC_ENUM_FRAMEINTERVALS` is optional. If it is 157 not implemented, then there are no special restrictions other than the 158 limits of the codec itself. 159 1605. Supported profiles and levels for the coded format currently set on 161 ``CAPTURE``, if applicable, may be queried using their respective controls 162 via :c:func:`VIDIOC_QUERYCTRL`. 163 1646. Any additional encoder capabilities may be discovered by querying 165 their respective controls. 166 167Initialization 168============== 169 1701. Set the coded format on the ``CAPTURE`` queue via :c:func:`VIDIOC_S_FMT`. 171 172 * **Required fields:** 173 174 ``type`` 175 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 176 177 ``pixelformat`` 178 the coded format to be produced. 179 180 ``sizeimage`` 181 desired size of ``CAPTURE`` buffers; the encoder may adjust it to 182 match hardware requirements. 183 184 ``width``, ``height`` 185 ignored (read-only). 186 187 other fields 188 follow standard semantics. 189 190 * **Return fields:** 191 192 ``sizeimage`` 193 adjusted size of ``CAPTURE`` buffers. 194 195 ``width``, ``height`` 196 the coded size selected by the encoder based on current state, e.g. 197 ``OUTPUT`` format, selection rectangles, etc. (read-only). 198 199 .. important:: 200 201 Changing the ``CAPTURE`` format may change the currently set ``OUTPUT`` 202 format. How the new ``OUTPUT`` format is determined is up to the encoder 203 and the client must ensure it matches its needs afterwards. 204 2052. **Optional.** Enumerate supported ``OUTPUT`` formats (raw formats for 206 source) for the selected coded format via :c:func:`VIDIOC_ENUM_FMT`. 207 208 * **Required fields:** 209 210 ``type`` 211 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 212 213 other fields 214 follow standard semantics. 215 216 * **Return fields:** 217 218 ``pixelformat`` 219 raw format supported for the coded format currently selected on 220 the ``CAPTURE`` queue. 221 222 other fields 223 follow standard semantics. 224 2253. Set the raw source format on the ``OUTPUT`` queue via 226 :c:func:`VIDIOC_S_FMT`. 227 228 * **Required fields:** 229 230 ``type`` 231 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 232 233 ``pixelformat`` 234 raw format of the source. 235 236 ``width``, ``height`` 237 source resolution. 238 239 other fields 240 follow standard semantics. 241 242 * **Return fields:** 243 244 ``width``, ``height`` 245 may be adjusted to match encoder minimums, maximums and alignment 246 requirements, as required by the currently selected formats, as 247 reported by :c:func:`VIDIOC_ENUM_FRAMESIZES`. 248 249 other fields 250 follow standard semantics. 251 252 * Setting the ``OUTPUT`` format will reset the selection rectangles to their 253 default values, based on the new resolution, as described in the next 254 step. 255 2564. Set the raw frame interval on the ``OUTPUT`` queue via 257 :c:func:`VIDIOC_S_PARM`. This also sets the coded frame interval on the 258 ``CAPTURE`` queue to the same value. 259 260 * ** Required fields:** 261 262 ``type`` 263 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 264 265 ``parm.output`` 266 set all fields except ``parm.output.timeperframe`` to 0. 267 268 ``parm.output.timeperframe`` 269 the desired frame interval; the encoder may adjust it to 270 match hardware requirements. 271 272 * **Return fields:** 273 274 ``parm.output.timeperframe`` 275 the adjusted frame interval. 276 277 .. important:: 278 279 Changing the ``OUTPUT`` frame interval *also* sets the framerate that 280 the encoder uses to encode the video. So setting the frame interval 281 to 1/24 (or 24 frames per second) will produce a coded video stream 282 that can be played back at that speed. The frame interval for the 283 ``OUTPUT`` queue is just a hint, the application may provide raw 284 frames at a different rate. It can be used by the driver to help 285 schedule multiple encoders running in parallel. 286 287 In the next step the ``CAPTURE`` frame interval can optionally be 288 changed to a different value. This is useful for off-line encoding 289 were the coded frame interval can be different from the rate at 290 which raw frames are supplied. 291 292 .. important:: 293 294 ``timeperframe`` deals with *frames*, not fields. So for interlaced 295 formats this is the time per two fields, since a frame consists of 296 a top and a bottom field. 297 298 .. note:: 299 300 It is due to historical reasons that changing the ``OUTPUT`` frame 301 interval also changes the coded frame interval on the ``CAPTURE`` 302 queue. Ideally these would be independent settings, but that would 303 break the existing API. 304 3055. **Optional** Set the coded frame interval on the ``CAPTURE`` queue via 306 :c:func:`VIDIOC_S_PARM`. This is only necessary if the coded frame 307 interval is different from the raw frame interval, which is typically 308 the case for off-line encoding. Support for this feature is signalled 309 by the :ref:`V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL <fmtdesc-flags>` format flag. 310 311 * ** Required fields:** 312 313 ``type`` 314 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``CAPTURE``. 315 316 ``parm.capture`` 317 set all fields except ``parm.capture.timeperframe`` to 0. 318 319 ``parm.capture.timeperframe`` 320 the desired coded frame interval; the encoder may adjust it to 321 match hardware requirements. 322 323 * **Return fields:** 324 325 ``parm.capture.timeperframe`` 326 the adjusted frame interval. 327 328 .. important:: 329 330 Changing the ``CAPTURE`` frame interval sets the framerate for the 331 coded video. It does *not* set the rate at which buffers arrive on the 332 ``CAPTURE`` queue, that depends on how fast the encoder is and how 333 fast raw frames are queued on the ``OUTPUT`` queue. 334 335 .. important:: 336 337 ``timeperframe`` deals with *frames*, not fields. So for interlaced 338 formats this is the time per two fields, since a frame consists of 339 a top and a bottom field. 340 341 .. note:: 342 343 Not all drivers support this functionality, in that case just set 344 the desired coded frame interval for the ``OUTPUT`` queue. 345 346 However, drivers that can schedule multiple encoders based on the 347 ``OUTPUT`` frame interval must support this optional feature. 348 3496. **Optional.** Set the visible resolution for the stream metadata via 350 :c:func:`VIDIOC_S_SELECTION` on the ``OUTPUT`` queue if it is desired 351 to be different than the full OUTPUT resolution. 352 353 * **Required fields:** 354 355 ``type`` 356 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 357 358 ``target`` 359 set to ``V4L2_SEL_TGT_CROP``. 360 361 ``r.left``, ``r.top``, ``r.width``, ``r.height`` 362 visible rectangle; this must fit within the `V4L2_SEL_TGT_CROP_BOUNDS` 363 rectangle and may be subject to adjustment to match codec and 364 hardware constraints. 365 366 * **Return fields:** 367 368 ``r.left``, ``r.top``, ``r.width``, ``r.height`` 369 visible rectangle adjusted by the encoder. 370 371 * The following selection targets are supported on ``OUTPUT``: 372 373 ``V4L2_SEL_TGT_CROP_BOUNDS`` 374 equal to the full source frame, matching the active ``OUTPUT`` 375 format. 376 377 ``V4L2_SEL_TGT_CROP_DEFAULT`` 378 equal to ``V4L2_SEL_TGT_CROP_BOUNDS``. 379 380 ``V4L2_SEL_TGT_CROP`` 381 rectangle within the source buffer to be encoded into the 382 ``CAPTURE`` stream; defaults to ``V4L2_SEL_TGT_CROP_DEFAULT``. 383 384 .. note:: 385 386 A common use case for this selection target is encoding a source 387 video with a resolution that is not a multiple of a macroblock, 388 e.g. the common 1920x1080 resolution may require the source 389 buffers to be aligned to 1920x1088 for codecs with 16x16 macroblock 390 size. To avoid encoding the padding, the client needs to explicitly 391 configure this selection target to 1920x1080. 392 393 .. warning:: 394 395 The encoder may adjust the crop/compose rectangles to the nearest 396 supported ones to meet codec and hardware requirements. The client needs 397 to check the adjusted rectangle returned by :c:func:`VIDIOC_S_SELECTION`. 398 3997. Allocate buffers for both ``OUTPUT`` and ``CAPTURE`` via 400 :c:func:`VIDIOC_REQBUFS`. This may be performed in any order. 401 402 * **Required fields:** 403 404 ``count`` 405 requested number of buffers to allocate; greater than zero. 406 407 ``type`` 408 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT`` or 409 ``CAPTURE``. 410 411 other fields 412 follow standard semantics. 413 414 * **Return fields:** 415 416 ``count`` 417 actual number of buffers allocated. 418 419 .. warning:: 420 421 The actual number of allocated buffers may differ from the ``count`` 422 given. The client must check the updated value of ``count`` after the 423 call returns. 424 425 .. note:: 426 427 To allocate more than the minimum number of OUTPUT buffers (for pipeline 428 depth), the client may query the ``V4L2_CID_MIN_BUFFERS_FOR_OUTPUT`` 429 control to get the minimum number of buffers required, and pass the 430 obtained value plus the number of additional buffers needed in the 431 ``count`` field to :c:func:`VIDIOC_REQBUFS`. 432 433 Alternatively, :c:func:`VIDIOC_CREATE_BUFS` can be used to have more 434 control over buffer allocation. 435 436 * **Required fields:** 437 438 ``count`` 439 requested number of buffers to allocate; greater than zero. 440 441 ``type`` 442 a ``V4L2_BUF_TYPE_*`` enum appropriate for ``OUTPUT``. 443 444 other fields 445 follow standard semantics. 446 447 * **Return fields:** 448 449 ``count`` 450 adjusted to the number of allocated buffers. 451 4528. Begin streaming on both ``OUTPUT`` and ``CAPTURE`` queues via 453 :c:func:`VIDIOC_STREAMON`. This may be performed in any order. The actual 454 encoding process starts when both queues start streaming. 455 456.. note:: 457 458 If the client stops the ``CAPTURE`` queue during the encode process and then 459 restarts it again, the encoder will begin generating a stream independent 460 from the stream generated before the stop. The exact constraints depend 461 on the coded format, but may include the following implications: 462 463 * encoded frames produced after the restart must not reference any 464 frames produced before the stop, e.g. no long term references for 465 H.264/HEVC, 466 467 * any headers that must be included in a standalone stream must be 468 produced again, e.g. SPS and PPS for H.264/HEVC. 469 470Encoding 471======== 472 473This state is reached after the `Initialization` sequence finishes 474successfully. In this state, the client queues and dequeues buffers to both 475queues via :c:func:`VIDIOC_QBUF` and :c:func:`VIDIOC_DQBUF`, following the 476standard semantics. 477 478The content of encoded ``CAPTURE`` buffers depends on the active coded pixel 479format and may be affected by codec-specific extended controls, as stated 480in the documentation of each format. 481 482Both queues operate independently, following standard behavior of V4L2 buffer 483queues and memory-to-memory devices. In addition, the order of encoded frames 484dequeued from the ``CAPTURE`` queue may differ from the order of queuing raw 485frames to the ``OUTPUT`` queue, due to properties of the selected coded format, 486e.g. frame reordering. 487 488The client must not assume any direct relationship between ``CAPTURE`` and 489``OUTPUT`` buffers and any specific timing of buffers becoming 490available to dequeue. Specifically: 491 492* a buffer queued to ``OUTPUT`` may result in more than one buffer produced on 493 ``CAPTURE`` (for example, if returning an encoded frame allowed the encoder 494 to return a frame that preceded it in display, but succeeded it in the decode 495 order; however, there may be other reasons for this as well), 496 497* a buffer queued to ``OUTPUT`` may result in a buffer being produced on 498 ``CAPTURE`` later into encode process, and/or after processing further 499 ``OUTPUT`` buffers, or be returned out of order, e.g. if display 500 reordering is used, 501 502* buffers may become available on the ``CAPTURE`` queue without additional 503 buffers queued to ``OUTPUT`` (e.g. during drain or ``EOS``), because of the 504 ``OUTPUT`` buffers queued in the past whose encoding results are only 505 available at later time, due to specifics of the encoding process, 506 507* buffers queued to ``OUTPUT`` may not become available to dequeue instantly 508 after being encoded into a corresponding ``CAPTURE`` buffer, e.g. if the 509 encoder needs to use the frame as a reference for encoding further frames. 510 511.. note:: 512 513 To allow matching encoded ``CAPTURE`` buffers with ``OUTPUT`` buffers they 514 originated from, the client can set the ``timestamp`` field of the 515 :c:type:`v4l2_buffer` struct when queuing an ``OUTPUT`` buffer. The 516 ``CAPTURE`` buffer(s), which resulted from encoding that ``OUTPUT`` buffer 517 will have their ``timestamp`` field set to the same value when dequeued. 518 519 In addition to the straightforward case of one ``OUTPUT`` buffer producing 520 one ``CAPTURE`` buffer, the following cases are defined: 521 522 * one ``OUTPUT`` buffer generates multiple ``CAPTURE`` buffers: the same 523 ``OUTPUT`` timestamp will be copied to multiple ``CAPTURE`` buffers, 524 525 * the encoding order differs from the presentation order (i.e. the 526 ``CAPTURE`` buffers are out-of-order compared to the ``OUTPUT`` buffers): 527 ``CAPTURE`` timestamps will not retain the order of ``OUTPUT`` timestamps. 528 529.. note:: 530 531 To let the client distinguish between frame types (keyframes, intermediate 532 frames; the exact list of types depends on the coded format), the 533 ``CAPTURE`` buffers will have corresponding flag bits set in their 534 :c:type:`v4l2_buffer` struct when dequeued. See the documentation of 535 :c:type:`v4l2_buffer` and each coded pixel format for exact list of flags 536 and their meanings. 537 538Should an encoding error occur, it will be reported to the client with the level 539of details depending on the encoder capabilities. Specifically: 540 541* the ``CAPTURE`` buffer (if any) that contains the results of the failed encode 542 operation will be returned with the ``V4L2_BUF_FLAG_ERROR`` flag set, 543 544* if the encoder is able to precisely report the ``OUTPUT`` buffer(s) that triggered 545 the error, such buffer(s) will be returned with the ``V4L2_BUF_FLAG_ERROR`` flag 546 set. 547 548.. note:: 549 550 If a ``CAPTURE`` buffer is too small then it is just returned with the 551 ``V4L2_BUF_FLAG_ERROR`` flag set. More work is needed to detect that this 552 error occurred because the buffer was too small, and to provide support to 553 free existing buffers that were too small. 554 555In case of a fatal failure that does not allow the encoding to continue, any 556further operations on corresponding encoder file handle will return the -EIO 557error code. The client may close the file handle and open a new one, or 558alternatively reinitialize the instance by stopping streaming on both queues, 559releasing all buffers and performing the Initialization sequence again. 560 561Encoding Parameter Changes 562========================== 563 564The client is allowed to use :c:func:`VIDIOC_S_CTRL` to change encoder 565parameters at any time. The availability of parameters is encoder-specific 566and the client must query the encoder to find the set of available controls. 567 568The ability to change each parameter during encoding is encoder-specific, as 569per the standard semantics of the V4L2 control interface. The client may 570attempt to set a control during encoding and if the operation fails with the 571-EBUSY error code, the ``CAPTURE`` queue needs to be stopped for the 572configuration change to be allowed. To do this, it may follow the `Drain` 573sequence to avoid losing the already queued/encoded frames. 574 575The timing of parameter updates is encoder-specific, as per the standard 576semantics of the V4L2 control interface. If the client needs to apply the 577parameters exactly at specific frame, using the Request API 578(:ref:`media-request-api`) should be considered, if supported by the encoder. 579 580Drain 581===== 582 583To ensure that all the queued ``OUTPUT`` buffers have been processed and the 584related ``CAPTURE`` buffers are given to the client, the client must follow the 585drain sequence described below. After the drain sequence ends, the client has 586received all encoded frames for all ``OUTPUT`` buffers queued before the 587sequence was started. 588 5891. Begin the drain sequence by issuing :c:func:`VIDIOC_ENCODER_CMD`. 590 591 * **Required fields:** 592 593 ``cmd`` 594 set to ``V4L2_ENC_CMD_STOP``. 595 596 ``flags`` 597 set to 0. 598 599 ``pts`` 600 set to 0. 601 602 .. warning:: 603 604 The sequence can be only initiated if both ``OUTPUT`` and ``CAPTURE`` 605 queues are streaming. For compatibility reasons, the call to 606 :c:func:`VIDIOC_ENCODER_CMD` will not fail even if any of the queues is 607 not streaming, but at the same time it will not initiate the `Drain` 608 sequence and so the steps described below would not be applicable. 609 6102. Any ``OUTPUT`` buffers queued by the client before the 611 :c:func:`VIDIOC_ENCODER_CMD` was issued will be processed and encoded as 612 normal. The client must continue to handle both queues independently, 613 similarly to normal encode operation. This includes: 614 615 * queuing and dequeuing ``CAPTURE`` buffers, until a buffer marked with the 616 ``V4L2_BUF_FLAG_LAST`` flag is dequeued, 617 618 .. warning:: 619 620 The last buffer may be empty (with :c:type:`v4l2_buffer` 621 ``bytesused`` = 0) and in that case it must be ignored by the client, 622 as it does not contain an encoded frame. 623 624 .. note:: 625 626 Any attempt to dequeue more ``CAPTURE`` buffers beyond the buffer 627 marked with ``V4L2_BUF_FLAG_LAST`` will result in a -EPIPE error from 628 :c:func:`VIDIOC_DQBUF`. 629 630 * dequeuing processed ``OUTPUT`` buffers, until all the buffers queued 631 before the ``V4L2_ENC_CMD_STOP`` command are dequeued, 632 633 * dequeuing the ``V4L2_EVENT_EOS`` event, if the client subscribes to it. 634 635 .. note:: 636 637 For backwards compatibility, the encoder will signal a ``V4L2_EVENT_EOS`` 638 event when the last frame has been encoded and all frames are ready to be 639 dequeued. It is deprecated behavior and the client must not rely on it. 640 The ``V4L2_BUF_FLAG_LAST`` buffer flag should be used instead. 641 6423. Once all ``OUTPUT`` buffers queued before the ``V4L2_ENC_CMD_STOP`` call are 643 dequeued and the last ``CAPTURE`` buffer is dequeued, the encoder is stopped 644 and it will accept, but not process any newly queued ``OUTPUT`` buffers 645 until the client issues any of the following operations: 646 647 * ``V4L2_ENC_CMD_START`` - the encoder will not be reset and will resume 648 operation normally, with all the state from before the drain, 649 650 * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the 651 ``CAPTURE`` queue - the encoder will be reset (see the `Reset` sequence) 652 and then resume encoding, 653 654 * a pair of :c:func:`VIDIOC_STREAMOFF` and :c:func:`VIDIOC_STREAMON` on the 655 ``OUTPUT`` queue - the encoder will resume operation normally, however any 656 source frames queued to the ``OUTPUT`` queue between ``V4L2_ENC_CMD_STOP`` 657 and :c:func:`VIDIOC_STREAMOFF` will be discarded. 658 659.. note:: 660 661 Once the drain sequence is initiated, the client needs to drive it to 662 completion, as described by the steps above, unless it aborts the process by 663 issuing :c:func:`VIDIOC_STREAMOFF` on any of the ``OUTPUT`` or ``CAPTURE`` 664 queues. The client is not allowed to issue ``V4L2_ENC_CMD_START`` or 665 ``V4L2_ENC_CMD_STOP`` again while the drain sequence is in progress and they 666 will fail with -EBUSY error code if attempted. 667 668 For reference, handling of various corner cases is described below: 669 670 * In case of no buffer in the ``OUTPUT`` queue at the time the 671 ``V4L2_ENC_CMD_STOP`` command was issued, the drain sequence completes 672 immediately and the encoder returns an empty ``CAPTURE`` buffer with the 673 ``V4L2_BUF_FLAG_LAST`` flag set. 674 675 * In case of no buffer in the ``CAPTURE`` queue at the time the drain 676 sequence completes, the next time the client queues a ``CAPTURE`` buffer 677 it is returned at once as an empty buffer with the ``V4L2_BUF_FLAG_LAST`` 678 flag set. 679 680 * If :c:func:`VIDIOC_STREAMOFF` is called on the ``CAPTURE`` queue in the 681 middle of the drain sequence, the drain sequence is canceled and all 682 ``CAPTURE`` buffers are implicitly returned to the client. 683 684 * If :c:func:`VIDIOC_STREAMOFF` is called on the ``OUTPUT`` queue in the 685 middle of the drain sequence, the drain sequence completes immediately and 686 next ``CAPTURE`` buffer will be returned empty with the 687 ``V4L2_BUF_FLAG_LAST`` flag set. 688 689 Although not mandatory, the availability of encoder commands may be queried 690 using :c:func:`VIDIOC_TRY_ENCODER_CMD`. 691 692Reset 693===== 694 695The client may want to request the encoder to reinitialize the encoding, so 696that the following stream data becomes independent from the stream data 697generated before. Depending on the coded format, that may imply that: 698 699* encoded frames produced after the restart must not reference any frames 700 produced before the stop, e.g. no long term references for H.264/HEVC, 701 702* any headers that must be included in a standalone stream must be produced 703 again, e.g. SPS and PPS for H.264/HEVC. 704 705This can be achieved by performing the reset sequence. 706 7071. Perform the `Drain` sequence to ensure all the in-flight encoding finishes 708 and respective buffers are dequeued. 709 7102. Stop streaming on the ``CAPTURE`` queue via :c:func:`VIDIOC_STREAMOFF`. This 711 will return all currently queued ``CAPTURE`` buffers to the client, without 712 valid frame data. 713 7143. Start streaming on the ``CAPTURE`` queue via :c:func:`VIDIOC_STREAMON` and 715 continue with regular encoding sequence. The encoded frames produced into 716 ``CAPTURE`` buffers from now on will contain a standalone stream that can be 717 decoded without the need for frames encoded before the reset sequence, 718 starting at the first ``OUTPUT`` buffer queued after issuing the 719 `V4L2_ENC_CMD_STOP` of the `Drain` sequence. 720 721This sequence may be also used to change encoding parameters for encoders 722without the ability to change the parameters on the fly. 723 724Commit Points 725============= 726 727Setting formats and allocating buffers triggers changes in the behavior of the 728encoder. 729 7301. Setting the format on the ``CAPTURE`` queue may change the set of formats 731 supported/advertised on the ``OUTPUT`` queue. In particular, it also means 732 that the ``OUTPUT`` format may be reset and the client must not rely on the 733 previously set format being preserved. 734 7352. Enumerating formats on the ``OUTPUT`` queue always returns only formats 736 supported for the current ``CAPTURE`` format. 737 7383. Setting the format on the ``OUTPUT`` queue does not change the list of 739 formats available on the ``CAPTURE`` queue. An attempt to set the ``OUTPUT`` 740 format that is not supported for the currently selected ``CAPTURE`` format 741 will result in the encoder adjusting the requested ``OUTPUT`` format to a 742 supported one. 743 7444. Enumerating formats on the ``CAPTURE`` queue always returns the full set of 745 supported coded formats, irrespective of the current ``OUTPUT`` format. 746 7475. While buffers are allocated on any of the ``OUTPUT`` or ``CAPTURE`` queues, 748 the client must not change the format on the ``CAPTURE`` queue. Drivers will 749 return the -EBUSY error code for any such format change attempt. 750 751To summarize, setting formats and allocation must always start with the 752``CAPTURE`` queue and the ``CAPTURE`` queue is the master that governs the 753set of supported formats for the ``OUTPUT`` queue. 754