1.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later 2.. c:namespace:: MC 3 4.. _media-request-api: 5 6Request API 7=========== 8 9The Request API has been designed to allow V4L2 to deal with requirements of 10modern devices (stateless codecs, complex camera pipelines, ...) and APIs 11(Android Codec v2). One such requirement is the ability for devices belonging to 12the same pipeline to reconfigure and collaborate closely on a per-frame basis. 13Another is support of stateless codecs, which require controls to be applied 14to specific frames (aka 'per-frame controls') in order to be used efficiently. 15 16While the initial use-case was V4L2, it can be extended to other subsystems 17as well, as long as they use the media controller. 18 19Supporting these features without the Request API is not always possible and if 20it is, it is terribly inefficient: user-space would have to flush all activity 21on the media pipeline, reconfigure it for the next frame, queue the buffers to 22be processed with that configuration, and wait until they are all available for 23dequeuing before considering the next frame. This defeats the purpose of having 24buffer queues since in practice only one buffer would be queued at a time. 25 26The Request API allows a specific configuration of the pipeline (media 27controller topology + configuration for each media entity) to be associated with 28specific buffers. This allows user-space to schedule several tasks ("requests") 29with different configurations in advance, knowing that the configuration will be 30applied when needed to get the expected result. Configuration values at the time 31of request completion are also available for reading. 32 33General Usage 34------------- 35 36The Request API extends the Media Controller API and cooperates with 37subsystem-specific APIs to support request usage. At the Media Controller 38level, requests are allocated from the supporting Media Controller device 39node. Their life cycle is then managed through the request file descriptors in 40an opaque way. Configuration data, buffer handles and processing results 41stored in requests are accessed through subsystem-specific APIs extended for 42request support, such as V4L2 APIs that take an explicit ``request_fd`` 43parameter. 44 45Request Allocation 46------------------ 47 48User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC` 49for the media device node. This returns a file descriptor representing the 50request. Typically, several such requests will be allocated. 51 52Request Preparation 53------------------- 54 55Standard V4L2 ioctls can then receive a request file descriptor to express the 56fact that the ioctl is part of said request, and is not to be applied 57immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that 58support this. Configurations set with a ``request_fd`` parameter are stored 59instead of being immediately applied, and buffers queued to a request do not 60enter the regular buffer queue until the request itself is queued. 61 62Request Submission 63------------------ 64 65Once the configuration and buffers of the request are specified, it can be 66queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor. 67A request must contain at least one buffer, otherwise ``ENOENT`` is returned. 68A queued request cannot be modified anymore. 69 70.. caution:: 71 For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for 72 output buffers, not for capture buffers. Attempting to add a capture buffer 73 to a request will result in an ``EBADR`` error. 74 75If the request contains configurations for multiple entities, individual drivers 76may synchronize so the requested pipeline's topology is applied before the 77buffers are processed. Media controller drivers do a best effort implementation 78since perfect atomicity may not be possible due to hardware limitations. 79 80.. caution:: 81 82 It is not allowed to mix queuing requests with directly queuing buffers: 83 whichever method is used first locks this in place until 84 :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is 85 :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier 86 a buffer was queued via a request or vice versa will result in an ``EBUSY`` 87 error. 88 89Controls can still be set without a request and are applied immediately, 90regardless of whether a request is in use or not. 91 92.. caution:: 93 94 Setting the same control through a request and also directly can lead to 95 undefined behavior! 96 97User-space can :c:func:`poll()` a request file descriptor in 98order to wait until the request completes. A request is considered complete 99once all its associated buffers are available for dequeuing and all the 100associated controls have been updated with the values at the time of completion. 101Note that user-space does not need to wait for the request to complete to 102dequeue its buffers: buffers that are available halfway through a request can 103be dequeued independently of the request's state. 104 105A completed request contains the state of the device after the request was 106executed. User-space can query that state by calling 107:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file 108descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a 109request that has been queued but not yet completed will return ``EBUSY`` 110since the control values might be changed at any time by the driver while the 111request is in flight. 112 113.. _media-request-life-time: 114 115Recycling and Destruction 116------------------------- 117 118Finally, a completed request can either be discarded or be reused. Calling 119:c:func:`close()` on a request file descriptor will make 120that file descriptor unusable and the request will be freed once it is no 121longer in use by the kernel. That is, if the request is queued and then the 122file descriptor is closed, then it won't be freed until the driver completed 123the request. 124 125The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it 126available again. No state is retained by this operation: the request is as 127if it had just been allocated. 128 129Example for a Codec Device 130-------------------------- 131 132For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used 133to associate specific controls to 134be applied by the driver for the OUTPUT buffer, allowing user-space 135to queue many such buffers in advance. It can also take advantage of requests' 136ability to capture the state of controls when the request completes to read back 137information that may be subject to change. 138 139Put into code, after obtaining a request, user-space can assign controls and one 140OUTPUT buffer to it: 141 142.. code-block:: c 143 144 struct v4l2_buffer buf; 145 struct v4l2_ext_controls ctrls; 146 int req_fd; 147 ... 148 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) 149 return errno; 150 ... 151 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 152 ctrls.request_fd = req_fd; 153 if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) 154 return errno; 155 ... 156 buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; 157 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; 158 buf.request_fd = req_fd; 159 if (ioctl(codec_fd, VIDIOC_QBUF, &buf)) 160 return errno; 161 162Note that it is not allowed to use the Request API for CAPTURE buffers 163since there are no per-frame settings to report there. 164 165Once the request is fully prepared, it can be queued to the driver: 166 167.. code-block:: c 168 169 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) 170 return errno; 171 172User-space can then either wait for the request to complete by calling poll() on 173its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will 174want to get CAPTURE buffers as soon as possible and this can be done using a 175regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`: 176 177.. code-block:: c 178 179 struct v4l2_buffer buf; 180 181 memset(&buf, 0, sizeof(buf)); 182 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 183 if (ioctl(codec_fd, VIDIOC_DQBUF, &buf)) 184 return errno; 185 186Note that this example assumes for simplicity that for every OUTPUT buffer 187there will be one CAPTURE buffer, but this does not have to be the case. 188 189We can then, after ensuring that the request is completed via polling the 190request file descriptor, query control values at the time of its completion via 191a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`. 192This is particularly useful for volatile controls for which we want to 193query values as soon as the capture buffer is produced. 194 195.. code-block:: c 196 197 struct pollfd pfd = { .events = POLLPRI, .fd = req_fd }; 198 poll(&pfd, 1, -1); 199 ... 200 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 201 ctrls.request_fd = req_fd; 202 if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls)) 203 return errno; 204 205Once we don't need the request anymore, we can either recycle it for reuse with 206:ref:`MEDIA_REQUEST_IOC_REINIT`... 207 208.. code-block:: c 209 210 if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT)) 211 return errno; 212 213... or close its file descriptor to completely dispose of it. 214 215.. code-block:: c 216 217 close(req_fd); 218 219Example for a Simple Capture Device 220----------------------------------- 221 222With a simple capture device, requests can be used to specify controls to apply 223for a given CAPTURE buffer. 224 225.. code-block:: c 226 227 struct v4l2_buffer buf; 228 struct v4l2_ext_controls ctrls; 229 int req_fd; 230 ... 231 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) 232 return errno; 233 ... 234 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 235 ctrls.request_fd = req_fd; 236 if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) 237 return errno; 238 ... 239 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 240 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; 241 buf.request_fd = req_fd; 242 if (ioctl(camera_fd, VIDIOC_QBUF, &buf)) 243 return errno; 244 245Once the request is fully prepared, it can be queued to the driver: 246 247.. code-block:: c 248 249 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) 250 return errno; 251 252User-space can then dequeue buffers, wait for the request completion, query 253controls and recycle the request as in the M2M example above. 254