1.. This file is dual-licensed: you can use it either under the terms 2.. of the GPL 2.0 or the GFDL 1.1+ license, at your option. Note that this 3.. dual licensing only applies to this file, and not this project as a 4.. whole. 5.. 6.. a) This file is free software; you can redistribute it and/or 7.. modify it under the terms of the GNU General Public License as 8.. published by the Free Software Foundation version 2 of 9.. the License. 10.. 11.. This file is distributed in the hope that it will be useful, 12.. but WITHOUT ANY WARRANTY; without even the implied warranty of 13.. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14.. GNU General Public License for more details. 15.. 16.. Or, alternatively, 17.. 18.. b) Permission is granted to copy, distribute and/or modify this 19.. document under the terms of the GNU Free Documentation License, 20.. Version 1.1 or any later version published by the Free Software 21.. Foundation, with no Invariant Sections, no Front-Cover Texts 22.. and no Back-Cover Texts. A copy of the license is included at 23.. Documentation/userspace-api/media/fdl-appendix.rst. 24.. 25.. TODO: replace it to GPL-2.0 OR GFDL-1.1-or-later WITH no-invariant-sections 26 27.. _media-request-api: 28 29Request API 30=========== 31 32The Request API has been designed to allow V4L2 to deal with requirements of 33modern devices (stateless codecs, complex camera pipelines, ...) and APIs 34(Android Codec v2). One such requirement is the ability for devices belonging to 35the same pipeline to reconfigure and collaborate closely on a per-frame basis. 36Another is support of stateless codecs, which require controls to be applied 37to specific frames (aka 'per-frame controls') in order to be used efficiently. 38 39While the initial use-case was V4L2, it can be extended to other subsystems 40as well, as long as they use the media controller. 41 42Supporting these features without the Request API is not always possible and if 43it is, it is terribly inefficient: user-space would have to flush all activity 44on the media pipeline, reconfigure it for the next frame, queue the buffers to 45be processed with that configuration, and wait until they are all available for 46dequeuing before considering the next frame. This defeats the purpose of having 47buffer queues since in practice only one buffer would be queued at a time. 48 49The Request API allows a specific configuration of the pipeline (media 50controller topology + configuration for each media entity) to be associated with 51specific buffers. This allows user-space to schedule several tasks ("requests") 52with different configurations in advance, knowing that the configuration will be 53applied when needed to get the expected result. Configuration values at the time 54of request completion are also available for reading. 55 56General Usage 57------------- 58 59The Request API extends the Media Controller API and cooperates with 60subsystem-specific APIs to support request usage. At the Media Controller 61level, requests are allocated from the supporting Media Controller device 62node. Their life cycle is then managed through the request file descriptors in 63an opaque way. Configuration data, buffer handles and processing results 64stored in requests are accessed through subsystem-specific APIs extended for 65request support, such as V4L2 APIs that take an explicit ``request_fd`` 66parameter. 67 68Request Allocation 69------------------ 70 71User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC` 72for the media device node. This returns a file descriptor representing the 73request. Typically, several such requests will be allocated. 74 75Request Preparation 76------------------- 77 78Standard V4L2 ioctls can then receive a request file descriptor to express the 79fact that the ioctl is part of said request, and is not to be applied 80immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that 81support this. Configurations set with a ``request_fd`` parameter are stored 82instead of being immediately applied, and buffers queued to a request do not 83enter the regular buffer queue until the request itself is queued. 84 85Request Submission 86------------------ 87 88Once the configuration and buffers of the request are specified, it can be 89queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor. 90A request must contain at least one buffer, otherwise ``ENOENT`` is returned. 91A queued request cannot be modified anymore. 92 93.. caution:: 94 For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for 95 output buffers, not for capture buffers. Attempting to add a capture buffer 96 to a request will result in an ``EBADR`` error. 97 98If the request contains configurations for multiple entities, individual drivers 99may synchronize so the requested pipeline's topology is applied before the 100buffers are processed. Media controller drivers do a best effort implementation 101since perfect atomicity may not be possible due to hardware limitations. 102 103.. caution:: 104 105 It is not allowed to mix queuing requests with directly queuing buffers: 106 whichever method is used first locks this in place until 107 :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is 108 :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier 109 a buffer was queued via a request or vice versa will result in an ``EBUSY`` 110 error. 111 112Controls can still be set without a request and are applied immediately, 113regardless of whether a request is in use or not. 114 115.. caution:: 116 117 Setting the same control through a request and also directly can lead to 118 undefined behavior! 119 120User-space can :ref:`poll() <request-func-poll>` a request file descriptor in 121order to wait until the request completes. A request is considered complete 122once all its associated buffers are available for dequeuing and all the 123associated controls have been updated with the values at the time of completion. 124Note that user-space does not need to wait for the request to complete to 125dequeue its buffers: buffers that are available halfway through a request can 126be dequeued independently of the request's state. 127 128A completed request contains the state of the device after the request was 129executed. User-space can query that state by calling 130:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file 131descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a 132request that has been queued but not yet completed will return ``EBUSY`` 133since the control values might be changed at any time by the driver while the 134request is in flight. 135 136.. _media-request-life-time: 137 138Recycling and Destruction 139------------------------- 140 141Finally, a completed request can either be discarded or be reused. Calling 142:ref:`close() <request-func-close>` on a request file descriptor will make 143that file descriptor unusable and the request will be freed once it is no 144longer in use by the kernel. That is, if the request is queued and then the 145file descriptor is closed, then it won't be freed until the driver completed 146the request. 147 148The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it 149available again. No state is retained by this operation: the request is as 150if it had just been allocated. 151 152Example for a Codec Device 153-------------------------- 154 155For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used 156to associate specific controls to 157be applied by the driver for the OUTPUT buffer, allowing user-space 158to queue many such buffers in advance. It can also take advantage of requests' 159ability to capture the state of controls when the request completes to read back 160information that may be subject to change. 161 162Put into code, after obtaining a request, user-space can assign controls and one 163OUTPUT buffer to it: 164 165.. code-block:: c 166 167 struct v4l2_buffer buf; 168 struct v4l2_ext_controls ctrls; 169 int req_fd; 170 ... 171 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) 172 return errno; 173 ... 174 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 175 ctrls.request_fd = req_fd; 176 if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) 177 return errno; 178 ... 179 buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; 180 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; 181 buf.request_fd = req_fd; 182 if (ioctl(codec_fd, VIDIOC_QBUF, &buf)) 183 return errno; 184 185Note that it is not allowed to use the Request API for CAPTURE buffers 186since there are no per-frame settings to report there. 187 188Once the request is fully prepared, it can be queued to the driver: 189 190.. code-block:: c 191 192 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) 193 return errno; 194 195User-space can then either wait for the request to complete by calling poll() on 196its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will 197want to get CAPTURE buffers as soon as possible and this can be done using a 198regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`: 199 200.. code-block:: c 201 202 struct v4l2_buffer buf; 203 204 memset(&buf, 0, sizeof(buf)); 205 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 206 if (ioctl(codec_fd, VIDIOC_DQBUF, &buf)) 207 return errno; 208 209Note that this example assumes for simplicity that for every OUTPUT buffer 210there will be one CAPTURE buffer, but this does not have to be the case. 211 212We can then, after ensuring that the request is completed via polling the 213request file descriptor, query control values at the time of its completion via 214a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`. 215This is particularly useful for volatile controls for which we want to 216query values as soon as the capture buffer is produced. 217 218.. code-block:: c 219 220 struct pollfd pfd = { .events = POLLPRI, .fd = req_fd }; 221 poll(&pfd, 1, -1); 222 ... 223 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 224 ctrls.request_fd = req_fd; 225 if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls)) 226 return errno; 227 228Once we don't need the request anymore, we can either recycle it for reuse with 229:ref:`MEDIA_REQUEST_IOC_REINIT`... 230 231.. code-block:: c 232 233 if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT)) 234 return errno; 235 236... or close its file descriptor to completely dispose of it. 237 238.. code-block:: c 239 240 close(req_fd); 241 242Example for a Simple Capture Device 243----------------------------------- 244 245With a simple capture device, requests can be used to specify controls to apply 246for a given CAPTURE buffer. 247 248.. code-block:: c 249 250 struct v4l2_buffer buf; 251 struct v4l2_ext_controls ctrls; 252 int req_fd; 253 ... 254 if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) 255 return errno; 256 ... 257 ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; 258 ctrls.request_fd = req_fd; 259 if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) 260 return errno; 261 ... 262 buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 263 buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; 264 buf.request_fd = req_fd; 265 if (ioctl(camera_fd, VIDIOC_QBUF, &buf)) 266 return errno; 267 268Once the request is fully prepared, it can be queued to the driver: 269 270.. code-block:: c 271 272 if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) 273 return errno; 274 275User-space can then dequeue buffers, wait for the request completion, query 276controls and recycle the request as in the M2M example above. 277