1.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later 2.. c:namespace:: V4L 3 4.. _overlay: 5 6*********************** 7Video Overlay Interface 8*********************** 9 10**Also known as Framebuffer Overlay or Previewing.** 11 12Video overlay devices have the ability to genlock (TV-)video into the 13(VGA-)video signal of a graphics card, or to store captured images 14directly in video memory of a graphics card, typically with clipping. 15This can be considerable more efficient than capturing images and 16displaying them by other means. In the old days when only nuclear power 17plants needed cooling towers this used to be the only way to put live 18video into a window. 19 20Video overlay devices are accessed through the same character special 21files as :ref:`video capture <capture>` devices. 22 23.. note:: 24 25 The default function of a ``/dev/video`` device is video 26 capturing. The overlay function is only available after calling 27 the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. 28 29The driver may support simultaneous overlay and capturing using the 30read/write and streaming I/O methods. If so, operation at the nominal 31frame rate of the video standard is not guaranteed. Frames may be 32directed away from overlay to capture, or one field may be used for 33overlay and the other for capture if the capture parameters permit this. 34 35Applications should use different file descriptors for capturing and 36overlay. This must be supported by all drivers capable of simultaneous 37capturing and overlay. Optionally these drivers may also permit 38capturing and overlay with a single file descriptor for compatibility 39with V4L and earlier versions of V4L2. [#f1]_ 40 41A common application of two file descriptors is the X11 42:ref:`Xv/V4L <xvideo>` interface driver and a V4L2 application. 43While the X server controls video overlay, the application can take 44advantage of memory mapping and DMA. 45 46Querying Capabilities 47===================== 48 49Devices supporting the video overlay interface set the 50``V4L2_CAP_VIDEO_OVERLAY`` flag in the ``capabilities`` field of struct 51:c:type:`v4l2_capability` returned by the 52:ref:`VIDIOC_QUERYCAP` ioctl. The overlay I/O 53method specified below must be supported. Tuners and audio inputs are 54optional. 55 56 57Supplemental Functions 58====================== 59 60Video overlay devices shall support :ref:`audio input <audio>`, 61:ref:`tuner`, :ref:`controls <control>`, 62:ref:`cropping and scaling <crop>` and 63:ref:`streaming parameter <streaming-par>` ioctls as needed. The 64:ref:`video input <video>` and :ref:`video standard <standard>` 65ioctls must be supported by all video overlay devices. 66 67 68Setup 69===== 70 71*Note: support for this has been removed.* 72Before overlay can commence applications must program the driver with 73frame buffer parameters, namely the address and size of the frame buffer 74and the image format, for example RGB 5:6:5. The 75:ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` and 76:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctls are available to get and 77set these parameters, respectively. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is 78privileged because it allows to set up DMA into physical memory, 79bypassing the memory protection mechanisms of the kernel. Only the 80superuser can change the frame buffer address and size. Users are not 81supposed to run TV applications as root or with SUID bit set. A small 82helper application with suitable privileges should query the graphics 83system and program the V4L2 driver at the appropriate time. 84 85Some devices add the video overlay to the output signal of the graphics 86card. In this case the frame buffer is not modified by the video device, 87and the frame buffer address and pixel format are not needed by the 88driver. The :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` ioctl is not privileged. An application 89can check for this type of device by calling the :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` 90ioctl. 91 92A driver may support any (or none) of five clipping/blending methods: 93 941. Chroma-keying displays the overlaid image only where pixels in the 95 primary graphics surface assume a certain color. 96 972. *Note: support for this has been removed.* 98 A bitmap can be specified where each bit corresponds to a pixel in 99 the overlaid image. When the bit is set, the corresponding video 100 pixel is displayed, otherwise a pixel of the graphics surface. 101 1023. *Note: support for this has been removed.* 103 A list of clipping rectangles can be specified. In these regions *no* 104 video is displayed, so the graphics surface can be seen here. 105 1064. The framebuffer has an alpha channel that can be used to clip or 107 blend the framebuffer with the video. 108 1095. A global alpha value can be specified to blend the framebuffer 110 contents with video images. 111 112When simultaneous capturing and overlay is supported and the hardware 113prohibits different image and frame buffer formats, the format requested 114first takes precedence. The attempt to capture 115(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) or overlay 116(:ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`) may fail with an ``EBUSY`` error 117code or return accordingly modified parameters.. 118 119 120Overlay Window 121============== 122 123The overlaid image is determined by cropping and overlay window 124parameters. The former select an area of the video picture to capture, 125the latter how images are overlaid and clipped. Cropping initialization 126at minimum requires to reset the parameters to defaults. An example is 127given in :ref:`crop`. 128 129The overlay window is described by a struct 130:c:type:`v4l2_window`. It defines the size of the image, 131its position over the graphics surface and the clipping to be applied. 132To get the current parameters applications set the ``type`` field of a 133struct :c:type:`v4l2_format` to 134``V4L2_BUF_TYPE_VIDEO_OVERLAY`` and call the 135:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` ioctl. The driver fills the 136struct :c:type:`v4l2_window` substructure named ``win``. It is not 137possible to retrieve a previously programmed clipping list or bitmap. 138 139To program the overlay window applications set the ``type`` field of a 140struct :c:type:`v4l2_format` to 141``V4L2_BUF_TYPE_VIDEO_OVERLAY``, initialize the ``win`` substructure and 142call the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl. The driver 143adjusts the parameters against hardware limits and returns the actual 144parameters as :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` does. Like :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`, the 145:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl can be used to learn 146about driver capabilities without actually changing driver state. Unlike 147:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` this also works after the overlay has been enabled. 148 149The scaling factor of the overlaid image is implied by the width and 150height given in struct :c:type:`v4l2_window` and the size 151of the cropping rectangle. For more information see :ref:`crop`. 152 153When simultaneous capturing and overlay is supported and the hardware 154prohibits different image and window sizes, the size requested first 155takes precedence. The attempt to capture or overlay as well 156(:ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`) may fail with an ``EBUSY`` error 157code or return accordingly modified parameters. 158 159 160.. c:type:: v4l2_window 161 162struct v4l2_window 163------------------ 164 165``struct v4l2_rect w`` 166 Size and position of the window relative to the top, left corner of 167 the frame buffer defined with 168 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`. The window can extend the 169 frame buffer width and height, the ``x`` and ``y`` coordinates can 170 be negative, and it can lie completely outside the frame buffer. The 171 driver clips the window accordingly, or if that is not possible, 172 modifies its size and/or position. 173 174``enum v4l2_field field`` 175 Applications set this field to determine which video field shall be 176 overlaid, typically one of ``V4L2_FIELD_ANY`` (0), 177 ``V4L2_FIELD_TOP``, ``V4L2_FIELD_BOTTOM`` or 178 ``V4L2_FIELD_INTERLACED``. Drivers may have to choose a different 179 field order and return the actual setting here. 180 181``__u32 chromakey`` 182 When chroma-keying has been negotiated with 183 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>` applications set this field 184 to the desired pixel value for the chroma key. The format is the 185 same as the pixel format of the framebuffer (struct 186 :c:type:`v4l2_framebuffer` ``fmt.pixelformat`` 187 field), with bytes in host order. E. g. for 188 :ref:`V4L2_PIX_FMT_BGR24 <V4L2-PIX-FMT-BGR32>` the value should 189 be 0xRRGGBB on a little endian, 0xBBGGRR on a big endian host. 190 191``struct v4l2_clip * clips`` 192 *Note: support for this has been removed.* 193 When chroma-keying has *not* been negotiated and 194 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 195 applications can set this field to point to an array of clipping 196 rectangles. 197 198 Like the window coordinates w, clipping rectangles are defined 199 relative to the top, left corner of the frame buffer. However 200 clipping rectangles must not extend the frame buffer width and 201 height, and they must not overlap. If possible applications 202 should merge adjacent rectangles. Whether this must create 203 x-y or y-x bands, or the order of rectangles, is not defined. When 204 clip lists are not supported the driver ignores this field. Its 205 contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` 206 are undefined. 207 208``__u32 clipcount`` 209 *Note: support for this has been removed.* 210 When the application set the ``clips`` field, this field must 211 contain the number of clipping rectangles in the list. When clip 212 lists are not supported the driver ignores this field, its contents 213 after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are undefined. When clip lists are 214 supported but no clipping is desired this field must be set to zero. 215 216``void * bitmap`` 217 *Note: support for this has been removed.* 218 When chroma-keying has *not* been negotiated and 219 :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>` indicated this capability, 220 applications can set this field to point to a clipping bit mask. 221 222It must be of the same size as the window, ``w.width`` and ``w.height``. 223Each bit corresponds to a pixel in the overlaid image, which is 224displayed only when the bit is *set*. Pixel coordinates translate to 225bits like: 226 227 228.. code-block:: c 229 230 ((__u8 *) bitmap)[w.width * y + x / 8] & (1 << (x & 7)) 231 232where ``0`` ≤ x < ``w.width`` and ``0`` ≤ y <``w.height``. [#f2]_ 233 234When a clipping bit mask is not supported the driver ignores this field, 235its contents after calling :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` are 236undefined. When a bit mask is supported but no clipping is desired this 237field must be set to ``NULL``. 238 239Applications need not create a clip list or bit mask. When they pass 240both, or despite negotiating chroma-keying, the results are undefined. 241Regardless of the chosen method, the clipping abilities of the hardware 242may be limited in quantity or quality. The results when these limits are 243exceeded are undefined. [#f3]_ 244 245``__u8 global_alpha`` 246 The global alpha value used to blend the framebuffer with video 247 images, if global alpha blending has been negotiated 248 (``V4L2_FBUF_FLAG_GLOBAL_ALPHA``, see 249 :ref:`VIDIOC_S_FBUF <VIDIOC_G_FBUF>`, 250 :ref:`framebuffer-flags`). 251 252.. note:: 253 254 This field was added in Linux 2.6.23, extending the 255 structure. However the :ref:`VIDIOC_[G|S|TRY]_FMT <VIDIOC_G_FMT>` 256 ioctls, which take a pointer to a :c:type:`v4l2_format` 257 parent structure with padding bytes at the end, are not affected. 258 259 260.. c:type:: v4l2_clip 261 262struct v4l2_clip [#f4]_ 263----------------------- 264 265``struct v4l2_rect c`` 266 Coordinates of the clipping rectangle, relative to the top, left 267 corner of the frame buffer. Only window pixels *outside* all 268 clipping rectangles are displayed. 269 270``struct v4l2_clip * next`` 271 Pointer to the next clipping rectangle, ``NULL`` when this is the last 272 rectangle. Drivers ignore this field, it cannot be used to pass a 273 linked list of clipping rectangles. 274 275 276.. c:type:: v4l2_rect 277 278struct v4l2_rect 279---------------- 280 281``__s32 left`` 282 Horizontal offset of the top, left corner of the rectangle, in 283 pixels. 284 285``__s32 top`` 286 Vertical offset of the top, left corner of the rectangle, in pixels. 287 Offsets increase to the right and down. 288 289``__u32 width`` 290 Width of the rectangle, in pixels. 291 292``__u32 height`` 293 Height of the rectangle, in pixels. 294 295 296Enabling Overlay 297================ 298 299To start or stop the frame buffer overlay applications call the 300:ref:`VIDIOC_OVERLAY` ioctl. 301 302.. [#f1] 303 In the opinion of the designers of this API, no driver writer taking 304 the efforts to support simultaneous capturing and overlay will 305 restrict this ability by requiring a single file descriptor, as in 306 V4L and earlier versions of V4L2. Making this optional means 307 applications depending on two file descriptors need backup routines 308 to be compatible with all drivers, which is considerable more work 309 than using two fds in applications which do not. Also two fd's fit 310 the general concept of one file descriptor for each logical stream. 311 Hence as a complexity trade-off drivers *must* support two file 312 descriptors and *may* support single fd operation. 313 314.. [#f2] 315 Should we require ``w.width`` to be a multiple of eight? 316 317.. [#f3] 318 When the image is written into frame buffer memory it will be 319 undesirable if the driver clips out less pixels than expected, 320 because the application and graphics system are not aware these 321 regions need to be refreshed. The driver should clip out more pixels 322 or not write the image at all. 323 324.. [#f4] 325 The X Window system defines "regions" which are vectors of ``struct 326 BoxRec { short x1, y1, x2, y2; }`` with ``width = x2 - x1`` and 327 ``height = y2 - y1``, so one cannot pass X11 clip lists directly. 328