xref: /linux/Documentation/userspace-api/media/v4l/crop.rst (revision cbecf716ca618fd44feda6bd9a64a8179d031fc5)
1.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
2
3.. _crop:
4
5*****************************************************
6Image Cropping, Insertion and Scaling -- the CROP API
7*****************************************************
8
9.. note::
10
11   The CROP API is mostly superseded by the newer :ref:`SELECTION API
12   <selection-api>`. The new API should be preferred in most cases,
13   with the exception of pixel aspect ratio detection, which is
14   implemented by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` and has no
15   equivalent in the SELECTION API. See :ref:`selection-vs-crop` for a
16   comparison of the two APIs.
17
18Some video capture devices can sample a subsection of the picture and
19shrink or enlarge it to an image of arbitrary size. We call these
20abilities cropping and scaling. Some video output devices can scale an
21image up or down and insert it at an arbitrary scan line and horizontal
22offset into a video signal.
23
24Applications can use the following API to select an area in the video
25signal, query the default area and the hardware limits.
26
27.. note::
28
29   Despite their name, the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`,
30   :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP
31   <VIDIOC_G_CROP>` ioctls apply to input as well as output devices.
32
33Scaling requires a source and a target. On a video capture or overlay
34device the source is the video signal, and the cropping ioctls determine
35the area actually sampled. The target are images read by the application
36or overlaid onto the graphics screen. Their size (and position for an
37overlay) is negotiated with the :ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>`
38and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctls.
39
40On a video output device the source are the images passed in by the
41application, and their size is again negotiated with the
42:ref:`VIDIOC_G_FMT <VIDIOC_G_FMT>` and :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>`
43ioctls, or may be encoded in a compressed video stream. The target is
44the video signal, and the cropping ioctls determine the area where the
45images are inserted.
46
47Source and target rectangles are defined even if the device does not
48support scaling or the :ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and
49:ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>` ioctls. Their size (and position
50where applicable) will be fixed in this case.
51
52.. note::
53
54   All capture and output devices that support the CROP or SELECTION
55   API will also support the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>`
56   ioctl.
57
58Cropping Structures
59===================
60
61
62.. _crop-scale:
63
64.. kernel-figure:: crop.svg
65    :alt:    crop.svg
66    :align:  center
67
68    Image Cropping, Insertion and Scaling
69
70    The cropping, insertion and scaling process
71
72
73
74For capture devices the coordinates of the top left corner, width and
75height of the area which can be sampled is given by the ``bounds``
76substructure of the struct :c:type:`v4l2_cropcap` returned
77by the :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` ioctl. To support a wide
78range of hardware this specification does not define an origin or units.
79However by convention drivers should horizontally count unscaled samples
80relative to 0H (the leading edge of the horizontal sync pulse, see
81:ref:`vbi-hsync`). Vertically ITU-R line numbers of the first field
82(see ITU R-525 line numbering for :ref:`525 lines <vbi-525>` and for
83:ref:`625 lines <vbi-625>`), multiplied by two if the driver
84can capture both fields.
85
86The top left corner, width and height of the source rectangle, that is
87the area actually sampled, is given by struct
88:c:type:`v4l2_crop` using the same coordinate system as
89struct :c:type:`v4l2_cropcap`. Applications can use the
90:ref:`VIDIOC_G_CROP <VIDIOC_G_CROP>` and :ref:`VIDIOC_S_CROP <VIDIOC_G_CROP>`
91ioctls to get and set this rectangle. It must lie completely within the
92capture boundaries and the driver may further adjust the requested size
93and/or position according to hardware limitations.
94
95Each capture device has a default source rectangle, given by the
96``defrect`` substructure of struct
97:c:type:`v4l2_cropcap`. The center of this rectangle
98shall align with the center of the active picture area of the video
99signal, and cover what the driver writer considers the complete picture.
100Drivers shall reset the source rectangle to the default when the driver
101is first loaded, but not later.
102
103For output devices these structures and ioctls are used accordingly,
104defining the *target* rectangle where the images will be inserted into
105the video signal.
106
107
108Scaling Adjustments
109===================
110
111Video hardware can have various cropping, insertion and scaling
112limitations. It may only scale up or down, support only discrete scaling
113factors, or have different scaling abilities in horizontal and vertical
114direction. Also it may not support scaling at all. At the same time the
115struct :c:type:`v4l2_crop` rectangle may have to be aligned,
116and both the source and target rectangles may have arbitrary upper and
117lower size limits. In particular the maximum ``width`` and ``height`` in
118struct :c:type:`v4l2_crop` may be smaller than the struct
119:c:type:`v4l2_cropcap`. ``bounds`` area. Therefore, as
120usual, drivers are expected to adjust the requested parameters and
121return the actual values selected.
122
123Applications can change the source or the target rectangle first, as
124they may prefer a particular image size or a certain area in the video
125signal. If the driver has to adjust both to satisfy hardware
126limitations, the last requested rectangle shall take priority, and the
127driver should preferably adjust the opposite one. The
128:ref:`VIDIOC_TRY_FMT <VIDIOC_G_FMT>` ioctl however shall not change
129the driver state and therefore only adjust the requested rectangle.
130
131Suppose scaling on a video capture device is restricted to a factor 1:1
132or 2:1 in either direction and the target image size must be a multiple
133of 16 × 16 pixels. The source cropping rectangle is set to defaults,
134which are also the upper limit in this example, of 640 × 400 pixels at
135offset 0, 0. An application requests an image size of 300 × 225 pixels,
136assuming video will be scaled down from the "full picture" accordingly.
137The driver sets the image size to the closest possible values 304 × 224,
138then chooses the cropping rectangle closest to the requested size, that
139is 608 × 224 (224 × 2:1 would exceed the limit 400). The offset 0, 0 is
140still valid, thus unmodified. Given the default cropping rectangle
141reported by :ref:`VIDIOC_CROPCAP <VIDIOC_CROPCAP>` the application can
142easily propose another offset to center the cropping rectangle.
143
144Now the application may insist on covering an area using a picture
145aspect ratio closer to the original request, so it asks for a cropping
146rectangle of 608 × 456 pixels. The present scaling factors limit
147cropping to 640 × 384, so the driver returns the cropping size 608 × 384
148and adjusts the image size to closest possible 304 × 192.
149
150
151Examples
152========
153
154Source and target rectangles shall remain unchanged across closing and
155reopening a device, such that piping data into or out of a device will
156work without special preparations. More advanced applications should
157ensure the parameters are suitable before starting I/O.
158
159.. note::
160
161   On the next two examples, a video capture device is assumed;
162   change ``V4L2_BUF_TYPE_VIDEO_CAPTURE`` for other types of device.
163
164Example: Resetting the cropping parameters
165==========================================
166
167.. code-block:: c
168
169    struct v4l2_cropcap cropcap;
170    struct v4l2_crop crop;
171
172    memset (&cropcap, 0, sizeof (cropcap));
173    cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
174
175    if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
176	perror ("VIDIOC_CROPCAP");
177	exit (EXIT_FAILURE);
178    }
179
180    memset (&crop, 0, sizeof (crop));
181    crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
182    crop.c = cropcap.defrect;
183
184    /* Ignore if cropping is not supported (EINVAL). */
185
186    if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
187	&& errno != EINVAL) {
188	perror ("VIDIOC_S_CROP");
189	exit (EXIT_FAILURE);
190    }
191
192
193Example: Simple downscaling
194===========================
195
196.. code-block:: c
197
198    struct v4l2_cropcap cropcap;
199    struct v4l2_format format;
200
201    reset_cropping_parameters ();
202
203    /* Scale down to 1/4 size of full picture. */
204
205    memset (&format, 0, sizeof (format)); /* defaults */
206
207    format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
208
209    format.fmt.pix.width = cropcap.defrect.width >> 1;
210    format.fmt.pix.height = cropcap.defrect.height >> 1;
211    format.fmt.pix.pixelformat = V4L2_PIX_FMT_YUYV;
212
213    if (-1 == ioctl (fd, VIDIOC_S_FMT, &format)) {
214	perror ("VIDIOC_S_FORMAT");
215	exit (EXIT_FAILURE);
216    }
217
218    /* We could check the actual image size now, the actual scaling factor
219       or if the driver can scale at all. */
220
221Example: Selecting an output area
222=================================
223
224.. note:: This example assumes an output device.
225
226.. code-block:: c
227
228    struct v4l2_cropcap cropcap;
229    struct v4l2_crop crop;
230
231    memset (&cropcap, 0, sizeof (cropcap));
232    cropcap.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
233
234    if (-1 == ioctl (fd, VIDIOC_CROPCAP;, &cropcap)) {
235	perror ("VIDIOC_CROPCAP");
236	exit (EXIT_FAILURE);
237    }
238
239    memset (&crop, 0, sizeof (crop));
240
241    crop.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;
242    crop.c = cropcap.defrect;
243
244    /* Scale the width and height to 50 % of their original size
245       and center the output. */
246
247    crop.c.width /= 2;
248    crop.c.height /= 2;
249    crop.c.left += crop.c.width / 2;
250    crop.c.top += crop.c.height / 2;
251
252    /* Ignore if cropping is not supported (EINVAL). */
253
254    if (-1 == ioctl (fd, VIDIOC_S_CROP, &crop)
255	&& errno != EINVAL) {
256	perror ("VIDIOC_S_CROP");
257	exit (EXIT_FAILURE);
258    }
259
260Example: Current scaling factor and pixel aspect
261================================================
262
263.. note:: This example assumes a video capture device.
264
265.. code-block:: c
266
267    struct v4l2_cropcap cropcap;
268    struct v4l2_crop crop;
269    struct v4l2_format format;
270    double hscale, vscale;
271    double aspect;
272    int dwidth, dheight;
273
274    memset (&cropcap, 0, sizeof (cropcap));
275    cropcap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
276
277    if (-1 == ioctl (fd, VIDIOC_CROPCAP, &cropcap)) {
278	perror ("VIDIOC_CROPCAP");
279	exit (EXIT_FAILURE);
280    }
281
282    memset (&crop, 0, sizeof (crop));
283    crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
284
285    if (-1 == ioctl (fd, VIDIOC_G_CROP, &crop)) {
286	if (errno != EINVAL) {
287	    perror ("VIDIOC_G_CROP");
288	    exit (EXIT_FAILURE);
289	}
290
291	/* Cropping not supported. */
292	crop.c = cropcap.defrect;
293    }
294
295    memset (&format, 0, sizeof (format));
296    format.fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
297
298    if (-1 == ioctl (fd, VIDIOC_G_FMT, &format)) {
299	perror ("VIDIOC_G_FMT");
300	exit (EXIT_FAILURE);
301    }
302
303    /* The scaling applied by the driver. */
304
305    hscale = format.fmt.pix.width / (double) crop.c.width;
306    vscale = format.fmt.pix.height / (double) crop.c.height;
307
308    aspect = cropcap.pixelaspect.numerator /
309	 (double) cropcap.pixelaspect.denominator;
310    aspect = aspect * hscale / vscale;
311
312    /* Devices following ITU-R BT.601 do not capture
313       square pixels. For playback on a computer monitor
314       we should scale the images to this size. */
315
316    dwidth = format.fmt.pix.width / aspect;
317    dheight = format.fmt.pix.height;
318