1.. SPDX-License-Identifier: GPL-2.0 2 3.. include:: <isonum.txt> 4 5======================================================== 6Intel Image Processing Unit 6 (IPU6) Input System driver 7======================================================== 8 9Copyright |copy| 2023--2024 Intel Corporation 10 11Introduction 12============ 13 14This file documents the Intel IPU6 (6th generation Image Processing Unit) 15Input System (MIPI CSI2 receiver) drivers located under 16drivers/media/pci/intel/ipu6. 17 18The Intel IPU6 can be found in certain Intel SoCs but not in all SKUs: 19 20* Tiger Lake 21* Jasper Lake 22* Alder Lake 23* Raptor Lake 24* Meteor Lake 25 26Intel IPU6 is made up of two components - Input System (ISYS) and Processing 27System (PSYS). 28 29The Input System mainly works as MIPI CSI-2 receiver which receives and 30processes the image data from the sensors and outputs the frames to memory. 31 32There are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an 33IPU6 common driver which does PCI configuration, firmware loading and parsing, 34firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) 35configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 36sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected 37to the IPU6 ISYS through V4L2 sub-device sensor drivers. 38 39.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more 40 information about the IPU6 hardware. 41 42Input system driver 43=================== 44 45The Input System driver mainly configures CSI-2 D-PHY, constructs the firmware 46stream configuration, sends commands to firmware, gets response from hardware 47and firmware and then returns buffers to user. The ISYS is represented as 48several V4L2 sub-devices as well as video nodes. 49 50.. kernel-figure:: ipu6_isys_graph.svg 51 :alt: ipu6 isys media graph with multiple streams support 52 53 IPU6 ISYS media graph with multiple streams support 54 55The graph has been produced using the following command: 56 57.. code-block:: none 58 59 fdp -Gsplines=true -Tsvg < dot > dot.svg 60 61Capturing frames with IPU6 ISYS 62------------------------------- 63 64IPU6 ISYS is used to capture frames from the camera sensors connected to the 65CSI2 ports. The supported input formats of ISYS are listed in table below: 66 67.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| 68 69.. flat-table:: 70 :header-rows: 1 71 72 * - IPU6 ISYS supported input formats 73 74 * - RGB565, RGB888 75 76 * - UYVY8, YUYV8 77 78 * - RAW8, RAW10, RAW12 79 80.. _ipu6_isys_capture_examples: 81 82Examples 83~~~~~~~~ 84 85Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this 86machine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can 87generate images at sBGGR10 with resolution 1280x800. 88 89Using the media controller APIs, we can configure ov01a10 sensor by 90media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. 91 92.. code-block:: none 93 94 # Example 1 capture frame from ov01a10 camera sensor 95 # This example assumes /dev/media0 as the IPU ISYS media device 96 export MDEV=/dev/media0 97 98 # Establish the link for the media devices using media-ctl 99 media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" 100 101 # Set the format for the media devices 102 media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" 103 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" 104 media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" 105 106Once the media pipeline is configured, desired sensor specific settings 107(such as exposure and gain settings) can be set, using the yavta tool. 108 109e.g 110 111.. code-block:: none 112 113 # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 114 export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") 115 116 yavta -w 0x009e0903 400 $SDEV 117 yavta -w 0x009e0913 1000 $SDEV 118 yavta -w 0x009e0911 2000 $SDEV 119 120Once the desired sensor settings are set, frame captures can be done as below. 121 122e.g 123 124.. code-block:: none 125 126 yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ 127 -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") 128 129With the above command, 10 frames are captured at 1280x800 resolution with 130sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. 131 132Here is another example of IPU6 ISYS RAW and metadata capture from camera 133sensor ov2740 on Lenovo X1 Yoga laptop. 134 135.. code-block:: none 136 137 media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]" 138 media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[1]" 139 media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[1]" 140 141 # set routing 142 media-ctl -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]" 143 144 media-ctl -V "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]" 145 media-ctl -V "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]" 146 media-ctl -V "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]" 147 media-ctl -V "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]" 148 149 CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0") 150 ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \ 151 -f SGRBG10 ${CAPTURE_DEV} 152 153 CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1") 154 ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \ 155 --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META} 156 157References 158========== 159 160.. [#f1] https://git.ideasonboard.org/media-ctl.git 161.. [#f2] https://git.ideasonboard.org/yavta.git 162
