1*ba124c8cSBingbu Cao.. SPDX-License-Identifier: GPL-2.0 2*ba124c8cSBingbu Cao 3*ba124c8cSBingbu Cao.. include:: <isonum.txt> 4*ba124c8cSBingbu Cao 5*ba124c8cSBingbu Cao======================================================== 6*ba124c8cSBingbu CaoIntel Image Processing Unit 6 (IPU6) Input System driver 7*ba124c8cSBingbu Cao======================================================== 8*ba124c8cSBingbu Cao 9*ba124c8cSBingbu CaoCopyright |copy| 2023--2024 Intel Corporation 10*ba124c8cSBingbu Cao 11*ba124c8cSBingbu CaoIntroduction 12*ba124c8cSBingbu Cao============ 13*ba124c8cSBingbu Cao 14*ba124c8cSBingbu CaoThis file documents the Intel IPU6 (6th generation Image Processing Unit) 15*ba124c8cSBingbu CaoInput System (MIPI CSI2 receiver) drivers located under 16*ba124c8cSBingbu Caodrivers/media/pci/intel/ipu6. 17*ba124c8cSBingbu Cao 18*ba124c8cSBingbu CaoThe Intel IPU6 can be found in certain Intel SoCs but not in all SKUs: 19*ba124c8cSBingbu Cao 20*ba124c8cSBingbu Cao* Tiger Lake 21*ba124c8cSBingbu Cao* Jasper Lake 22*ba124c8cSBingbu Cao* Alder Lake 23*ba124c8cSBingbu Cao* Raptor Lake 24*ba124c8cSBingbu Cao* Meteor Lake 25*ba124c8cSBingbu Cao 26*ba124c8cSBingbu CaoIntel IPU6 is made up of two components - Input System (ISYS) and Processing 27*ba124c8cSBingbu CaoSystem (PSYS). 28*ba124c8cSBingbu Cao 29*ba124c8cSBingbu CaoThe Input System mainly works as MIPI CSI-2 receiver which receives and 30*ba124c8cSBingbu Caoprocesses the image data from the sensors and outputs the frames to memory. 31*ba124c8cSBingbu Cao 32*ba124c8cSBingbu CaoThere are 2 driver modules - intel-ipu6 and intel-ipu6-isys. intel-ipu6 is an 33*ba124c8cSBingbu CaoIPU6 common driver which does PCI configuration, firmware loading and parsing, 34*ba124c8cSBingbu Caofirmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) 35*ba124c8cSBingbu Caoconfiguration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 36*ba124c8cSBingbu Caosub-device interfaces. The IPU6 ISYS driver supports camera sensors connected 37*ba124c8cSBingbu Caoto the IPU6 ISYS through V4L2 sub-device sensor drivers. 38*ba124c8cSBingbu Cao 39*ba124c8cSBingbu Cao.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more 40*ba124c8cSBingbu Cao information about the IPU6 hardware. 41*ba124c8cSBingbu Cao 42*ba124c8cSBingbu CaoInput system driver 43*ba124c8cSBingbu Cao=================== 44*ba124c8cSBingbu Cao 45*ba124c8cSBingbu CaoThe Input System driver mainly configures CSI-2 D-PHY, constructs the firmware 46*ba124c8cSBingbu Caostream configuration, sends commands to firmware, gets response from hardware 47*ba124c8cSBingbu Caoand firmware and then returns buffers to user. The ISYS is represented as 48*ba124c8cSBingbu Caoseveral V4L2 sub-devices as well as video nodes. 49*ba124c8cSBingbu Cao 50*ba124c8cSBingbu Cao.. kernel-figure:: ipu6_isys_graph.svg 51*ba124c8cSBingbu Cao :alt: ipu6 isys media graph with multiple streams support 52*ba124c8cSBingbu Cao 53*ba124c8cSBingbu Cao IPU6 ISYS media graph with multiple streams support 54*ba124c8cSBingbu Cao 55*ba124c8cSBingbu CaoThe graph has been produced using the following command: 56*ba124c8cSBingbu Cao 57*ba124c8cSBingbu Cao.. code-block:: none 58*ba124c8cSBingbu Cao 59*ba124c8cSBingbu Cao fdp -Gsplines=true -Tsvg < dot > dot.svg 60*ba124c8cSBingbu Cao 61*ba124c8cSBingbu CaoCapturing frames with IPU6 ISYS 62*ba124c8cSBingbu Cao------------------------------- 63*ba124c8cSBingbu Cao 64*ba124c8cSBingbu CaoIPU6 ISYS is used to capture frames from the camera sensors connected to the 65*ba124c8cSBingbu CaoCSI2 ports. The supported input formats of ISYS are listed in table below: 66*ba124c8cSBingbu Cao 67*ba124c8cSBingbu Cao.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| 68*ba124c8cSBingbu Cao 69*ba124c8cSBingbu Cao.. flat-table:: 70*ba124c8cSBingbu Cao :header-rows: 1 71*ba124c8cSBingbu Cao 72*ba124c8cSBingbu Cao * - IPU6 ISYS supported input formats 73*ba124c8cSBingbu Cao 74*ba124c8cSBingbu Cao * - RGB565, RGB888 75*ba124c8cSBingbu Cao 76*ba124c8cSBingbu Cao * - UYVY8, YUYV8 77*ba124c8cSBingbu Cao 78*ba124c8cSBingbu Cao * - RAW8, RAW10, RAW12 79*ba124c8cSBingbu Cao 80*ba124c8cSBingbu Cao.. _ipu6_isys_capture_examples: 81*ba124c8cSBingbu Cao 82*ba124c8cSBingbu CaoExamples 83*ba124c8cSBingbu Cao~~~~~~~~ 84*ba124c8cSBingbu Cao 85*ba124c8cSBingbu CaoHere is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this 86*ba124c8cSBingbu Caomachine, ov01a10 sensor is connected to IPU ISYS CSI-2 port 2, which can 87*ba124c8cSBingbu Caogenerate images at sBGGR10 with resolution 1280x800. 88*ba124c8cSBingbu Cao 89*ba124c8cSBingbu CaoUsing the media controller APIs, we can configure ov01a10 sensor by 90*ba124c8cSBingbu Caomedia-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. 91*ba124c8cSBingbu Cao 92*ba124c8cSBingbu Cao.. code-block:: none 93*ba124c8cSBingbu Cao 94*ba124c8cSBingbu Cao # Example 1 capture frame from ov01a10 camera sensor 95*ba124c8cSBingbu Cao # This example assumes /dev/media0 as the IPU ISYS media device 96*ba124c8cSBingbu Cao export MDEV=/dev/media0 97*ba124c8cSBingbu Cao 98*ba124c8cSBingbu Cao # Establish the link for the media devices using media-ctl 99*ba124c8cSBingbu Cao media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" 100*ba124c8cSBingbu Cao 101*ba124c8cSBingbu Cao # Set the format for the media devices 102*ba124c8cSBingbu Cao media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" 103*ba124c8cSBingbu Cao media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" 104*ba124c8cSBingbu Cao media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" 105*ba124c8cSBingbu Cao 106*ba124c8cSBingbu CaoOnce the media pipeline is configured, desired sensor specific settings 107*ba124c8cSBingbu Cao(such as exposure and gain settings) can be set, using the yavta tool. 108*ba124c8cSBingbu Cao 109*ba124c8cSBingbu Caoe.g 110*ba124c8cSBingbu Cao 111*ba124c8cSBingbu Cao.. code-block:: none 112*ba124c8cSBingbu Cao 113*ba124c8cSBingbu Cao # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 114*ba124c8cSBingbu Cao export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") 115*ba124c8cSBingbu Cao 116*ba124c8cSBingbu Cao yavta -w 0x009e0903 400 $SDEV 117*ba124c8cSBingbu Cao yavta -w 0x009e0913 1000 $SDEV 118*ba124c8cSBingbu Cao yavta -w 0x009e0911 2000 $SDEV 119*ba124c8cSBingbu Cao 120*ba124c8cSBingbu CaoOnce the desired sensor settings are set, frame captures can be done as below. 121*ba124c8cSBingbu Cao 122*ba124c8cSBingbu Caoe.g 123*ba124c8cSBingbu Cao 124*ba124c8cSBingbu Cao.. code-block:: none 125*ba124c8cSBingbu Cao 126*ba124c8cSBingbu Cao yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ 127*ba124c8cSBingbu Cao -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") 128*ba124c8cSBingbu Cao 129*ba124c8cSBingbu CaoWith the above command, 10 frames are captured at 1280x800 resolution with 130*ba124c8cSBingbu CaosBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. 131*ba124c8cSBingbu Cao 132*ba124c8cSBingbu CaoHere is another example of IPU6 ISYS RAW and metadata capture from camera 133*ba124c8cSBingbu Caosensor ov2740 on Lenovo X1 Yoga laptop. 134*ba124c8cSBingbu Cao 135*ba124c8cSBingbu Cao.. code-block:: none 136*ba124c8cSBingbu Cao 137*ba124c8cSBingbu Cao media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]" 138*ba124c8cSBingbu Cao media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[5]" 139*ba124c8cSBingbu Cao media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[5]" 140*ba124c8cSBingbu Cao 141*ba124c8cSBingbu Cao # set routing 142*ba124c8cSBingbu Cao media-ctl -v -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]" 143*ba124c8cSBingbu Cao 144*ba124c8cSBingbu Cao media-ctl -v "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]" 145*ba124c8cSBingbu Cao media-ctl -v "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]" 146*ba124c8cSBingbu Cao media-ctl -v "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]" 147*ba124c8cSBingbu Cao media-ctl -v "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]" 148*ba124c8cSBingbu Cao 149*ba124c8cSBingbu Cao CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0") 150*ba124c8cSBingbu Cao ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \ 151*ba124c8cSBingbu Cao -f SGRBG10 ${CAPTURE_DEV} 152*ba124c8cSBingbu Cao 153*ba124c8cSBingbu Cao CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1") 154*ba124c8cSBingbu Cao ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \ 155*ba124c8cSBingbu Cao --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META} 156*ba124c8cSBingbu Cao 157*ba124c8cSBingbu CaoReferences 158*ba124c8cSBingbu Cao========== 159*ba124c8cSBingbu Cao 160*ba124c8cSBingbu Cao.. [#f1] https://git.ideasonboard.org/media-ctl.git 161*ba124c8cSBingbu Cao.. [#f2] https://git.ideasonboard.org/yavta.git 162