1.. SPDX-License-Identifier: GPL-2.0 2 3Writing camera sensor drivers 4============================= 5 6CSI-2 7----- 8 9Please see what is written on :ref:`MIPI_CSI_2`. 10 11Handling clocks 12--------------- 13 14Camera sensors have an internal clock tree including a PLL and a number of 15divisors. The clock tree is generally configured by the driver based on a few 16input parameters that are specific to the hardware:: the external clock frequency 17and the link frequency. The two parameters generally are obtained from system 18firmware. **No other frequencies should be used in any circumstances.** 19 20The reason why the clock frequencies are so important is that the clock signals 21come out of the SoC, and in many cases a specific frequency is designed to be 22used in the system. Using another frequency may cause harmful effects 23elsewhere. Therefore only the pre-determined frequencies are configurable by the 24user. 25 26ACPI 27~~~~ 28 29Read the "clock-frequency" _DSD property to denote the frequency. The driver can 30rely on this frequency being used. 31 32Devicetree 33~~~~~~~~~~ 34 35The currently preferred way to achieve this is using "assigned-clock-rates" 36property. See Documentation/devicetree/bindings/clock/clock-bindings.txt for 37more information. The driver then gets the frequency using clk_get_rate(). 38 39This approach has the drawback that there's no guarantee that the frequency 40hasn't been modified directly or indirectly by another driver, or supported by 41the board's clock tree to begin with. Changes to the Common Clock Framework API 42are required to ensure reliability. 43 44Frame size 45---------- 46 47There are two distinct ways to configure the frame size produced by camera 48sensors. 49 50Freely configurable camera sensor drivers 51~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 52 53Freely configurable camera sensor drivers expose the device's internal 54processing pipeline as one or more sub-devices with different cropping and 55scaling configurations. The output size of the device is the result of a series 56of cropping and scaling operations from the device's pixel array's size. 57 58An example of such a driver is the smiapp driver (see drivers/media/i2c/smiapp). 59 60Register list based drivers 61~~~~~~~~~~~~~~~~~~~~~~~~~~~ 62 63Register list based drivers generally, instead of able to configure the device 64they control based on user requests, are limited to a number of preset 65configurations that combine a number of different parameters that on hardware 66level are independent. How a driver picks such configuration is based on the 67format set on a source pad at the end of the device's internal pipeline. 68 69Most sensor drivers are implemented this way, see e.g. 70drivers/media/i2c/imx319.c for an example. 71 72Frame interval configuration 73---------------------------- 74 75There are two different methods for obtaining possibilities for different frame 76intervals as well as configuring the frame interval. Which one to implement 77depends on the type of the device. 78 79Raw camera sensors 80~~~~~~~~~~~~~~~~~~ 81 82Instead of a high level parameter such as frame interval, the frame interval is 83a result of the configuration of a number of camera sensor implementation 84specific parameters. Luckily, these parameters tend to be the same for more or 85less all modern raw camera sensors. 86 87The frame interval is calculated using the following equation:: 88 89 frame interval = (analogue crop width + horizontal blanking) * 90 (analogue crop height + vertical blanking) / pixel rate 91 92The formula is bus independent and is applicable for raw timing parameters on 93large variety of devices beyond camera sensors. Devices that have no analogue 94crop, use the full source image size, i.e. pixel array size. 95 96Horizontal and vertical blanking are specified by ``V4L2_CID_HBLANK`` and 97``V4L2_CID_VBLANK``, respectively. The unit of these controls are lines. The 98pixel rate is specified by ``V4L2_CID_PIXEL_RATE`` in the same sub-device. The 99unit of that control is Hz. 100 101Register list based drivers need to implement read-only sub-device nodes for the 102purpose. Devices that are not register list based need these to configure the 103device's internal processing pipeline. 104 105The first entity in the linear pipeline is the pixel array. The pixel array may 106be followed by other entities that are there to allow configuring binning, 107skipping, scaling or digital crop :ref:`v4l2-subdev-selections`. 108 109USB cameras etc. devices 110~~~~~~~~~~~~~~~~~~~~~~~~ 111 112USB video class hardware, as well as many cameras offering a similar higher 113level interface natively, generally use the concept of frame interval (or frame 114rate) on device level in firmware or hardware. This means lower level controls 115implemented by raw cameras may not be used on uAPI (or even kAPI) to control the 116frame interval on these devices. 117 118Power management 119---------------- 120 121Always use runtime PM to manage the power states of your device. Camera sensor 122drivers are in no way special in this respect: they are responsible for 123controlling the power state of the device they otherwise control as well. In 124general, the device must be powered on at least when its registers are being 125accessed and when it is streaming. 126 127Existing camera sensor drivers may rely on the old 128:c:type:`v4l2_subdev_core_ops`->s_power() callback for bridge or ISP drivers to 129manage their power state. This is however **deprecated**. If you feel you need 130to begin calling an s_power from an ISP or a bridge driver, instead please add 131runtime PM support to the sensor driver you are using. Likewise, new drivers 132should not use s_power. 133 134Please see examples in e.g. ``drivers/media/i2c/ov8856.c`` and 135``drivers/media/i2c/smiapp/smiapp-core.c``. The two drivers work in both ACPI 136and DT based systems. 137 138Control framework 139~~~~~~~~~~~~~~~~~ 140 141``v4l2_ctrl_handler_setup()`` function may not be used in the device's runtime 142PM ``runtime_resume`` callback, as it has no way to figure out the power state 143of the device. This is because the power state of the device is only changed 144after the power state transition has taken place. The ``s_ctrl`` callback can be 145used to obtain device's power state after the power state transition: 146 147.. c:function:: int pm_runtime_get_if_in_use(struct device *dev); 148 149The function returns a non-zero value if it succeeded getting the power count or 150runtime PM was disabled, in either of which cases the driver may proceed to 151access the device. 152 153Controls 154-------- 155 156For camera sensors that are connected to a bus where transmitter and receiver 157require common configuration set by drivers, such as CSI-2 or parallel (BT.601 158or BT.656) bus, the ``V4L2_CID_LINK_FREQ`` control is mandatory on transmitter 159drivers. Receiver drivers can use the ``V4L2_CID_LINK_FREQ`` to query the 160frequency used on the bus. 161 162The transmitter drivers should also implement ``V4L2_CID_PIXEL_RATE`` control in 163order to tell the maximum pixel rate to the receiver. This is required on raw 164camera sensors. 165