mirror of
https://github.com/torvalds/linux
synced 2024-09-22 20:37:18 +00:00
media: staging/ipu3-imgu: Address documentation comments
Address comments on the documentation after Yong's original patch. Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
This commit is contained in:
parent
d0276a2b83
commit
5e44465ccf
|
@ -175,4 +175,4 @@ video node in ``V4L2_BUF_TYPE_META_CAPTURE`` format.
|
||||||
Intel IPU3 ImgU uAPI data types
|
Intel IPU3 ImgU uAPI data types
|
||||||
===============================
|
===============================
|
||||||
|
|
||||||
.. kernel-doc:: include/uapi/linux/intel-ipu3.h
|
.. kernel-doc:: drivers/staging/media/ipu3/include/intel-ipu3.h
|
||||||
|
|
|
@ -9,67 +9,180 @@ Copyright |copy| 2018 Intel Corporation
|
||||||
Introduction
|
Introduction
|
||||||
============
|
============
|
||||||
|
|
||||||
This file documents Intel IPU3 (3rd generation Image Processing Unit) Imaging
|
This file documents the Intel IPU3 (3rd generation Image Processing Unit)
|
||||||
Unit driver located under drivers/media/pci/intel/ipu3.
|
Imaging Unit drivers located under drivers/media/pci/intel/ipu3 (CIO2) as well
|
||||||
|
as under drivers/staging/media/ipu3 (ImgU).
|
||||||
|
|
||||||
The Intel IPU3 found in certain Kaby Lake (as well as certain Sky Lake)
|
The Intel IPU3 found in certain Kaby Lake (as well as certain Sky Lake)
|
||||||
platforms (U/Y processor lines) is made up of two parts namely Imaging Unit
|
platforms (U/Y processor lines) is made up of two parts namely the Imaging Unit
|
||||||
(ImgU) and CIO2 device (MIPI CSI2 receiver).
|
(ImgU) and the CIO2 device (MIPI CSI2 receiver).
|
||||||
|
|
||||||
The CIO2 device receives the raw bayer data from the sensors and outputs the
|
The CIO2 device receives the raw Bayer data from the sensors and outputs the
|
||||||
frames in a format that is specific to IPU3 (for consumption by IPU3 ImgU).
|
frames in a format that is specific to the IPU3 (for consumption by the IPU3
|
||||||
CIO2 driver is available as drivers/media/pci/intel/ipu3/ipu3-cio2* and is
|
ImgU). The CIO2 driver is available as drivers/media/pci/intel/ipu3/ipu3-cio2*
|
||||||
enabled through the CONFIG_VIDEO_IPU3_CIO2 config option.
|
and is enabled through the CONFIG_VIDEO_IPU3_CIO2 config option.
|
||||||
|
|
||||||
The Imaging Unit (ImgU) is responsible for processing images captured
|
The Imaging Unit (ImgU) is responsible for processing images captured
|
||||||
through IPU3 CIO2 device. The ImgU driver sources can be found under
|
by the IPU3 CIO2 device. The ImgU driver sources can be found under
|
||||||
drivers/media/pci/intel/ipu3 directory. The driver is enabled through the
|
drivers/staging/media/ipu3 directory. The driver is enabled through the
|
||||||
CONFIG_VIDEO_IPU3_IMGU config option.
|
CONFIG_VIDEO_IPU3_IMGU config option.
|
||||||
|
|
||||||
The two driver modules are named ipu3-csi2 and ipu3-imgu, respectively.
|
The two driver modules are named ipu3_csi2 and ipu3_imgu, respectively.
|
||||||
|
|
||||||
The driver has been tested on Kaby Lake platforms (U/Y processor lines).
|
The drivers has been tested on Kaby Lake platforms (U/Y processor lines).
|
||||||
|
|
||||||
The driver implements V4L2, Media controller and V4L2 sub-device interfaces.
|
Both of the drivers implement V4L2, Media Controller and V4L2 sub-device
|
||||||
Camera sensors that have CSI-2 bus, which are connected to the IPU3 CIO2
|
interfaces. The IPU3 CIO2 driver supports camera sensors connected to the CIO2
|
||||||
device are supported. Support for lens and flash drivers depends on the
|
MIPI CSI-2 interfaces through V4L2 sub-device sensor drivers.
|
||||||
above sensors.
|
|
||||||
|
|
||||||
ImgU device nodes
|
CIO2
|
||||||
=================
|
====
|
||||||
|
|
||||||
The ImgU is represented as two V4L2 subdevs, each of which provides a V4L2
|
|
||||||
subdev interface to the user space.
|
|
||||||
|
|
||||||
Each V4L2 subdev represents a pipe, which can support a maximum of 2
|
|
||||||
streams. A private ioctl can be used to configure the mode (video or still)
|
|
||||||
of the pipe.
|
|
||||||
|
|
||||||
This helps to support advanced camera features like Continuous View Finder
|
|
||||||
(CVF) and Snapshot During Video(SDV).
|
|
||||||
|
|
||||||
CIO2 device
|
|
||||||
===========
|
|
||||||
|
|
||||||
The CIO2 is represented as a single V4L2 subdev, which provides a V4L2 subdev
|
The CIO2 is represented as a single V4L2 subdev, which provides a V4L2 subdev
|
||||||
interface to the user space. There is a video node for each CSI-2 receiver,
|
interface to the user space. There is a video node for each CSI-2 receiver,
|
||||||
with a single media controller interface for the entire device.
|
with a single media controller interface for the entire device.
|
||||||
|
|
||||||
Media controller
|
The CIO2 contains four independent capture channel, each with its own MIPI CSI-2
|
||||||
----------------
|
receiver and DMA engine. Each channel is modelled as a V4L2 sub-device exposed
|
||||||
|
to userspace as a V4L2 sub-device node and has two pads:
|
||||||
|
|
||||||
The media device interface allows to configure the ImgU links, which defines
|
.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
|
||||||
the behavior of the IPU3 firmware.
|
|
||||||
|
.. flat-table::
|
||||||
|
|
||||||
|
* - pad
|
||||||
|
- direction
|
||||||
|
- purpose
|
||||||
|
|
||||||
|
* - 0
|
||||||
|
- sink
|
||||||
|
- MIPI CSI-2 input, connected to the sensor subdev
|
||||||
|
|
||||||
|
* - 1
|
||||||
|
- source
|
||||||
|
- Raw video capture, connected to the V4L2 video interface
|
||||||
|
|
||||||
|
The V4L2 video interfaces model the DMA engines. They are exposed to userspace
|
||||||
|
as V4L2 video device nodes.
|
||||||
|
|
||||||
|
Capturing frames in raw Bayer format
|
||||||
|
------------------------------------
|
||||||
|
|
||||||
|
CIO2 MIPI CSI2 receiver is used to capture frames (in packed raw Bayer format)
|
||||||
|
from the raw sensors connected to the CSI2 ports. The captured frames are used
|
||||||
|
as input to the ImgU driver.
|
||||||
|
|
||||||
|
Image processing using IPU3 ImgU requires tools such as raw2pnm [#f1]_, and
|
||||||
|
yavta [#f2]_ due to the following unique requirements and / or features specific
|
||||||
|
to IPU3.
|
||||||
|
|
||||||
|
-- The IPU3 CSI2 receiver outputs the captured frames from the sensor in packed
|
||||||
|
raw Bayer format that is specific to IPU3.
|
||||||
|
|
||||||
|
-- Multiple video nodes have to be operated simultaneously.
|
||||||
|
|
||||||
|
Let us take the example of ov5670 sensor connected to CSI2 port 0, for a
|
||||||
|
2592x1944 image capture.
|
||||||
|
|
||||||
|
Using the media contorller APIs, the ov5670 sensor is configured to send
|
||||||
|
frames in packed raw Bayer format to IPU3 CSI2 receiver.
|
||||||
|
|
||||||
|
# This example assumes /dev/media0 as the CIO2 media device
|
||||||
|
|
||||||
|
export MDEV=/dev/media0
|
||||||
|
|
||||||
|
# and that ov5670 sensor is connected to i2c bus 10 with address 0x36
|
||||||
|
|
||||||
|
export SDEV=$(media-ctl -d $MDEV -e "ov5670 10-0036")
|
||||||
|
|
||||||
|
# Establish the link for the media devices using media-ctl [#f3]_
|
||||||
|
media-ctl -d $MDEV -l "ov5670:0 -> ipu3-csi2 0:0[1]"
|
||||||
|
|
||||||
|
# Set the format for the media devices
|
||||||
|
media-ctl -d $MDEV -V "ov5670:0 [fmt:SGRBG10/2592x1944]"
|
||||||
|
|
||||||
|
media-ctl -d $MDEV -V "ipu3-csi2 0:0 [fmt:SGRBG10/2592x1944]"
|
||||||
|
|
||||||
|
media-ctl -d $MDEV -V "ipu3-csi2 0:1 [fmt:SGRBG10/2592x1944]"
|
||||||
|
|
||||||
|
Once the media pipeline is configured, desired sensor specific settings
|
||||||
|
(such as exposure and gain settings) can be set, using the yavta tool.
|
||||||
|
|
||||||
|
e.g
|
||||||
|
|
||||||
|
yavta -w 0x009e0903 444 $SDEV
|
||||||
|
|
||||||
|
yavta -w 0x009e0913 1024 $SDEV
|
||||||
|
|
||||||
|
yavta -w 0x009e0911 2046 $SDEV
|
||||||
|
|
||||||
|
Once the desired sensor settings are set, frame captures can be done as below.
|
||||||
|
|
||||||
|
e.g
|
||||||
|
|
||||||
|
yavta --data-prefix -u -c10 -n5 -I -s2592x1944 --file=/tmp/frame-#.bin \
|
||||||
|
-f IPU3_SGRBG10 $(media-ctl -d $MDEV -e "ipu3-cio2 0")
|
||||||
|
|
||||||
|
With the above command, 10 frames are captured at 2592x1944 resolution, with
|
||||||
|
sGRBG10 format and output as IPU3_SGRBG10 format.
|
||||||
|
|
||||||
|
The captured frames are available as /tmp/frame-#.bin files.
|
||||||
|
|
||||||
|
ImgU
|
||||||
|
====
|
||||||
|
|
||||||
|
The ImgU is represented as two V4L2 subdevs, each of which provides a V4L2
|
||||||
|
subdev interface to the user space.
|
||||||
|
|
||||||
|
Each V4L2 subdev represents a pipe, which can support a maximum of 2 streams.
|
||||||
|
This helps to support advanced camera features like Continuous View Finder (CVF)
|
||||||
|
and Snapshot During Video(SDV).
|
||||||
|
|
||||||
|
The ImgU contains two independent pipes, each modelled as a V4L2 sub-device
|
||||||
|
exposed to userspace as a V4L2 sub-device node.
|
||||||
|
|
||||||
|
Each pipe has two sink pads and three source pads for the following purpose:
|
||||||
|
|
||||||
|
.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}|
|
||||||
|
|
||||||
|
.. flat-table::
|
||||||
|
|
||||||
|
* - pad
|
||||||
|
- direction
|
||||||
|
- purpose
|
||||||
|
|
||||||
|
* - 0
|
||||||
|
- sink
|
||||||
|
- Input raw video stream
|
||||||
|
|
||||||
|
* - 1
|
||||||
|
- sink
|
||||||
|
- Processing parameters
|
||||||
|
|
||||||
|
* - 2
|
||||||
|
- source
|
||||||
|
- Output processed video stream
|
||||||
|
|
||||||
|
* - 3
|
||||||
|
- source
|
||||||
|
- Output viewfinder video stream
|
||||||
|
|
||||||
|
* - 4
|
||||||
|
- source
|
||||||
|
- 3A statistics
|
||||||
|
|
||||||
|
Each pad is connected to a corresponding V4L2 video interface, exposed to
|
||||||
|
userspace as a V4L2 video device node.
|
||||||
|
|
||||||
Device operation
|
Device operation
|
||||||
----------------
|
----------------
|
||||||
|
|
||||||
With IPU3, once the input video node ("ipu3-imgu 0/1":0,
|
With ImgU, once the input video node ("ipu3-imgu 0/1":0, in
|
||||||
in <entity>:<pad-number> format) is queued with buffer (in packed raw bayer
|
<entity>:<pad-number> format) is queued with buffer (in packed raw Bayer
|
||||||
format), IPU3 ISP starts processing the buffer and produces the video output
|
format), ImgU starts processing the buffer and produces the video output in YUV
|
||||||
in YUV format and statistics output on respective output nodes. The driver
|
format and statistics output on respective output nodes. The driver is expected
|
||||||
is expected to have buffers ready for all of parameter, output and
|
to have buffers ready for all of parameter, output and statistics nodes, when
|
||||||
statistics nodes, when input video node is queued with buffer.
|
input video node is queued with buffer.
|
||||||
|
|
||||||
At a minimum, all of input, main output, 3A statistics and viewfinder
|
At a minimum, all of input, main output, 3A statistics and viewfinder
|
||||||
video nodes should be enabled for IPU3 to start image processing.
|
video nodes should be enabled for IPU3 to start image processing.
|
||||||
|
@ -79,126 +192,58 @@ Each ImgU V4L2 subdev has the following set of video nodes.
|
||||||
input, output and viewfinder video nodes
|
input, output and viewfinder video nodes
|
||||||
----------------------------------------
|
----------------------------------------
|
||||||
|
|
||||||
The frames (in packed raw bayer format specific to IPU3) received by the
|
The frames (in packed raw Bayer format specific to the IPU3) received by the
|
||||||
input video node is processed by the IPU3 Imaging Unit and is output to 2
|
input video node is processed by the IPU3 Imaging Unit and are output to 2 video
|
||||||
video nodes, with each targeting different purpose (main output and viewfinder
|
nodes, with each targeting a different purpose (main output and viewfinder
|
||||||
output).
|
output).
|
||||||
|
|
||||||
Details on raw bayer format specific to IPU3 can be found as below.
|
Details onand the Bayer format specific to the IPU3 can be found in
|
||||||
Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
|
:ref:`v4l2-pix-fmt-ipu3-sbggr10`.
|
||||||
|
|
||||||
The driver supports V4L2 Video Capture Interface as defined at :ref:`devices`.
|
The driver supports V4L2 Video Capture Interface as defined at :ref:`devices`.
|
||||||
|
|
||||||
Only the multi-planar API is supported. More details can be found at
|
Only the multi-planar API is supported. More details can be found at
|
||||||
:ref:`planar-apis`.
|
:ref:`planar-apis`.
|
||||||
|
|
||||||
|
Parameters video node
|
||||||
parameters video node
|
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
The parameter video node receives the ISP algorithm parameters that are used
|
The parameters video node receives the ImgU algorithm parameters that are used
|
||||||
to configure how the ISP algorithms process the image.
|
to configure how the ImgU algorithms process the image.
|
||||||
|
|
||||||
Details on raw bayer format specific to IPU3 can be found as below.
|
Details on processing parameters specific to the IPU3 can be found in
|
||||||
Documentation/media/uapi/v4l/pixfmt-meta-intel-ipu3.rst
|
:ref:`v4l2-meta-fmt-params`.
|
||||||
|
|
||||||
3A statistics video node
|
3A statistics video node
|
||||||
------------------------
|
------------------------
|
||||||
|
|
||||||
3A statistics video node is used by the ImgU driver to output the 3A (auto
|
3A statistics video node is used by the ImgU driver to output the 3A (auto
|
||||||
focus, auto exposure and auto white balance) statistics for the frames that
|
focus, auto exposure and auto white balance) statistics for the frames that are
|
||||||
are being processed by the ISP to user space applications. User space
|
being processed by the ImgU to user space applications. User space applications
|
||||||
applications can use this statistics data to arrive at desired algorithm
|
can use this statistics data to compute the desired algorithm parameters for
|
||||||
parameters for ISP.
|
the ImgU.
|
||||||
|
|
||||||
CIO2 device nodes
|
|
||||||
=================
|
|
||||||
|
|
||||||
CIO2 is represented as a single V4L2 sub-device with a video node for each
|
|
||||||
CSI-2 receiver. The video node represents the DMA engine.
|
|
||||||
|
|
||||||
Configuring the Intel IPU3
|
Configuring the Intel IPU3
|
||||||
==========================
|
==========================
|
||||||
|
|
||||||
The Intel IPU3 ImgU driver supports V4L2 interface. Using V4L2 ioctl calls,
|
The IPU3 ImgU pipelines can be configured using the Media Controller, defined at
|
||||||
the ISP can be configured and enabled.
|
:ref:`media_controller`.
|
||||||
|
|
||||||
The IPU3 ImgU pipelines can be configured using media controller APIs,
|
Firmware binary selection
|
||||||
defined at :ref:`media_controller`.
|
-------------------------
|
||||||
|
|
||||||
Capturing frames in raw bayer format
|
The firmware binary is selected using the V4L2_CID_INTEL_IPU3_MODE, currently
|
||||||
------------------------------------
|
defined in drivers/staging/media/ipu3/include/intel-ipu3.h . "VIDEO" and "STILL"
|
||||||
|
modes are available.
|
||||||
|
|
||||||
IPU3 MIPI CSI2 receiver is used to capture frames (in packed raw bayer
|
Processing the image in raw Bayer format
|
||||||
format) from the raw sensors connected to the CSI2 ports. The captured
|
|
||||||
frames are used as input to the ImgU driver.
|
|
||||||
|
|
||||||
Image processing using IPU3 ImgU requires tools such as v4l2n [#f1]_,
|
|
||||||
raw2pnm [#f1]_, and yavta [#f2]_ due to the following unique requirements
|
|
||||||
and / or features specific to IPU3.
|
|
||||||
|
|
||||||
-- The IPU3 CSI2 receiver outputs the captured frames from the sensor in
|
|
||||||
packed raw bayer format that is specific to IPU3
|
|
||||||
|
|
||||||
-- Multiple video nodes have to be operated simultaneously
|
|
||||||
|
|
||||||
Let us take the example of ov5670 sensor connected to CSI2 port 0, for a
|
|
||||||
2592x1944 image capture.
|
|
||||||
|
|
||||||
Using the media contorller APIs, the ov5670 sensor is configured to send
|
|
||||||
frames in packed raw bayer format to IPU3 CSI2 receiver.
|
|
||||||
|
|
||||||
# This example assumes /dev/media0 as the ImgU media device
|
|
||||||
|
|
||||||
export MDEV=/dev/media0
|
|
||||||
|
|
||||||
# and that ov5670 sensor is connected to i2c bus 10 with address 0x36
|
|
||||||
|
|
||||||
export SDEV="ov5670 10-0036"
|
|
||||||
|
|
||||||
# Establish the link for the media devices using media-ctl [#f3]_
|
|
||||||
media-ctl -d $MDEV -l "ov5670 ":0 -> "ipu3-csi2 0":0[1]
|
|
||||||
|
|
||||||
media-ctl -d $MDEV -l "ipu3-csi2 0":1 -> "ipu3-cio2 0":0[1]
|
|
||||||
|
|
||||||
# Set the format for the media devices
|
|
||||||
media-ctl -d $MDEV -V "ov5670 ":0 [fmt:SGRBG10/2592x1944]
|
|
||||||
|
|
||||||
media-ctl -d $MDEV -V "ipu3-csi2 0":0 [fmt:SGRBG10/2592x1944]
|
|
||||||
|
|
||||||
media-ctl -d $MDEV -V "ipu3-csi2 0":1 [fmt:SGRBG10/2592x1944]
|
|
||||||
|
|
||||||
Once the media pipeline is configured, desired sensor specific settings
|
|
||||||
(such as exposure and gain settings) can be set, using the yavta tool.
|
|
||||||
|
|
||||||
e.g
|
|
||||||
|
|
||||||
yavta -w 0x009e0903 444 $(media-ctl -d $MDEV -e "$SDEV")
|
|
||||||
|
|
||||||
yavta -w 0x009e0913 1024 $(media-ctl -d $MDEV -e "$SDEV")
|
|
||||||
|
|
||||||
yavta -w 0x009e0911 2046 $(media-ctl -d $MDEV -e "$SDEV")
|
|
||||||
|
|
||||||
Once the desired sensor settings are set, frame captures can be done as below.
|
|
||||||
|
|
||||||
e.g
|
|
||||||
|
|
||||||
yavta --data-prefix -u -c10 -n5 -I -s2592x1944 --file=/tmp/frame-#.bin
|
|
||||||
-f IPU3_GRBG10 media-ctl -d $MDEV -e ipu3-cio2 0
|
|
||||||
|
|
||||||
With the above command, 10 frames are captured at 2592x1944 resolution, with
|
|
||||||
sGRBG10 format and output as IPU3_GRBG10 format.
|
|
||||||
|
|
||||||
The captured frames are available as /tmp/frame-#.bin files.
|
|
||||||
|
|
||||||
Processing the image in raw bayer format
|
|
||||||
----------------------------------------
|
----------------------------------------
|
||||||
|
|
||||||
Configuring ImgU V4L2 subdev for image processing
|
Configuring ImgU V4L2 subdev for image processing
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The ImgU V4L2 subdevs have to be configured with media controller APIs to
|
The ImgU V4L2 subdevs have to be configured with media controller APIs to have
|
||||||
have all the video nodes setup correctly.
|
all the video nodes setup correctly.
|
||||||
|
|
||||||
Let us take "ipu3-imgu 0" subdev as an example.
|
Let us take "ipu3-imgu 0" subdev as an example.
|
||||||
|
|
||||||
|
@ -206,40 +251,38 @@ media-ctl -d $MDEV -r
|
||||||
|
|
||||||
media-ctl -d $MDEV -l "ipu3-imgu 0 input":0 -> "ipu3-imgu 0":0[1]
|
media-ctl -d $MDEV -l "ipu3-imgu 0 input":0 -> "ipu3-imgu 0":0[1]
|
||||||
|
|
||||||
media-ctl -d $MDEV -l "ipu3-imgu 0":2 -> "output":0[1]
|
media-ctl -d $MDEV -l "ipu3-imgu 0":2 -> "ipu3-imgu 0 output":0[1]
|
||||||
|
|
||||||
media-ctl -d $MDEV -l "ipu3-imgu 0":3 -> "viewfinder":0[1]
|
media-ctl -d $MDEV -l "ipu3-imgu 0":3 -> "ipu3-imgu 0 viewfinder":0[1]
|
||||||
|
|
||||||
media-ctl -d $MDEV -l "ipu3-imgu 0":4 -> "3a stat":0[1]
|
media-ctl -d $MDEV -l "ipu3-imgu 0":4 -> "ipu3-imgu 0 3a stat":0[1]
|
||||||
|
|
||||||
Also the pipe mode of the corresponding V4L2 subdev should be set as
|
Also the pipe mode of the corresponding V4L2 subdev should be set as desired
|
||||||
desired (e.g 0 for video mode or 1 for still mode) through the
|
(e.g 0 for video mode or 1 for still mode) through the control id 0x009819a1 as
|
||||||
control id 0x009819a1 as below.
|
below.
|
||||||
|
|
||||||
e.g
|
yavta -w "0x009819A1 1" /dev/v4l-subdev7
|
||||||
|
|
||||||
v4l2n -d /dev/v4l-subdev7 --ctrl=0x009819A1=1
|
RAW Bayer frames go through the following ImgU pipeline HW blocks to have the
|
||||||
|
processed image output to the DDR memory.
|
||||||
|
|
||||||
RAW bayer frames go through the following ISP pipeline HW blocks to
|
RAW Bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> Geometric
|
||||||
have the processed image output to the DDR memory.
|
|
||||||
|
|
||||||
RAW bayer frame -> Input Feeder -> Bayer Down Scaling (BDS) -> Geometric
|
|
||||||
Distortion Correction (GDC) -> DDR
|
Distortion Correction (GDC) -> DDR
|
||||||
|
|
||||||
The ImgU V4L2 subdev has to be configured with the supported resolutions
|
The ImgU V4L2 subdev has to be configured with the supported resolutions in all
|
||||||
in all the above HW blocks, for a given input resolution.
|
the above HW blocks, for a given input resolution.
|
||||||
|
|
||||||
For a given supported resolution for an input frame, the Input Feeder,
|
For a given supported resolution for an input frame, the Input Feeder, Bayer
|
||||||
Bayer Down Scaling and GDC blocks should be configured with the supported
|
Down Scaling and GDC blocks should be configured with the supported resolutions.
|
||||||
resolutions. This information can be obtained by looking at the following
|
This information can be obtained by looking at the following IPU3 ImgU
|
||||||
IPU3 ISP configuration table.
|
configuration table.
|
||||||
|
|
||||||
https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/master
|
https://chromium.googlesource.com/chromiumos/overlays/board-overlays/+/master
|
||||||
|
|
||||||
Under baseboard-poppy/media-libs/arc-camera3-hal-configs-poppy/files/gcss
|
Under baseboard-poppy/media-libs/cros-camera-hal-configs-poppy/files/gcss
|
||||||
directory, graph_settings_ov5670.xml can be used as an example.
|
directory, graph_settings_ov5670.xml can be used as an example.
|
||||||
|
|
||||||
The following steps prepare the ImgU ISP pipeline for the image processing.
|
The following steps prepare the ImgU pipeline for the image processing.
|
||||||
|
|
||||||
1. The ImgU V4L2 subdev data format should be set by using the
|
1. The ImgU V4L2 subdev data format should be set by using the
|
||||||
VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained above.
|
VIDIOC_SUBDEV_S_FMT on pad 0, using the GDC width and height obtained above.
|
||||||
|
@ -257,14 +300,14 @@ For the ov5670 example, for an input frame with a resolution of 2592x1944
|
||||||
for input feeder, BDS and GDC are 2592x1944, 2592x1944 and 2560x1920
|
for input feeder, BDS and GDC are 2592x1944, 2592x1944 and 2560x1920
|
||||||
respectively.
|
respectively.
|
||||||
|
|
||||||
Once this is done, the received raw bayer frames can be input to the ImgU
|
Once this is done, the received raw Bayer frames can be input to the ImgU
|
||||||
V4L2 subdev as below, using the open source application v4l2n.
|
V4L2 subdev as below, using the open source application v4l2n [#f1]_.
|
||||||
|
|
||||||
For an image captured with 2592x1944 [#f4]_ resolution, with desired output
|
For an image captured with 2592x1944 [#f4]_ resolution, with desired output
|
||||||
resolution as 2560x1920 and viewfinder resolution as 2560x1920, the following
|
resolution as 2560x1920 and viewfinder resolution as 2560x1920, the following
|
||||||
v4l2n command can be used. This helps process the raw bayer frames and
|
v4l2n command can be used. This helps process the raw Bayer frames and produces
|
||||||
produces the desired results for the main output image and the viewfinder
|
the desired results for the main output image and the viewfinder output, in NV12
|
||||||
output, in NV12 format.
|
format.
|
||||||
|
|
||||||
v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4
|
v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4
|
||||||
--fmt=type:VIDEO_OUTPUT_MPLANE,width=2592,height=1944,pixelformat=0X47337069
|
--fmt=type:VIDEO_OUTPUT_MPLANE,width=2592,height=1944,pixelformat=0X47337069
|
||||||
|
@ -281,7 +324,7 @@ v4l2n --pipe=4 --load=/tmp/frame-#.bin --open=/dev/video4
|
||||||
where /dev/video4, /dev/video5, /dev/video6 and /dev/video7 devices point to
|
where /dev/video4, /dev/video5, /dev/video6 and /dev/video7 devices point to
|
||||||
input, output, viewfinder and 3A statistics video nodes respectively.
|
input, output, viewfinder and 3A statistics video nodes respectively.
|
||||||
|
|
||||||
Converting the raw bayer image into YUV domain
|
Converting the raw Bayer image into YUV domain
|
||||||
----------------------------------------------
|
----------------------------------------------
|
||||||
|
|
||||||
The processed images after the above step, can be converted to YUV domain
|
The processed images after the above step, can be converted to YUV domain
|
||||||
|
@ -290,7 +333,7 @@ as below.
|
||||||
Main output frames
|
Main output frames
|
||||||
~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.out /tmp/frames.out.pnm
|
raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.out /tmp/frames.out.ppm
|
||||||
|
|
||||||
where 2560x1920 is output resolution, NV12 is the video format, followed
|
where 2560x1920 is output resolution, NV12 is the video format, followed
|
||||||
by input frame and output PNM file.
|
by input frame and output PNM file.
|
||||||
|
@ -298,7 +341,7 @@ by input frame and output PNM file.
|
||||||
Viewfinder output frames
|
Viewfinder output frames
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.vf /tmp/frames.vf.pnm
|
raw2pnm -x2560 -y1920 -fNV12 /tmp/frames.vf /tmp/frames.vf.ppm
|
||||||
|
|
||||||
where 2560x1920 is output resolution, NV12 is the video format, followed
|
where 2560x1920 is output resolution, NV12 is the video format, followed
|
||||||
by input frame and output PNM file.
|
by input frame and output PNM file.
|
||||||
|
@ -315,7 +358,7 @@ The source can be located under hal/intel directory.
|
||||||
References
|
References
|
||||||
==========
|
==========
|
||||||
|
|
||||||
include/uapi/linux/intel-ipu3.h
|
.. [#f5] include/uapi/linux/intel-ipu3.h
|
||||||
|
|
||||||
.. [#f1] https://github.com/intel/nvt
|
.. [#f1] https://github.com/intel/nvt
|
||||||
|
|
||||||
|
|
|
@ -21,3 +21,14 @@ staging directory.
|
||||||
Further clarification on some ambiguities such as data type conversion of
|
Further clarification on some ambiguities such as data type conversion of
|
||||||
IEFD CU inputs. (Sakari)
|
IEFD CU inputs. (Sakari)
|
||||||
Move acronyms to doc-rst file. (Mauro)
|
Move acronyms to doc-rst file. (Mauro)
|
||||||
|
|
||||||
|
- Switch to yavta from v4l2n in driver docs.
|
||||||
|
|
||||||
|
- Elaborate the functionality of different selection rectangles in driver
|
||||||
|
documentation. This may require driver changes as well.
|
||||||
|
|
||||||
|
- More detailed documentation on calculating BDS, GCD etc. sizes needed.
|
||||||
|
|
||||||
|
- Document different operation modes, and which buffer queues are relevant
|
||||||
|
in each mode. To process an image, which queues require a buffer an in
|
||||||
|
which ones is it optional?
|
||||||
|
|
Loading…
Reference in a new issue