SAMA7G5 Image Acquisition pipeline


Introduction

This page is a complete reference to SAMA7G5 image acquisition pipeline capabilities, focusing on the software for it.
SAMA7G5 can capture image either from a serial sensor using the MIPI CSI-2 interface, or from a parallel sensor using the PIO parallel interface.
The MIPI CSI-2 pipeline is a way of capturing image from a sensor using a serial interface, named CSI-2 (camera serial interface 2) which is standardized by the MIPI Alliance.
Previous MPU products had a simple parallel pipeline, in which there was a parallel sensor (like ov2640, ov7740) connected directly through I/O pins to the MPU to the Image Sensor Controller or Image Sensor Interface hardware block.
SAMA7G5 still supports a parallel sensor connection, with slight changes (the csi2dc hardware block as described below).
Pointing hand SAMA7G5 pipeline configuration requires media controller pipeline and media controller user space configuration. For media controller tutorial, check this page.

Drivers

This is a summary of the drivers required for SAMA7G5 image acquisition

Microchip eXtended Image Sensor Controller (XISC)

This driver is for the Image sensor controller, the top video driver (root video driver) that is the direct interface with the user. The XISC is the component that can convert pixels into data in memory, which can be saved into a file.
Driver can be enabled in menuconfig:
screen1.jpg
The XISC, once probed correctly, will prepare a device /dev/video0 and a device /dev/media0 into the devfs. (it can be /dev/video1 , /dev/video2 for example if you have another camera driver loaded, like an USB camera).
Pointing hand The /dev/video is called the video node, and /dev/media0 the media controller node.

Microchip CSI2 Demux Controller (CSI2DC)

This driver is a demux controller that controls the input into XISC. It is positioned between a possible PHY and the XISC.
Driver can be enabled in menuconfig:
screen2.jpg
The CSI2DC will register a subdevice node into devfs, /dev/v4l-subdev1 (not mandatory to be number 1)

Synopsys DWC CSI2HOST

This driver is a CSI2 Host controller interface. It is used only if the pipeline includes a serial CSI2 MIPI sensor. This sensor is connected to the CSI2 PHY , and this PHY is connected to the CSI2HOST .
Pointing hand If one connects a parallel sensor to the XISC I/O pins, this block is unused.
Warning, important But the CSI2DC is still required even in case a parallel sensor is connected.
Driver can be enabled in menuconfig:

screen3.jpg
The CSI2HOST will register a subdevice node into devfs, /dev/v4l-subdev2 (not mandatory to be number 2)

Synopsys DWC CSI2 D-PHY

This is a driver for the Synopsys CSI2 D-PHY. It is used only if the pipeline includes a serial CSI2 MIPI sensor. The sensor is directly connected to this PHY. The connections include the MIPI data lanes and clock lane (D0+, D0-, D1+, D1-, CLK+, CLK-). The PHY maintains the required electrical levels for MIPI CSI2 communication.
Pointing hand The PHY is unidirectional , from sensor to the D-PHY. There is no possible communication from the D-PHY back to the sensor.
The driver can be enabled in menuconfig:

screen4.jpg
Pointing hand The PHY does not register a subdevice node in /dev and thus it cannot be accessed from userspace. It is fully transparent for the user.

Sensors

The de-facto sensor for this platform is Sony IMX219 MIPI CSI2 sensor

Sony IMX219

Also known as Raspberry Pi Camera module v2.1, the Sony IMX219 is a MIPI CSI2 sensor with 10 bits precision that can stream data in Raw BAYER format with a resolution up to 8 Mpix , at 3280x2464 .
Pointing hand Communication with the sensor is done via i2c bus. The D-PHY is used to capture pixels, but sensor initialization , start/stop , configuration, is done via i2c bus.
The driver can be enabled in menuconfig:

screen5.jpg
The sensor driver will register a subdevice node as /dev/v4l-subdev3 (not mandatory to be number 3)
The imx219 is used as an example in the media controller tutorial. Check the specific page for examples how to configure the pipeline with this sensor.

Sony IMX274

This Sony sensor is a 4-lane MIPI CSI2 serial sensor with 10/12 bits precision that can stream data in Raw BAYER format with a resolution up to 8.59 Mpix at 3840x2160 (wide)
Pointing hand Communication with the sensor is done via i2c bus. The D-PHY is used to capture pixels, but sensor initialization , start/stop , configuration, is done via i2c bus.
Warning, important IMX274 is a 4-lane sensor so an additional patch is required to make the driver operate the sensor in MIPI HALF mode. This patch is already applied in Linux4SAM kernel but not in the mainline.
The driver can be enabled in menuconfig:

screen6.jpg
The sensor driver will register a subdevice node as /dev/v4l-subdev3 (not mandatory to be number 3)
Examples on how to configure Sony IMX274 with the media controller pipeline can be found in this page.

Omnivision OV7740

This Omnivision sensor is a parallel 8 bit Raw BAYER sensor with a resolution up to 640x480 (VGA). It can also stream direct YUYV format.
Pointing hand Communication with the sensor is done via i2c bus. The parallel lines are used to capture pixels, but sensor initialization , start/stop , configuration, is done via i2c bus.
The driver can be enabled in menuconfig:
screen7.jpg


The sensor driver will register a subdevice node as /dev/v4l-subdev3 (not mandatory to be number 3)
Examples on how to configure OV7740 with the media controller pipeline can be found in this page.

WebFaqBaseForm
Boards Sam9x75Curiosity, Sama7g5-ek
Components Kernel, linux-5.15-mchp, linux-6.1-mchp
Summary Detailed explanation of the sama7g5 image acquisition pipeline
Topic attachments
I Attachment Action Size Date Who Comment
JPEGjpg screen1.jpg manage 56.2 K 2021-11-19 - 14:45 UnknownUser  
JPEGjpg screen2.jpg manage 54.6 K 2021-11-19 - 14:57 UnknownUser  
JPEGjpg screen3.jpg manage 54.2 K 2021-11-19 - 15:04 UnknownUser  
JPEGjpg screen4.jpg manage 55.7 K 2021-11-19 - 15:10 UnknownUser  
JPEGjpg screen5.jpg manage 47.2 K 2021-11-19 - 15:15 UnknownUser  
JPEGjpg screen6.jpg manage 45.4 K 2021-11-19 - 15:22 UnknownUser  
JPEGjpg screen7.jpg manage 50.0 K 2021-11-19 - 16:24 UnknownUser  
r10 - 09 Nov 2023 - 09:13:38 - VarshiniRajendran
 
Linux & Open Source for AT91 Microchip Microprocessors

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.

Microchip® and others, are registered trademarks or trademarks of Microchip Technology Inc. and its subsidiaries. This site is powered by the TWiki collaboration platform

Arm® and others are registered trademarks or trademarks of Arm Limited (or its affiliates). Other terms and product names may be trademarks of others.

Ideas, requests, contributions ? Connect to LinksToCommunities page.

Syndicate this siteRSS ATOM