UsingISC (Image Sensor Controller)


Introduction

This page describes how to configure and use of the ISC in SAMA5D2 SoC with the image sensors.
  • The Image Sensor Controller(ISC) manages incoming data from a sensor up to 12-bit parallel interface.
    • The ISC can generate the isc_mck output clock to the image sensor.
      • The isc_mck clock has three selectable clock sources and one programmable clock divider.
    • The ISC can handle pixel clock up to 96MHz, with the maximum resolution of 2592x1944.
  • The ISC includes a video pipeline consist of the following sub-modules:
    • PFE: Parallel Front End to sample the camera sensor input stream
    • WB: Programmable white balance in the Bayer domain
    • CFA: Color filter array interpolation module
    • CC: Programmable color correction
    • GAM: Gamma correction
    • CSC: Programmable color space conversion
    • CBC: Contrast and Brightness control
    • SUB: This module performs YCbCr444 to YCbCr420 chrominance subsampling
    • RLP: This module performs rounding, range limiting and packing of the incoming data
  • The sub-modules in video pipeline can handle the following input pixel formats:
    • Raw Bayer convert to YCbCr
    • YCbCr (YCbCr 444, YCbCr 422, YCbCr 420),
    • Luminance (Black and White) Pixel Format.
    • 16 bpp RGB, 32 bpp RGB, 16 bpp.
  • For output format,
    • For YCbCr output, the ISC includes a DMA interface which allows YCbCr data organized in packed, semi planar or planar format.
    • It can also convert YCbCr to RGB.
    • It can also be configured as by pass mode, to store original data received from the sensor.

Prerequisites

Pointing hand Starting with Linux Kernel 4.9, we provide and support kernel drivers for ISC.

Pointing hand Starting with Linux Kernel 4.14 (included in Linux4SAM 6.0) , we offer the Device Tree support for the ISC and sensors as an Overlay for the SAMA5D2 Xplained board only. Overlays are required for extension boards, like camera sensors in our case. For more information, visit the DT-overlays page.

Pointing hand Starting with Linux4SAM 6.2 , we offer the Device Tree support for the ISC and sensors as an Overlay for the SAMA5D2 Xplained, SAMA5D27 SOM1 EK and SAMA5D27 WLSOM1 EK . Overlays are required for extension boards, like camera sensors in our case. For more information, visit the DT-overlays page.

Pointing hand Starting with Linux4SAM 2020.04 , we offer additional support for the MT9V022 monochrome Aptina sensor.

Advanced features

Pointing hand Starting with Linux Kernel 4.19 (included in Linux4SAM 6.1) , we offer extended support for the White Balance features.

Configuration in kernel

The ISC driver in Linux kernel provides the upper layer interface to the Video4Linux (V4L2) framework. To make it work, you should enable the following options in Kernel menuconfig.
  • Enable V4L2 framework.
    • Under the menu: "Device Drivers ---> Multimedia support"
      multimedia_support_4.13.0.png
      • Select the menu "Device Drivers ---> Multimedia support ---> Media Controller API"
      • Select the menu "Device Drivers ---> Multimedia support ---> V4L2 sub-device userspace API"
      • Deselect the menu "Device Drivers ---> Multimedia support ---> Autoselect ancillary drivers (tuners, sensors, i2c, spi, frontends)"

Enable ISC and sensor drivers

  • Select device driver
    • V4L platform device can be found in the menu: "Device Drivers ---> Multimedia support ---> V4L platform devices"
      v4l_platform_devices_4.13.0.png
      • Deselect the menu "Device Drivers ---> Multimedia support ---> V4L platform devices ---> SoC camera support"
      • Select the menu "Device Drivers ---> Multimedia support ---> V4L platform devices ---> ATMEL Image Sensor Controller (ISC) support"
  • Select sensor driver
    • Sensor drivers can be found in the menu "Device Drivers ---> Multimedia support ---> I2C Encoders, decoders, sensors and other helper chips "
      Screenshot_from_2019-04-23_14-01-23.png
    • We have tested with OV7670 , OV7740 and OV5640 image sensors. Starting with Linux4SAM 2020.04 we also tested with mt9v022

Configuration in U-boot

Starting with Linux Kernel 4.14 (included in Linux4SAM 6.0) and U-boot 2018.07 , we offer the Device Tree support for the ISC and sensors as an Overlay for SAMA5D2 Xplained SAMA5D27 SOM1 EK and SAMA5D27 WLSOM1 EK . Overlays are required for extension boards, like camera sensors in our case. For more information, visit the DT-overlays page.

U-boot, if instructed to do so, will load the Device Tree Overlay for the Image sensor controller (ISC) and the Device Tree Overlay for the specific sensor which we support.

To instruct U-boot to load the additional required Device Tree Overlays, proceed with the following steps:

SAMA5D2 Xplained board

  • Step 1: The boot command needs to have an additional overlay for the ISC controller itself. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc
  • Step 2: The boot command needs to have an additional overlay for the sensor. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc#ov7670
or
bootm 0x24000000#kernel_dtb#isc#ov7740
or
bootm 0x24000000#kernel_dtb#isc#ov5640
or
bootm 0x24000000#kernel_dtb#isc#mt9v022

Pointing hand Note that both overlay configurations are required, chained as in the command in step 2 above. the 'isc' overlay will add the setup for the ISC, and the ov7670/ov7740/ov5640/mt9v022 will add the setup for the sensor itself.

The following table summarizes the possible FIT image overlay configurations that are needed:

Overlay configuration Effect Requirements Starting with Linux4SAM version
Image Sensor Controller isc Adds the Image Sensor Controller setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration Linux4SAM 6.0
Omnivision ov7670 ov7670 Adds the Omnivision ov7670 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 6.0
Omnivision ov7740 ov7740 Adds the Omnivision ov7740 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 6.0
Omnivision ov5640 ov5640 Adds the Omnivision ov5640 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 6.1
Aptina MT9V022 mt9v022 Adds the Aptina mt9v022 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 2020.04

Pointing hand Note that adding just an overlay for the sensor is not enough. The 'isc' overlay is a mandatory prerequisite .
Pointing hand Note that SAMA5D2 Xplained board is used as a reference, and the software support is created and tested for this board. The SAMA5D2 SoC supports ISC, but to have it up and running, please check your board schematics.

SAMA5D27 SOM1 EK board

Pointing hand This support is only available starting with Linux Kernel 4.19 (included in Linux4SAM 6.1)

  • Step 1: The boot command needs to have an additional overlay for the sensor. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#ov5640
or
bootm 0x24000000#kernel_dtb#isc#mt9v022

Pointing hand Note that only the support for ov5640 and mt9v022 are currently available.

The following table summarizes the possible FIT image overlay configurations that are needed:

Overlay configuration Effect Requirements Starting with Linux4SAM version
Omnivision ov5640 ov5640 Adds the Omnivision ov5640 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration Linux4SAM 6.1
Aptina MT9V022 mt9v022 Adds the Aptina mt9v022 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 2020.04

Pointing hand Note that the connector on this board is different from the one on SAMA5D2 Xplained. Different wiring/extension board are required .
Pointing hand Note that SAMA5D27 SOM1 EK board is used as a reference, and the software support is created and tested for this board. The SAMA5D2 SoC supports ISC, but to have it up and running, please check your board schematics.

SAMA5D27 WLSOM1 EK board

Pointing hand This support is only available starting with Linux4SAM 6.2

* Step 1: The boot command needs to have an additional overlay for the ISC controller itself. This is an additional FIT image overlay configuration:

bootm 0x24000000#kernel_dtb#isc
  • Step 2: The boot command needs to have an additional overlay for the sensor. This is an additional FIT image overlay configuration:
bootm 0x24000000#kernel_dtb#isc#ov7670
or
bootm 0x24000000#kernel_dtb#isc#ov7740
or
bootm 0x24000000#kernel_dtb#isc#ov5640
or
bootm 0x24000000#kernel_dtb#isc#mt9v022

Pointing hand Note that both overlay configurations are required, chained as in the command in step 2 above. the 'isc' overlay will add the setup for the ISC, and the ov7670/ov7740/ov5640/mt9v022 will add the setup for the sensor itself.

The following table summarizes the possible FIT image overlay configurations that are needed:

Overlay configuration Effect Requirements Starting with Linux4SAM version
Image Sensor Controller isc Adds the Image Sensor Controller setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration Linux4SAM 6.2
Omnivision ov7670 ov7670 Adds the Omnivision ov7670 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 6.2
Omnivision ov7740 ov7740 Adds the Omnivision ov7740 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 6.2
Omnivision ov5640 ov5640 Adds the Omnivision ov5640 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 6.2
Aptina MT9V022 mt9v022 Adds the Aptina mt9v022 setup configuration Base Kernel device Tree Blob 'kernel_dtb' configuration and 'isc' configuration Linux4SAM 2020.04

Pointing hand Note that the connector on this board is different from the one on SAMA5D2 Xplained. Different wiring/extension board are required .
Pointing hand Note that SAMA5D27 WLSOM1 EK board is used as a reference, and the software support is created and tested for this board. The SAMA5D2 SoC supports ISC, but to have it up and running, please check your board schematics.

Connect the sensor to ISC

Hardware connection

Pointing hand Note that the connector on SAMA5D27 SOM1 EK is different from the one on SAMA5D2 Xplained. Different wiring/extension board are required .
SAMA5D2 Xplained includes an image sensor connector:
ISC_connect_schematic.png

It uses two peripherals from SAMA5D2:

  • ISC: The DVP interface consists of:
    • Pixel Clock (VSYNC).
    • Horizontal Sync (HSYNC).
    • Vertical Sync (VSYNC).
    • 8 to 12-bits parallel data interface
  • TWI: an I2C interface to control the sensor's behavior.
    • And reset and power down control of the sensor.

Connection specification in driver

The ISC driver uses the device tree to specify the ISC and sensor device, please refer to the binding document atmel-isc.txt.
The binding document explains:
  • How the DVP interface of the sensor is connected to SAMA5D2.
  • How the sensor's I2C control channel is connected to SAMA5D2.
    • We should specify the sensor device under the I2C bus as a child node. (for example, the ov7670 must be connected under I2C1 on SAMA5D2 Xplained)
    • At the same time, we need to specify the pins to control the sensor's reset and power down. (for example, PB11 as reset, PB12 as power down on SAMA5D2 Xplained).
    • Pay attention to the conflicts with GPIOs. The Device Tree needs to disable the conflicting nodes.
       reset-gpios = <&pioA 43 GPIO_ACTIVE_LOW>;
       powerdown-gpios = <&pioA 44 GPIO_ACTIVE_HIGH>;
       

Pointing hand Note that the hardware connections are highly dependent on your board. Check your board schematics to see how the wiring is done.

Starting with Linux Kernel 4.14 (included in Linux4SAM 6.0) and U-boot 2018.07 , we offer the Device Tree support for the ISC and sensors as an Overlay for SAMA5D2 Xplained. Overlays are required for extension boards, like camera sensors in our case. For more information, visit the DT-overlays page.
The DT-overlay for the ISC and for the sensors can be consulted in our Github repository.
Here are links to the overlays themselves, for the SAMA5D2 Xplained board:

Starting with Linux Kernel 4.19 (included in Linux4SAM 6.1) we offer additional support for omnivision ov5640 for SAMA5D2 Xplained but also for SAMA5D27 SOM1 EK. Here are links to the overlay itself, for the SAMA5D2 Xplained board:

Here are links to the overlay itself, for the SAMA5D27 SOM1 EK board:

Starting with Linux4SAM 6.2 we offer support via overlays for all the three sensors (ov7670, ov5640, ov7740) for the SAMA5D27 WLSOM1 EK board.

Starting with Linux4SAM 2020.04 we offer support for the Aptina sensor monochrome mt9v022 for all the three boards.

The boot message

If everything is okay, you can see the log while the system boot-up, it indicates the ov7670 sensor successfully probed.
   ov7670 2-0021: chip found @ 0x42 (AT91)
   

Device file and sysfs

After boot up, you can see a character device file named /dev/video:
   root@sama5d2-xplained:/# ls -l /dev/video0
   crw-rw---- 1 root video 81, 1 Jul 12 11:56 /dev/video0
   
The more information is in the sysfs filesystem.
   root@sama5d2-xplained:/# ls sys/class/video4linux/video0
   dev  dev_debug  device  index  name  power  subsystem  uevent
   root@sama5d2-xplained:/# cat /sys/class/video4linux/video0/name
   atmel_isc
   root@sama5d2-xplained:/# cat /sys/class/video4linux/v4l-subdev0/name
   ov7670 2-0021
   

User space software

Media controller paradigm

Since linux4sam-2022.04 , using Linux Kernel 5.15 , userspace needs to use the media controller paradigm of configuring the image capture pipeline.
Warning, important without properly setting up the media controller entities and links, image capture will fail.
Warning, important the ISC driver will no longer setup the sensor with a corresponding format, as in the media controller paradigm, this is no longer possible.

Use the v4l-utils

The v4l-utils are a series of packages for handling the video4linux devices.

v4L2-compliance

You can use v4l2-compliance tool to test the v4l2 API compliance of the ISC driver.
# v4l2-compliance
v4l2-compliance 1.22.1, 32 bits, 32-bit time_t

Compliance test for microchip-isc device /dev/video0:

Driver Info:
        Driveratmel-     : microchip-isc
        Card type        : Atmel Image Sensor Controller
        Bus info         : platform:atmel-sama5d2-isc f000
        Driver version   : 5.15.32
        Capabilities     : 0xa4200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : atmel_isc_commo
        Model            : atmel,sama5d2-isc
        Serial           :
        Bus info         : platform:atmel-sama5d2-isc f000
        Media version    : 5.15.32
        Hardware revision: 0x00000137 (311)
        Driver version   : 5.15.32
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x0000000a (10)
        Name             : atmel_isc_common
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x0100000b   : 0: Sink
          Link 0x02000010: from remote pad 0x1000003 of entity 'atmel_isc_scaler' (Video Scaler): Data, Enabled, Immutable

Required ioctls:
        test MC information (see 'Media Driver Info' above): OK
        test VIDIOC_QUERYCAP: OK
        test invalid ioctls: OK

Allow for multiple opens:
        test second /dev/video0 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
sama5d2-isc f0008000.isc: =================  START STATUS  =================
atmel-sama5d2-isc f0008000.isc: Contrast: 256
atmel-sama5d2-isc f0008000.isc: Brightness: 0
        tatmefor unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
l-sama5d2-isc f0008000.isc: Gamma: 2
atmel-sama5d2-isc f0008000.isc: White Balance, Automatic: true
atmel-sama5d2-isc f0008000.isc: Red Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Blue Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Red Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Blue Component Gain: 512 inactive volatile
atmel-sama5d2-isc f0008000.isc: Red Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: Blue Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Red Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: Green Blue Component Offset: 0 inactive volatile
atmel-sama5d2-isc f0008000.isc: ==================  END STATUS  ==================
        test VIDIOC_LOG_STATUS: OK

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
        test VIDIOC_QUERYCTRL: OK
        test VIDIOC_G/S_CTRL: OK
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 6 Private Controls: 8

Format ioctls (Input 0):
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
        test VIDIOC_G/S_PARM: OK (Not Supported)
        test VIDIOC_G_FBUF: OK (Not Supported)
        test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
        test Cropping: OK (Not Supported)
        test Composing: OK (Not Supported)
        test Scaling: OK

Codec ioctls (Input 0):
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)
        test TIME32/64: OK

Total for microchip-isc device /dev/video0: 47, Succeeded: 47, Failed: 0, Warnings: 0

v4l2-ctl

  • Display all information available.
# v4l2-ctl --all
Driver Info:
        Driver name      : microchip-isc
        Card type        : Atmel Image Sensor Controller
        Bus info         : platform:atmel-sama5d2-isc f000
        Driver version   : 5.15.32
        Capabilities     : 0xa4200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : atmel_isc_commo
        Model            : atmel,sama5d2-isc
        Serial           :
        Bus info         : platform:atmel-sama5d2-isc f000
        Media version    : 5.15.32
        Hardware revision: 0x00000137 (311)
        Driver version   : 5.15.32
Interface Info:
        ID               : 0x0300000c
        Type             : V4L Video
Entity Info:
        ID               : 0x0000000a (10)
        Name             : atmel_isc_common
        Function         : V4L2 I/O
        Flags            : default
        Pad 0x0100000b   : 0: Sink
          Link 0x02000010: from remote pad 0x1000003 of entity 'atmel_isc_scaler' (Video Scaler): Data, Enabled, Immutable
Priority: 2
Video input : 0 (atmel_isc_common: ok)
Format Video Capture:
        Width/Height      : 640/480
        Pixel Format      : 'AR12' (16-bit ARGB 4-4-4-4)
        Field             : None
        Bytes per Line    : 1280
        Size Image        : 614400
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :

User Controls

                     brightness 0x00980900 (int)    : min=-1024 max=1023 step=1 default=0 value=0 flags=slider
                       contrast 0x00980901 (int)    : min=-2048 max=2047 step=1 default=256 value=256 flags=slider
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1 flags=update
               do_white_balance 0x0098090d (button) : value=0 flags=inactive, write-only, volatile, execute-on-write
                          gamma 0x00980910 (int)    : min=0 max=2 step=1 default=2 value=2 flags=slider
             red_component_gain 0x009819c0 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
            blue_component_gain 0x009819c1 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
       green_red_component_gain 0x009819c2 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
      green_blue_component_gain 0x009819c3 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
           red_component_offset 0x009819c4 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
          blue_component_offset 0x009819c5 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
     green_red_component_offset 0x009819c6 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
    green_blue_component_offset 0x009819c7 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
#


  • Display all controls and their values
# v4l2-ctl --list-ctrls

User Controls

                     brightness 0x00980900 (int)    : min=-1024 max=1023 step=1 default=0 value=0 flags=slider
                       contrast 0x00980901 (int)    : min=-2048 max=2047 step=1 default=256 value=256 flags=slider
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1 flags=update
               do_white_balance 0x0098090d (button) : value=0 flags=inactive, write-only, volatile, execute-on-write
                          gamma 0x00980910 (int)    : min=0 max=2 step=1 default=2 value=2 flags=slider
             red_component_gain 0x009819c0 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
            blue_component_gain 0x009819c1 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
       green_red_component_gain 0x009819c2 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
      green_blue_component_gain 0x009819c3 (int)    : min=0 max=8191 step=1 default=512 value=512 flags=inactive, slider, volatile
           red_component_offset 0x009819c4 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
          blue_component_offset 0x009819c5 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
     green_red_component_offset 0x009819c6 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
    green_blue_component_offset 0x009819c7 (int)    : min=-4095 max=4095 step=1 default=0 value=0 flags=inactive, slider, volatile
#

  • Get / Set the value of the controls
# v4l2-ctl --get-ctrl=contrast
contrast: 256
# v4l2-ctl --set-ctrl=contrast=200
# v4l2-ctl --get-ctrl=contrast
contrast: 200
#

media-ctl

Media-ctl is a tool that is used to query and set media controller pipeline (entities, links, etc.). More information about the media controller and a tutorial about it can be found here here.

Example of media-ctl -p output with mt9v022 sensor connected

# media-ctl -p
Media controller API version 5.15.32

Media device information
------------------------
driver          atmel_isc_commo
model           atmel,sama5d2-isc
serial
bus info        platform:atmel-sama5d2-isc f000
hw revision     0x137
driver version  5.15.32

Device topology
- entity 1: atmel_isc_scaler (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb
                 crop.bounds:(0,0)/2592x1944
                 crop:(0,0)/2592x1944]
                <- "mt9v032 1-0048":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb]
                -> "atmel_isc_common":0 [ENABLED,IMMUTABLE]

- entity 4: mt9v032 1-0048 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:Y10_1X10/752x480 field:none colorspace:srgb
                 crop:(1,5)/752x480]
                -> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]

- entity 10: atmel_isc_common (1 pad, 1 link)
             type Node subtype V4L flags 1
             device node name /dev/video0
        pad0: Sink
                <- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]

#

Example of media-ctl -p output with ov7740 sensor connected


# media-ctl -p
Media controller API version 5.18.0

Media device information
------------------------
driver          atmel_isc_commo
model           atmel,sama5d2-isc
serial
bus info        platform:atmel-sama5d2-isc f000
hw revision     0x137
driver version  5.18.0

Device topology
- entity 1: atmel_isc_scaler (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb
                 crop.bounds:(0,0)/2592x1944
                 crop:(0,0)/2592x1944]
                <- "ov7740 1-0021":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:SBGGR8_1X8/2592x1944 field:none colorspace:srgb]
                -> "atmel_isc_common":0 [ENABLED,IMMUTABLE]

- entity 4: ov7740 1-0021 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:YUYV8_2X8/640x480@1/60 field:none colorspace:srgb]
                -> "atmel_isc_scaler":0 [ENABLED,IMMUTABLE]

- entity 10: atmel_isc_common (1 pad, 1 link)
             type Node subtype V4L flags 1
             device node name /dev/video0
        pad0: Sink
                <- "atmel_isc_scaler":1 [ENABLED,IMMUTABLE]

#

Video-capture-at91 scripts

  • Video-capture-at91 is a collection of scripts present on Github that is part of the Linux4SAM distribution, and offers examples and scripts that automatically configure the media controller pipeline.

  • The scripts are available in your home directory, and are organized in directories corresponding to each sensor supported.
    • There is a directory for each of the supported sensors:
# ls -la video-capture-at91/
total 24
drwxr-xr-x    6 root     root          4096 May 10  2022 .
drwx------    3 root     root          4096 Jan  1 02:08 ..
drwxr-xr-x    2 root     root          4096 May 10  2022 mt9v022
drwxr-xr-x    2 root     root          4096 May 10  2022 ov5640
drwxr-xr-x    2 root     root          4096 May 10  2022 ov7670
drwxr-xr-x    2 root     root          4096 May 10  2022 ov7740

  • There is a script that prepares the media controller pipeline for capturing for each supported resolution:

  • There are also demo scripts available for each resolution, on how to use fswebcam to capture video in different formats, and how to configure the video format accordingly.

  • Example on how to use scripts for sama5d2 + mt9v022 sensor:

# ./video-capture-at91/mt9v022/
752x480_Y10.sh       fswebcam_752x480.sh
# ./video-capture-at91/mt9v022/752x480_Y10.sh
Preparing MT9V022 in Y10 10bits mode
Ready to capture at 752x480
#
#
# ./video-capture-at91/mt9v022/fswebcam_752x480.sh
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.37 seconds. (56 fps)
--- Processing captured image...
Writing JPEG image to 'tinyGREY.png'.
#

  • Example on how to use scripts for sama5d2 + ov7740 sensor in raw bayer mode:

# ./video-capture-at91/ov7740/640x480_raw.sh
Preparing OV7440 in RAW BAYER MODE
Ready to capture at 640x480
# ./video-capture-at91/ov7740/fswebcam_640x480.sh
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (55 fps)
--- Processing captured image...
Writing JPEG image to 'tinyRGB565.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (55 fps)
--- Processing captured image...
Writing JPEG image to 'tinyYUYV.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (54 fps)
--- Processing captured image...
Writing JPEG image to 'tinyABGR32.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (54 fps)
--- Processing captured image...
Writing JPEG image to 'tinyGREY.png'.
--- Opening /dev/video0...
Trying source module v4l2...
/dev/video0 opened.
No input was specified, using the first.
--- Capturing frame...
Skipping 20 frames...
Capturing 1 frames...
Captured 21 frames in 0.38 seconds. (54 fps)
--- Processing captured image...
Writing JPEG image to 'tinyYUV420P.png'.
#

Using ffmpeg

  • ffmpeg can be used to capture video from the v4l2 device.

  • To list possible pixel formats use :
ffmpeg -pix_fmts

Warning, important Media controller pipeline must be properly configured before using ffmpeg

Example at 640x480 , RGB565 format

v4l2-ctl -v pixelformat=RGBP,height=480,width=640         # corresponding v4l2 format

ffmpeg -s vga -pix_fmt rgb565be -f video4linux2 -i /dev/video0 video_vga_rgb565be.avi

Example at 640x480, YUYV format

v4l2-ctl -v pixelformat=YUYV,height=480,width=640         # corresponding v4l2 format

ffmpeg -s vga -pix_fmt yuyv422 -f video4linux2 -i /dev/video0 video_vga_yuyv422.avi

r25 - 16 May 2023 - 09:40:46 - VarshiniRajendran
 
Linux & Open Source for AT91 Microchip Microprocessors

Copyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.

Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.

Microchip® and others, are registered trademarks or trademarks of Microchip Technology Inc. and its subsidiaries. This site is powered by the TWiki collaboration platform

Arm® and others are registered trademarks or trademarks of Arm Limited (or its affiliates). Other terms and product names may be trademarks of others.

Ideas, requests, contributions ? Connect to LinksToCommunities page.

Syndicate this siteRSS ATOM