Case studies

MRTech key projects

Airbus Defence and Space

9×7 Digital Cinema Camera

Robotic Systems Lab

Embedded Vision cases

We present here several embedded vision cases those built on XIMEA digital cameras, NVIDIA Jetson computers, and with MRTech IFF software. For each case, a brief description of the hardware / software configuration and performance results is provided.

Every case (or kit) includes one or several XIMEA cameras, a separate NVIDIA embedded module, an appropriate carrier for the module, and necessary accessories.

Software from MRTech arranges standard or customized image processing pipeline running on NVIDIA module with following common processing steps:

  • acquisition images from cameras
  • color pre-processing including black level, white balance, demosaic
    • for GPU processing, we usually utilize Fastvideo SDK
  • additional options such as autoexposure, gain control when needed
  • H.264/H.265 encoding, RTSP streaming via Ethernet or Wi-Fi.

RTSP streams from cameras can be received using VLC app on one of the devices such as Macbook with MacOS, desktop / notebook with Windows or Linux OS, Apple devices such as iPhone, iPad, etc. Nonetheless, for all these cases we used one or another receiving computer equipped with NVIDIA graphic card, 144Hz refresh rate display, and MRTech software for receiving streams and rendering images to measure glass-to-glass (G2G) latency. Other performance figures show the processing pipeline time not including acquisition, data transfer times and network performances.

3-cam TX2 kit

  • MX200CG-CM XIMEA camera and
  • 2x MX031CG-SY XIMEA camera
  • Nvidia Jetson TX2 with XIMEA xEC2 carrier
  • Dell XPS laptop as a receiving station

Standard processing pipeline:

  • 12-bit acquisition
  • color pre-processing
  • H.264 encoding and RTSP streaming

Operating results:

  • 2x FullHD 1080p (1920×1080) ROI @ 30 FPS
  • WQHD 1440p (2560×1440) binning @ 30 FPS

2-cam TX2 kit

  • 2x MX031CG-SY XIMEA camera
  • Nvidia Jetson TX2 with XIMEA xEC2 carrier
  • Laptop or desktop as a receiving station

Standard processing pipeline:

  • 8-bit acquisition
  • Color pre-processing
  • H.264 encoding and RTSP streaming

Operating results:

  • 2x Full frame 3.1 MP (2064×1544) @ 60 FPS
  • processing time: 15ms for both cameras
  • G2G latency: < 60 ms (with 144 Hz monitor)

High resolution with TX2

  • MX500CG-CM-X4G2 XIMEA camera
  • NVIDIA Jetson TX2 module
  • Connect Tech Elroy Carrier for TX2

Customized processing pipeline:

  • 8/10/12 acquisition
  • RAW processing
  • color pre-processing
  • H.265 encoding
  • RTSP multi streaming

Processing results:

  • 47.5 MP, 7920 x 6004 @ 5 FPS

Two USB3 cameras with TX2

  • 2x MC031CG-SY-UB XIMEA camera
  • NVIDIA Jetson TX2 module
  • Connect Tech Elroy carrier
  • desktop GPU computer as a receiving station

Standard processing pipeline:

  • 8-bit acquisition from both cameras
  • Color pre-processing
  • H.264 encoding and RTSP streaming

Operating modes:

  • Full frame images:
    • 2x 3.1 MP (2064×1544) @ 50 FPS
    • G2G latency: 80-100 ms
  • Full HD images:
    • 2x FullHD 1080p (1920×1080) ROI @ 60 FPS
    • IFF processing: 15ms for both cameras
    • G2G latency: 70 – 80 ms

One camera Nano kit

  • MQ013CG-CM XIMEA camera
  • Jetson Nano Developer Kit
  • Dell XPS laptop as a receiving station

 

Standard processing pipeline:

  • 8-bit acquisition
  • Color pre-processing
  • H.264 encoding
  • RTSP streaming via Ethernet

Operating modes:

  • 1.3MP 1280 × 1024 @ 60 FPS
  • IFF processing time: 15ms
  • G2G latency: 80 ms

Jetson Nano camera kit

3-cam setup with Xavier

  • 2x MC031CG-SY XIMEA camera
  • MX124CG-SY-FL(FV) XIMEA camera
  • Jetson AGX Xavier Developer Kit
  • GPU desktop as a receiving station

Standard full frame processing:

  • 8/10/12-bit acquisition
  • color pre-processing
  • H.264/H.265 encoding, RTSP streaming

Operating results:

  • 8-bit – up to 60/60/60 FPS
  • 12-bit – up to 60/60/45 FPS with
    • processing latency ~ 30/30/45 ms

High performance with Xavier

  • CB262CG-GP-X8G3 XIMEA camera with
    • GPixel GMAX0505 sensor
  • NVIDIA Jetson AGX Xavier dev. kit

Standard demo application:

  • 12-bit acquisition
  • color pre-processing
  • render on a screen

Example of operating results:

  • 26.2 MP 5120 x 5120 (12-bit) @ 39 FPS
  • total GPU + CPU processing: 29 ms
  • G2G latency: 90 – 150 ms

Xavier NX camera kit

High performance image processing, RAW recording

Sufficiently multi-camera case

  • 8x MX124CG-SY-X2G2-FV XIMEA camera
  • XS-8P-X2G2-X8G3 XIMEA xSwitch
  • powerful x86 computer with AMD Ryzen or Intel i9 CPU as Video Capture Node
  • desktop GPU station to control 100+ cams

Processing:

  • 8/10/12-bit RAW images recording
  • control and monitoring (under development)

Operation modes:

  • recording RAW images without loss
    • 8x 12.4 MP 4112×3008 @ 60 FPS (8-bit)

High performance scientific case

  • 3x CB042MG-GP-BSI XIMEA camera
    • Gpixel GSENSE2020 BSI
    • typical readout noise: 1.2 e-
  • computer with Intel Core i9 – 9960X CPU

Customized image processing:

  • 3x 4.2 MP 2048 x 2048 @ 90 FPS
    • with reconstruction of 16-bit images
  • processing for scientific purposes
  • recording
  • GUI

RAW processing of 65MP images

  • CB654CG-GP-X8G3 XIMEA camera
  • xiCX XIMEA computer
  • high speed NVMe SSD array up to 8 TB
  • MRTech application to control recording

Processing:

  • 10/12-bit RAW images recording
  • control and monitoring

Operating modes:

  • 65 MP (9344 x 7000) 10-bit @ 70 FPS
  • 65 MP (9344 x 7000) 12-bit @ 30 FPS