XIMEA camera systems

XIMEA is a very well-known manufacturer of innovative machine vision cameras and camera systems. Following the motto “Imaging beyond the standard!” XIMEA develops, manufactures, and distributes high-end industrial and scientific cameras, getting the best out of the latest sensors in the most compact design.

MRTech is an authorised technology partner of XIMEA with decades of experience in the development and implementation of XIMEA camera-based machine vision systems.

Embedded vision system with high-performance XIMEA camera

XIMEA cameras with PCI Express Gen 3 interface stand out through breakneck speed based on the throughput of up to 64 Gbits per second. The whole range of models offers DMA (Direct Memory Access) and data transfer to GPU possible on Linux. This helps with extremely low latency.

The arrival of new Orin modules, the world’s most powerful AI edge computers from NVIDIA opens up new potential for using XIMEA high-performance PCIe cameras in embedded vision applications.

We used the XIMEA MX245CG-SY-X4G3 camera together with the Jetson AGX Orin module for the common case where we stream images from the camera to a remote host computer.

The MX245CG-SY-X4G3 camera has 24.5 MP 5320×4600 image resolution with color Sony Pregius S™ 4th Generation sensor IMX530, and among other things supports the HDR function enabled by this sensor. We worked with 4K (4096×4096) ROI images (or lower) as this is the maximum image size that the Jetson AGX Orin video encoder (NVENC) can handle.

The complete system configuration and the results achieved are shown below.

  1. XIMEA MX245CG-SY-X4G3-FF-HDR camera
  2. NVIDIA Jetson AGX Orin 32GB module
  3. MSI Raider laptop with NVIDIA GPU as a receiving computer
  4. Orin software stack:
    • Jetpack 5.1.2 [L4T 35.4.1]
    • Image processing IFF SDK from MRTech
    • Application based on SDK `farsight` sample

This software allowed us to implement the following image processing pipeline:

  • image acquisition from XIMEA camera with ROI preset
  • color pre-processing on GPU:
    – black level subtraction
    – histogram calculation
    – white balance
    – high-quality MG demosaicing algorithm with processing 23×23 window
    – color correction
    – gamma
    – image format conversion
  • automatic control of exposure time and white balance
  • 265 encoding and RTSP streaming over the network

Case with full 4K (4096×4096) image at 60 FPS

  • Input image: 16.8 Mpixel @ RAW16 @ 60 FPS
  • Input transfer speed: 16.1 Gbps
  • Image processing latency on Orin: 40 ms
  • Average Orin CPU load: 7%
  • Average Orin GPU load: 76%
  • Orin power: 45W
  • Encoder bitrate: 40 Mbps
  • Image processing latency on the receiving side: 12 ms
  • G2G latency: ~100 ms

Case with 4K UHD (3840×2160) image at 120 FPS

  • Input image: 7.7 Mpixel @ RAW16 @ 120 FPS
  • Input transfer speed: 14.9 Gbps
  • Image processing latency on Orin: 21.2 ms
  • Average Orin CPU load: 5%
  • Average Orin GPU load: 70%
  • Orin power: 46W
  • Encoder bitrate: 100 Mbps
  • Image processing latency on the receiving side: 9 ms
  • G2G latency: 50-60 ms

Case with full 4K (4096×4096) image in Dual ADC HDR mode at 42.6 FPS
Note that 42.6 FPS is the maximum framerate for Dual ADC Non-Combined camera mode

  • Input image: 16.78 Mpixel @ RAW16 @ 42.6 FPS
  • Input transfer speed: 22.9 Gbps
  • Image processing latency on Orin: 45.5 ms
  • Average Orin CPU load: 38%
  • Average Orin GPU load: 60%
  • Orin power: 44W
  • Encode bitrate: 30 Mbps
  • Image processing latency on the receiving side: 12 ms
  • G2G latency: ~110 ms

Multi-camera systems

XIMEA PCIe cameras and MRTech image processing IFF SDK are perfect for implementing high-load video applications and systems with a large number of cameras. These solutions are required in 360 panorama, virtual and augmented reality, sports broadcasting, and entertainment applications.

Generally, such systems use high-resolution cameras with high frame rates, multiple viewpoints and synchronous capturing. An application running on a system of this type should be capable of capturing images from many high-resolution cameras, sometimes transmitting the data for long distances with very high bandwidth, processing, and storing images without loss. It’s also necessary to synchronize many cameras with lights, to manage and monitor the filming process.

Our software solutions and our expertise in hardware make it possible to address all these challenges by bringing together modern technological capabilities and customer requirements. Here is an example of such an implementation from the Argus project to create the 3D scanning system.

Camera Node

  • Eight XIMEA PCIe cameras MX124CG-SY-X2G2
  • XIMEA xSwitch for multi-camera connection:
    – Some switch models can connect more cameras.
    – Extremely high 64 Gbit/s upstream bandwidth allows bundling multiple camera data streams at full speed directly to the host computer memory (DMA) over a single copper or fiber optic cable.
    – An integrated IO circuitry allows synchronizing all cameras without complex external wiring.
  • Powerful computer with AMD Ryzen Threadripper CPU as Video Capture Node with:
    – an array of Samsung 960Pro 1 TB SSD to record captured RAW images,
    – inexpensive NVIDIA Quadro GPU card to pre-process images for previewing.
  • Custom image processing application based on MRTech IFF SDK

Image processing pipeline

  • Image acquisition 8x 12.4 MP (4112×3008) @ 60 FPS
    – total image transfer speed: 48 Gbps
  • Recording raw images to SSD at 6 GB/s without frame loss
  • Image processing for previewing:
    – dropping, downscaling, resizing if needed
    – color pre-processing
    – H.265 encoding
    – RTSP streaming over the network

 

Scaling and recording control

The system architecture with one node and 8 cameras connected allows scaling up to a system with more than 100 cameras. A single computer control node makes it possible to have a preview of all cameras available in the system, configure the selected camera and control image recording.

Please contact us if you would like to know how it works in more detail.

High-end scientific XIMEA cameras

XIMEA develops and manufactures innovative scientific cameras built around Scientific CMOS front and back-illuminated sensors coupled with compact Peltier TE Cooling and USB3 or PCI Express interface for speed.

As the XIMEA technological partner, MRTech helps its customers to implement sophisticated ideas and projects at the new frontiers of science.

Here’s what we’re doing to make it happen:

  • Guidance in Industrial and Scientific camera area, pre-project expertise.
  • HW / SW design, optimization of vision system architecture and low-level integration of new technologies.
  • Development of highly specialised software applications for image acquisitions, processing, recording, etc.
  • Customer support.

Please, write to us about your project and ask if you need a consultation.

Request to MRTech