MIPI cameras

MRTech projects and solutions with MIPI cameras

Streaming system with digital zoom for an object tracking

 

Digital zoom is a method of enlarging a section of an image or video by cropping the original high-resolution image. This allows, for example, a remote drone operator to examine a specific area of the captured image in great detail, by reducing the field of view and increasing the resolution. It is important, however, to have a high-resolution input image and a high-performance system.

When recording the attached video, we used a system with lightweight, compact, high-resolution MIPI camera and a high-performance NVIDIA Jetson Orin NX module mounted on a compact carrier board. Most importantly, we used MRTech IFF SDK, which greatly simplifies implementation.

So, what this system demonstrates:

  • Jetson Orin NX module enables acquisition of 48 MP images at 19FPS from the camera.
  • It also provides color pre-processing, encoding and streaming of the captured images to the receiving computer. There are two streams there: 4×4 downscaled full frame camera view and selected cropped area at full resolution.
  • The receiving computer receives, decodes and renders the image streams onto the screen.
  • The system operator can use a joystick to control the position of the cropped area.

The video is a screencast from the receiving computer that shows:

  • a downscaled full frame camera view,
  • the cropped area at the full resolution,
  • an overview of the system captured by a separate camera connected to the receiving computer.

The system provides high-quality images with good color reproduction, low glass-to-glass latency, denoising, as well as HTTP control of the image acquisition and processing and RTSP and WebRTC streaming.

Furthermore, using the IFF SDK makes it easy to customise the system, for example, to control the cropped area from an external tracking application. You can also use other cameras, other Jetson modules such as Jetson Nano, Thor, or even alternative hardware platforms.

So, the system configuration details:

  • Based on Orin NX 16GB module on compact carrier board
  • Sony IMX586 MIPI camera (48 MP @ 19 FPS)
  • MRTech V4L driver for Jetson
  • Two streams from the camera: 4×4 downscaled full frame view and crop area with full resolution
  • The position of the cropped area can be controlled by joystick or external tracking application
  • Good color quality, denoising, very low glass-to-glass latency

Low power AI system on NXP SoC

 

This system run an anonymization application with the hardware acceleration of neural network algorithms, including person detection, pose estimation, hand bone position estimation, face detection, and face expression classification. It is based on I-Pi SMARC development kit powered by NXP i.MX 8M Plus module with a miniature MIPI camera from Vision Components connected.

The system is a prototype of a smart camera with low power consumption, no more than 5 watts.

 

System highlights:

  • Hardware-accelerated neural network (object detection and classification)
  • Low power system (5W board + camera power consumption)
  • NXP development kit with Sony IMX462 MIPI camera (2.1 MP @ 30 FPS) connected

 

 

System prototype of 4K stereo MIPI camera for entertainment

 

MRTech team developed and tested a prototype of a 4K stereoscopic camera based on IMX477 sensors with MIPI interface and NVIDIA Xavier module.

System highlights:

  • NVIDIA Jetson Xavier NX 16GB with Auvidea carrier board
  • Two MIPI camera modules with Sony IMX477 sensor
  • High quality color pre-processing on Jetson GPU, H.265 encoding, RTSP streaming
  • Output 4K stereo side-by-side images: 2x 1920×2160 @ 60 FPS

4K stereo-cameras

A massive MIPI-multi-sensor system for medical applications

 

 

The system consists of an array of 32 high-resolution MIPI sensors placed on a specially designed sensor carrier board and connected to the NVIDIA AGX Xavier module.

The software provides image acquisition with 5 GB/s total bandwidth, custom image processing pipeline, and an API for remote control.

 

 

MIPI multi-sensor array