MRTech IFF SDK for Machine Vision

MRTech IFF SDK (Image Flow Framework SDK) is a powerful cross-platform toolkit that facilitates the development of high-performance machine vision and image processing applications. The core of IFF SDK was initially developed by MRTech in 2016 and has been continuously improved and used in many projects ever since.

The main feature of IFF SDK is delivering images to the customer’s application code in the most efficient way. The toolkit helps achieve maximum performance for any configuration of the customer’s image processing system.

SDK highlights

Production-ready, high-quality code, field-proven in multiple real-life projects
High-performance image processing with low latency and low overhead
SDK architecture that makes for easy development and customization of the target application
Upon request, MRTech provides consulting services and technical support (including implementation assistance)

Basic Functionality

Textual descriptions of pipeline configurations to build image processing workflows
Exporting and importing images from the SDK pipeline to the customer application
Controlling acquisition and processing parameters at runtime
Seamless integration with the target application using C or Python API as well as REST HTTP interface to control image processing
Performing accelerated image processing on dedicated and embedded NVIDIA GPUs
Providing input from multiple machine vision cameras with PCIe, USB3, Ethernet, and MIPI interfaces

Processing modules:

FFC, white balance, histogram
Auto-exposure, white balance
Gamma correction, LUT
High performance demosaicing
Spatial denoiser
Image cropping, resizing
Color space transformation
Color correction

Control interfaces:

JSON configuration
C/Python API
HTTP REST API

Compression and decompression:

H.264, H.265
JPEG
JPEG2000

Input capabilities:

Machine vision cameras
RTSP video source
File data input

Output capabilities:

TIFF/DNG image recording
RTSP video stream, WebRTC
Export to user applications

Supported Hardware

Cameras:
- Machine vision cameras from XIMEA, Basler, and other vendors
- MIPI sensors
Platforms and operating systems:
- 64-bit Intel x86, Linux and Windows
- 64-bit ARM, NVIDIA Jetson family, Linux
Acceleration devices:
- CUDA processing on NVIDIA GPUs, including Jetson platform
- Hardware video encoding/decoding on NVIDIA GPUs and Jetsons

Use cases

High-speed image processing for digital cinema cameras

The developed system includes either one or two high-performance PCIe cameras with 65-megapixel resolution at up to 70 FPS and an embedded Intel Core i7 computer.

The MRTech software solution enables the recording of high-framerate, high-resolution clips with write speed of up to 10 GB/s and focus on reliability, previewing clips while recording, controlling the camera, and converting recorded images to CinemaDNG and TIFF formats.

A multi-camera system for medical applications

The system consists of an array of 32 high-resolution MIPI sensors placed on a specially designed sensor carrier board and connected to the NVIDIA AGX Xavier module.

The software provides image acquisition with 5 GB/s total bandwidth, custom image processing pipeline, and an API for remote control.

A large-scale video system for entertainment and 3D scanning

This scalable system is composed of several high-performance Intel/AMD capturing nodes with eight 12-megapixel machine vision cameras per computer.

The software used in the system ensures 64 GB/s bandwidth for each node.

The entire infrastructure is controlled by a dedicated computer.

Low-latency video solutions for embedded AI systems

We have a variety of solutions with low latency streaming applications that run on the NVIDIA Jetson platform.

The following results were achieved for NVIDIA Jetson AGX Orin module:

4K UHD (3840×2160) images at 100 FPS with G2G latency of less than 60 ms.
Full HD (1920×1080) images at 250 FPS with G2G latency of less than 30 ms.

How it works

Follow these steps to tap into all the benefits IFF SDK has to offer:

1. Select cameras and system hardware

This step is one of the most important in creating a machine vision system. The right choice of equipment is crucial for project success.

Please contact us if you have any questions. The MRTech team happily shares its extensive hands-on experience, know-hows, and best practices to help our customers successfully implement even the most complex machine vision projects.

2. Order and get the IFF SDK software package with:

Binary libraries for your system configuration
C/C++ header files, Python interface module
Sample applications with the source code
Technical manual in PDF and HTML formats
The SDK manual is also available on our GitHub account.

The IFF SDK package under a Trial license for evaluation can be provided upon request.

3. Read the technical manual to explore how to use the IFF SDK

The technical manual contains a detailed description of the library components and explains how to use the SDK efficiently.

4. Try out sample applications included in the SDK software package and available on GitHub:

farsight is the first and most general sample application of the SDK.
It supports various functionalities, including acquisition from a machine vision camera, color pre-processing, auto-exposure and white balance, writing to disk, H.265 encoding, and RTSP streaming.
imagebroker is another sample application that shows how to export images to the user code across IFF SDK library boundaries. Additionally, it provides an example code to render an image on the screen using OpenCV.

5. Design the required image processing pipeline using IFF SDK components and describe the pipeline in the JSON format.

The figure on the right represents an example pipeline.

6. Build your own application

Below is an example of a basic C application that illustrates the user-friendly, low-code approach implemented in IFF SDK interface:

iff_initialize(base_cfg);
void* handle = iff_create_chain(chain_cfg, error_handler);
iff_execute(handle, "{\"writer\":{\"command\":\"on\"}}");

10 main reasons to need IFF SDK for your project:

1. Your project requires a top-end performance of the camera system. With IFF SDK you will be able, for example, to capture raw 8K 10-bit images at 115 FPS and process them (applying black level subtraction, histogram calculation, white balance, demosaicing, color correction, etc.) in real-time or write image streams from two such cameras (in raw, TIFF or DNG formats) to RAID storage with a 10 GB/s throughput and without frame loss.

2. You need ultra-low processing and streaming latency, unnoticeable to the human eye. With IFF SDK it is possible to achieve 35ms Glass-to-Glass latency by receiving FullHD raw images from a camera, processing (including H.265 encoding) on a NVIDIA Jetson Orin system and streaming to a receiving computer (also, optionally, powered by IFF SDK) using the RTSP (or alternatively WebRTC) network protocol.

3. You want to quickly get first results. Your project can get a head start by re-using the source code and configuration files of the IFF SDK sample applications. There are examples for image acquisition, color pre-processing, writing to disk, H.264/H.265 encoding, RTSP/WebRTC streaming, integrating the processing pipeline with custom OpenCV code (on-screen rendering implementation is provided).

4. You need excellent image quality and faithful colors. IFF SDK has high-performance modules for high-quality demosaicing, color processing (DNG-compatible) and denoising. A handy tool to create DNG color profiles (DCP) is also provided with the IFF SDK package.

5. You want freedom to choose a camera manufacturer for your system or maybe even a possibility to easily swap camera models without changing your code. IFF SDK supports a variety of machine vision cameras with USB3, Ethernet and PCIe interfaces using GenTL or vendor APIs. IFF SDK also works with MIPI cameras through Video4Linux2 API and supports the GMSL camera interface. In general only configuration file requires adjustment for specific camera model, while code stays the same. Of course your system can also have multiple (potentially different) cameras, limited only by the performance of your hardware.

6. You want an option to have either real-time (with input directly from the camera) or offline processing (with input from the files). IFF SDK supports various input pixel formats, such as 8/10/12/14/16-bit raw or RGB, and output image formats, such as H.264/H.265 and TIFF/DNG/EXR (compatible, for example, with professional video editing application DaVinci Resolve).

7. IFF SDK is easy to learn, as it has a simple C API with just five basic functions plus an HTTP interface to control the image processing pipeline and a configuration file in a JSON format. A comprehensive technical manual and several quick-start and how-to articles are also available online on the MRTech website, which guarantees a quick learning curve.

8. You want a cross-platform solution in case you deploy your code on multiple platforms or for easier and faster code development. For example, you can develop your machine vision application under Windows and deploy it on a Linux machine or NVIDIA Jetson module.

9. You want production-ready code right from the start, allowing fast transition from prototype stage to deployment. High quality of the IFF SDK code is ensured by continuous testing in-house and field-proven in multiple real-life projects.

10. You need comprehensive technical support for your image acquisition and processing tasks. Our MRTech team provides extensive IFF SDK support, assistance in implementation and pre-project consulting, all powered by our know-how accumulated over 10+ years of experience in machine vision field.

To summarise, IFF SDK makes images flow so you can focus on your ultimate goal!

You can take MRTech IFF SDK for a free test drive right now.