When it comes to advances in edge intelligence, one can draw parallels to the evolution of the human brain itself. Consider where we are today; edge devices are becoming aware of their environment, dealing with several concepts (or inputs) at the same time and imagining the future consequences of their actions. That almost sounds like the teenage years. It is incredibly exciting when you think about the future possibilities, and the edge is just getting started. Using advances in machine learning (ML) and neural networks, developers are creating industrial machine vision, image classification, object detection, speech recognition, natural language processing and hand gesture and emotion detection applications that are transforming how we live, work and learn. These and many other ML innovations deployed on edge devices are adding new dimensions of safety, intelligence and convenience to our homes, offices, factories and cities.
Think of machine learning as the way to leverage compute engines to handle complex human tasks but faster and even autonomously without our direct intervention. ML apps run on a torrent of mathematical operations totaling trillions of operations per second. Machine learning once involved costly, complex cloud computing or “cloud-based AI.” But nowadays much of the ML action can and should happen at the edge where smart, connected devices provide faster runtime responses, lower latency (i.e. no round-trip data journeys to the cloud), and greater reliability, privacy and security.
Today’s engines for edge-based ML are powerful multicore processors packed with features and capabilities to handle neural network, vision, voice and multimedia processing – all in a single, system-on-chip (SoC) device. Enter the i.MX 8M Plus applications processor family shipping now in volume to customers worldwide. The i.MX 8M Plus is a centerpiece of NXP EdgeVerse™ portfolio and the popular i.MX 8 series.
We packed the i.MX 8M applications processor with the right mix of heterogeneous cores, subsystems and interfaces that developers need. The i.MX 8M Plus is the first i.MX processor to integrate a dedicated neural processing unit (NPU), accelerating ML inferencing for the industrial and internet of things (IoT) edge. It also integrates dual image signal processors (ISPs), quad or dual Arm® Cortex-A53 cores running at up to 1.8 GHz, an independent real-time subsystem with an 800 MHz Cortex-M7 core, an 800 MHz HiFi 4 audio DSP for voice and natural language acceleration and a high-precision graphics processing unit (GPU). Developers can offload ML inference functions to the NPU while tapping the combined power of the Cortex-A and Cortex M cores, DSP and GPU to execute other system-level or application tasks.
i.MX 8M Plus Applications Processor Block Diagram
Rounding out its ML and multimedia capabilities, the i.MX 8M Plus has a sharp set of eyes and ears with dual-camera ISPs and eight microphone inputs. Robust control networks supported by dual CAN-FD and dual Gigabit Ethernet with time-sensitive networking (TSN) support an array of industrial automation designs. Error correction code (ECC) technology protects internal memories and helps to ensure high reliability for demanding factory floor applications.
The i.MX 8M Plus processor enables developers to bring their ML workloads right to the edge, which is the closest point to the actual sensing whether it is voice, vision or anomaly detection. Developers can access the on-chip NPU, with 2.3 trillion operations per second (TOPS) performance, to handle ML tasks approximately 30 times faster than the Arm core. Optimizations for compression and sparsity (i.e., eliminating unnecessary 'multiply by zero' calculations) further enhance the performance of ML algorithms. Data compression is also useful because it eliminates system memory bottlenecks, while memory transfers race along with a DRAM controller capable of 4 giga transfers per second (GT/s).
Edge-based ML applications often incorporate machine vision powered by camera-driven image data. What if high-resolution cameras are located far from the edge device? No problem for devices built with the i.MX 8M Plus. To minimize latency and image degradation, the dedicated ISPs easily handle image acceleration on-chip, while two camera inputs enable stereo vision, mimicking combined left/right eye human vision.
The integrated ISPs provide ample power to handle complex image processing tasks such as rapid multiple exposures for high dynamic range (HDR) or simultaneous wide-angle and zoom views from two different cameras. Dewarping is another important feature handled by the ISPs. A low-cost camera with a fisheye lens can warp images beyond human comprehension. The processor’s dewarping engine normalizes the image for accurate ML processing. The ISPs also handle image enhancements such as “de-noising,” which is critical in low-light situations that cause blurry images. Using ML algorithms, the ISPs detect edges that need to be sharp while keeping other image areas naturally blurry. For advanced multimedia and display, the i.MX 8M Plus also includes a video decoder/encoder supporting the H.265 compression algorithm to save large images for offline storage and future processing.
The i.MX 8M Plus reveals features tuned for industrial applications including ECC, a must-have for detecting and correcting system memory bit errors. For example, alpha particles, a type of radiation in some industrial environments, can induce soft errors in SRAM, causing memory issues that can crash software. ECC algorithms detect and correct these errors, resulting in much lower soft error rates (SER).
Many complex factory processes require precisely timed, deterministic Ethernet data transmissions. For this reason, the i.MX 8M Plus supports TSN Ethernet connectivity, which helps guarantee that data is transmitted at the right time for time-sensitive functions on the factory floor. In addition, two CAN-FD peripherals support low latency communications and networking functions. Combined with all the other on-chip cores, peripherals and interfaces, the i.MX 8M Plus enables a well-rounded industrial system and provides a range of edge-based ML capabilities for Industry 4.0.
While the electronics industry constantly evolves, there’s still a constant need for reliability, robust operation in harsh conditions and product longevity. The i.MX 8M Plus applications processor is part of NXP's longevity program providing a minimum of fifteen years of supply from product launch. You can count on i.MX 8M Plus processors powering your ML applications for many years to come.
NXP supports i.MX 8M Plus machine learning, vision, multimedia and industrial IoT applications with a wide range of audio boards, camera modules and evaluation kits, along with a rich ecosystem of partner tools and solutions. To get started, order an i.MX 8M Plus evaluation kit today.
For product information please visit i.MX 8M Plus
To learn more about image processing in the i.MX 8M Plus, watchTwelve Minutes smarter – Machine Learning and Images Processing Demystified
To get started quickly, lean on one of our early access partners profiled in the companion blog here.
Amanda McGregor leads product innovation for NXP applications processors. She has driven the definition, enablement and adoption of i.MX applications processors for more than a decade and led the development of NXP 14LPC FinFET applications processors for consumer, industrial and IoT applications.
