Brainchip Extends AI, Machine Learning In Space And Time With Bio-Inspired Neural Networks

With artificial intelligence and machine learning (AI/ML) processors and coprocessors roaring across the embedded edge product landscape, the quest continues for high-performance technology that can run a wide range of AI/ML models with very low power consumption. Brainchip has been marketing a line of unique, bio-inspired Akida line of licensable, configurable neural processing IP for a while now. The company’s synthesizable IP is designed to efficiently implement AI/ML workloads as an on-chip CPU coprocessor that demands little CPU intervention. Now, the company has introduced its second-generation Akida AI/ML neural-processing coprocessor architecture, which improves upon the first-generation IP architecture in several ways.

The company’s Akida IP runs standard neural-network models built with standard AI/ML tool flows, but this IP uniquely leverages the energy-efficient organic brain architectures produced over more than 100 million years of evolution, implementing cognitive AI/ML processing using a bio-inspired but fully digital approach to neural processing. This is a radically different design approach compared to the large arrays of power-hungry multiplier/accumulators (MACs) embedded in most of today’s AI/ML neural processing units (NPUs). As a result of Brainchip’s unique design approach to AI/ML processing, the company’s Akida neural processing IP is able to deliver real-time processing with equal or better accuracy than mainstream AI/ML solutions, but with greatly reduced power requirements.

The Akida architecture can solve a wide range of AI/ML challenges in a number of embedded edge applications including:

• Video object detection and vision transformer networks using either high- or low-resolution images
• Advanced sequence prediction
• Object detection, classification, and localization
• Vital signs prediction
• Advanced audio classification, including keyword spotting and beyond
• Gesture recognition
• Vibration analysis and anomaly detection

These applications span many different embedded edge market segments including automotive, smart home, consumer, healthcare, digital security and surveillance, and industrial automation.

There are several forces driving the rise in edge AI/ML processing versus cloud-based AI/ML processing. First, cloud-based services represent an ongoing operating expense, and those service costs are rising. At the same time, cloud-based solutions have responsiveness and latency challenges that sometimes cannot be overcome if AI/ML processing is restricted to the cloud. This problem is exacerbated by the rapidly increasing use of cloud-based processing for many workloads, not just AI/ML, which pits competing workloads on data center servers against each other and burdens the infrastructure needed to transport sensor data to these servers. AI/ML processing that’s localized to an embedded device is untethered from the cloud and the processing continues whether or not the device is connected to a network, which can be a tremendous benefit in unconnected or intermittently connected applications or when in poor coverage areas.

Although data centers and networks are scalable, incorporating AI/ML processing into embedded edge devices creates a solution that automatically scales with load increases because, as more devices needing AI/ML processing are added to the edge, more AI/ML processing capability is added by each new device as well. As a further benefit, keeping sensor data localized in the embedded edge device and performing the AI/ML processing locally also circumvents potential privacy and security issues because private and secure data is inaccessible to and never traverses the network.

The core element of Brainchip’s Akida architecture is a spiking NPU, which emulates the operation of an organic brain’s myriad neurons and synapses. The Akida NPU is event-driven. It only operates when there’s data to process, which conserves power. In addition, each Akida NPU has its own local memory, which eliminates the need to swap AI/ML model weights in and out of external memory. This feature further reduces power consumption while boosting performance and eliminating external memory bottlenecks caused by AI/ML traffic on the memory buses.

The Akida architecture bundles four NPUs into a node and connects multiple nodes with a mesh NOC (network on chip). This architecture can implement standard CNNs (convolutional neural networks), DNNs (deep neural networks), RNNs (recurrent neural networks), sequence prediction networks, vision transformers, and other types of neural networks in addition to the Akida NPU’s native networks, SNNs (spiking neural networks).

The 2^nd-generation Akida platform architecture adds optimized hardware called Vision Transformer nodes, which work with the existing event-based neuromorphic components to create vision transformers. These Vision Transformer networks attain excellent results in vision-recognition, object detection, and image classification applications compared to competing, state-of-the-art convolutional networks with substantially fewer computational resources or electrical power. The new 2^nd-generation architecture has hardware support for long-range skip connections and now supports 8-, 4-, 2- and 1-bit weights and activations, which allows AI/ML development teams to tune model accuracy, memory usage, and power consumption to application requirements.

The company has also added support for what the company calls Temporal Event-Based Neural Networks (TENNs), which reduce the memory footprint and number of operations needed for workloads, including sequence prediction and video object detection, by orders of magnitude when handling 3D data or time-series data. What makes the TENNs particularly interesting is the ability take raw sensor data without preprocessing, allowing for radically simpler audio or healthcare monitoring or predictive devices. Brainchip’s MetaTF software framework automatically converts other neural networks into SNNs. MetaTF works with industry-standard frameworks like TensorFlow/Keras and development platforms like Edge Impulse.

There are three distinct licensable IP products based on Brainchip’s Akida platform architecture, as shown in the figure below:

The Max Efficiency variant (Akida-E) provides as many as four nodes (or 16 NPUs), runs as fast as 200MHz, delivers the equivalent of 200 GOPS (giga-operations per second) of performance, and needs only milliwatts of power for operation. The company says that this smaller variant is designed for running simpler AI/ML networks and is intended for use in continuously operating equipment where power consumption is at a premium. The Sensor Balanced variant (Akida-S) can be configured with as many as eight nodes (32 NPUs), runs as fast as 500MHz, and delivers the equivalent of 1 TOPS (trillion operations per second) of performance, which is capable of running object detection and classification workloads. The Performance variant (Akida-P) accommodates as many as 128 nodes (512 NPUs), operates as fast as 1.5GHz, and delivers the equivalent of 50 TOPS. The most capable version of the Performance variant includes optional hardware support for vision transformer networks that take the form of additional nodes in the internal Akida mesh network.

The high-end variant can run the full gamut of AI/ML models in Brainchip’s model zoo to perform tasks including classification, detection, segmentation, and prediction. Together, these Akida variants allow a design team to use one AI/ML architecture that scales from low-power configurations that consume mere microwatts of power to high-performance configurations that deliver dozens of TOPS, and according to BrainChip, can perform HD video object detection at 30 frames per second while consuming less than 75 milliwatts, which could result in very compelling portable vision solutions.

Brainchip’s bio-inspired Akida platform is certainly an unusual way to tackle AI/ML applications. While most other NPU vendors are figuring out how many MACs they can fit – and power – on the head of a pin, Brainchip is taking an alternative approach that’s been proven by Mother Nature to work over many tens of millions of years.

In Tirias Research’s opinion, it’s not the path taken to the result that’s important, it’s the result that counts. If Brainchip’s Akida event-based platform succeeds, it won’t be the first time that a radical new silicon technology has swept the field. Consider DRAMs (dynamic random access memories), microprocessors, microcontrollers, and FPGAs (field programmable gate arrays), for example. When those devices first appeared, there were many who expressed doubts. No longer. It’s possible that Brainchip has developed yet another breakthrough that could rank with those previous innovations. Time will tell.