Companies Work to Bring Generative AI to the Edge

BrainChip, a small public firm specializing in neuromorphic technology, has released the second generation of its Akida processor designed for generative AI in edge devices, highlighting a market that is rapidly gaining traction among startups as well as established IT component providers.

BrainChip specializes in technology that digitally mimics the patterns of intelligence in the human brain by generating artificial neurons and synapses in complex neural networking patterns. Its Akida platform is fully customizable and can run generative AI models in Internet of Things (IoT) environments, such as smart cars, manufacturing robots, and smart home systems. Using a technique called Temporal Event Based Neural Nets (TENNs), BrainChips says Akida provides a means of running large language models (LLMs) in edge devices with minimal power consumption, a small form factor, and with full accuracy.

BrainChip is part of a rapidly growing group of firms interested in bringing generative AI to the network edge, where it can automate applications across a range of industries. Other companies include Numenta, which offers a Numenta Platform for Intelligent Computing (NuPIC) that delivers a training module and inference server via Docker containers to an enterprise’s Intel CPUs.

“With our optimized inference server, model library, and training module, you can select the right model for your unique business needs, fine-tune them on your data, and run them at extremely high throughput and low latency on CPUs, significantly faster than on an NVIDIA A100 GPU— all with utmost security and privacy,” boasted Subutai Ahmad, CEO of Numenta, in a press release last month.

Source: Numenta

Another company, Innatera, founded in 2018 as a spin-off from Delft University of Technology in the Netherlands, claims its processors feature “up to 10,000X higher performance per watt than conventional microprocessors and digital accelerators.”

Larger Players Involved in Neuromorphic Chips

Neuromorphic technology isn’t restricted to a handful of esoteric companies. Major IT players are also exploring ways to bring generative AI processing out of the supercomputer realm and into the edge. IBM, for instance, recently released information about research into a chip that can mimic the human brain using less energy than today's components for generative AI processing. Intel too has devoted considerable resources to neuromorphic computing and has developed a Loihi chip available for evaluation to qualified parties. Qualcomm has long worked in the field as well. NVIDIA itself acknowledges the need for “edge AI” that depends in part on advances in neural networking and compute infrastructure.

All of which prompts questions about why all these chips aren’t yet generally available. The answer is that progress in this field is slow going.

Progress on a Slow Track

BrainChip, for instance, was spun off out of a larger company in 2004 by Peter Van Der Made (ex-IBM, now CTO) and Anil Mankar (ex-Western Digital, Conexant Systems, now chief development officer). It is based in Sydney, Australia, and for reasons related to its complex history trades publicly on the Australian Securities Exchange (ASX) under the ticker BRN.

So far, it hasn’t done well. For the second half of 2023, BrainChip reported in August a net loss of over $17 million on paltry revenues of $115,606. CEO Sean Hehir called the results “unacceptable” but noted that the company’s would-be customers, though plentiful, have been waiting in the wings for certain capabilities that were released with the second Akida generation last month.

Similarly, Numenta has also taken a long time to get to market. Founded in 2005 by Jeff Hawkins (ex-Palm), Donna Dubinsky (ex-Palm, now board chairman), and Dileep George, the company claims just 27 employees on LinkedIn, and its press releases are largely concerned with publications and demonstrations of its technologies.

All of which indicates that despite the recognized need to bring generative AI to the edge, actual progress has been slow. Still, recent announcements indicate that this nascent market could be on the verge of fresh progress. It’s worth watching.