Microsoft, Others Drive AI to the Edge

Microsoft (MSFT) has announced a series of products geared to simplifying the use of artificial intelligence (AI) in low-power, edge-enabled cameras and listening devices. Dubbed Azure Percept, the series includes hardware accelerators and no-code/low-code interfaces for developing secure Azure-based Internet of Things (IoT) applications.

The announcement was made as part of Microsoft’s Ignite conference for developers and IT pros that’s happening virtually this week. The Percept series, which is currently available for preview only, includes ready-to-OEM Intel (INTC) and Arm silicon and Asus endpoint gear. Everything is compatible with existing Azure products, including Azure AI, Cognitive Services, Machine Learning, Live Video Analytics, and bundled Azure IoT products and services.

The move comes shortly after news of Microsoft's partnership with Infiot, in which Infiot's palm-sized Intelligent Access devices will connect IoT sensors over LTE and 5G cellular networks with Azure IoT cloud. That news and today's Percept announcement are firm acknowledgement by Microsoft that AI and its cousin machine learning (ML) are part of a growing trend to streamline analytics by processing data at the network edge. [Editor's note: Infiot's innovation earned it a spot on our recently released Futuriom 40 list of hot private companies. Read more there.]

“When I talk to customers, it doesn’t matter what industry, size, or shape the customer is, everyone seems to have a business scenario that has them looking to leverage AI at the edge,” said Erin Chapple, corporate vice president of Azure Compute, during an analyst briefing last week.

Competitors AWS and Google also offer AI-edge wares in a range of ways. Google’s Edge TPU ASIC, for instance, is part of the cloud vendor’s series of developer tools for putting AI into edge apps. And AWS offers software to equip edge applications with ML for IoT deployments. So far, though, Azure’s Percept seems to be a first in terms of packaging OEM-ready, specifically focused edge devices with chips and software.

More on AI-at-the-Edge

As the Percept announcement makes clear, AI growth is expected to be robust in IoT and industrial IoT (IIoT) scenarios, where it’s become practical for performance and security reasons to process as much data as possible close to endpoints before directing it to central analytics engines.

“You're going to see smart lawnmowers, smart tractors, smart air conditioners, smart elevators, smart buildings, smart warehouses, robotic retail stores. The entire retail store is like a robot and they will all have autonomous capability, they'll all be driven by AI,” said Jensen Huang, CEO of NVIDIA (NVDA), during the company’s latest earnings call.

Chip developers like NVIDIA are responding to demand with products such as NVIDIA’s EGX, a 5G-compatible chip which is geared to sifting endpoint data for use in AI-based analytics applications, as illustrated in the diagram below:

Source: NVIDIA


NVIDIA, Intel, and Arm aren’t the only big challengers in the game. AMD and Qualcomm (QCOM) also have products, as do many others. And newer chip companies with innovative designs have emerged, such as EdgeQ, a well-funded startup that boasts a 5G “base-station-on-a-chip,” and Mythic, which claims its AI chip design uses analog computing to achieve faster performance.

“In the digital world, even with clever architectures, memory is the main bottleneck,” states Mythic’s website. “Squeezing more performance in the presence of severe memory bottlenecks quickly drives up the power and system cost….” states Mythic's website.

Besides bringing hefty compute capabilities onto endpoint devices that have limited power, companies also compete on differentiators of cost and performance. Flex Logix Technologies, for example, claims techniques that speed up the neural networking models used in edge applications for robotics, industrial automation, medical imaging, scientific research, banking, retail, autonomous vehicles, and aerospace.

Startups also are addressing the specific kinds of AI needed to power edge applications. Visual objects, for example, require approaches that enable an endpoint to sift images — from traffic cameras, say — and only send relevant ones to the central system. According to startup Hailo, this calls for thoughtful work by developers, since there are various tradeoffs involved, particularly when high resolution images are required.

The companies mentioned above represent a growing ecosystem that includes many other established and new players, and many can be found in an organization called the Edge AI and Vision Alliance.

So far, however, none of the big public cloud companies belong to that alliance, though, like Microsoft, they are no doubt intent on working with established and new firms to advance the edge AI cause. Expect to see many more announcements that engage cloud titans with component suppliers and OEMs to meet growing demand.