Intel open sources its NPU Acceleration library enabling compact AI models to run on

[harlan4096]

Intel open sources NPU Acceleration software for developers



Developers seeking to improve their software for next-gen Intel processors should consider looking into a new library released by the company. The software called NPU Acceleration Library is open source, and it is now available for download.

The library is written in Python, and it is designed to help developers boost their applications through Intel’s newest NPU (Neural Processing Unit) as available in Meteor Lake chips. Currently, no other Intel CPU features NPU, but by the end of this year Intel is also expected to launch Arrow Lake and Lunar Lake with built-in NPU units.

As per Tony Mongkolsmai, who serves as Intel’s Software Architect and Technical Evangelist, this library allows for the seamless local execution of Gemma-2b or TinyLlama compact language models. In theory, this enables developers to easily create local AI chat applications without running into significant challenges. Intel demonstrated this feature during the Meteor Lake launch.

Tony has presented this feature by running the AI model on MSI Prestige 16 AI laptop powered by Intel Core Ultra processor:


...

Continue Reading