AMD explains how easy it is to run local AI chat powered by Ryzen CPUs and Radeon GPU

[harlan4096]

“Chat with Ryzen/Radeon”

AMD guides how to run local AI chats with their hardware. 



AMD does not have its own tool alike to NVIDIA Chat with RTX. NVIDIA came up with a simple tool that can be used to run locally smaller LLM (Large Language Models) powered by GeForce GPUs. While AMD has no such tool, the company suggests one should try using a 3rd party software that runs on CPUs and GPUs.

AMD Ryzen 7040 “Phoenix” and 8040 “Hawk Point” are equipped with XDNA AI accelerator, which can offer anything from 10 TOPS to 16 TOPS (Trillion of operations per second) of AI inference. Soon those AI processors will be utilized broadly and thanks to upcoming changes to Windows, the importance of AI cores will only grow.

Meanwhile, rather than develop their own tool, AMD is recommending that users should check the LM Studio software, which is a closed source, cross-platform and cross-vendor solution. It also comes with easier model management, but also many options that Chat with RTX simply does not have.



LM Studio, Source: VideoCardz

LM Studio works with Apple M-Series, CPUs that have AVX2 support and can also take advantage of AMD and NVIDIA GPU acceleration. For GPUs, the tool has a 16GB VRAM recommendation, while CPU based ML algorithms should have at least 6GB of memory available.
...

Continue Reading