AMD unveils its first small language model, AMD-135M — AI performance enhanced by spe
#1
Information 
Quote:General and coding-optimized models released.
 
As AMD flexes its muscles in the AI game, it is not only introducing new hardware but is betting on software too, trying to hit new market segments not already dominated by Nvidia. 

Thus, AMD has unveiled its first small language model, AMD-135M, which belongs to the Llama family and is aimed at private business deployments. It is unclear whether the new model has to do anything with the company's recent acquisition of Silo AI (as the deal has to be finalized and cleared by various authorities, so probably not), but this is a clear step in the direction of addressing the needs of specific customers with a pre-trained model done by AMD - using AMD hardware for inference.  

 The main reason why AMD's models are fast is because they use so-called speculative decoding. Speculative decoding introduces a smaller 'draft model' that generates multiple candidate tokens in a single forward pass. Tokens are then passed to a larger, more accurate 'target model' that verifies or corrects them. On the one hand, this approach allows for multiple tokens to be generated simultaneously, yet on the other hand this comes at the cost of power due to increased data transactions.  

AMD's new release comes in two versions: AMD-Llama-135M and AMD-Llama-135M-code, each designed to optimize specific tasks by accelerating inference performance by using speculative decoding technology, a logical thing to do for a small-language model-based AI service. Somehow, both prevail in performance tests conducted by AMD.
  • The base model, AMD-Llama-135M, was trained from the ground up on 670 billion tokens of general data. This process took six days using four 8-way AMD Instinct MI250-based nodes (in AMD's nomenclature these are just 'four AMD MI250 nodes'). 
  • In addition, AMD-Llama-135M-code was fine-tuned with an extra 20 billion tokens specifically focused on coding, completing this task in four days using the same hardware.
AMD believes that further optimizations can lead to even better performance. Yet, as the company shares benchmark numbers of its previous-generation GPUs, we can only imagine what its current-generation (MI300X) and next-generation (MI325X) could do. 
...
Continue Reading
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)
[-]
Welcome
You have to register before you can post on our site.

Username/Email:


Password:





[-]
Recent Posts
Intel to release another microcode upda...
Intel confirms 13th/...harlan4096 — 09:12
Adobe will disable Adobe Elements 2025 t...
Adobe announced ne...harlan4096 — 09:11
iOS 18.0.1 fixes iPhone touch screen pro...
Apple has released...harlan4096 — 09:09
K-Lite Codec Pack 18.5.5 / 18.5.7 Update
Changes in 18.5.7:...harlan4096 — 09:32
AdGuard Browser Extension 5.0.128
AdGuard Browser Ex...harlan4096 — 09:31

[-]
Birthdays
Today's Birthdays
avatar (47)Konradsuiz
avatar (37)haupeadvoge
avatar (41)RobertBrura
Upcoming Birthdays
avatar (46)Michaelaceve
avatar (36)QuadirLigh
avatar (37)Mblippek
avatar (40)guerigGep
avatar (43)viecontAceve
avatar (48)wohnkwagVib
avatar (43)Michaelsen
avatar (36)armaMIZ
avatar (47)lambrnag
avatar (48)AlbertDig
avatar (48)beipHit
avatar (44)Wendigortup
avatar (37)Timothynet
avatar (40)MichaelHig
avatar (36)icebywiwy
avatar (44)Cameronreoxy
avatar (40)BrendaRusia
avatar (46)maggiebz16
avatar (37)MarthaWes
avatar (46)vikgoMam
avatar (39)Michaelcrini
avatar (37)DonaldSoM

[-]
Online Staff
harlan4096's profile harlan4096
Administrator

>