1.5 Million Threads: AMD-Powered Archer 2 Supercomputer With 12,000 EPYC Rome CPUs

[harlan4096]

1.5 million threads of EPYC CPUs all under one roof.

What would you do with 1.5 million threads of computational horsepower? The UK Research and Innovation (UKRI) center has an idea.

The Archer supercomputer system based in Edinburgh was first introduced in 2013 based on the Cray XC30 design and has since been in use by researchers in need of serious compute power. It was once among the top 20 supercomputers in the world, but that’s no longer the case and the time has come to replace it.

As such, the center announced that it has contracted with Cray to build the Archer 2 supercomputer, which will be based on AMD EPYC Rome processors.

The center will build the system out of 5,848 Shasta compute nodes (deep dive here). Each node will contain two second-gen AMD EPYC Rome processors with 64 cores apiece, with each clocked at 2.2GHz (likely the base frequency).

This adds up to 11,969 CPUs, and a grand total of 748,544 CPU cores and about 1.5 million threads (we rounded that number up simply because we’ve always wanted to).

Depending on how it is measured, averaged out over a handful of benchmarks, the new supercomputer will perform an average of 11 times faster than its predecessor.

Full technical specifications as posted on HPC Wire below:

Peak performance estimated at ~ 28 PFLOP/s

System Design

5,848 compute nodes, each with dual AMD Rome 64 core CPUs at 2.2GHz, for 748,544 cores in total and 1.57 PBytes of total system memory
23x Shasta Mountain direct liquid cooled cabinets
14.5 PBytes of Lustre work storage in 4 file systems
1.1 PByte all-flash Lustre BurstBuffer file system
1+1 PByte home file system in Disaster Recovery configuration using NetApp FAS8200
Cray next-generation Slingshot 100Gbps network in a diameter-three dragonfly topology, consisting of 46 compute groups, 1 I/O group and 1 Service group
Shasta River racks for management and post processing
Test and Development System (TDS) platform, to be installed in advance
Collaboration platform with 4 x compute nodes attached to 16 x Next Generation AMD GPUs

Software stack:

Cray Programming Environment including optimizing compilers and libraries for the AMD Rome CPU
Cray Linux Environment optimized for the AMD CPU blade based on SLES 15
Shasta Software Stack
SLURM work load manager
CrayPat as profiler
GDB4HPC as debugger

The center will install the Archer 2 supercomputer in the same room as the supercomputer it is replacing, meaning there will be a period of downtime. Archer will be shut down on February 18, 2020, and Archer 2 will be booted up 78 days later on May 6, if all goes according to schedule.
...

Continue Reading