Numascale, Supermicro, and AMD Announce the World‚Äôs Largest Shared Memory System to Date

Sys­tem fea­tures 5184 CPU cores and 20.7 TBytes of shared memo­ry, hand­les mas­si­ve amounts of data as the first part of a new lar­ge cloud com­pu­ting faci­li­ty for Big Data Analytics.

New Orleans, Novem¬≠ber 18, 2014

Numas¬≠ca¬≠le today announ¬≠ced the suc¬≠cessful instal¬≠la¬≠ti¬≠on of a lar¬≠ge shared memo¬≠ry Numascale/Supermicro/AMD sys¬≠tem at a cus¬≠to¬≠mer dat¬≠a¬≠cen¬≠ter faci¬≠li¬≠ty in North Ame¬≠ri¬≠ca. The sys¬≠tem is the first part of a lar¬≠ge cloud com¬≠pu¬≠ting faci¬≠li¬≠ty for ana¬≠ly¬≠tics and simu¬≠la¬≠ti¬≠on of sen¬≠sor data com¬≠bi¬≠ned with his¬≠to¬≠ri¬≠cal data.

The Numas¬≠ca¬≠le sys¬≠tem, instal¬≠led over the last two weeks, con¬≠sists of 108 Super¬≠mi¬≠cro 1U ser¬≠vers con¬≠nec¬≠ted in a 3D torus with Numa¬≠Connect‚ĄĘ, using three cabi¬≠nets with 36 ser¬≠vers apie¬≠ce in a 6x6x3 topo¬≠lo¬≠gy. Each ser¬≠ver has 48 cores three AMD Opte¬≠ron‚ĄĘ 6386 CPUs and 192 GBytes memo¬≠ry, pro¬≠vi¬≠ding a sin¬≠gle sys¬≠tem image and 20.7 TBytes to all 5184 cores. The sys¬≠tem was desi¬≠gned to meet user demand for ‚Äúvery lar¬≠ge memo¬≠ry‚ÄĚ hard¬≠ware solu¬≠ti¬≠ons run¬≠ning a stan¬≠dard sin¬≠gle image Linux OS on com¬≠mo¬≠di¬≠ty x86 based servers.

‚ÄúSupermicro‚Äôs 1U 4‚ÄĎway ser¬≠vers (AS-1042G-LTF) with Hyper¬≠Trans¬≠port (HTX) con¬≠nec¬≠ti¬≠vi¬≠ty pro¬≠vi¬≠de users with fle¬≠xi¬≠ble com¬≠pu¬≠ting resour¬≠ces, inclu¬≠ding capa¬≠bi¬≠li¬≠ties for hand¬≠ling very lar¬≠ge data sets such as tho¬≠se found in appli¬≠ca¬≠ti¬≠ons for hand¬≠ling near real-time sen¬≠sor inputs, com¬≠bi¬≠ned with mas¬≠si¬≠ve amounts of his¬≠to¬≠ri¬≠cal data,‚ÄĚ said Tau Leng, Vice Pre¬≠si¬≠dent of HPC at Super¬≠mi¬≠cro. ‚ÄúOur 4‚ÄĎway sys¬≠tems with can be used as a sin¬≠gle sys¬≠tem or par¬≠ti¬≠tio¬≠ned into smal¬≠ler sys¬≠tems whe¬≠re each par¬≠ti¬≠ti¬≠on runs one ins¬≠tance of the OS. With pro¬≠per Numa-awa¬≠re¬≠ness, appli¬≠ca¬≠ti¬≠ons with high band¬≠width requi¬≠re¬≠ments will be able to uti¬≠li¬≠ze the com¬≠bi¬≠ned band¬≠width of all of the memo¬≠ry con¬≠trol¬≠lers and still be able to share data with low laten¬≠cy access through the coher¬≠ent shared memory.‚ÄĚ

Sys¬≠tems equip¬≠ped with Numa¬≠Connect pro¬≠vi¬≠de high per¬≠for¬≠mance shared memo¬≠ry pro¬≠gramming capa¬≠bi¬≠li¬≠ties with the same cost struc¬≠tu¬≠re as a clus¬≠ter. Numa¬≠connect‚ĄĘ also eli¬≠mi¬≠na¬≠tes the dif¬≠fi¬≠cul¬≠ty of MPI coding for big data pro¬≠blems and can increase pro¬≠gramm¬≠er pro¬≠duc¬≠ti¬≠vi¬≠ty. This alter¬≠na¬≠ti¬≠ve repres¬≠ents a com¬≠pel¬≠ling solu¬≠ti¬≠on for sci¬≠en¬≠tists who curr¬≠ent¬≠ly work with their shared memo¬≠ry codes on x86 desk¬≠tops and lap¬≠tops. The¬≠se users can now sca¬≠le up their data sets wit¬≠hout any extra effort within a fami¬≠li¬≠ar, stan¬≠dard Linux OS environment.

‚ÄúAMD is proud to have been sel¬≠ec¬≠ted to par¬≠ti¬≠ci¬≠pa¬≠te in this exci¬≠ting pro¬≠ject,‚ÄĚ said Suresh Gopa¬≠l¬≠a¬≠krish¬≠n¬≠an, Gene¬≠ral Mana¬≠ger and Cor¬≠po¬≠ra¬≠te Vice Pre¬≠si¬≠dent, Ser¬≠ver Busi¬≠ness Unit, AMD.  ‚ÄúThis col¬≠la¬≠bo¬≠ra¬≠ti¬≠on with Numas¬≠ca¬≠le and Super¬≠mi¬≠cro brings tog¬≠e¬≠ther the right tech¬≠no¬≠lo¬≠gies to sol¬≠ve the chal¬≠len¬≠ging pro¬≠blem of lar¬≠ge shared memo¬≠ry implementation.‚ÄĚ

Einar Rustad, CTO of Numas¬≠ca¬≠le, agrees. ‚ÄúThe sin¬≠gle memo¬≠ry image clus¬≠ter pro¬≠vi¬≠des both shared memo¬≠ry ‚ÄĒ inclu¬≠ding threads and OpenMP ‚ÄĒ and MPI pro¬≠gramming opti¬≠ons,‚ÄĚ he said. ‚ÄúThe sca¬≠lable sys¬≠tem takes advan¬≠ta¬≠ge of low cost com¬≠mo¬≠di¬≠ty x86 hard¬≠ware to offer signi¬≠fi¬≠cant savings when com¬≠pared to con¬≠ven¬≠tio¬≠nal shared memo¬≠ry sys¬≠tems. With Numa¬≠Connect‚ĄĘ, sys¬≠tem admi¬≠nis¬≠tra¬≠ti¬≠on is iden¬≠ti¬≠cal to that of a sin¬≠gle ser¬≠ver becau¬≠se the¬≠re are no sepa¬≠ra¬≠te node images to main¬≠tain and dis¬≠tri¬≠bu¬≠te. We are very proud to have been part of the pro¬≠ject that enab¬≠led us to build this ground¬≠brea¬≠king sys¬≠tem in clo¬≠se coope¬≠ra¬≠ti¬≠on with Super¬≠mi¬≠cro and AMD.‚ÄĚ

Numa¬≠Connect works with AMD Opteron‚ĄĘ-based ser¬≠vers and pro¬≠vi¬≠des up to 256 TBytes of sys¬≠tem-wide shared memo¬≠ry using cache cohe¬≠ren¬≠cy logic with a direc¬≠to¬≠ry-based pro¬≠to¬≠col that sca¬≠les to 4096 nodes. The cache cohe¬≠ren¬≠cy logic is imple¬≠men¬≠ted in an ASIC tog¬≠e¬≠ther with inter¬≠con¬≠nect fabric cir¬≠cuit¬≠ry with rou¬≠ting tables for mul¬≠ti-dimen¬≠sio¬≠nal Torus topo¬≠lo¬≠gies. This type of fabric is very sca¬≠lable and the same topo¬≠lo¬≠gy is used in the world‚Äôs lar¬≠gest supercomputers.

Visit Numas¬≠ca¬≠le in booth 1923 at SC14 to see live pro¬≠duct demos.

About Advan­ced Micro Devices, Inc. (AMD)

AMD (NYSE: AMD) designs and inte­gra­tes tech­no­lo­gy that powers mil­li­ons of intel­li­gent devices, inclu­ding per­so­nal com­pu­ters, tablets, game con­so­les, and cloud ser­vers that defi­ne the new era of sur­round com­pu­ting. AMD solu­ti­ons enable peo­p­le ever­y­whe­re to rea­li­ze the full poten­ti­al of their favo­ri­te devices and appli­ca­ti­ons to push the boun­da­ries of what is pos­si­ble. For more infor­ma­ti­on, visit

About Super¬≠mi¬≠cro Com¬≠pu¬≠ter, Inc. 

Super¬≠mi¬≠cro¬ģ (NASDAQ: SMCI), the lea¬≠ding inno¬≠va¬≠tor in high-per¬≠for¬≠mance, high-effi¬≠ci¬≠en¬≠cy ser¬≠ver tech¬≠no¬≠lo¬≠gy, is a pre¬≠mier pro¬≠vi¬≠der of advan¬≠ced ser¬≠ver Buil¬≠ding Block Solu¬≠ti¬≠ons¬ģ for Data Cen¬≠ter, Cloud Com¬≠pu¬≠ting, Enter¬≠pri¬≠se IT, Hadoop/Big Data, HPC, and Embedded Sys¬≠tems world¬≠wi¬≠de. Super¬≠mi¬≠cro is com¬≠mit¬≠ted to pro¬≠tec¬≠ting the envi¬≠ron¬≠ment through its ‚ÄúWe Keep IT Green¬ģ‚ÄĚ initia¬≠ti¬≠ve and pro¬≠vi¬≠des cus¬≠to¬≠mers with the most ener¬≠gy-effi¬≠ci¬≠ent, envi¬≠ron¬≠men¬≠tal¬≠ly-fri¬≠end¬≠ly solu¬≠ti¬≠ons available on the mar¬≠ket. For more infor¬≠ma¬≠ti¬≠on, visit

About Numas­ca­le

With offices in Euro­pe, Asia, and the USA, Numascale’s ground­brea­king Numa­Connect inter­con­nect tech­no­lo­gy is the back­bone for the most sca­lable ser­ver com­pu­ter sys­tems. Numa­Connect allows high-volu­me manu­fac­tu­red ser­ver boards to be used as buil­ding blocks for sys­tems with fea­tures that are only found in high-end enter­pri­se ser­vers. Numa­Connect includes full sup­port for vir­tua­liza­ti­on of pro­ces­sing, memo­ry, and I/O resour­ces, and can be used with stan­dard ope­ra­ting systems.

Numas­ca­le is sup­port­ed by: Sta­toil, Pro­Ven­ture, Inves­tinor, Inno­va­ti­on Nor­way, Nor­ges forsk­nings­råd, and Euro­stars. For more infor­ma­ti­on, visit