Samsung Develops Industry’s First High Bandwidth Memory with AI Processing Power

Korea on Febru¬≠ary 17, 2021

The new architecture will deliver over twice the system performance
and reduce energy consumption by more than 70%

Sam¬≠sung Elec¬≠tro¬≠nics, the world lea¬≠der in advan¬≠ced memo¬≠ry tech¬≠no¬≠lo¬≠gy, today announ¬≠ced that it has deve¬≠lo¬≠ped the industry‚Äôs first High Band¬≠width Memo¬≠ry (HBM) inte¬≠gra¬≠ted with arti¬≠fi¬≠ci¬≠al intel¬≠li¬≠gence (AI) pro¬≠ces¬≠sing power ‚ÄĒ the HBM-PIM. The new pro¬≠ces¬≠sing-in-memo¬≠ry (PIM) archi¬≠tec¬≠tu¬≠re brings powerful AI com¬≠pu¬≠ting capa¬≠bi¬≠li¬≠ties insi¬≠de high-per¬≠for¬≠mance memo¬≠ry, to acce¬≠le¬≠ra¬≠te lar¬≠ge-sca¬≠le pro¬≠ces¬≠sing in data cen¬≠ters, high per¬≠for¬≠mance com¬≠pu¬≠ting (HPC) sys¬≠tems and AI-enab¬≠led mobi¬≠le applications.


Kwan¬≠gil Park, seni¬≠or vice pre¬≠si¬≠dent of Memo¬≠ry Pro¬≠duct Plan¬≠ning at Sam¬≠sung Elec¬≠tro¬≠nics sta¬≠ted, ‚ÄúOur ground¬≠brea¬≠king HBM-PIM is the industry‚Äôs first pro¬≠gramma¬≠ble PIM solu¬≠ti¬≠on tail¬≠o¬≠red for diver¬≠se AI-dri¬≠ven workloads such as HPC, trai¬≠ning and infe¬≠rence. We plan to build upon this breakth¬≠rough by fur¬≠ther col¬≠la¬≠bo¬≠ra¬≠ting with AI solu¬≠ti¬≠on pro¬≠vi¬≠ders for even more advan¬≠ced PIM-powered applications.‚ÄĚ


Rick Ste¬≠vens, Argonne‚Äôs Asso¬≠cia¬≠te Labo¬≠ra¬≠to¬≠ry Direc¬≠tor for Com¬≠pu¬≠ting, Envi¬≠ron¬≠ment and Life Sci¬≠en¬≠ces com¬≠men¬≠ted, ‚ÄúI‚Äôm deligh¬≠ted to see that Sam¬≠sung is addres¬≠sing the memo¬≠ry bandwidth/power chal¬≠lenges for HPC and AI com¬≠pu¬≠ting. HBM-PIM design has demons¬≠tra¬≠ted impres¬≠si¬≠ve per¬≠for¬≠mance and power gains on important clas¬≠ses of AI appli¬≠ca¬≠ti¬≠ons, so we look for¬≠ward to working tog¬≠e¬≠ther to eva¬≠lua¬≠te its per¬≠for¬≠mance on addi¬≠tio¬≠nal pro¬≠blems of inte¬≠rest to Argon¬≠ne Natio¬≠nal Laboratory.‚ÄĚ


Most of today‚Äôs com¬≠pu¬≠ting sys¬≠tems are based on the von Neu¬≠mann archi¬≠tec¬≠tu¬≠re, which uses sepa¬≠ra¬≠te pro¬≠ces¬≠sor and memo¬≠ry units to car¬≠ry out mil¬≠li¬≠ons of intri¬≠ca¬≠te data pro¬≠ces¬≠sing tasks. This sequen¬≠ti¬≠al pro¬≠ces¬≠sing approach requi¬≠res data to con¬≠stant¬≠ly move back and forth, resul¬≠ting in a sys¬≠tem-slo¬≠wing bot¬≠t¬≠len¬≠eck espe¬≠ci¬≠al¬≠ly when hand¬≠ling ever-incre¬≠asing volu¬≠mes of data.


Ins¬≠tead, the HBM-PIM brings pro¬≠ces¬≠sing power direct¬≠ly to whe¬≠re the data is stored by pla¬≠cing a DRAM-opti¬≠mi¬≠zed AI engi¬≠ne insi¬≠de each memo¬≠ry bank ‚ÄĒ a sto¬≠rage sub-unit ‚ÄĒ enab¬≠ling par¬≠al¬≠lel pro¬≠ces¬≠sing and mini¬≠mi¬≠zing data move¬≠ment. When appli¬≠ed to Samsung‚Äôs exis¬≠ting HBM2 Aquabolt solu¬≠ti¬≠on, the new archi¬≠tec¬≠tu¬≠re is able to deli¬≠ver over twice the sys¬≠tem per¬≠for¬≠mance while redu¬≠cing ener¬≠gy con¬≠sump¬≠ti¬≠on by more than 70%. The HBM-PIM also does not requi¬≠re any hard¬≠ware or soft¬≠ware chan¬≠ges, allo¬≠wing fas¬≠ter inte¬≠gra¬≠ti¬≠on into exis¬≠ting systems.


Samsung‚Äôs paper on the HBM-PIM has been sel¬≠ec¬≠ted for pre¬≠sen¬≠ta¬≠ti¬≠on at the renow¬≠ned Inter¬≠na¬≠tio¬≠nal Solid-Sta¬≠te Cir¬≠cuits Vir¬≠tu¬≠al Con¬≠fe¬≠rence (ISSCC) held through Feb. 22. Samsung‚Äôs HBM-PIM is now being tes¬≠ted insi¬≠de AI acce¬≠le¬≠ra¬≠tors by lea¬≠ding AI solu¬≠ti¬≠on part¬≠ners, with all vali¬≠da¬≠ti¬≠ons expec¬≠ted to be com¬≠ple¬≠ted within the first half of this year.