Meet the world's largest chip manufactured by TSMC 16nm

[AthlonXP1800]

https://wccftech.com/meet-cerebras-...ore-than-56-times-the-size-of-an-nvidia-v100/





Nvidia CEO said TSMC 300mm wafer had limited size up to 850mm2 for chips, it cant go beyond 850mm2. Volta V100 was the world largest 815mm2 GPU chip on TSMC 12nm process.

But Jensen Huang got it wrong so now Volta had lost world largest chip crown to Cerebras Systems WSE Wafer Scale Engine's 46,225mm2 AI chip contained 1.2 trillion transistors, 400,000 cores, 18GB SRAM with 9 PB/s memory bandwidth (yes that is 9 petabyte memory bandwidth per second ), 100 Pb/s fabric bandwidth (yes that is 100 petabyte fabric bandwidth per second ), 9 and the chip manufactured on TSMC 16nm process consumed 15 KW power.

I found Cerebras has website.

https://www.cerebras.net/​

I been read about Cerebras a year ago that the CEO was seemed very confident claimed their first chip will win AI, machine learning and data center war and reckon it will beat Nvidia but I dismissed it as utter nonsense so after read the news and I changed my view that Cerebras CEO is very serious about the chip. Guess AMD, Intel and Nvidia are in huge trouble, the big three cant compete with 1.2 trillion chip now. Cerebras WSE chip devastated and unimaginable 100 petabyte fabric bandwidth and 9 petabyte memory bandwidth just made NVLink and future HBM3 and HBM4 looked like obsolete.

Hopefully Jensen Huang had changed his view a while ago after heard about Cerebras Systems WSE realised Jensen can create GPU far beyond 850mm2 and Nvidia can develop Volta successor with 1.2 trillion GPU for data center and the GPU can do full scene ray tracing consume a lot less than 15KW power in 2020.

 

[AthlonXP1800]

I think you are being a bit enthusiastic there - it is essentially still a bunch of chips stitched together and given the issues cooling it the resulting performance from keeping the frequency to feasible levels probably won't outperform a bunch of separate chips stitched together via a longer interconnect for compute stuff.

I can't see it competing with those 3 - it will probably be king of a specific niche but not have broad application.

Accorded to Cerebras CEO Andrew Feldman, a single Nvidia DGX-2 system cant compete with a Cerebras WSE silicon so Nvidia will need 20 to 30 DGX-2 systems in NV switches to compete with Cerebras WSE silicon. 1 DGX-2 system has 2 petaflops performance while 20 to 30 DGX-2 systems in NV-Switches has 40 to 60 petaflops performance. AMD's future Frontier supercomputer due in 2021 will have 100 cabinets with 1.5 exaflops performance while a single cabinet will have 15 petaflops mean Cerabras WSE will be up to 4 times faster than EPYC 4 CPUs and Arcturis GPU successor that do not exist yet

https://medium.com/syncedreview/hot...-ai-chip-as-big-as-a-notebook-why-4d068429349

It seemed Cerebras did an amazing job stitched chips together really beautiful, very clean and very neat resembled of bank card security chip, much better than AMD shoddy poorly job with HBM, HBM2, CPU chiplets and I/O die. Look like it probably has 84 chips so each chip would have 214MB, 107TB/s memory bandwidth and 1.19PB/s fabric bandwidth. Wow insanely ridiculous fast than NVLink Fabric's 900GB/s bandwidth and non existed HBM3, HBM4.

If Cerebras WSE will do a good job with benchmarks and lots of press in the next few months then I think Nvidia will have plan to acquire Cerebras.

 

[AthlonXP1800]

Sounds like a good dose of speculation in there, give it a year or so and we might see what it actually does.

A year?

Accorded to sources it will be in September or in the next few months.

 

[AthlonXP1800]

Oh wow that power draw though!

Equates to about 20 pounds a day for electricity running for 8 hours

A single Cerebras WSE silicon has 15KW power. 1 Nvidia DGX-2 system has 10KW power so to match Cerebras WSE performance would need 20 Nvidia DGX-2 systems with 200KW power or 30 Nvidia DGX-2 systems with 300KW power. Much faster and 20 times more energy efficient than AMD upcoming Frontier 2021 supercomputer cabinet with 300KW power.

 

[AthlonXP1800]

https://www.zdnet.com/article/cerebras-ceo-big-implications-for-deep-learning-in-companys-big-chip/

"The more interesting thing would be to divide up work so that some of your 400,000 cores work on one layer, and some on the next layer, and some on the third layer, and so on, so that all layers are being worked on in parallel," he muses. One effect of that is to vastly multiply the size of the parameter state that can be handled for a neural network, he says. With a GPU's data parallelism, any one GPU might be able to handle a million parameters, say. "If you put two GPUs together [in a multi-processing system], you get two machines that can each handle a million parameters," he explains, "but not a machine that can handle 2 million parameters — you don't get a double."

With the single WSE, it's possible to support a four-billion parameter model. Cluster the machines together, he suggests, and "you can now solve an eight-billion or 16-billion parameter network, and so it allows you to solve bigger problems by adding resources."

Wow a single WSE can handle data with 4 billion parameter model that is 4000 times more than a Nvidia Volta V100 GPU, Intel upcoming Xe GPU for datacentre and AMD Radeon Instinct Vega, Navi GPU and even the fastest supercomputers with thousands of GPUs still limited with 1 million parameter model. 2 WSE machines on cluster can handle 8 billion parameter and 4 WSE machines together can handle 16 billion parameter. Bloody hell, AMD upcoming Frontier supercomputer due in 2021 with 100 cabinets with thousands of AMD Radeon Instinct GPUs on network are limited with 1 million parameter data model will use over 30 MW power. I cant imagine 100 cabinets water cooling 100 WSE will handle 400 billion parameter data model to solve biggest problems ever used only just 1.5 MW power.

Jensen Huang, Lisa Su and Raja Koduri all three will have excessive sweating and very concern now.