AI At An Inflection Point Drives Storage And Memory Demand

At the 2023 Nvidia Global Technology Conference (GTC) CEO Jensen Huang said that artificial intelligence (AI) is at an inflection point and that generative AI is generating a new wave of opportunities with various diverse data and many applications. He mentioned that Chat GPT, a generative AI, generated over 100 million users in just a few months.

Below is a summary slide from his presentation on AI showing various new hardware and software products as well as some of the company’s partnerships, particularly for providing various AI services through the cloud (e.g. big data centers). In his talk Jansen showed applications in medical science including genomic analysis and designing new biochemicals, semiconductor manufacturing, product design, automobile design and manufacturing, digital twins for designing factory and warehouse robotic automation and for creating and editing professional image and videos.

Jensen announced new products and services that will support more sophisticated and powerful AI for more industries. These new services provide productivity boosts using AI. One example being that that one can develop working programs using human language that the generative AI can turn into software. This means that anyone can do some level of programming, without extensive technical training. Jensen said that NVIDIA wants to reinvent itself as a provider of software driven services and provide acceleration libraries for lots of applications.

AI training, in particular, consumes a great amount of digital storage for modelling data as well as memory to support the processing of that data. NVIDIA’s GRACE CPU for AI and cloud workflows includes 1TB of memory. The company’s GRACE Hopper CPU Superchip for big AI and high performance computing (HPC) applications designed to provide 10X higher performance than past devices for applications using terabytes of data, includes 96GB of high bandwidth memory (HBM) close to the processor chip. This chip connected to a GPU is shown below.

It runs all the NVIDIA software stacks and platforms including the NVIDIA HPC SDK, as well as AI and Omniverse. The product includes the company’s BlueField 3 digital processing unit (DPU) and 4^th generation NVLink.

At the GTC DDN announced the compatibility of their A³I storage appliances with the next generation of NVIDIA DGX to support AI training models requiring large data models and high-speed throughput. According to DDN, “Offered as part of DDN’s A³I infrastructure solution for AI deployments, customers can scale to support larger workloads with multiple DGX systems. DDN also supports the latest NVIDIA Quantum-2 and Spectrum-4 400Gb/s networking technologies. Validated with NVIDIA QM9700 Quantum-2 InfiniBand and NVIDIA SN4700 Spectrum-4 400GbE switches.” Double IO performance with the DGX H100 systems requires high performance storage solutions that can support that performance. The DDN AI400X2 storage applianceis shown below.

DDN also said that in addition to these on-premises deployment options, DDN is also announcing a partnership with Lamda to deliver a scalable data solution based on NVIDIA DGX SuperPOD with over 31 DGX H100 systems. Lambda intends to use the systems to allow customers to reserve between two and 31 DGX instances backed by DDN’s parallel storage and the full 3200 Mbps GPU fabric. This hosted offering supplies rapid access to GPU-based computing without a commitment to a large data center deployment.

The NVIDIA GTC showed the company’s continuing support for AI modeling and inference infrastructure as well as new software and service offerings. This infrastructure requires significant storage and memory to train and run these models. DDN showed their latest storage appliance for the DGX H100 systems.