SK Hynix and Samsung’s early bet on AI memory chips pays off

Receive free Semiconductors updates

We’ll send you a myFT Daily Digest email rounding up the latest Semiconductors news every morning.

Shares in South Korea’s leading chipmakers rose this week despite heavy operating losses, as investor excitement builds over their dominance of a niche technology that has emerged as vital for the development of generative artificial intelligence.

SK Hynix and Samsung Electronics control 90 per cent of the global market in “high bandwidth memory” chips, which are crucial components in the systems needed to train AI systems such as OpenAI’s ChatGPT.

Both Korean companies announced plans to double their HBM chip production next year even as they scale back elsewhere in the memory sector amid a supply glut that has slashed operating profits.

“HBM chips are a critical component for AI servers,” said Daniel Kim, an analyst at Macquarie. “You can’t make large language models without them.”

On Thursday, Samsung announced a second quarter operating profit of Won 668.5bn ($522mn), a 95 per cent drop on the same period last year. But it also reported a surge in orders for high-end memory technology, including the latest generation “HBM3” chips, as it outperformed guidance predicting a quarterly operating profit of Won 600bn.

The previous day, SK Hynix, which has pioneered the technology since it released the world’s first HBM chip in 2015, announced a surge in AI-related demand even as it reported a $2.28bn operating loss for the second quarter of this year.

Shares in the company rose by as much as 12 per cent over the course of this week, while Samsung’s rose by 2.4 per cent on Thursday before falling back. Both companies have also benefited from expectations of a wider recovery in the memory sector later this year.

“AI servers use at least double to eight times more memory compared to traditional servers for faster computational processing, and adopts high-performance memory products such as HBM, which not only drives demand but also positively impacts profitability,” Kim Woohyun, SK Hynix’s CFO said during an earnings call on Wednesday.

According to researchers at TrendForce, SK Hynix had a 50 per cent share of the global HBM market last year, followed by Samsung at 40 per cent and US rival Micron trailing at 10 per cent.

Initially used for high-end gaming graphics cards, HBM chips still account for a small share of the global “dynamic random access memory” (Dram) market. They use stacking technology to improve the bandwidth and performance of graphic processing units in high speed computers.

“In terms of volume, HBM chips will account for about 3 per cent of overall Dram shipments this year, and 5 per cent next year,” said Kim of Macquarie. “But given their high prices, they will account for 8-9 per cent of Dram revenues this year.”

SK Hynix last year became the first to mass produce cutting edge fourth generation HBM3 chips used in Nvidia’s H100 GPUs — demand for which has pushed the US chipmaker’s market capitalisation above $1tn in May.

According to industry experts, HBM supply is expected to remain tight over the next two years as it takes time to adjust capacity, while increasing HBM production quickly is difficult because it requires specialised production lines.

An Nvidia employee said it is still purchasing the chips despite rising prices, and has booked enough capacity to meet its GPU orders — contributing to a shortage for smaller market participants.

“It has become harder to obtain HBM3 chips, as generative AI services become more popular and Nvidia is taking most of them for its GPUs,” said Oh Jinwook, chief technology officer at Korean AI accelerator start-up Rebellions.

According to two people close to SK Hynix, Nvidia and AMD have also requested samples of the Korean company’s next generation HBM3E chips, which are not yet in mass production.

Nvidia has asked SK Hynix to supply HBM3E as soon as possible, and is willing to pay a “premium price”, they added.

“They are a necessity for the GPUs used in data centres and AI training, so it is necessary to buy them even if they are more expensive,” a sourcing agent told the Financial Times.

Nvidia declined to comment. AMD did not respond to a request for comment.

Samsung, which is the only other chipmaker to mass produce HBM3 chips, is also working on its own next generation HBM3P chip, which it plans to release later this year.

Jaejune Kim, executive vice-president of Samsung’s memory business, told analysts this week that the company will double its HBM capacity between 2023 and 2024. Samsung insists that it, not SK Hynix, is presently the global market leader.

Meanwhile US chipmaker Micron is at least one generation behind its Korean rivals, according to analysts, but this week announced it had begun sampling its HBM3 Gen 2 chip in a bid to close the gap.

TrendForce expects global demand for HBM chips will grow 60 per cent in 2023, and another 30 per cent next year. It projects that by 2024 SK Hynix will have a 53 per cent share of the HBM market, followed by Samsung at 38 per cent and Micron at 9 per cent.

“This is the first time that SK Hynix is ahead of Samsung in the memory technology race,” said Kim of Macquarie. “SK Hynix bet on HBM chips early on and it worked out well.”