In the AI Era, Why We Need PIM Beyond HBM

With the arrival of ChatGPT bringing AI deep into our daily lives, memory technology, which serves as the brain of the computer, is also undergoing rapid change. As existing memory struggled to handle AI's enormous data-processing demands, an ultra-fast memory called HBM (High Bandwidth Memory) emerged. But as AI advances further, even HBM is revealing its limits, and the industry is now turning its attention to an innovative technology called PIM (Processing-in-Memory).
1. The fundamental problem of computer architecture - The limits of an 80-year-old design Every computer we use today follows the architecture designed by John von Neumann in 1945. In this architecture, the memory that stores data and the processor that performs calculations are separated, so time and energy are consumed every time data is exchanged.
The problem is that the process of physically moving data takes hundreds of times more energy than the actual calculation. AI must constantly process huge amounts of data, and with the existing architecture, the time the processor spends waiting for data becomes too long.
2. The emergence and achievements of HBM - An innovative solution HBM is a 3D memory technology that stacks multiple memory chips vertically to maximize data transfer speed. It has the advantages of being much faster than existing memory while also being more power-efficient and smaller.
- Explosive growth in the market Along with the AI boom, HBM has become a new growth engine for the memory industry. In particular, as it became essential to NVIDIA's GPUs that run ChatGPT, the sales of related companies have surged. Memory makers such as Micron have even announced that their HBM products are already sold out through 2025.
3. The limits of HBM and new challenges - The wall of cost and complexity Although HBM is innovative in terms of performance, its manufacturing process is very complex and costly. Multiple layers of memory must be stacked precisely, and it generates a lot of heat, requiring advanced cooling systems.
More importantly, although HBM increased data transfer speed, it has not solved the fundamental problem of having to move data.
4. PIM: A paradigm shift
- Processing right where the data is PIM takes an entirely different approach. Instead of moving data, it puts small processors inside the memory to process calculations right where the data is stored. It is like reading a book right inside the library instead of borrowing it and reading it at home.
- Remarkable performance improvements In an experiment conducted jointly by Samsung and AMD, PIM technology doubled GPU performance while saving more than 70% of energy consumption. A PIM product developed by SK hynix also showed processing speeds 16 times faster than before on certain tasks.
5. Technology development and commercialization outlook - Competition among major companies After Samsung unveiled the world's first product applying PIM to HBM in 2021, SK hynix also announced its own PIM technology. Both companies have announced plans to apply PIM technology to their next-generation HBM products.
6. Active government support The Korean government has also recognized the importance of PIM technology and decided to invest a total of 1 trillion won by 2029, and recently announced that it will put in an additional 400 billion won by 2028.
7. PIM-related investment points < Direct beneficiaries > ①SK hynix: developed the world's first commercial PIM product, more than 60% share of the HBM market ②Samsung Electronics: leader in HBM-PIM technology, world No. 1 in memory semiconductors < Equipment and component companies > ①Hanmi Semiconductor: supplies bonding equipment for PIM, supplies NVIDIA ②EO Technics: supplies laser equipment for PIM production ③Winpac: SK hynix partner, supplies test equipment
8. Opportunity factors: ① Growth potential: the PIM market is in its early stages and is expected to grow explosively over the next 10 years ② Government support: national-level investment of 1 trillion won is expected to accelerate technology development ③ Barriers to entry: high technological capability is required, so the oligopoly of existing leaders will be maintained
9. Risk factors: ① Profitability: commercialization will take time, so short-term profitability is uncertain ② Market conditions: semiconductor investment may be reduced if the global economy slows ③ Technical difficulty: there is a risk of technology development failure
💡10. Conclusion💡
The rapid development of AI is changing computer architecture itself. If HBM solved the speed problem of early AI, PIM is a fundamental solution that also addresses energy efficiency. PIM's philosophy, "Don't move the data; process it where the data is," is simple but revolutionary. In the future, how smart a memory you can make will determine competitiveness more than increasing memory capacity. As Korean memory makers are leading the world in PIM technology, this could be a new opportunity for us. However, since it will take time for the technology to be fully commercialized, it is advisable for investors to approach it from a long-term perspective.