Reinventing Semiconductors for the AI Era

Patrick Moorhead:
Welcome to day two, the Six Five Summit. It’s our fifth year after the AI build out in 2023 of a lot of infrastructure and tools. This year’s summit is all about expanding the AI footprint into the data center, into the edge, and delivering real value across enterprises and consumers and customers. So semiconductors have really been the tip of the spear for the AI discussion, and Daniel and I have joked that semiconductors are eating the world versus software that was the meme last decade. They really are. And whether it’s compute, memory, networking, and storage, AI is pressing the limits of all these chips. And with that said, it is my distinct pleasure to introduce our semiconductor track opening speaker and friend of the Six Five Jinman Han, president of Samsung Semiconductor. Welcome back to the Six Five.

Jinman Han:
Oh, thank you. Thank you, Pat, for having me. I’d like to start by saying that we have to remind ourselves that about 60 years ago, Roy Amara, Stanford scientist, he said that the impact of technology is overestimating in the short term, but underestimated in the long run. I think we really have to think about that, but whether we are now in an era of overestimating ai, we are not so sure. But in the long run, I’m pretty sure this will be the start of AI era.

Patrick Moorhead:
Yeah, it is amazing how far it’s come. And in fact, in the 1960s, the first AI algorithms were created. We didn’t do much with it for about 30 to 40 years. And then, through advances in semiconductor technology, programming, expertise and a whole other confluence, here we are in the world of generative AI. Can you talk about, from your point of view, as literally one of the largest semiconductor companies on the planet, how is generative AI redefining your space?

Jinman Han:
I don’t really want to talk about the market situation, because AI is one of the very few bright spots that we are seeing huge demand from the market, but from the semiconductor point of view, I believe that this AI will redefine our manufacturing in addition to design processes in a whole different way. I believe this is a kind of virtual cycle, because we are now providing semiconductors, both memory and logic, to the industry. But this will create new application or use cases for AI. But at the same time, this will help us reoptimize our design process and manufacturing process, which is working kind of a virtual cycle. So it’s going to be very unpredictable what this interaction will bring in the future, but it’s going to be pretty good and pretty exciting.

Patrick Moorhead:
Yeah, it’s interesting. As an industry analyst, I really have to work hard to call something a fad or a trend, and I’ll typically go to the end user in the research that we do, to determine that and then go out through the entire value chain. And it’s absolutely a trend. I met with some end users on the enterprise side that were seeing a 50% increase in efficiency in their creativity process. And we’re seeing this in multiple verticals, in multiple spaces. And we haven’t even touched the consumer side yet. A little bit, but on PCs and smartphones, we believe that the best is yet to come. So, Samsung as a foundry and a designer and IDM, you have to make huge bets years in advance on what’s going to accompany, what is going to be driving a demand, and where. And I’m curious, so you have to be very closely tuned into industry trends. What are some of the industry trends that you see right now changing in the semiconductor market?

Jinman Han:
So one thing that we are noticing right now is that, suddenly, the demand for high speed SSD is going up. It was not like this last year, because, last year, everyone was talking about compute and the memory. So everybody was wondering what’s happening with storage area? And suddenly, from early this year, we are starting to see huge enterprise grade SSD demand going up. And this is all because of this high performance GPU and increased number of parameters for AI models. And also at the same time, you also have to make sure that your trained data has to be saved every hour. So you really need very fast SSSD technologies. And this is really interesting. And right now we are seeing which one is making limitation on this whole system from end to end. So sometimes you see memory, sometimes GPU, CPU, and the storage. It’s just fighting for each other so that it cannot be a bottleneck for this whole system. That’s the thing I’m seeing right now.

Patrick Moorhead:
Yeah, I’m so glad you brought that up. And I like to call that the infrastructure quadrangle, and I said it in the run-up, which is, every big inflection point we’ve seen really required a balanced approach to the compute memory, storage and networking. And if one of those four is not in alignment, you probably won’t get the results that you want. And that’s where… Everything I’ve seen from, my gosh, all the way to the smartphone, to the AI PC, to what’s going on inside of the data center memory and storage has become a bottleneck. And therefore faster and more is going to be what’s going to happen this year and beyond. So I think that should be very good for Samsung Semiconductor. Yeah. So no, please go ahead.

Jinman Han:
But behind all these technological requirements, we keep hearing from our customers about power reduction, power efficiency because this power model requires more demand from GPU and even memory and storage. And this power issue is getting critical right now. I believe that in United States, about 4% of electricity consumption is caused by data center. 4% is not big number. But when you think about the concentration of data center in a very specific region is a huge demand for the region. So power-efficient chips requirement is getting higher and higher. Although, we know that more efficient chips will stimulate more demand. So people might buy more chips, but eventually, we’re going to hit the wall, but at least it can allow us to have more function using the same power. So it’s good for all of us. And you know what, Pat? This week is earth weak. So we have to think about this power green energy issues.

Patrick Moorhead:
Yeah, I’m glad you brought that up. Sometimes we throw around performance, capacity and we don’t hit on… Literally within a year we’re supposed to double the amount of power draw from data centers. And a lot of that has to do with accelerators, and that’s real CapEx that people are looking at. And anything that can be done on the foundry side, the memory side, and even the storage side means that, I like to call it power sloshing, you can provide that power savings either to the compute, the GPU or the processor, or you can just reduce your overall footprint. And for all those who are listening out there, it is really hard to deliver high performance, increased capacity, and lower power all at the same time. But welcome to Jinham’s world here. So no, it’s a huge challenge.

Jinman Han:
Right. It’s a huge challenge. We try to produce, develop power-efficient chips. Everybody knows about that, that’s for sure. But also, I believe that we also have to think about new process technologies, which can endure high temperature in the harsh workload environments. Otherwise, we have to invest too much CapEx on cooling system. I know a lot of companies are working on next generation cooling system, liquid cooling, but the semiconductor is not conducive to heat. So we also have to think about how to develop new process so that our transistors can be more resistant to any heat caused by this demanding workload. And also, we are going to develop power efficient architecture so that we can reduce power consumption in the first place. So that two-track approach might be needed in the future from semiconductor industry, including Samsung.

Patrick Moorhead:
Yeah. One of the key trends too is supply chain diversification. I’m curious what your thoughts are about… I’m here in Austin, Texas and Taylor, Texas, you’re making a multi tens of billions. I think the overall commitment was $100 billion. The recent commitment was $40 billion. So you are definitely one who’s diversifying your manufacturing in multiple parts of the globe. Can you talk a little bit about that?

Jinman Han:
Oh, yeah. We had a very nice ceremony last week acknowledging the funding from US government. It’s the most expensive money I talked before, because US taxpayers’ money, right? We have to spend very wisely. And so our main goal is to have a big presence in the United States so that we can not only produce chips, but also we want to do R&D, as well as advanced packaging. And eventually we really want to promote raising this workforce in semiconductor ecosystem. So that’s a huge goal for us. But the last week was the first step. We got the funding from US government. We really appreciate that.

And you said, really have to also think about not only geopolitical diversification, but we also have to think about geographical diversification.

Patrick Moorhead:
Right.

Jinman Han:
All different things we didn’t expect before it’s happening. It’s like Black Swan event, and we believe that Samsung has to be very resilient in terms of supply chain by having multiple locations as a provider for semiconductor, including Korea and US, of course.
Also, we really want to emphasize not just big companies like Samsung, but we also have to think about collaboration between big players and small players like Setup so that we can, how many recognize value of this data diversification, and especially for this data center, AI solutions. I think there are a lot of things we can do together, not just as a supplier or customer, but as a ecosystem builder. I think in that, I believe something can play a significant role in the United States.

Patrick Moorhead:
There’s one thing, digital twin technology. I’m curious if that’s intersecting your world. The sophistication of your manufacturing, the design you have to do, I would expect that you must be looking into this?

Jinman Han:
Oh yeah. So before this digital simulation is really bread and butter for our development process. We have to simulate our chips, even though we don’t have any process ready. And then we also have to simulate all the devices, how it’s going to work, when to process the chip. And right now, because of this great technological advancement, we believe we’ll be able to simulate the whole chip a whole fab. We can simulate the whole fab, so we can see what’s going to happen to the process, how we can optimize all these equipment placement, or how many wafers can we produce by changing these processes. It’s going to be amazing. So even though we don’t have any fab ready in the next two or three years, we might be building our fab in the virtual environment. We may be able to run the simulations, you see what’s going to happen. It’s going to be amazing, right?

Patrick Moorhead:
Yeah, it really is. The tools particularly, for design validation are pretty amazing. And interestingly enough, similar to other areas in terms of performance improvement, even on creativity tools using Adobe Firefly and things like that, or some of the AI that Salesforce has. So it’s super impressive. So with all of this new technology coming out, some of the new popular things are chiplets, which is revolutionizing, allowing smaller companies to innovate, and be brought to market in a big company foundry, custom HBM, advanced packaging, these all have the ability to speed up the innovation process. And I’m curious, how are you pulling elements like that together to increase time to market and enhance innovation?

Jinman Han:
Yes, Pat, that’s one of the reasons why our customers are so excited about Samsung’s so-called one-stop solution, because we have memory logic, process foundry business, as well as advanced packaging. So we believe we can combine all these technologies in a much more chemical way, so that we can provide a holistic solution to our customers, specific to their needs.

I think there are not many companies in the world can provide their solution. And then this chiplet approach is very flexible. It’s kind of a mix-and-match approach, like a Lego block, but I think it just started, I think it’s just tip of the iceberg. What I’m thinking in the future is that we are going to explore more dimensions to add. We are now talking about 2.5D, 3D, but it’s all between memories and logics, small chiplets. But when you really think about it harder, there are a lot of areas that we can put, add more dimensions. Maybe we can add IP chiplets, maybe we can have 3D interconnections. So there are a lot of room for us to grow into in the future. So this is just the starting, and I believe there are a lot of players in this ecosystem, but Samsung is in unique position to provide very specific, tailored, bespoke approach to each individual customers.

Patrick Moorhead:
And I really like that you’re an IDM, and you’re also a foundry as well. So you have experience on what the end product would look like. And if AI has shown us anything, it’s that a very tight connection, tighter connections between the compute, other types of IP, high bandwidth memory, and then all of the solution put together in advanced packaging is really a requirement, that with Moore’s Law slowing down, we had to find ways to deliver the end performance that we needed. It’s so funny, when I worked for a semiconductor manufacturer years ago, packaging was a sidelight, and now, it’s a first-order citizen along there with the design. I think that’s super exciting. And also an opportunity for Samsung Semiconductor.

Jinman Han:
We are very excited to have this advanced packaging business unit. We just created a team last year, and then it’s getting bigger and bigger, but there are lots of different requirements from our customers. So I think our engineers should work harder to meet all those demands, but it’s in an area that can lift us out of this end of the Moore’s Law wall. I think we’ll see a lot of great breakthroughs in this area.

Patrick Moorhead:
Yeah, I’m super excited on all this. Chiplets to me, I’d love to see small companies who get paired up with foundries, like Samsung Semiconductor capability where they can have a design, have it qualified and ready for the final product to be brought together. And by the way, these new packages blow my mind because, typically, packages added power and it decreased performance in connecting chips. And now we actually have an architecture with chiplets that, through packaging, we keep the performance and we keep the power low. So yeah, packaging is like this first-order citizens.

Jinman Han:
Right.

Patrick Moorhead:
So we’ve talked a lot about your thoughts on the industry today, how we got here with AI, some of the unique parts that Samsung Semiconductor brings to the table. But let’s talk about the future. What types of technology and markets do you see driving the future? You have to make $50 billion commitments upfront and beyond before you even know what the market is, three to four years in advance of the market even being there. It still blows my mind, even though I work for a semiconductor company for over a decade.

Jinman Han:
Pat, I do predictions a lot as other peoples do, but problem is accuracy. And even two years ago, we didn’t expect this open AI [inaudible 00:20:18] thing to be like this. But in three years, we may see great things. I heard that there are lots of disruptive AI models being developed at the lab. I don’t know what that really are, but I’m very excited to see what those models will be.

And so in semiconductor, it’s fairly comfortable for us to predict one or two quarters ahead, because you have to put your wave frame in the fast, so you know what’s going to happen. But after two quarters, it’s very hard to predict. But we believe that in the future, this quantum computing might be one of the areas that will give us huge growth, as well as healthcare as use cases. That’s for sure. I think everybody’s agreed upon on that. But I’m also thinking about these robot technologies. Hey, you know what? In South Korea, every child women give birth to is about 0.6 or 0.7. We need 2.1 to maintain our population.

I don’t think this is unique to South Korea.

Patrick Moorhead:
It’s not.

Jinman Han:
I think except for some continents like Africa, the population is not growing as fast as before. So we really need to compensate for that. Like automotive, self-driving car, they’re all robots. And with this great VR, AR technology and robots and this AI, these are all dots. When you connect all together, we may see great way of compensating for this population decrease. Also, at the same time, this will allow us to have higher productivity to make this economy going forward. So I don’t know what’s going to happen within the next three or four years, but it’s going to be really great technology.

Patrick Moorhead:
Yeah. It really is. And I’m convinced too that with all the action that we’re seeing in the data center, and my firm has been calling the start of a PC super cycle starting in the middle of this year, and really ramping in the first half ’25, I also believe that if we do what we need to do on the smartphone side and with on-device AI with, by the way, more memory, more storage, more NPU compute, we’re going to be driving a lot of cool use cases to drive that.

And one thing we haven’t talked about yet that I’m excited about is, and you hit it with robotics, is generative AI in the industrial IOT. I think what we found is that if you process data on the edge, let’s say it’s smart factory, smart transportation, it makes the investment of replacing what’s on the edge a lot more meaningful, because it basically pays for itself. And I know people aren’t talking a lot about that now, but watch this space as AI goes from the data center to PC to smartphones, and then to the industrial IT, landlocked places, like you said, in robotics.

Jinman Han:
Right. Exactly. And behind all of these great technological advancement, we need semiconductor.

Patrick Moorhead:
Oh, it starts everything. There are people that said software is eating the world. And I always like to say, “Yeah, but software can’t run on air.” And Jinman the last 40 years of technological information, semiconductors and hardware has leaped over each other, right? It’s been this virtual cycle, which has been the hardware and the semiconductors do this big build out. The developers have a lot more to work out, work with. They drive a ton of software development, and then they’re like, “I need more.” So it’s fun to see the overlap here, and Samsung is taking a huge role in driving this generation of generative AI. So I want to thank you for coming on, Jinman, I really appreciate that.

Jinman Han:
Thank you for having me. Always a pleasure.

Patrick Moorhead:
Yeah, absolutely. And so you heard it here, the kickoff for the Six Five Summit 2024 semiconductor track. Appreciate you tuning in today. And hopefully you see the value of memory and storage in the quadrangle of AI and how Foundry and foundry services make this all happen. And essentially, delivering the AI economy, because without these semiconductors, it’s not going to happen. So stay tuned for all the day two coverage today, and make sure you check out day one, because it’s on simulcast, and of course, check in for day three. Thanks for checking in. Appreciate that.