AMD: Advancing AI from Cloud to Edge to Endpoints

Patrick Moorhead:
Welcome back to the Six Five Summit 2024. We are talking cloud infrastructure. And as you probably know, the theme of this year’s summit is all about AI and how enterprises and consumers are taking advantage of that. And of course, we’re still building out the infrastructure as we move along here. And it is my distinct pleasure to introduce Robert Hormuth from AMD. Robert, man, it’s great to see you. What have we known each other for 15 years or something?

Robert Hormuth:
At least 15, if not more, Patrick. It’s great to see you and great to be on the show.

Patrick Moorhead:
It’s been great. And Robert, this whole data center AI thing has been absolutely amazing, but what’s as important as the technology is dialing out on the strategy side what enterprises are doing and why they’re doing it. And so, a pretty broad question to start here. We believe that AI is really a journey, not a destination. It’s not like, “Okay, we’re here.” And I’m just curious, because you talk to a lot of enterprises, where do you see them positioned along this AI timeline?

Robert Hormuth:
It’s a great question because you’re absolutely right, AI is a journey and if we look back at all the historic technology waves, from the mainframe through the PC, all these waves have come and they’ve all been bigger than the last. And there’s a strong belief that this is going to be the biggest one yet ever, which has always been the case. But I do think we’re super early on the journey, because I look at it and go, “Well, what are we using AI for today?” And if you look at it today, it’s mostly human influence.

It’s buy, click, watch, ad, sell, very human influence based. We’re moving into the human Q&A, we’re starting to move into with Copilot, we’re starting to move into human productivity. And with some of the video and things like that are being that we’re actually getting into reasonable content creation, other than a PowerPoint summary. But where we’re moving to is what I like to think of is that the far end of human influence is true human assistance. If you’re old enough like me and I grew up watching The Jetsons and Rosie the Robot, that’s where we’re going.

Patrick Moorhead:
Which by the way, I did. I am Gen X, Robert, so I’m there with you.

Robert Hormuth:
So, I think that that true human assistance is going to just be huge. And it’s hard to say who’s going to be the winners, who are going to be the losers, but it’s going to be a very big field. And I think that the enterprises are just, they’re looking at today, where can they get the best return to impact their business, whether it be fraud detection in banking or new user experiences for helping consumers on their platform with enhanced chatbots and things. But I think we’re really early on that journey towards that end state of really truly helping humans live better, healthier, longer lives and doing useful stuff for us, so I’m excited.

Patrick Moorhead:
Robert, I remember, I started my career when there were still mini computers, but we were moving to Unix servers, not Linux servers, but Unix servers and PCs. And it’s interesting, this definitely feels like something is bigger than dare I say, even the internet. I’m getting that feeling right now, and that build-out feels like there’s even more reality on the benefit behind it. So, AI definitely continues to evolve and advance. And I was at a bunch of conferences this week and there were some people that are using this whole notion of AI time, like AI time, pre-AI time. It was a year. And AI time, it’s a quarter, three months or a month. And it’s got to be hard for enterprise IT to know how do I get best positioned to be able to implement AI? What kind of recommendations do you have for them?

Robert Hormuth:
So, for enterprises, what we’ve been talking a lot about is, for them to get ready, there’s four or five things, six things they need to really think about in terms of doing and/or partnering. And one of the biggest ones right now is, if you look at the power infrastructure landscape of data centers today, we’re at a historic low vacancy rate. It’s 1.7% vacancy rate on data centers. So, there is no space at the end. And if you look at the data centers in construction, something like 84, 85% of the new construction that hasn’t finished is already leased. So we have this power capacity space problem brewing that enterprises have to really figure out. And so, one of the biggest things that they can do to make space and create capacity for AI is deeper consolidation. So, drive performance and efficiency with EPIC, you can get 5:1 or 8:1 consolidation of those five-year-old servers.

I did the math and it’s estimated there’s about a hundred million five-year-old servers out there, and that is about 43-ish so gigawatts of power. And if we consolidate it all that down to EPIC, that cuts it from that 100 million to about 21 million servers at about 14 gigawatts. So, there’s 30 gigawatts right there that we can go create space power to go do AI. So, I think that’s step one. The second step, and kind of goes hand in hand efficiency and performance, you can’t have efficiency and sacrifice performance, so, you’ve got to do those two together. And for us, part of that journey too with enterprise is trying to make it really easy, how to make easy consolidation or easy AI adoption. And so we’re addressing that both on the EPIC side and the Instinct side. We put out a tool with VMware to help ease the migration across clusters called VAMT, VMware Architecture Migration Tool.

So, we help automate that consolidation. And then, on the Instinct side, we’re trying to make it really easy to just work out of the box at the AI framework level. So, if you’re operating at the framework level, we just work right out of the box. So, we’re trying to make it easy. And then, the last part of this we talk about is picking a partner that’s really driving the innovation curve with advanced packaging, advanced semiconductor, and that can execute. And execution has been a hallmark for AMD the last five… We’ve executed four generations up EPIC on time. The fifth is about out of the house, the next one is coming soon. And so, I was joking at Dell Tech World, we have a word for that in Texas, strategy without execution is all hat, no cattle.

Patrick Moorhead:
That’s good.

Robert Hormuth:
So we have hat and cattle at AMD. We have a strategy and we’re executing to it.

Patrick Moorhead:
I like that. So, you brought up a really unique point that I don’t hear a lot, which was that, we’re out of room and out of power, consolidate maybe your non-AI AI servers to make room for the AI-AI part of it. I’m surprised we don’t hear that a lot more. And Robert, just to put an exclamation point on the power, in Reston, Virginia, there literally is no more power on the grid. There’s no more power for more EVs, more homes, more data centers. They are tapped out and it takes, what, five years best case to get a new power station up and running. So, I really appreciate you framing that. And it sounds like, when it comes to Instinct, your play is the easy button or was that EPIC?

Robert Hormuth:
Well, it’s actually we’re trying to make it easy for… For us, at AMD, we view it as, we want to make it easy for customers to adopt AMD technology. With EPIC, it’s, in the EPIC world, how do we make consolidation easy and be easy in the x86 ecosystem of software and all that? So that’s great for x86, but we still have to make the consolidation easy to go drive that goodness. And then, on the Instinct side, it’s about supporting the open source, supporting the frameworks out of the box. It’s about having the same libraries at the bottom, it’s about having a rock’ em… If people are operating at the Triton level, we have a rock’ em Triton backend. And so that again, where we’re just right out of the box. And so, we have to have to be easy on both sides to do that.

Patrick Moorhead:
That’s great. So let’s talk about differentiation, Robert. Maybe we take the next few minutes, talk through that. How is AMD distinctive as it relates to AI infrastructure?

Robert Hormuth:
I think if you look at the journey that we were on with EPIC, we tackled some of the biggest challenges in terms of driving core count up and driving massive differentiation in core count, IO count and memory bandwidth. And that’s been a really good recipe for EPIC to drive a clear performance and for per watt leadership perspective. And if you think about where we’re going with Instinct, we get the advantages of all those years of advanced packaging that we’ve been doing. And now, we get to do it over on Instinct. And then, if you look at the 300X, what’s the big value, the differentiator of 300X? We’ve got two and a half times more onboard memory, and we’ve got 1.6x the ATM bandwidth compared to the volume competitor right now.

And so, what that translates to from a TCO, it’s really simple. If you needed eight of theirs to fit a model, you can do it in three and a half of hours. That’s pretty easy math to go see the memory advantage. So, we’re going to keep pushing that memory advantage, that memory bandwidth. And I’d say the other thing that is part of our strategy on Instinct that is we started a while back, which was addressed in the open. AMD is always about open industry, open source, open standards. So, we started the Ultra Ethernet Consortia with a bunch of our friends to really go drive ethernet at scale, have an open choice ecosystem of nicks and switches. And that journey is going great, Patrick. The amount of participation in the industry and the leaders in there, everybody’s in. You remember Jimmy Pike, and this is-

Patrick Moorhead:
Oh, yeah.

Robert Hormuth:
One of Jimmy-isms, “Never bet against ethernet.” We are not betting against ethernet at all for ScaleOut.

Patrick Moorhead:
And I’d say, I don’t bet against ethernet either. There are some technologies and high performance computing that make sense to connect, but it’s amazing, every time you count ethernet out, ethernet finds a way of lowering latency, increasing performance, reducing power in many different ways because it’s almost an, I don’t want to call it an open source technology, but the community comes together to make that happen. And whether it’s the hyperscalers or the big enterprise data centers, ethernet that makes a lot of sense.

Robert Hormuth:
And then on the other part of the open, so ultra ethernet great for the ScaleOut networking. If we wind the clock back to December of last year when we announced the 300X, we stood up on stage and said we would open up part of Infinity Fabric for collaborators and strategic innovators. There’ll be more of that to come soon, but I’m happy to say, we are fulfilling that promise to go address, to go do that, to help on that scale up to create bigger pods of GPUs within a pod. And so, we’re going to go address it with ecosystem partners and go advance that in the industry.

Patrick Moorhead:
It’s great stuff. And by the way, some people might say, “Well, hey, Robert, what gives you the permission to do this?” And it’s like… By the way, congratulations. You now have eclipsed, with EPIC, the market share that, when I was at AMD with Opteron, I think it was 32%. So, congratulations on that. And quite frankly, your MI product line is on an absolute rocket ship. And it was really interesting, at a conference that I was covering, Satya said that the MI300X was the best price performance GBU instance that they offer. And that’s, first of all, quite a testimonial. It’s not theory, it’s an action. But also, when I look at the starting point of how long that product line has been out, that’s like a rocket ship, man. It’s been fun to watch.

Robert Hormuth:
And I think a lot of that goes to the testament of what Lisa and Mark and Forrest put in place in terms of the architectural choices, the advanced packaging, the being on the forefront of chiplets gives up a lot more late binding decisions and optionality as we go. And getting back to, well, what gives us the right? Look at Frontier. Frontier’s been the number one supercomputer for what, three years now?

Patrick Moorhead:
Yeah.

Robert Hormuth:
And there’s another one on the top five list now. But, if you’ve looked at, it’s first per watt. It’s not very attractive. And there was, in this year’s at the ISC, there was a small run of the 300A, which will be the next number one. That’s basically combining the EPIC cores and the Instinct cores all on package and a true APU and a true unified shared memory. And I am really looking forward to seeing how El Capitan, so we have a fair amount of experience building really big machines, solving really complex problems for some very demanding customers both for the US government. So, I think we’ve earned a right to play in this field given what we’ve been investing in and tackling some of the largest problems in the world with… Frontier, to me, every time I see the stats on Frontier, I’m just amazed at the power, the density, the cabling, the miles of this and that. It’s just like, “Holy cow, we built that.” Or we helped build that with HPE.

Patrick Moorhead:
Well, Robert, you and your team, a lot of people just cranking out at AMD, it looks like a fun thing to be doing and changing the industry and Robert, I really want to thank you for coming on the show and helping us kick off this cloud infrastructure section here. I really appreciate that.

Robert Hormuth:
Absolutely. It was a pleasure, Patrick. Always a pleasure to talk.

Patrick Moorhead:
Sounds good. This is Pat Moorhead and Robert from AMD signing off here, AI infrastructure, man, it is cool. Everybody’s talking about it. It is a core piece of building out all of these amazing user experiences that wind up powering and improving enterprises and, as Robert said, improving people’s life on the consumer side. Take care and check out all of the cloud infrastructure highlights from this track. Take care.