Empowering Businesses with Scalable Custom Supercomputing Solutions

In a world increasingly driven by AI access to high-powered computing is seeing unprecedented demand. Host Keith Townsend is joined by Dell Technologies' Director of Product Management, Saurabh Kapoor and Hot Aisle's CEO, Jon Stevens for this episode of Six Five On The Road at SC24. They share thoughts on empowering businesses with scalable custom supercomputing solutions. Tune in for more on 👇

• Democratizing access to supercomputing: Hot Aisle is making top-tier GPU compute available to everyone, not just the hyperscalers. Think hourly rentals and low-level hardware access for devs.
• Simplifying AI infrastructure: Dell's AI Factory is a game-changer, offering a flexible, open architecture with a choice of GPUs, NICs, and connectivity options. Plus, they're championing open standards like SONiC, bringing hyperscaler-level networking to the enterprise.
• Solving the AI talent shortage: Dell provides expert services and training to empower businesses with the knowledge they need to succeed in AI. Hot Aisle fosters a developer ecosystem that drives innovation and collaboration.

Learn more at Dell Technologies and Hot Aisle. Watch the video below at Six Five Media at SC24 and be sure to subscribe to our YouTube channel, so you never miss an episode.

Keith Townsend:
All right. If you’ve heard Dan and Pat talk big industry picture, the value of AI, the value of all of this celebrated compute, and you think to yourself, “How do you actually make that happen?” We’re having those conversations here at SuperCompute 2024. I have with me Saurabh and Jon. Jon from Hot Aisle, and you from Dell.

Saurabh Kapoor:
Dell Technologies. Thank you for having us here. Good to be here, and so much action going on there. Yeah.

Keith Townsend:
It is a lot of action going on in AI. And you guys have a huge truck on the show floor, the AI Factory. We’ll get into the AI Factory, the concept, you have a actual mobile version of it and we’ll get into both in a second. But Hot Aisle, I got to love the name first, Jon.

Jon Stevens:
Thank you so much. Yeah. Good morning.

Keith Townsend:
Talk to us about one, where did it come from and what do you do?

Jon Stevens:
Yeah, so kind of just came to me after probably a month of searching for names. You Google all over the internet, and everybody squatted all the good names, right? And so, one day I was just thinking about what is cool things in a data center, because I’m building a data center company? So what’s nice in there? I was like, “Oh, Hot Aisle. That’s so great.” And so yeah, it ended up it was available, registered it, and yeah, it came out to be a great name, so.

Keith Townsend:
And what do you folks do?

Jon Stevens:
So yeah, we are a cloud service provider. On one side of the business is a cloud service provider. We deploy GPU Compute, top-end, best in class. Everything is set up in a way to not only de-risk our business, but de-risk our customers as well. And then what we do is we take that as a blueprint and we deploy it for companies that want to deploy their own supercomputers.

Keith Townsend:
So through Dell and a cloud service provider, I thought you were a private infrastructure company. What’s going on with this relationship?

Saurabh Kapoor:
Well, the world of AI, we have a concept called Dell AI Factory, and the concept is to champion customers like Jon to simplify AI infrastructure. And what we are doing with AI Factory is we’re bringing the marriage of products from compute storage and networking packaged together under open architectures where you have choice of GPU and choice of NIC and choice of AI connectivity all packaged together and delivered with Dell services and capabilities to simplify AI rollout. So that’s the concept, and we’re seeing a lot of success with that. AI is evolving very quickly, a lot of new technologies coming in. It’s good to see all of these in action at Hot Aisle.

Jon Stevens:
Yeah, so we work with Dell Technologies to do these deployments, and they enable us to have access to any equipment that our customers are asking us for. So right now we’re focused on AMD GPUs, but we can deploy any type of GPU. And as long as it is best in class and latest and greatest, we can work with our partners, Dell Technologies, and get it done.

Keith Townsend:
So this moves fast for customers. I have to imagine it all moves extremely faster for a service provider. So tell me the importance of the Dell relationship in open standards.

Jon Stevens:
Right. So it allows us to have this very close partnership. Our business model is to always be deploying the latest and greatest. So by working with Dell, we can always get access to that because they have the relationships with AMD and Intel and NVIDIA and all these other big companies that enable us to quickly deploy compute. And as long as we’re deploying the latest and greatest, developers who want to use our systems can then get access to the latest and greatest. So right now, a lot of companies really focus on HPC compute, the big supercomputers, El Capitan, Frontier. Those systems are amazing systems.

Keith Townsend:
Yeah, they are.

Jon Stevens:
I can’t wait to deploy El Capitan myself. But the problem with these systems is that they’re really locked up. There’s only a very small amount of developers who can get access to them today. So we’re working on democratizing compute, enabling anyone to get access to supercomputer level of compute but just as an hourly rental. And when I started this business, nobody was doing this really. I mean, we had the hyperscaler level of compute, but not the stuff that we’re doing which gives developers even low-level access to the hardware. We allow developers access to the BIOS, and they can really configure the machine and tune it to their needs and their applications.

Keith Townsend:
So what type of developer or what’s the typical use case for a Hot Aisle service?

Jon Stevens:
Yeah. So we basically, because we’re only buying the latest and greatest and we deploy the top top-end, everybody, they don’t have an excuse not to use us, right? Because they can’t get any better than that. So it’s really any use case, it’s HPC, AI. We want to do computational fluid dynamics. AMD GPUs are extremely good at floating-point 64, so they can do any wide range of computation. And then AI of course, because that’s the big hot thing today. So these GPUs can really be used for anything. And I think that by giving access to developers in a very democratized way and a wider range, people will start to invent new things. I mean, just a few years ago, nobody was really talking about AI the way we’re talking about it today. Who knows what in another couple of years people are going to be building?

Keith Townsend:
So Saurabh, the AI Factory, the Dell AI Factory, it has to be pretty extensible to support customers like Hot Aisle, and I’ve talked to many enterprise customers that have adopted it. How are you able to have such flexibility in a design that is supporting HPC use cases and traditional AI use cases in enterprise?

Saurabh Kapoor:
True. True. Yeah. I mean, we are talking to all spectrum of customers, from GPU farms and AI factories, which are super large clusters. And these are thousands of GPUs that are put together to build these supercomputers to all the way enterprises who are now getting onto the AI bandwagon and looking at more fine-tuning and fencing-based use cases. So how we are approaching this is to create a choice of GPUs with AMD, NVIDIA, and Intel. Then we have a choice of NIC and connectivity, and we are also championing a lot of ethernet technologies, right? Enterprises have been using ethernet forever. And ethernet, yes, it is now getting ready for AI use cases. There’s a lot of optimization that has come from an AI perspective. We see all these high switching now from 400 to 800 gig. We’re working with our partners, NVIDIA and Broadcom, to provide these ethernet-based technologies, and we’re championing that with open standards.

So SONiC is software for open networking the cloud, it’s an open-source network operating system, and we are extending this technology which is running one of some of the largest hyperscaler ecosystems to now enterprises. So adding tons of feature set, putting support model around it, and then also bringing a lot of supply chain and services efficiency into the system to cater to the customers, because delivering the technology is as important as the technology itself, right? So bringing all of that together under the Dell umbrella, under Dell AI Factory, and putting validated solutions and architectures around it so that it’s wash, rinse, and repeat and plug and play for enterprises to take these best practices and move faster than AI.

Keith Townsend:
So Jon, Saurabh just mentioned a couple of things that kind of got my network engineer-

Jon Stevens:
Yeah. Yeah. I heard a few things too. And we’re excited.

Keith Townsend:
Yeah. So one of which is SONiC.

Jon Stevens:
Yeah, SONiC.

Keith Townsend:
Are you folks using SONiC in your day-to-day?

Jon Stevens:
Absolutely. Yeah. So we love SONiC. My background is open source. I started the or co-founded the Java at Apache project, so I was one of the first people to bring Java to the server. And so having that open source background and SONiC being open source, it’s just a fantastic product and we can work with it to automate our deployment. So our customers, when we onboard a customer, we just run a script that segments the network, sets up the machine, and the customer can be onboarded in 30 minutes.

Keith Townsend:
So how important is an idea like SONiC, this open source network operating system, to an idea such as the AI Factory in this open approach to AI?

Jon Stevens:
Yeah. So we actually, we use the enterprise version of SONiC from Dell on our Z9864 Switch, which is-

Keith Townsend:
Oh, I love model numbers.

Jon Stevens:
I know. It’s like-

Keith Townsend:
That’s when you know you have a CTO advisor on the set. We get model numbers, but please.

Jon Stevens:
I’m a CTO, but I’m also very technical, so yeah. So we run the enterprise version of SONiC on there and we love it. I mean, it’s just a fantastic product. It works, and that doesn’t give us any trouble, right? So as long as it’s moving us forward in a clear path, it’s a pleasure to work with, and has all the features that we need.

Saurabh Kapoor:
And Jon mentioned, right? I mean, one of the big advantages with SONiC is it’s very API-centric. Anything and everything in the box can be exposed out to any application that the user is using in-house tools, open source technologies, enterprise applications. It provides you streaming telemetry. So in the world of AI where you’re moving a lot of data across or you’re reducing number of flows, but those are elephant flows now, so automation and telemetry are extremely important, right? You’re running these products or solutions at highest capacity, training workloads for weeks and fine-tuning and inferencing. So having deep understanding or deep insights into the solutions really helps you to be better prepared and deliver better.

Keith Townsend:
So one of the other big themes happening at the show has been talent. There is a massive shortage of talent from both enterprise service providers and even big companies training LLMs. How is Dell’s AI Factory and Hot Aisle helping to solve or alleviate some of this challenge? We’ll start first with Dell.

Saurabh Kapoor:
Yeah. No, so as part of the Dell AI Factory offering, we also have a big umbrella of services, right? And that service is while we provide 24 by 7 support on products and solutions, we also provide consulting services, solution architectures, residency services and things like that. And our objective here is to provide subject matter expertise, right? AI is evolving very quickly, and we are working with our partners and customers very closely to be aware of their requirements and also with partners to make sure that we have an understanding of what’s coming next. And we provide those capabilities and get the systems ready with services capabilities, and also with trainings and education services offerings to make sure that we spread the knowledge and get the community along with us.

Jon Stevens:
And that’s actually one of the benefits of Hot Aisle, is that we have these partnerships with companies like Dell Technologies, AMD, Broadcom, where we can rely on them for help. Right? These are the experts, so we’re using their equipment and we can get feedback from the developers and create a developer flywheel where the developer is giving us feedback, we give it back to AMD or Dell or Broadcom, and then Broadcom makes changes which improve their products, which then encourage developers to write more code for these products, and then we just keep this flywheel going where we’re buying more and more and more because we’re getting more developers on our systems. So it’s a really good circular ecosystem that we’re building here at Hot Aisle.

Keith Townsend:
So Saurabh, Dell has this fancy two-trailer AI Factory set up and on the show floor. I think this is city number 38 of a 44 city tour. I talked to that team. They’re energized, but I got to say, that’s a pretty aggressive schedule, two cities a week. What should customers expect to see when they visit the Dell AI Factory, whether we’re talking about the truck and the EBC or one of your partners, what will they see?

Saurabh Kapoor:
So AI is influencing every spectrum of the customer, every domain. So we have brought in AI from our PCs to infrastructures and data center technologies, and we demonstrate all of that on the truck, right? The goal here is to simplify AI rollout, make workforce productivity, efficiency, bring new revenue streams and things like that. So you see all of those in different products and solutions from the PCs to compute storage, networking, and much beyond then that, right? And what we are also kind of working on is what’s next for AI, right? We’re looking at these big machines, and these are power-hungry devices, so how to make sure that liquid cooling and the next set of technologies are in place? So this is an opportunity for us to demonstrate what’s on the truck, but also engage with the users to better understand what would like to see more and engage in conversation on the future of these solutions that we’re working on.

Keith Townsend:
I’d like to thank both of you for stopping by Six Five On The Road. You’ve heard it here, we have semi trucks, we have Hot Aisle using AMD, NVIDIA, all the latest and greatest technologies. We’re listening to how Dell is enabling their ecosystem of partners, customers with their AI Factories. We’re going to continue the conversation. Me and my co-host, Dave Nicholson, will be here all day interviewing Dell, Dell’s partners, Dell’s customers to help you better understand how you take these executive-level initiatives, bring them down to the technology level and securely implement these solutions. Stay tuned for more coverage from SuperCompute 2024 in Atlanta, Georgia.