SharonAI: Revolutionizing Conversations with Advanced AI

How is Sharon AI adapting its GPU-as-a-service offerings for organizations of all sizes? On this episode of the Six Five On The Road at SC24, hosts Keith Townsend and David Nicholson are joined by Lenovo's Sinisa Nikolic and Sharon AI'sAndrew Leece for a conversation on how SharonAI, a leading GPU-as-a-service provider along with Lenovo's innovative technology, is driving the AI research boom. Catch the full into this episode and: Discover how Sharon AI differentiates itself with its cutting-edge, informative, and engaging conversation capabilities Learn how Sharon AI is optimizing its infrastructure for high-performance computing (HPC), serving higher education, research, and now expanding into enterprise and government sectors Learn about the critical role of Lenovo's TruScale in enabling Sharon AI's rapid and capital-efficient scaling Explore the exciting advancements in water-cooling technology and its impact on data center sustainability and performance Get details on how Lenovo TruScale has powered SharonAI's offerings and what future developments to anticipate from the company Learn more at Lenovo and Sharon AI. Watch the video below at Six Five Media at SC24 and be sure to subscribe to our YouTube channel, so you never miss an episode.

Keith Townsend:
All right, Dave. I don’t know if it’s only me that finds it ironic that we’re in Atlanta talking about cooling.

David Nicholson:
It’s raining today though.

Keith Townsend:
It is raining, so there’s a bit of liquid cooling going on here in Atlanta. SuperCompute 2024. We’re in the Lenovo booth talking AI at SuperCompute 2024. I have with me representatives from both Lenovo and Sharon AI. We practiced the names before. I’m going to do the easy one first. Andrew.

Andrew Leece:
I thought you were going to lead with him, for sure.

Keith Townsend:
You thought I was going to lead with Sinisa.

Sinisa Nikolic:
That’s pretty good. That’s pretty good. We’ll do that.

Keith Townsend:
From Lenovo. And from Sharon AI, Andrew. Andrew, tell us, what is Sharon AI?

Andrew Leece:
Sharon AI is a leading GPU as a service provider in Australia, and we’ve got expansion into the U.S. in the very near future. So we’ve got full vertically integrated infrastructure, and we’ve just signed a joint venture now to build a 90 MW data center here in Texas.

Keith Townsend:
So talk to me about the differentiator for Sharon AI, because we’re at SuperCompute. Every booth we go to, they’re talking AI. What’s the differentiator for Sharon AI?

Andrew Leece:
There’s plenty of GPU providers around. We’ve built our infrastructure and optimized that for high performance computing. All of our customers from day one have been from this segment. They’ve been higher education and researchers. We are moving into enterprise and gov, but our background as a storage provider originally has all been to service HPC segment.

David Nicholson:
So Keith and I think in terms of the stack, often both physical and logical. First of all, GPU is a service. Do you have specific GPU partners that you only focus on or so what flavor GPUs, and then how high up in that abstraction stack do you go when delivering that as a service to your customers?

Andrew Leece:
Yeah, so to take the first point is around the GPU infrastructure. It’s all leading edge NVIDIA and AMD GPUs. So the predominant fleet that we have is NVIDIA H100s, L40Ss and soon to be H200s. And in fact, in partnership with Lenovo, we’re developing a full 1K NVIDIA cluster that’ll be water-cooled, direct-to-chip water-cooled. But as far as stack goes and abstraction away from the metal, we’ve got solutions architects that are building and have built a library of tools specific to research and higher ed that will allow people to one-click run all sorts of different applications on our equipment. So as far as differentiation goes, we are moving further up the stack, but we stop at LLMs. We don’t produce LLMs. We take open source LLMs, all private LLMs, and we can work with the end customer to optimize that for our equipment. But for the most part, we’re taking pre-optimized LLMs through NVIDIA NIMs and alike to run specifically on our hardware.

Keith Townsend:
So Sinisa, talk to me about that special relationship. You’re supporting many CSPs. How is Lenovo enabling this network of providers?

Sinisa Nikolic:
Oh, we’ve been years now together as partners. So it started in storage, storage as a service, way back when, maybe three or four years ago plus, enabled by TruScale scale as a offering technology, that it allows operating, an operating expense world. So in the storage world, architecture, all of these things to enable DSS, which is what they were at the time, to go to market to offer these technologies to clients in terms of their expansion. With this whole jump into capacity and GPU as a service, then the acceleration began. And one of the unique, I guess the unique value proposition of the way Lenovo works with its clients was whilst this is a CSP offering as a service, we brought technology skills around HPC. We were at a HPC event today, Supercomputing 2024. So we brought those skills to bear as we’re architecting, designing, consulting on what’s coming, what’s today and what’s coming in the future.

So Andrew touched on it just then. This growth in AI is driving sustainability related issues in data centers amongst other things, so now we’re going to be progressing into water cooling, which is what you started with at the beginning. So there’s all of those types of things that we do. And to work with Sharon AI, the way they verticalize their business is a true value differentiator in the market. And for us, it’s quite exciting to work in that space versus what I would say is like a bare metal, a bare metal as a service infrastructure. Working with them, this verticalization and going to clients that way is completely and utterly unique in the market.

David Nicholson:
So if I’m an individual enterprise data center customer, or a cloud service provider, I can engage Lenovo and I can say, “I want to buy a server. I want to buy some storage.” Right?

Sinisa Nikolic:
Yeah.

David Nicholson:
Or I can get to that sort of rack scale level where I’m buying things designed that way. What does that relationship look like between Sharon and Lenovo? Are you buying things that are designed at the rack scale? Are you buying at the component level? What does that look like, Sinisa?

Sinisa Nikolic:
It sort of starts off at the base level, right? Which is, what do we want to do? What do you want to do? Where is the capacity going? What is the architectural design point that you want? So are we H200s? Are we moving into Blackwell? Are we talking MI300? What is the client aspect of it? Where are we going to market? How are you going to go to market with that? We then work very closely with Andrew’s architecture team and our architecture team to actually design that. And when you’re going into these heavy cloud-based environments, network becomes very important, how many leafs you have, all sorts of things, the way the system hangs together. Because always in this business, you have to have sort of an end point or a horizon position so that you’re not just doing a lump here. Oh, now I have to re-architect and go on. You’re always designing for the fact that you can integrate as quickly as possible, because the market dynamics are such that they’re moving very quickly, and Andrew and the team need capacity on the floor sometimes incredibly quickly. So you don’t want to go through a re-architecture. So we’re always continually looking at that advancement, but hand in hand with Andrew and his architecture team.

Andrew Leece:
Yeah, I would just add to that is that obviously we started out as buying the individual component level, but now we’ve very quickly progressed to building a whole 1K reference architecture clusters. And we can still deliver that in stages. But what is engineered from day one with NVIDIA and Lenovo is the network infrastructure. Because the network infrastructure is critical to deliver the optimum performance of the equipment. There are a lot of people out there buying just individual boxes, and we started that way too. But then that doesn’t deliver the full potential of that environment. So the network layer is critical. And so right now, we’ve got two 1K cluster designs that we’re implementing or in final design stages. And whilst we might start with 256 GPUs in that cluster, we develop the whole spine and leaf architecture for the network on day one. And that allows us to deliver, even with only 256 nodes, the optimum performance out of that equipment.

Keith Townsend:
So we’ve talked about the transformation that happened at Sharon AI, going from a storage provider into kind of this AI service provider model. Those are two very different types of workloads. You need to scale for storage and serving up storage latency that may not matter as much. Now we’re talking about moving our AI to the data, moving our data to the AI. You already have the data. Talk to me about how TruScale has enabled this transition.

Andrew Leece:
Yeah, So TruScale has enabled us to scale in a very capital efficient manner, and do so in a very responsive manner as well. So obviously we want to have infrastructure on the ground on a just in time basis. We don’t want to let equipment hanging around, especially in the compute space at significant expense per GPU. So as a TruScale customer, maybe I’m not allowed to say, but I believe there is a preferential delivery schedule because there is a commitment to have X amount of-

David Nicholson:
He just said it.

Keith Townsend:
Yeah, he did just say it. I thought I was the only one.

David Nicholson:
Continue, continue.

Keith Townsend:
Yes, please

David Nicholson:
Continue.

Keith Townsend:
Tell us the secrets.

Andrew Leece:
Yeah, well-

Sinisa Nikolic:
It’s because we love Sharon AI.

Andrew Leece:
There’s a contractual requirement on Lenovo’s part to have X amount of capacity available to us at all times. And so we might push that boundary sometimes and say, “Hey, that X amount of capacity was actually X-plus.” And there’s always been a great commitment on behalf of Lenovo to deliver that. So capital efficiency and availability, and then even services to install. If we’re deploying in a remote location or remote to our headquarters, then there are services available for us to actually do the deployments. But I did want to just come back to your point around the transition from data storage to compute.

Yes, architecturally, they are different. The workloads that we’re experiencing are the same, we’re just participating in both parts of that workload. And bringing compute to the data has been a really complimentary, well now skill set, but proposition to our customers. Because as we know, the data management is one of the most difficult and costly components of these operations. So bringing data to compute is often very expensive. In many cases, there are sovereignty issues associated with that, so you need to bring compute to the data. And having the data management skills and capabilities within our team has been enormously valuable to our compute customers.

Keith Townsend:
So talk to us about their future. If you had a crystal ball, it’d probably be wrong, because this is AI. But talk to us about how you’re thinking about the future of Sharon AI.

Andrew Leece:
Yeah. Our 90 MW data center joint venture that we’ve just done probably is going to be too small in three weeks, but we’re circa 2.5 MW come following our next deployment in terms of compute and storage capacity. So we have a significant expansion capacity here in the U.S. as well as in Australia. And we have all of our operations in tier three or tier four data centers. They’re in Equinix and NextDC data centers. So we offer a genuine enterprise service level, and we expect to continue to do that even with our own vertically integrated data centers. But the future is unknown. The product lifecycle of the GPUs use is probably speeding up. There are advancements in CPU as well. We’re not limited to GPU. We have large CPU arrays as well for our customers.

But we’re just excited about those advancements. We stay at the forefront of all of those. We solve a lot of problems for our customers. A lot of the university customers that we work with, they might have budget to go and buy a handful of these, but two or three nodes doesn’t give you the opportunity that several hundred nodes gives you in terms of throughput and raw horsepower. But one of the big problems, as we know, is density and power availability. And a lot of the universities are capping out their on-prem capacity. Even their data center partners are struggling to respond quickly enough to density requirements.

David Nicholson:
Well, Sinisa, you’re working with companies all over the world.

Sinisa Nikolic:
Yeah.

David Nicholson:
And do you see a difference between the constraints on power in Australia versus North America, or is it the same everywhere?

Sinisa Nikolic:
It’s literally the same. There’s just different levels of it. When you see these next generation systems, they run hot. They spin fast, they run hot. Everyone’s worried about climate change, cost of power. You have to cool these things. The hotter things run, the more unreliable, I guess. You want to get them cooler things. As you cool things, reliability increases. You get rid of CPU jitter, all these other things.

Andrew Leece:
The performance curve in the direct water-cooled environment goes up, flat lines, and it comes down again. In the air-cooled, you get the performance jitter, right?

Sinisa Nikolic:
You get this performance jitter, the CPU jitter that jumps around. So the future that we’ve been talking, rather you spoke about, is around water-cooling, direct water-cooling, and we have our technology’s actually on the stand today, and you can see it on the web.

David Nicholson:
Neptune. Boom, boom.

Andrew Leece:
Neptune.

Sinisa Nikolic:
But when you start looking at that, that’s where things really change. It’s where things escalate in terms of what you can put on the floor, the density that we were talking about, to deliver that type of capacity to Sharon AI’s clients. That’s where the world changes. And we’re seeing that everywhere through Europe, Middle East, North America, Asia Pacific. That’s the change. You can’t do it anymore.

Keith Townsend:
So Sinisa, Andrew, thank you so much for stopping by Six Five On The Road. I’m Keith Townsend. For my co-host, Dave Nicholson, we want to encourage you to continue to explore this fascinating world of SuperCompute. We continue our coverage throughout the week here in Atlanta.