AI-Powered Innovation in the Cloud: Strategies for Breakthrough Results

Daniel Newman:
Hey everyone. Welcome back to The Six Five Summit. It’s day one, we’re in the cloud infrastructure track. Really appreciate everybody tuning in and being part of this event. For this next session, I’m very excited to bring in Sachin Gupta. Sachin is a VP and GM of the Infrastructure and Solutions Group at Google Cloud. And this conversation is going to be all about innovation in the cloud, a big topic, something that every company has focused on, and we’re going to be looking at everything from public cloud to edge to sovereign cloud, cross-cloud, and so much more. Sachin, welcome to The Six Five Summit. Excited to have you here.

Sachin Gupta:
Thanks, I’m excited to be here.

Daniel Newman:
So I’m going to get started right away, Sachin. I think this is a really exciting time. I mean, look, AI is on the tip of everybody’s tongue, it has risen to the conscious of every enterprise, every business on the planet, and of course Google is at the center of this. You guys are advancing everything from silicon to the next wave of applications and everything in between.

I’d love to get a little bit of a perspective from you overall on what are the advancements you’re most focused on? What are you seeing customers get really excited about? And then of course, are there industries where you are seeing these use cases for the cloud rising quicker than others?

Sachin Gupta:
Yes, absolutely. So our approach is to understand what customers are trying to do with AI, are they trying to do foundation model training? Are they fine-tuning based on a mainstream model? Are they inferencing and serving for their enterprise application? And giving them an integrated system that’s optimized at every single layer.

So what we introduced at Google Cloud Next is our AI hyper-computer. This isn’t something that came together very quickly, this is something that we’ve been working on through decades of research. And so at the very bottom layer of that hyper-computer is our silicon. So this is providing choice with the latest generation of Nvidia GPUs, but also providing the latest TPU technology. In fact, we just recently at I/O announced Trillium. This is our sixth-generation of TPUs. So yeah, it’s not the first, second generation, they are the sixth generation of TPUs. In fact, the new models we announced, like Gemini, Flash, the new Imogen models, the new Gemma 2 model have all been trained and are served on our TPUs. So commitment to get the latest from Nvidia, but also provide options at that bottom layer with our own TPUs and other silicon choices that will become available.

And then on top of that, making sure that there’s open software choices. Most customers access this through Kubernetes with GKE, but if they have other open-source capabilities, like if they want to use Run, for example, if they want to use other open frameworks of any kind, and making sure that those are optimized on top of GPUs and TPUs, that’s super important for us.

And then finally, they want to consume it in different ways. I need to run a training job for six months, I need to serve and I need burst capability, I need to go up and down. So how they consume it, we provide great choices.

And then finally, we want to make sure that they have access not just to our Gemini models, the latest and greatest models that Google provides, but we will support over 100 models through our Model Garden with Vertex so that they can incorporate third party open-source Google models, build them into their development chain and deliver agents and enterprise applications on the other side. So optimizing this at every single layer and then providing that integrated system, I think, differentiates us, but provides tremendous value to customers.

Daniel Newman:
Yeah, Sachin, that was a really good and interesting explanation. When I was at Google Cloud Next, one of the things that really caught my attention was the amount of work that you were doing on your own silicon innovation, the TPU. When I found out that Gemini’s newest models were trained entirely on TPU, I shared that out. It actually created a ton of buzz across the social spheres when I was talking about that because I don’t think people fully realize and appreciate that there is an alternative. And I understand your whole perspective about homegrown versus obviously merchants. And of course you have a great relationship with Nvidia, Nvidia is doing really great things, but there is optionality there, and I think that’s going to continue to expand.

And then of course, software is really important too because you want model adoption and of course you want people using Gemini. It’s a very powerful model, it’s got a lot of mainstream attention. But there are all kinds of different model varieties that are going to be out to the market. We’re hearing more about small language, we’re hearing more about industry-specific. And then of course there are other open source large language that people are building on, and that’s going to be critical, that Vertex and that Google doesn’t limit, but really is democratizing access to all the tools and technology. So great to hear all that, definitely in line with the assessment that we’ve made over at Futurum Group, and of course on various conversations we’ve had on The Six Five.

Now I want to go in a little bit of a different direction, I want to talk about the slower moving highly regulated industries. So we have customers that are moving to the cloud, but they’re all moving at different paces. AI is accelerating, pushing them forward, but at the same time, in these highly regulated industries, there’s all kinds of sensitivity, everything from the privacy of data to sovereignty to where data resides in residency. And of course we heard from Thomas Kurian at Next that there’s accreditations within Google, they’re working across top secret and secret workloads for the US government. I’d love to hear a little more about that. What are you doing for government? What are you doing for regulated industries around the globe? And then of course, for this particular customers, for this particular group of customers, how are they thinking about AI?

Sachin Gupta:
Yeah, so this might be a bit of a longer answer, but the first thing I just want to make clear is in our public cloud regions, we offer a set of controls like where does your data reside? Who owns the encryption keys? The customers own those encryption keys. Who has access to that data? I need to keep it in this region in my country, who has access? You can control all of that. We also are very clear that when you feed your data into our models in our public cloud, we don’t train our model based on that data. Your data is your data, those enhancements are for you in your environment, we’re not learning from that. And so there’s a set of protections there.

But as you said, there are reasons like you’re a government customer, defense, intelligence, or you’re a highly regulated, let’s say, central bank or an energy company. And then what happens is there’s simply no way that the regulations will allow you to move that data into the public cloud. So for that, we built a product called Google Distributed Cloud, and think of that as the ability to take our services such as translation, translating from dozens or hundreds of languages, speech to text APIs, optical character recognition, vision, prediction capabilities, and moving those into a 100% air gapped cloud environment. So it’s your own cloud that delivers infrastructure services, database, data management services, AI services locally for you on-premise, okay?

And so the latest actually example that we hear here is about. “I want to be able to do search, it can be multimodal on multimodal content, multimodal data, I need to be able to do search, but it needs to stay within my premise and it needs to be on my own data.” And that’s what Google Distributed Cloud enables through a complete solution that we provide. We also use a product of ours, which is AlloyDB, as a vector database as part of that solution. We use our own open source model, Gemma, but we can use any third party open source model that we can run on this system. So a very powerful AI data, highly secure solution that runs 100% air gapped on-prem. So that’s the defense use case.

I just did want to add one other example. So on the defense side, by the way, we’re very proud, we have CSIT who provide services to Singaporean government that’s deploying Google Distributed Cloud. We’re also working with Proximus, who’s deploying this in Belgium and Luxembourg to provide a highly secure private cloud air gapped cloud services. So seeing great momentum in those segments.

And then you have a very different example, which is, you need your data on-prem, but it doesn’t need to be air gapped, meaning operations can be done centrally. So think of this when you have hundreds of sites or thousands of sites and you want to provide some kind of data AI capabilities in those locations, and maybe because of latency or survivability of the link, like the link may go down, you still need to have services locally, that’s where you want to make sure that you’ve got cloud infrastructure in each of those sites. So one example of this is McDonald’s, another example is Orange, where with McDonald’s they have I think almost 40,000 locations where they want to be able to get operational simplicity, get all of their data points, make their tool smarter, but really make their customer experience better, make the experience of their crews better in those stores. And so McDonald’s is working with us to deploy Google Distributed Cloud in all of those stores. And then they can build AI capabilities on top of that, such as automated order taking as one venue add that many retailers are absolutely looking at.

And then Orange has operations in 26 different countries. They need to be able to keep data, analyze that, apply ML to it, but it needs to stay in each of those countries. And we’re able to deploy Google Distributed Cloud in each of those countries to serve as a use case.

So we’re absolutely about enabling AI anywhere. So the best place to do it, you get massive scale, all the latest and greatest technology is our public cloud regions, and we have a set of controls there. But if for latency, survivability, cost, or for regulatory reasons, you need something that’s outside of our public regions, we have Google Distributed Cloud.

Daniel Newman:
Sachin, there’s no doubt that Google has continued to advance its services capabilities to be able to address both the various complexities of highly regulated industries and of course dealing with the distributed and hybrid multi clouds, and that’s going to be the norm going forward. Congratulations on receiving those clearances and being able to work with those various governments. I think those applications are going to be important. And of course, governments are going to want to benefit from what AI can do. It’s going to accelerate every industry, no different to the highly regulated and defense contractors and defense organizations around the world.

Now, having said that, you sort of lent into a bit, you were talking a bit about the kind of distributed options, and I want to talk about that because we all know there’s significant technical debt, we all know that every workload is in the cloud. One of the things we talk about regularly on the pod, Patrick and I, is kind of about workloads on-prem versus in the cloud. And I think over time the TAM is expanding, we’re seeing more workloads go to the cloud, but we’re also seeing the overall workloads grow. There’s still a lot of demand for certain amounts on-prem, and there’s also a certain amount of demand for multi-cloud. I mean, Google has accelerated really quickly in the era of AI, but companies have already had workloads in other clouds.

So I would love to kind of understand how you are approaching and engaging those customers and really helping them deal with this in time because maybe they’re seeing Google as a great opportunity to address the AI challenges and they want to move more there, but they may very well have workloads in other clouds, edges, on-prem. What is the kind of current state of your approach for multi and cross-cloud?

Sachin Gupta:
That’s an excellent question. And especially with AI, you’re talking about BigQuery being leveraged for data analytics and AI as well, what you find is how you move your data securely between other clouds or multiple clouds and on-premise, because in many cases it’s still sitting in on-premise data centers, is a complicated question for customers. And so we’re solving this by introducing something called Cross-Cloud Network. And what the Cross-Cloud Network does is at the very fundamental level, is makes it simpler, makes it cost-effective for you to connect all of these environments. So think of connecting your on-prem environments, your multi-cloud environments, your SaaS environments, all into the Google backbone. And the Google network is now your network as an enterprise customer.

So to make this real, there’s some capabilities we produce like cross-cloud interconnect where we can take away a lot of the pain of putting routers and firewalls in into CoLOS to provide secure simple connectivity between clouds. Many customers are already using this. And then we actually introduced something, and we’re working with Scotiabank on this actually, where we can take all of the services that are sitting on-prem or in another cloud provider or inside Google Cloud, for example, you mentioned BigQuery, that’s a service they’re using, they’re using ISV services in our cloud, they have wallet services and credit card services that they integrate with. And you can create a representation of that service in a, I’m getting technical here, but like a service landing PPC. And think of that as the ability to apply policy in terms of who has access, how do developers access these services securely, and simply to build the apps that they’re trying to build.

So it’s about simplifying data movement, it’s about offering a whole suite of security services because they have to secure that data. And it’s about representation of all of the different environments that customers have in one consistent way so that the network teams and the security teams can move much more quickly. And the developers are not encumbered by those complexities anymore. And so Scotiabank is just one of the customers seeing great value in this. And we’re thinking about really making the Google tremendous network the backbone that we have, our sub C table investments that we have, we run all of our great multi-billion user products on it, we want that to be the network that enterprises can use as their own.

Daniel Newman:
Sachin, I really appreciate you covering so much ground there from the architecture of your chips all the way to the networking of multi-cloud. You covered a lot of ground, you were great. It was really a lot of fun to have this conversation here at The Six Five Summit. Appreciate it, let’s have you back sometime soon.

Sachin Gupta:
Thank you so much.

Daniel Newman:
And for everyone out there, we appreciate you being part of The Six Five Summit. We’ve had a great day one so far, and we hope you stick with us. We have lots of content here. Be part of our community, stick with us for all three days if you’d like or you’ll be able to check the sessions out on demand. But for this one, for myself, I got to go say goodbye. We’ll see you all soon.