Making Edge AI Devices Smaller, Faster, Cooler, Dustproof and Waterproof

Patrick Moorhead:
It’s day three here at the Six Five Summit 2024. We are talking Modern Work and if you’ve tuned into any of our sessions, day one, two, or three here, you’ll know it’s all about AI. It’s not only enabling the build out of AI, like real customer benefit from AI, but also building out the ecosystem and the infrastructure for that. And as we know, with AI comes increased processing power, more memory, more storage, and even changes on the networking side, but the one thing we haven’t talked about is how do I keep all of this infrastructure cool, but we’re going to talk about that because I have here Seshu from Frore Systems. We have had him on the Six Five before, but welcome back.

Seshu Madhavapeddy:
Thank you Patrick. It’s great to be back.

Patrick Moorhead:
Can you talk about the company and the types of problems you’re trying to solve?

Seshu Madhavapeddy:
Sure. We are a five-year-old Silicon Valley startup and we set out to create a solid state active cooling chip. Now this is unheard of because when it comes to thermal, the only options that people have had so far is to use mechanical fans.

Patrick Moorhead:
That’s right.

Seshu Madhavapeddy:
But that’s centuries old technology. With the rapid improvement in computing, there was a huge need for bringing new technology and new innovation into thermal. So we started this company to build the world’s first solid state active cooling chip and that’s exactly what we did. It took us four years to build a chip and we were able to proudly introduce it to the industry at the CS that I was talking about last year. Yeah. So the chip itself is, as you rightly described it last time we talked, it’s a chip that cools other chips.

It’s kind of like when you think about it, it’s obvious, that everything should move to semiconductors when it comes to building these state-of-the-art computing platforms, but thermal had not and we are the company that is moving thermal into the domain of semiconductors because what that means is that you can make them much thinner, you can make them silent, and also you can actually innovate much faster so that every year you can continue to improve the performance of the active cooling chip much faster than you would see in the domain of the current technologies like fans.

Patrick Moorhead:
Yeah, it is cool. Chips cooling other chips, that’s very meta and high-tech. Seshu, the whole theme of the Six Five Summit is about AI and getting more out of AI and increasing this just torrid pace of build out here. So what are your views on on-device AI? I’m assuming it creates an opportunity for you that may not have been nearly as big before.

Seshu Madhavapeddy:
It was a big opportunity for us before, but now the need for a solid state active cooling chip is just increasing exponentially because think about it, if you look at any computing platform, particularly at the edge, you have very sophisticated systems on Chip, SOCs, that have CPUs and GPUs and all sorts of other dedicated hardware cores in them and they are individually very, very powerful and they’re running at full hilt in order to deliver the user experience that these devices are able to offer. But now bring in AI and you introduced a new subsystem called the NPU, or the Neural Processing Unit, and that is now going to demand a lot of power as well. So you’ve got the CPUs and the GPUs that they have their day job. Now you’ve got to punch that with an NPU, which also is going to consume a lot of power.

So the amount of power budget and the thermal budget and the thermal envelope your device needs is just double. So how are you going to do that? Many of these devices are very compact and waterproof and dustproof so you couldn’t possibly think about putting a fan in them. So the only choice that you have is really to be able to use a technology that is cutting edge like ours, the Frore Systems AirJet solid state active cooling chip, which is only 2.5 millimeters in thickness and can slide in and improve the thermal envelope of even the tiniest and the slimmest devices that you have in the market today.

Patrick Moorhead:
Yeah. I think the first time we talked too about your technology, I think I said it’s a no-brainer for not only what it was when you started four years ago, but even more as we find ourselves in this conundrum of we can’t get enough performance, we want to keep form factors of any device thinner, and we want it to be a lot quieter. So let’s turn up the contrast ratio on fans versus AirJet here. Can you talk just very distinctly and comparatively on the uniqueness about AirJet compared to fans?

Seshu Madhavapeddy:
Absolutely. Absolutely. The first thing is that we’re going into devices that are too thin for fans and also devices that need to be dustproof and water resistance. We did a great showcase where basically we took the Apple MacBook Air, which is the highest selling laptop out there in the market today. And the MacBook Air is super thin. It is the thinnest laptop that’s available in the market today and it’s so thin that it’s fanless because you really couldn’t put a fan in there. But it comes at the penalty that because you don’t have a fan, it has a much smaller thermal envelope. And so the M3 processor that runs in the MacBook Air is basically only getting half the performance that it is really capable of. And you can see the difference because if you bought a MacBook Pro with the same M3 processor, you’d get twice the performance as you would in a MacBook Air, which has the same processor.

The difference is really the thermal solution. One doesn’t have a fan, the other has a fan. One doesn’t have a fan because it’s too thin. So what we did was we took the same MacBook Air and without changing the ID one IOTA, we were able to put in our AirJet mini chips in it and we were able to show that you can double the performance and reach the same performance as a MacBook Pro. So that is I think a clear cut example that illustrates how the AirJet Mini chip can significantly improve the performance of ultra-thin devices that are too thin for a fan, but because AirJet is only 2.5 millimeters in thickness, you can slide them in and get all the performance that you deserve.

Patrick Moorhead:
Yeah, it’s really cool. I mean, customers can even kind of have their choice in how they want a design. You can have same form factor higher performance, you can have a form factor with a noisy fan and you can turn that into basically no noise. Yeah. I think those are the two major options. So it sounds super flexible. So we talked about the MacBook Air I’ll call it test example. Great way to show the capability. Which devices feature AirJet or which devices do you feel like your platform offers a superior experience?

Seshu Madhavapeddy:
Great question. We are currently designed into multiple devices. So I’ll walk them through one at a time. But before I do that, let me explain to you what the company’s journey has been over the last 15 months ever since we came out of stealth. We came out of stealth, we introduced our product to the industry, and the feedback has been overwhelmingly positive. Everybody looked at the potential of our technology and wanted to have it. So we got the right feedback saying that we were on the right track. So we spent the next several months ensuring that we complete the design of the product, bring it to commercial quality, which meant going through rigorous reliability testing of our product, and then ramping up on volume manufacturing. That was our journey all through 2023. So we ended 2023 with the first commercial products with AirJet hitting the market and they’re now a quick and good cadence of products that are beginning to come out.

The first product that came out was actually a compact mini PC. It’s compact and fanless mini PC, therefore it had very low performance. By putting two AirJet minis into it, we were able to triple the performance in the same form factor. So that was a great showcase example. The next ones are also additional mini PCs that are using our technology and we also have a bunch of SSD storage products that are integrating our technology into them because with Thunderbolt and NVME, you can actually triple, quadruple, the data, read write throughputs, but you can’t do it because of thermal issues and we are able to solve that problem in SSD.

So these are some of the early adapters of our technology and that has given us field experience to see exactly how our product would function live in people’s hands. So now building on that track record, we’ve started engaging with much larger customers who are building premium tablets and smartphones and IoT devices, edge AI devices, devices that use NVIDIA Jetson send Processor for example, that are also designed in our product. So we are in the process of maturing their platforms with our technology integrated into it and you’ll see a continuous stream of new product announcements coming out all through this year and into next year.

Patrick Moorhead:
No, that’s great. If I look at the four subsystems that typically, well yeah, four subsystems that seem to get challenged every inflection point we have, whether it was the dawn of the internet with the mobile phone, it’s the compute, memory, storage, and networking and it seems like in all four of those we’re trying to crank up the performance, reduce heat, and keep the experience at least the same from an auditory point of view. So without divulging what the future, your future roadmap, your future design wins, but hey, you can if you want, what’s next for AirJet? You talked a little bit about some of the form factors, but I want give you the chance to give even more detail.

Seshu Madhavapeddy:
Certainly. We have Computex coming up in a month’s time. So we’ve got some huge announcements there that we are going to come up with, customer product launches certainly, but also new products that we’re introducing. We are building on the strength of our AirJet mini chip to introduce the AirJet pack at Computex, which is essentially a completely self-contained cartridge that has up to five AirJet minis and can remove anywhere from five to 25 watts of heat and it’s super thin, but it’s cartridge. So we want to reduce the complexity of adding AirJets into edge AI device or an on-device or an AI edge device to the simplicity of what it takes to plug in or replace your battery. So we want to come up with a cartridge concept where it becomes very easy for device manufacturers to integrate our solution into their platforms. So we’re going to announce the AirJet pack at the Computex in June in Taiwan.

Patrick Moorhead:
That’s exciting. Exciting stuff on the move here.

Seshu Madhavapeddy:
You mentioned that computing runs on CPU, GPU, memory, networking, but I would add a fifth one to your list, the short list, that without-

Patrick Moorhead:
Yeah I was waiting for that.

Seshu Madhavapeddy:
You are not going to get anywhere either. So that’s just as important.

Patrick Moorhead:
No. I appreciate that and I’m going to think through that because what is happening is we keep busting out of these four different points and a conversation that goes along with that is how are we going to power this, but how are we going to cool this well? And in the data center it’s about the cooling and the amount of energy going in and the amount of water that people have to bring in now and the noise isn’t really discussed, but when it comes to the edge and whether that’s the edge PC, a tablet, a smartphone, or even an IoT device, you just can’t be driving that level of noise because on the consumer side as we know, it’s annoying and it drowns out some of the enjoyments you’re trying to have with all those fun things you’re trying to do, but then on the commercial side, it’s either a distraction to the worker or it’s a distraction to the customer who has to be around some noisy devices.

And in our research, Seshu, that we’re doing about around the industrial IoT, whether it’s retail or in an airline, you just can’t be having a noisy solution out there. So yeah. I will take this under advisement, Seshu, on a fifth element here.

Seshu Madhavapeddy:
Thank you. And it’s not just noise, Patrick, it’s not just noise. If you are forced to use a fan, then inevitably your device is going to be bigger, it’s going to be thicker, it’s going to be more voluminous, and nobody wants that. The second thing is noise. The third is you have to surrender a couple of other attributes, dust. You are no longer dustproof because if you use a fan, you need to have huge vents to let air in and out and you also have to surrender water resistance as an attribute that you have for your device. So it’s all four. And also there’s a limit to how much ultimately performance fan can deliver, no matter how big it gets, which is the reason in data center they’re moving to water and liquid. So there is a ceiling to how much performance you can deliver and you have four negatives if you used a fan.

And we overcome all of those. We overcome all of those in terms of using semiconductor technology to be the first in the industry to deliver an active solid state chip for cooling. And our first generation is already on par with fans and our subsequent generations are going to get better and better and better. So we believe that we are at the beginning of a new industry that we are creating that is going to continue to pay dividends for the industry and make sure that, just as compute gets more and more powerful, so does the thermal solution and you no longer constrained in terms of delivering a high performance device that is thin, silent, dustproof, and waterproof. That’s our vision and that’s what we’re executing to make happen.

Patrick Moorhead:
I like the vision. I haven’t been shy about sharing that, but it’s super exciting and Seshu, thanks again for coming on the Six Five. Again, I think this is your third time and thank you for doing the Summit 2024 here and educating us all on the value of alternative cooling solutions in this age of AI.

Seshu Madhavapeddy:
Thank you very much, Patrick. It’s my pleasure.

Patrick Moorhead:
So here to here, Six Five Summit day three we are talking Modern Work and that can come in the many different forms. It can be smartphones, PCs, tablets, industrial IoT devices, all bursting with AI goodness. You have to cool that in an appropriate way, which we talked about for the last 20 minutes. Check out all the rest of the day three speakers here in Modern Work and go back to day one and two. You can go back and review all of those, whether you’re interested in data center, enterprise SaaS, all those goodies under the sun. Thanks and take care.