Innovation in DataOps and AI: BMC’s CTO on Orchestration and Data Efficiency
What is the current state of DataOps in the enterprise? Host Mike Vizard and BMC‘s Ram Chakravarti, Senior Vice President & Chief Technology Officer, share thoughts with Six Five Media at BMC Connect on the evolving landscape of DataOps, AI, and their critical roles in enterprise orchestration and data efficiency.
Their discussion covers:
- The current state and the future of DataOps in enterprises
- How AI technologies are being integrated into orchestration tools
- Strategies for enhancing data efficiency and security in cloud environments
- The role of automation in managing IT operations and data workflows
- Insights into BMC’s initiatives towards advancing AI and DataOps solutions
Learn more at BMC Software.
Watch all of our coverage at Six Five Media at BMC Connect, and be sure to subscribe to our YouTube channel, so you never miss an episode.
Transcript
Mike Vizard: Hello everybody. I’m Mike Vizard and we’re back at Six Five On The Road and we’re at the BMC Connect event in Las Vegas at the lovely Fountain Blue Hotel. And we’re here with my old friend Ram and we’re going to talk about Data Ops. Ram, welcome to the show.
Ram Chakravarti: Thanks, Mike. Always a pleasure to have a chat with you.
Mike Vizard: There was this thing that happened last year, it was this AI, it was everywhere. It’s a bright new shiny object. I think this year we kind of figured out it doesn’t really work without data.
Ram Chakravarti: Absolutely.
Mike Vizard: So is data ops kind of going to be the new place where the cool kids are hanging out because the AI model isn’t worth much without data?
Ram Chakravarti: I think it’s a fair statement. I’ll go back to what I said in last year’s Connect events. AI and data are in a cosmic dance, but one without the other does not make any sense. One enables the other, they feed off of each other and jointly create value. So you cannot talk about one or the other in isolation. So specifically a question about data ops. Data ops is all about operationalizing your data management and data analytics use cases. It’s great to use technology, but unless you can operationalize those use cases, you’re not going to be able to get value from them. That’s the premise behind data ops. So it’s a must have for AI.
Mike Vizard: You and I have been talking about data management as long as I can remember.
Ram Chakravarti: Guilty as charged.
Mike Vizard: What is it about data ops that’s different than what we used to think of as data management in the first place?
Ram Chakravarti: Great question. I mean, if you look at traditional data management, it’s the collection of data, ingestion, integration, storage, analytics, visualization, all those things. What it doesn’t account for is how do you get a handle from source to insights and keep everything in place such that if there is any break or flaw in the continuum from source to insights, how do you manage that? How do you mitigate the risk? And that’s the key to operationalizing those use cases. So when you deploy in a production environment, you want the rigor and discipline and enterprise scale resilience. And that’s where data pipeline orchestration comes in as an enabler of data ops and with Control-M, which is our industry leading solution in that space, and Helix Control-M, which is it’s SaaS counterpart, we have the best data pipeline orchestration solutions bar none for enterprise-grade customers.
Mike Vizard: I also saw this new kind of data assurance tool that you guys were showing. And what is that exactly?
Ram Chakravarti: Yeah, so it had its genesis in the BMC Innovation labs, which I established and I oversee. And what we do in the labs is we solicit ideas from customers, we look for their pain points and evolving needs, future strategy and direction, and we take a subset of those, synthesize them into ideas. And this was one that came from such conversations. So data assurance, think of it as a complement to data pipeline orchestration, builds on the notion of business data observability. So think about applying observability concepts to analytics pipelines through which data traverses. So if you can get a handle on the health and performance of data as it traverses complex pipelines and institute course corrections before the proverbial things go hit the ceiling, then you’re in a good place. So that’s the premise behind data assurance.
Mike Vizard: Managing all that data is a complex endeavor. Are we going to get to the point where I’m going to have something that feels like an AI agent to help me manage the data that I’m using to go build AI models?
Ram Chakravarti: That’s a great question, Mike. I mean, I think the short of it is AI and generative AI can be applied to pretty much every use case. But the trade-off is what is the value? What is the burden of implementation? Are you going to get a return on investment? What are the associated risks? Needless to say, Control-M is ripe for augmentation with generative AI and AI. And that’s definitely a key focus area for us to build on top of our existing automation capabilities with generative AI-based workflows. We are working on another cool one which exploits metadata using generative AI to provide previously unavailable insights to business services. So those are many of the things that we’re looking at with the power of AI and gen AI. Again, AI feeds data, data feeds the AI. Together, they are in a cosmic dance.
Mike Vizard: And that’s kind of how we’re going to operationalize all this because one of the things you hear is that we don’t have enough data engineers, but then the next logical question is, well, do we need a data engineer for everything or can we democratize this to the point where mere mortals can do this?
Ram Chakravarti: I absolutely think it can be democratized. If we can institute code generation capabilities on traditional workflows for data-specific workflows, you start with 50, 60% of the code or 70% of the code, your dependence on data engineers suddenly goes down. Don’t get me wrong, we are never going to eliminate those people because the insights and the experience that they bring, the human touch is going to continue to be fundamentally important. But you can alleviate your own talent shortage challenges by augmenting with gen AI.
Mike Vizard: Do you think maybe the way IT teams are organized is going to change in the future? Because right now it takes a village to do anything and we don’t have a village.
Ram Chakravarti: I mean, change is inevitable as the cliche goes, right? So it’s absolutely going to happen. In fact, it’s already happening even within our own organization, in our IT organization run by our CIO. They’re re-marshaling and pivoting resources and augmenting with gen AI so that the productivity gains are faster and the time to bring to market solutions is that much shorter.
Mike Vizard: So there are lots of folks out there and they’re trying to figure out what their job is going to be and how it’s going to evolve and what is their role going to be? I don’t think anybody’s going to be replaced per se, but what I was doing yesterday is not what I’m going to be doing tomorrow.
Ram Chakravarti: Yeah. The nature of our work is going to change. The work itself is not going to go away. We are going to evolve and we are evolving towards doing higher value added work as opposed to traditional types of work that we’ve become accustomed to over the last few years. So I think in some ways it is the techies version of an Ironman coder. That’s how I would look at it.
Mike Vizard: Are we going to have a better understanding of what data is valuable? And I ask this question because a lot of times IT people, they manage data, but it’s often all the same to them. We now have data that is unstructured, semi-structured, structured. Are we going to be able to ascertain which data has more value to drive into these AI models?
Ram Chakravarti: That’s a great question. Okay, there’s multiple dimensions to your question. Number one, you need to know that you’re using the right data and the authorized data for each use case. Even something as simple as, is this data authorized for public use or some other kind of broader use in training and AI model? That’s one set of considerations. But beyond that, specific to your question is it’s not just about the business data, it’s about the data about the data which is otherwise called the metadata. This is data about code, about configuration, lineage, versioning, and so much more. This is pretty complex and dispersed across the technology layers, but you can use generative AI to link it to business services, understand interdependencies between your business services and data about the data and that can provide killer insights and give you outcomes that were previously not possible. So absolutely doable.
Mike Vizard: All right folks. You heard it here. It’s all about the data at the end of the day. And if you really want a job in AI, think about being in the data management side of the equation. You’ve been watching Six Five On The Road here in Las Vegas. We’ll be back with some more episodes. By all means, check them out. Ram, thanks for being on the show.
Ram Chakravarti: Mike, always a pleasure. I’ll leave you with this. Data is the fuel that powers AI.
Mike Vizard: All right.
Ram Chakravarti: Thank you.
Mike Vizard: Back in a minute.