Powering the SOC of the Future

Will Townsend:
Hi, I want to welcome all of our viewers to the Six Five Summit. We’re in the cybersecurity track, and I have the pleasure of speaking with Splunk.

And from my perspective, the landscape is ever evolving when it comes to cybersecurity and defenders. Bad actors are becoming more sophisticated. They’re leaning into tools like generative AI to become more sophisticated in their attacks. Also, IoT is expanding threat surfaces tremendously.

And with that in mind, I’m going to be spending time with David. David, it’s great to see you, thanks for taking the time.

David Dalling:
No, thank you so much. This is something that I’ve been dealing with and working with for many years, so I was happy to jump in and help with.

Will Townsend:
That’s great. So let’s start off with the whole notion of your vision for the evolution of the SOC. And I know in speaking with you, this has been a decades long process and really started with the future of the SOC, but really where it’s moving is to more resiliency. So, can you speak to that and what Splunk is doing to address that?

David Dalling:
Yeah, definitely. I started out in federal government and before SOCs were really a thing and help really think about what is that next generation for SOC. And, so, I helped build it for Department of Homeland Security, I helped build it for CBP, I helped build it for ICE. And as I’ve been growing, it’s always been the next gen.

And so I really thought back on what is that next? And to me, and to Splunk, it’s really about resiliency. It’s really about building the next resilient SOC, being able to recover faster. As you know, new tactics are coming in, they’re constantly changing tactics, and so we have to be able to respond to it faster.

And so the interesting thing is, it’s things that we’ve been doing forever, now just really bringing them in to a TLDR standpoint. Bringing in the machine learning, bringing in the AI with AI assistance, bringing in automation and being a part of it and not going full bore where you’re going in and are meeting everything through automation or through AI, but bringing it in back to the analyst and helping them.

We all know that the staffing and the workforce for cybersecurity is really low and it’s getting harder and harder to staff. It’s needing more and more skills. And now we’re responsible, personally responsible, if things are happening, and so it’s getting harder and harder. And so that’s where Splunk’s really coming in. Is we’re really trying to make sure we’re bringing in constant detections. We’re bringing in Talos, we’re bringing in now free threat intel into EOS. We’re bringing in automation through SOAR. We’re bringing in machine learning through MLTK and then AI assistance as well. So, that’s kind of our approach, is really going and helping and building those analysts to make it more resilient.

Will Townsend:
It makes perfect sense. And one of the superpowers that I see with generative AI is the ability to leverage GenAI to onboard these analysts even much more quickly.

To your point, there’s still a huge deficit of talent. Universities are focused on curriculum in grade schools, they’re focused on grooming the next generation of defenders. But it’s a huge challenge. And I know that the company has done a lot of primary research, the state of security report, and I’m wondering, can you spend a little bit of time and highlight some of the important points there that sort of point to this need to drive further resiliency within the SOC?

David Dalling:
Yeah, definitely. If you look at the report, I don’t have in front of me, so I’m not going to quote any metrics for you, but we’re finding that more and more people are trying to leverage and use AI. And the issue that they’re also having is we don’t know where it’s going. We don’t necessarily know the impacts.

I remember when ChatGPT first came out, everyone jumped onto it and then we realized that everything you post, everything that you do is now public, right? Oh, all the companies pull back and, “No, we can’t do that.” And so understanding that there’s multiple different types of AI, there’s multiple different types of the way to use AI and use it, especially in security operations, is kind of changing the narrative. We need to use more specific AI models for very specific tasks and not necessarily open it all up.

Where generative AI is really coming in handy and especially using dedicated versions of generative AI for the analyst is it’s really helping them think through things that they wouldn’t necessarily think, probing them with the right questions that they wouldn’t necessarily think about, helping them integrate and pull in from other tools and other sources.

So it actually has been a big help and we’re getting there, but we are not even close to where we can go. And so we went way far real quick and now we’re pulling back really thinking through that and taking that to the next step, and really doing the baby steps as we make sure we know the impact that it’ll bring.

Will Townsend:
Well, the alert fatigue is incredible. And at RSA conference, I had my first opportunity to spend time with security practitioners and really understand what they’re up against. And it’s incredibly challenging with the alert fatigue. And to your point, the disparate tool sets. There is a consolidation movement happening that I see as an analyst. It’s an amazing stat. The average mid-market organization is managing upwards of 75 to a hundred security point solutions. And that’s really untenable long-term, it’s difficult to manage that.

And so I like the point that you made around using AI and automation in logical ways to sort of reduce that fatigue, but still provide analysts the necessary tools so that it’s not just completely automated. Because I think that’s also one of the concerns that may be warranted or unwarranted around will generative AI basically replace a SOC analyst? And I know that that’s sort of an unscripted point, but I’d love to get your take on that sort of notion or idea.

David Dalling:
Yeah, and interesting fact, I do have a lot of experience with this. So I started about five, six years ago is when I started bringing in automation and SOAR into our SOC, very, very mature. And we’ve also started bringing in a lot of AI, and we’ve been doing AI in it for about three plus years. And it’s really interesting looking back after the fact, it completely changed the way we were working.

But looking back on the fact, there was a lot of lessons learned that I think people do need to think about.

So for first, before we started, there are studies done and we followed along with this, but there’s about 20% of alerts actually get investigated. And so that goes to that alert for take. There’s just so much. We were dealing with millions of alerts just flowing in, millions of logs flowing in, thousands of alerts. We had a very large implementation, but we couldn’t do that.

So with automation, we brought in risk-based alerting. We fixed that. We were actually able to do a hundred percent of alert triage. And so that really helped. And so we could actually say that we are finally alerting on everything.

But then what we found out is in that automation and starting to bring in that, things were being closed and correctly or there wasn’t enough data and they were just being marked as false positives. And so then you’re like, “Oh, well we can’t.” So now we needed to pull back on that automation and really change the analyst portion. We didn’t have tier one, tier two or tier three analysts anymore. We had case managers. And so they would get the case, they would review, find out what’s missing, and then do an investigation or kickoff different automations.

And then we brought in AI. And AI really helped with that because it would help to determine this is something we can’t make a determination on. And it kind of went from that signature base to more of that anomaly base. It’s like, “Oh, hey, here are things that we want you to actually take a look at.” And that actually really helped too.

But then what we found is a couple of years later after that, we were finding that, one, my analysts were kind of losing some of the skills, because they were relying too much on the automation and AI. But when we started to investigate a lot of those alerts, they weren’t fully investigated correctly or completely. It can only take you so far. And so again, we had to pull back and change how the AI worked and how did the automation worked.

I don’t think anytime soon we will get to the point where we can remove the analysts out. Our big approach at Splunk and something I’m firm believer on is automation, AI, generative AI, machine learning, deep learning, all are there to support the analyst to make them more effective, to make them more efficient and more accurate and consistent. So I don’t think it’s going to happen anytime soon where we replace. I know people want that, but there’s just too many unknowns to really make that happen.

Will Townsend:
I totally agree with you, Chris, and I think one of the other superpowers with GenAI is the ability to generate sitreps, situation reports. Security has got to be a team sport, and you’ve got to be able to communicate that to all parts of the organization, legal, human resources, engineering, executive management, and the ability to use GenAI as a complimentary tool to generate that. What can be sort of a mundane task for an analyst, as an example, can free them up to do the more important investigation work. So I think that’s another superpower.

David Dalling:
Agree.

Will Townsend:
As we sort of wind down our conversation, I’d love to touch on specifically what Splunk is doing to make this all a reality, because certainly you’re known for your federated approach to data as well as unifying the whole notion of security operations. So can you go a little bit deeper and talk about, can you point to some things that you’re focused on today to sort of drive that home?

David Dalling:
Yeah, definitely. So, we have ES, which has been table stake for enterprise or for security with Splunk. And then we’ve really started adding on a lot more features around ES to really start helping and making that unified, and really maximizing the use of the logs that we’re bringing in, and really helping to enable those analysts. With the tech analyzer, being able to do malware detonation and pull in additional threat intel and look through files and things like that, has been something that really has helped the analysts.

Then we released ARI, which is our asset risk intelligence, and now that’s really just going and getting all of the history and background of all of your assets. Where have they been? What have they logged into? What have they connected to? what have been the IPS over the last 30 days? Really, again, giving that intelligence to those analysts.

And then we had SOAR already, but with the launch of ES-8 coming out, actually today when this goes live, is we are combining all of these together and bringing it in, adding case management, adding AI assistance like you said, to really help with that report generation and the summary generation. And so we’re looking at what does an analyst do day-to-day and how can we make sure they have all the tools at their fingertips?

And then going beyond that is, like you said, the federated search. The federated model is we know not all data is going to be in Splunk. We know you’ve got all of the other tools, there’s all the hundreds of tools that everyone has, and maximizing the location, knowing what needs to come in for that fast detection and response and knowing what you don’t need that you use for your investigation.

And so, I think, that whole process is something we’re really trying to build and bring to the market and making it easier for our analysts to be more resilient and more effective.

Will Townsend:
From my perspective, what you do from an observability standpoint with data and logs and that sort of thing is exceptional. I really can’t think of any other company that does it as well as you do, and that is super critical just to sort of understand the gaps and what needs to be addressed there.

And I’ve written about you numerous occasions on Forbes and have spoken to that point, but I do think it’s a superpower that, and I keep using the superpower theme. I don’t know, I’m kind of on a Marvel kick today, but it truly is a superpower.

But hey, David, thank you for the time. It’s been a very enlightening conversation. Any final thoughts to leave with our viewers before we sign off?

David Dalling:
Yeah, definitely one of the things that I really think we should hit on as well is just that Cisco and that Splunk better together story. Cisco, massive with network telemetry, really big on observability. And you hit on that with the observability as well. With us combining the observability products, with us bringing in their network telemetry, network tools and being really integrated with ES and with Splunk and with their XDR, like Talos that I mentioned earlier, is we really feel like this was the merger needed to really take Splunk to that next level and really become that security household name, and knowing that we know how to bring all of that data and logs together.

I just think that’s something that sometimes that’s scaring people having us come together, but honestly from the outside and then coming in and then being a part of this, I’m really excited about where we’ll be able to go.

Will Townsend:
Yeah, it’s a super powerful combination and I think there’s more to come in the near future. But hey, David, thanks for the time. It’s been a great conversation.

Well, I want to thank all of our viewers for tuning in to this Splunk session during the cybersecurity track at Six Five Summit. We’ve got a lot of other great tracks as well within cybersecurity, as well as other categories, so be sure to tune in. And, David, thanks again for your time.

David Dalling:
No problem. Thank you so much for letting me join.