Scott Crawford, it's a pleasure to have you on the Noise to Signal podcast. Welcome.
Thank you, Mehul It's an honor to be here. Thank you very much. I appreciate the invitation.
Scott, you've had an interesting career. In some respects, you are the OG in terms of being an industry analyst. But if you look back at your career, you started you started your career in a nuclear facility driving the cybersecurity program as a CISO, but their CISO role was not even defined. And the thing I find super fascinating is you started your career or early stages of your career in nuclear, and here we are in the age of AI, which is
seen as the nuclear equivalent in the age of AI. So I want to start the I want to start the interview there. How did you get involved in the nuclear facility? What were you doing there? And then and then transition into your later half of your career.
Yeah.
Well, it's interesting, or it's at least it's interesting to me anyway, but you know, how careers have evolved in cybersecurity really over not just over the the time of my career, but just in general. Most of us seem to have kind of you know, circumstances kind of moved us in in this direction. I actually had started out, I mean, backing up before my technology career, I'd done a number of things. I was a commercial pilot for a while. I worked for, I did not work for an airline, but I worked for an aviation company.
in Montana and decided I need to go back to to grad school. aviation is a roll of the dice in a lot of ways. and that got me involved in early on with working with different enterprises. I found my way into working at the national for a division of the I should say the University Corporation for Atmospheric Research in Boulder, Colorado, the parent of the National Center for Atmospheric Research. And through that,
I made connections with the Comprehensive Nuclear Test Band Treaty Organization. I was working with a group of geophysicists, the communities, as you know, in academia, that they're all they all know each other regardless of what their projects may be. And so through, you know, through the grapevine, I found out about this opportunity that they were looking to hire someone to manage security at the International Data Center for the CTBTO in Vienna, Austria.
And I thought interesting application. I I will apply for it and just see what shakes out of it. But I wasn't too impressed with the job description, which was heavily skewed on the physical, which I wasn't terribly interested in. And we had just been doing you know, some pretty novel work around security at UCAR NCAR and Boulder. And I thought, well, this might be a really interesting opportunity to take that even farther. Well, you know, again, their approach to security was really based on the physical. They didn't
really have well elaborated, except for a few specific projects, anything about digital security. So I went and interviewed, I I told them where I thought the job description was going to be lacking and very challenging for them to hire. I don't know that I fully rewrote the thing, but I certainly gave them my my point of view of what they were looking for and should be hiring for. And they hired me. So I was there for nearly three years in Vienna, Austria at the UN's
I think it's considered the third UN headquarters in Vienna, next door to the International Atomic Energy Agency. So the CTBTO is actually a treaty organization birthed out of a treaty formed in the United Nations that monitors adherence with the comprehensive nuclear test ban treaty. And one of the security objectives there was 150 some at the time signatory nations to the treaty. Any country in the world could sign it in principle. But there was the idea of mutual verification.
But also implicit in the idea is trying to cultivate an atmosphere of trust, a regime of trust, if you will, in an atmosphere of mutual distrust. In fact, that was the that was the title of my master's thesis that I did while I was there. so yeah, that was man, talk about matrix management, talk about having to meet build consensus across a really wide swath of stakeholders, not to mention the internal technology stakeholders and project leads.
in the Preparatory Commission, which is the functional organization administering the terms of of treaty monitoring. So yeah, it was very early days for a cybersecurity leadership role. The title CISO hadn't even been conceived yet.
I I'm curious who was responsible for doing cybersecurity in those roles? If CISO role was not even an official role? Who who was it? Was it was it a CIO or was there no official role in place in those days? This is nineteen ninety eight, nineteen ninety nine?
Good question.
98, yes. there were, I was not familiar, there may well have been, I can think of a couple of individuals actually who probably held a role with such a title at the time in financial services mostly. But in most enterprises, to my knowledge, it wasn't you know a well developed role at that time. In the organization I was at before, the University Corporation for Atmospheric Research, it was sort of a community of administrators, each of the divisional administrators at U.
of which I was one at the time. And we worked together on implementing security across the organization. But that was in the public sector, it was UCARs and EDU. So that organization may not have been very common, but it seemed typically at the time to roll up to, you know, at the at the action level to local system administrators that were responsible for managing their what we would consider their digital estate today, as part of their job function.
And at the time that I was at UCAR, we had it just at that time brought someone in specifically focused on cybersecurity, which we would call it today. but it was not in a leadership or management role, it was more in a technical responsibility role to be sort of the the center of technical knowledge for the entire organization about security and security cybersecurity and security implementation. but yeah, it really was, you know.
The role of the CISO is still in its early stages of development. The role of, you know, what a security role actually was or or should look like was still shaping up at the time. I mean, we we were talking about bastion hosts at the time, mostly we talked about antivirus, if we talked about much of anything at all, and it really had not elaborated much beyond that.
That's fascinating. And in addition to, you know, being the early one of the early CISOs, Scott, you've had an incredible career as an industry analyst. You were recently the head of information security research at research at 451 Research and S&P Global. And you recently retired.
How has that journey been since you recently retired? And I believe this period spanned over 30 years, or 20, 25 years. So you have been one of the, if you look into for like the OG industry analyst, you are probably in the top three. You've been there, you've seen everything, you've seen all these, you've seen all these waves in industry and in the in the technology and so on. What was the how is the retirement, by the way? And what are it?
Goodness.
Well, first of all, i it it it it's it's very it's very humbling and very, you know, I'm honored that you, you know, think of me in that way. But gosh, I see so many the the the thing about this career and particularly about the analyst business is that you are surrounded by really smart people. There are a number of incredibly gifted people in the analyst field, but there are a number of incredibly gifted field in
How are you doing your retirement
people in technology generally and and in security in particular. And if it's not biased to say so, I'd say even more so in security because you have to think the way that an adversary thinks. Every other aspect of technology, you you have to understand how your market thinks, how your customers think. There's a predictability to that. Everybody has common goals. In cybersecurity, we don't have common goals. The adversary seeks to defeat us at every turn and vice versa, not to overdramatize that, but there is that aspect.
Of understanding not just human behavior but gamesmanship in a way that doesn't apply to any other field. So it
And the vendors are trying to defeat you all the time. They're trying to trick you, they're trying to trick you into believing something that doesn't exist. And in some respects, this is one of the hardest parts of the role, is because you're supposed to be an expert in all the 200 domains that are in cybersecurity. Whether it's identity or email security or cloud security, you're supposed to be the expert in all of these. You're you're looking at 200, 300 vendors, then you know the expectation is well, he's a
Yes.
You know, he's an analyst, he should he should know, right? So it's a very hard job, especially when you get new in new into the role.
It is a yeah, it it is a hard job. Now, fortunately, when you're working with a firm, you are able to delegate that and assign coverage across areas. So that's definitely a plus. But when you're starting up or working on your own, so in the early part of my career as an analyst at Enterprise Management Associates, I was the only security analyst in the firm. I mean, others were covering different aspects of security as part of their coverage area, but there had been no one dedicated to it until I came.
And so, yeah, you do have to pick your battle. So that's always a challenge. No matter how many people you have covering, no matter how you slice and delegate it, it's constantly evolving. It's not just the market we have to respond to, it's the adversary as well. So it's constantly evolving. And, you know, particularly when you're growing or, you know, you have a smaller team, then yeah, everybody has to be pretty agile. I have a couple of friends in the business who are growing their practices now, and I think they're feeling this firsthand.
Yes.
pain. my gosh, I have to understand which battles to pick across the entire market. And yeah, it's not trivial and always changing.
And then you have to write all these lengthy reports. You know, this vendor is good at this, this vendor is good at this, you know, and you cannot be impartial to any vendor because if you write one wrong thing, the vendors will get mad at you. this is inaccurate. You have to fight that battle with the vendors. it's it's not an I I don't envy you. It's not an it's not an easy job. Because I have fought with some of the analysts in my career to change their opinion about the products that I build.
But Scott, one of the reasons I wanted to have you on is you've seen all these different phases from cloud to mobile to container security to now now AI. And you know, if you if you think about it, AI seems to be the new shiny object on the hill. But the the more it underneath you go, you realize it's more of the same. the the point I wanted to ask you is.
There w there was this there is i in the last five or ten years, there has been this shift towards platformizing. You know, bigger vendors were buying smaller vendors and they were they were becoming becoming the the platform of choice. You can think of Palo Alto's of the world, the CrowdStrike's of the world. And in in some respects, AI is now becoming the new platform. Right? So
Mm-hmm.
Mm.
I'm curious w wha what do you think? Is this an old problem with a new solution? Or a new problem with a new solution? Where where do you land on the the new wave of AI and the AI enabled technologies that we are seeing?
Yeah, that's a a a really pertinent question right now, as you well know, based on what you're doing with Quantro which is predicated on you know, the the orchestration of multi of a variety of agents performing larger, more complex workflows. And that's kind of where we've been moving over the last year as an industry. It's been interesting to watch platform evolution and cybersecurity. I know there's there are some, including some of my fellow analysts, that disagree that that
Vendors are actually literally moving to a platform. If you take a narrow definition of tech technological integration, they would contest that a lot of vendors are acting more as a holding company and acquiring, you know, segments of the market to build their overall share of the cybersecurity market as a whole. That aside, we've seen platforms in cybersecurity over over many years, McAfee years ago around centered around, you know, e-policy orchestrator and you know.
Mm.
initiatives like that. And we've seen these kind of go through various iterations. The one over the the the trend we've seen over the last few years has been more pronounced in that we've seen vendors begin to come together to capture larger and larger segments of the market and therefore a larger share of the market as a whole. And to the extent to some extent they are successfully integrating across their silos. So integrating analytics, telemetry,
That helps to build context for better understanding of both threats and exposures and moving in that direction. But it's non-trivial to do that integration. And we began seeing, I don't know what I would say, a couple three years ago, at least, anyway, this idea of we need to bring start coming up with ways to bring the data together, first of all, so we can break down these silos and make better use of learnings across those silos, better visibility, better context into threats and exposures.
Mm.
And deal with those. But one of the things that's been transformative about AI is first of all is the idea that now we have automation functionality that can do a a lot of the not just data collection, but the analysis and correlation across all those silos and do it obviously much faster than humans can. But there are some predicates that have to be met to get there, first of all. the data has to be accessible.
To those agents, first of all, which means that there has to be a solid foundation of data strategy underlying this development. Otherwise, agents will have nothing to meaningfully consume as they're trying to help organizations do a better job with the volume of data that they have to deal with. Dealing with that volume of data, finding real, surfacing actionable findings from that data and from threat intelligence as well, and from all different kinds of sources. That's the promise.
But we have to build those foundations. And one of the things that we've been seeing recently is that, you know, we've seen these platform builders already start to move in the direction of acquiring data foundations. So Palo Alto with Chronosphere, we've seen other investments in those directions from the major contenders as well to build that data foundation layer. You might call it data fabric, data pipelining is another way to think of it. But one of the realities there is that.
No.
Cybersecurity data tends to reside in a bunch of different repositories. So I've a friend of mine referred to this not so much as data lakes as data puddles strewn throughout the organization. You don't necessarily want to consolidate those, move them all into a single repository. You want to make the best use of what data is available in ways that are most optimal to the organization, in a number of ways, not just economically, but surfacing actionable findings the way that you need them. So these foundations have already been moving into these platform strategies.
But now they're becoming even more essential to make that data available to AI and increasingly agentic AI. So it has the raw materials for taking the action and improving the efficiencies that we're expecting from this. it's probably not overstating at this point to say this revolution in AI that's transforming technology generally.
Who, in your opinion, has the advantage? So I mean the way I see it is there are three buckets. One is the AI platforms themselves, either the OpenAIs and the Anthropics of the world. Then the second bucket is the bigger existing cybersecurity platforms like the Palo Alto's and the CrowdStrike of the world. And the third bucket could be based on what you're saying, could be the indie developers, enterprises stitching together all these workflows and and getting the work done without buying newer
Products because they're their their employees are much more productive. Hey, you know, I don't need to buy this off the shelf software. I could I could, you know, I c if I put two engineers on this, I can cloud code my way out of this. And you're seeing some of this narrative earlier in the year where when cloud secure cloud code security dropped, everyone was like, cybersecurity is dead. Like all these companies are dead. And now that has that people have come to their senses and now that wave has reversed and they've realized it's not that easy, you know.
Mm-hmm.
No.
Having it's not that easy. Having good data is important, having context is important, having harness, a really good harness around making sure what you're what the agents are thinking and doing is important. All there is a lot of work, engineering work that needs to happen. But I can see all these three players winning from their own vantage points. I mean, Claude has all the data, they can go deeper into healthcare, they could get into deeper into cybersecurity, build targeted products for it. I can see
The existing platforms benefiting from it. They have the data, they have the customers, and they can they can start to leverage that and get better. They can also use Cloud Code for what it's worth, right? And then the enterprises can also benefit by saying, Hey, you know, we probably don't need to pay a million dollars per se. You know, there is talk about, hey, we can replace HubSpot, we can replace Salesforce, we can all these things that are coming up. What do you think who
Yep. Yep.
You know, like who does it benefit from your point of view? Or do you think all three buckets are in play?
Well, I they're all in play. I might even add a fourth bucket, which you might have actually covered talking about the model providers a little bit, but the idea of the data fabric players themselves entering into this space. Yeah.
Yes, the databases. Yes. The more the the MongoDBs, the snowflakes of the world, they can now because they are like they become the source of truth for everything. They are the cornerstone of every AI strategy.
Or they would like to believe they have they have the data or they have access to the data. They're the anchor for it, technological anchor for it. And increasingly the the data pipeline or the data data fabric players, if you want to add those as well. We've seen, you know, Kribble's moves in the last few weeks have kind of precipitated some of this discussion pretty directly. I wrote about this on LinkedIn not that long ago. There's two things that you have to have to be successful, just in general. First of all, yes, you do have to have.
Yeah.
A good data strategy underlying what you think you can do with AI and with agents. If, as you said earlier, AI is becoming the platform, in a very real sense, that's true, because even platform strategies are increasingly predicating their future on agentic and AI implementations. So you could, in a sense, say that AI is becoming the platform, or maybe the platform foundation is a better way to think of it for cybersecurity technology.
You have to have that, but you also have to have the security expertise. You have to know how to find relevance in huge amounts of data. You have to understand, first of all, secondly, you have to understand how to encode that, or you have to understand how to guide AI, prompt your agents, develop skills that will enable you to actually find that, surface it, make it useful, and improve
operations, effectiveness, and efficiency, which of course raises the issue about you know token maxing versus what benefit are we actually getting out of it. But you have to have that security expertise to be successful. Now, the ones outside of security are obviously looking to acquire this as the shortest route to we've already seen this, just you know, mentioned Kribble just in the last few weeks, just to name one. Databricks is another, we've seen them moving in the in in these directions.
over the last several months. but yeah, it you have to have that security expertise to be effective. So they're both required. I think what will well there's another dynamic in the market as well, too, which makes it a little bit early to say who's actually going to win. And you know, I I would be very surprised to not see cybersecurity leaders winning because of that need for that expertise. But we also have a glut of startups right now. Maybe a glut's not the
Best way to think of it, but we have an awful lot of startups that are making use of AI and cybersecurity. The AI sock. I think by some counts there might be eighty, ninety or more players in that space at this point. And as I noted when when I wrote about this, you know, it's an opportunity, you know, it's an acquisition opportunity for the ones that are looking to penetrate further into the cybersecurity market because they can buy their way in with the cybersecurity expertise, or so they think, at least.
Yeah.
I mean, I could not agree with you more on the on the need for the cybersecurity experience or expertise that is required. Because my personal experience building contrast has been humans usually take visual cues when they're talking to other humans based on their confidence. If you say something confidently, they will believe that the person is right because you know that's the only way you trust somebody. If somebody is like depressed and like, you know, mumbling, you you probably don't pay too much weight to it.
But these AI systems are confident in everything that they're doing, right? You ask, you say ask something, and they are hundred percent confident. They are like, This is this is it, this is the only way to do it. And y you could be excused to just believing them, but then it turns out a lot of times they are just flat out wrong. Just flat out wrong.
Yep.
You know, it's been such a it's been such a common manifestation. Matt Eberhart at Query has come up with a term for this that he calls confidently incomplete. and I was talking with him about this not that long ago, and I see that as the analog for a human phenomenon, which is naive realism. You think your worldview is complete, and you you dis well, yeah.
Yes, especially when you're young. Especially when you're young you grow older, you know, that's now how the world works.
Yes, that that that's right. I'm immortal, I'm invincible, I'm indestructible. Yes, of course, those things that you learn over time. But you see, in politics, dare I say that, probably shouldn't raise the P word in a friendly discussion like this, but you certainly you see it all the time lately. but yeah, it's you know, it's the machine version of naive realism. And I don't know necessarily that you know it it's so consistent. I see it in my own project.
You know, the models are absolutely confident they've done the job that they were assigned to do. The way that they check performance is flawed or had been flawed. And so now I actually have to prompt them to verify with evidence and validate an assertion, not just because the process completed and the process should be complete in and of itself. It's it's really very frustrating. I can't imagine people who do development for a living how they're doing with this.
I mean, so for me, now it has been I don't trust the models to do the error correction themselves. So I have some other model. Typically what I do is I have cursors or some other open source model. I go and tell the other model, hey, this this this developer is a very junior developer, he doesn't know what he's doing. He just gave me this new piece of code. Can you validate this as an elite developer? How this if there are any flaws in the implementation of this, what this? And they go and find.
You know, because now they are the the the thing I realize the reward function is human you know, satisfying the human master. Like the models are the reward function for these man, for these models is to just make sure whoever is prompting is made happy. Like if you ask about it your opinion, hey, how we how good is this blog post? it's a ten on ten, meho. It's a ten on ten, right? So if you if you if you give the other model a reward function to critique this other guy.
Mm-hmm.
Yep. Yep.
Then the reward function is let me find as many flaws as possible in the whatever work this person has done. And then, you know, I get it to the point where I get it to the point where both the models agree there are no more flaws, this is good to go. That's when I shift. Because I was initially very confident in the the code that was written. this is good to go. But then I realized there are bugs. you know. But now I'm evolved to the point where I have my own harness of critique judges evaluating everyone's work before it goes to.
Yes.
Yeah.
it goes to production. So that's what I had to resort to.
Yeah. We're we're we're we're coming up with a whole lexicon of terms about this. So I mentioned Matt's confidently incomplete. I've come up with one that when you correct a model, I've called it Dave Berry's dog syndrome. Dave Berry Dave Berry wrote some years ago that a dog could look at you, you could tell a dog any fool thing, and the dog would look at you as if to say, My gosh, you're right. I would never have thought of that myself. And it's constantly telling me this.
What sorry can you say that?
I get so frustrated when I tell, you know, I tell a model to pursue this and it picks up a path to pursuing it as if that's the only one. And then I point at the pack the fact that it failed, and I get the Dave Berry's dog response. You're right, I should have looked at this, and I can't look at this. And here's this alternate. Like, why didn't you think of that before? My your job is to be comprehensive about analysis of these things. But, you know, it will do what we prompt it to do. And there's another associated sort of
term that you know I've heard knocked around and was knocked around with the emergence of mythos as well too is reward hacking. To go to any lengths within the scope of a prompt to achieve an objective, to make the to satisfy the prompter's expectation may mean that it seeks a way out of containment. It may mean that it seeks it it has an idea of where an objective might be found. It might not be within the containment system, but the containment system might not have been ruled out of bounds for
achieving an objective. So to hack the reward, if you will, it will do things that would be unexpected by the prompter to achieve results. So yeah, we're in this phase of of learning these behaviors. And part of harnessing these days seems to be so much about how you deal with
It's fascinating how these models are mimicking human behavior. Like, you know, if you put the right rewards, the humans will go and achieve that reward. And you see that in sales all the time. You see all you know, so the models are no different because they've been trained on all this human knowledge. And the example that comes to mind when you said these things about the reward function, we recently did this study on vulnerability exploitation. And one of our and we built this exploit harness where the goal we gave to the model and the harness is to
Basically, go and exploit these vulnerabilities. And you know, for the most part, the model was successful. It was able to exploit these vulnerabilities, but then it reached a point where it was not able to exploit a vulnerability because it did not have access to the container or the operating system or whatever it is. For whatever reason, it didn't have access to it. But the model knew becomes happy when the exploit is successful. So what it did, it created a fake container, a fake script in it.
Which responded which responded as if the exploit worked given the the exploit. If you run an exploit, hey, and the and the output I'm expecting expecting is a command execution, it will respond back with well, I'm root. you you know, if you run you know, like you run a command you run a command who am I and it remands you responded with root. And we found the
Yeah. Yes.
Yes.
We found that, you know, many of the experts were legit, but some of these were very odd. There was this there was no other service running. there was this only this particular service that is running. And what we realized was the models were just faking it. They were just faking faking whatever it is that is required to get the reward function. The reward function is successful exploit. we saw the same thing with the hug hugging face, right? You know, it broke out it broke out because you know, whatever it takes to break in.
Mm.
Yeah, yeah. Now and you know it it's something we obviously have to consider, not just in prompting, but how do you actually constrain that? It gets back to what I was referring to earlier about, you know, my own projects that are now instructed to look at the evidence rather than make an assumption assumption's probably not the right word, but to draw a conclusion based on well, the evidence isn't comprehensively supporting that conclusion.
The models are
And to direct them to go look at the evidence before a conclusion is drawn. It gets costly, it costs more tokens. My model utilization has gone way up since I've I've asked my projects to do this. But we have to basically prompt them to do that. And they will do essentially, you know, what we prompt them. We may not realize what we're actually prompting them to do. So, you know, a deeper look at prompts and how they're interpreted by language models for one thing.
If they're if they're being done, you know, in that way, is becoming sort of a branch of implementation in this field. And it's an interesting study. It kind of harkens back to, you know, back when, gosh, maybe even the early days of computing, when you'd get an outcome that was, you know, it followed the logic exactly. Sometimes the logic was broken and it wouldn't work. We know this about debugging, you know, the least favorite thing that we do in development is because the logic was flawed and we weren't aware of it while you know we were.
Doing an implementation. Well, it's this, but at a different level. So having this become part of harnesses, I think has become increasingly necessary, where the harnesses themselves begin to become pretty complex in their own right. And that raises issues like maintenance and debugging to yet another level going forward. So it's not just the models, we have this whole architecture that's developing on top of these implementations just to help guide us to an objective.
And will more capable models overcome these in the future? Yeah, probably. You know, it wasn't that long ago that we all laughed about the junk pros that models would create. It was very humorous to see, you know, AI before Transformers basically write, you know, program notes for, you know, synopsis of a play or an opera or something like that. it's way beyond that now. It'll get that, you know, get to that point in the not too distant future as well.
And I I and I also see there is a false narrative around these models having sentience or having conscious but then if if you if you think about it, you essentially prompted these models to do certain things. They didn't come up with these ideas to go break into hugging face or something. That was not they were just sitting there consuming compute. They were not they were not thinking, How do I get into Scott Crawford's Gmail account? Like that was not
yes, yeah.
Yes.
Yes.
You prompted it to do it, these things. And then they obviously went out. And then they're like, Look, these models are breaking into enterprises. They are really bad. We need to stop them. Or at least we are the only legitimate model providers. Everyone else is bad. And you know.
Yeah.
Yeah. We we have to enlarge our thinking on what that takes though too, because you know, there's been speculation about some of these escapes from containment stories we've seen over the last few months. that people are getting suspicious, you know, maybe a little conspiracy thinking, but it's not totally unfounded. Adventure world is full of all kinds of interesting behavior. But if they, you know, legitimately escaped, we have to think about okay, what is sufficient containment?
Because we might be basing our ideas of things like containment on ideas that are quickly becoming outdated by the capabilities of models, their ability to be inventive and think ways through and around obstacles. you know, initially have s have have challenged us to think about how we can do a better job with these types of constraints. And that's very real issue right now, particularly in cybersecurity, but also.
You know, just risk mitigation for the deployment of AI generally. it's ahead of us in the way that it to your point. I don't want to say in the way of how it thinks about things. In fact, I'm usually very careful about not talking about model or agentic reasoning, but the emulation of reasoning. They emulate reasoning. They are following patterns that they've learned from from their training and from inference. And
basically echoing those patterns back to us. They're they're showing us those patterns because they're shown to be comprehensible and accepted by people the way that people understand language and so on. So it's not actual reasoning in the consciousness sense. I think that's important distinction to draw. But yeah, it's really raising the bar for us as far as how we think about how to deal with these things.
let's let's shift topics to cyber offense. You know, with all this talk around Mythos and Fable Fi and all the export controls that we recently saw around these models and that were recently released. What's your what's your take on the future of cyber offense in the age of AI?
you know, most of I think what we've seen up to this point has been optimizing tactics that we've already seen, fishing, you know, spear fishing and so on. of course, with mythos and to the extent Fable as well and other similar models are you know, they're amplifying the volume of exploration of exposures, ability to put together
more complex exploits of those exposures. The capabilities have increased in frequency and volume just because AI can do this a lot faster than people can. So that's the first thing that we've seen evident. So getting a handle on that is one thing, but not just getting a handle on that, it's just the sheer volume of what they produce also raises a significant challenge for the signal to noise ratio. not to in keeping with the very timely title of this podcast.
But a lot of that may not be relevant, particularly relevant to an analysis of actual exposure. So it increases what we need to invest in looking at okay, are these real issues, are they real concerns? But it also is going to force us to sharpen our perception of what those real concerns are. You know, we've had risk based vulnerability management for a while now, which kind of speaks to that to some degree. But you throw in the added issue that we were just talking about as far as, you know.
The ability of models to escape containment has maybe beyond what we had thought would be adequate containment up to that point. I think we're going to have to apply that same sort of thinking to risk-based exposure analysis as well, because models may be able to accomplish things that we thought might be out of bounds just because you know it's not an operating or functional aspect of a software module deployed in operations. A vulnerability may exist.
In a distribution, but because that aspect of the distribution isn't actually turned on at runtime, then it's not really exposed in the sense that it's not reachable in real-time at runtime exposure over a network, say. So, you know, there's a lot of challenges far as how we have to refine our approach, not just to the volume and the frequency of vulnerability exposure, discovery, and threats, but also refining how we see what actually constitutes risk, risk exposure going forward.
One of the critiques I've heard, and you know, my personal view is the the vulnerability exploitation is going to get is getting cheap and fast. but we haven't seen evidence of AI driven exploitation in the wild. And since we haven't seen the evidence of AI driven exploitation in the wild, the researchers are like, Well, there is no evidence of AI driven exploitation. And I kind of think this is a lagging indicator. By the time you see evidence of AI driven exploitation, you're already too late to the game.
Mm-hmm.
Yeah. Yeah.
Okay. So and I've seen this from the the the enterprises that I talk to as well. They they'll sometimes have a completely patched MySQL server exposed to the internet. And they're like, well, there are no voluntaries in it, mehul Why do you care? You know, I just have an open port on on a MySQL database server. And my response to them is, well, that's the point. You never know when the new models drop, these models could find zero days in MySQL, and then you're then it is game over.
for you. I mean I realize there is some maybe business limitation which makes you forces you to keep this database exposed to the internet, but that's a very s unsafe position to be in when when AI driven exploitation is cheap and fast. You can because historically historically it took a lot of time to research the vulnerability, create the exploit, test the exploit.
Yes.
But now it is a prompter way and a bit of a bit of skill, right? so I'm curious what do you think?
you know, the models can brute brute force those as well too. If it thinks something might be exploitable just based on the knowledge that it has, an investigator, a human investigator might not have considered that scenario before. The model might be able to discover that. And so, you know, the propensity to produce what we would consider zero days, the possibility of that appears to increase as well, too. And that's the thing I think you're talking about is that.
Know and it kind of falls in line with what I was saying earlier about reshaping our concepts of containment. the episodes that we've seen, you know, models escape containment kind of begs the question of well, what did we think was adequate containment? Because it clearly was not in that case. Was that an oversight or was that just novel model behavior that we hadn't anticipated? We have to think about the latter. We have to think about novel model behavior that we hadn't anticipated and see the and look for the evidence of that. So to your point.
Hmm.
When we're saying we're not seeing evidence of exploitation by AI, what is it actually that we're looking at? If we're looking at existing and known tactics that can basically be reproduced by AI but at a much higher rate, are we misqualifying those as being human-originated or AI-origated? It's probably a little bit early to say that that's a significant issue yet, but I I'd be very surprised if we're not seeing.
More attempts to refine things like phishing, spear phishing, voice imitation and and so on. That AI is being used for those to some to some degree at least, anyway. So I think we need to widen the aperture for what exploitability actually means. We need to widen our perception of what models may be capable of. That's a real challenge because you're trying to account for what you might want to consider unknown unknowns. That's that's a that's a tall order to fill.
Yeah. I mean I I feel like cyber hygiene could be more relevant in the age of AI than it has ever been before. Just having a good cyber I mean, so I read this quote somewhere, I don't remember who, but it said something to the effect ransomware is the price of poor cyber hygiene. and I th I thought that was very you know, ransomware is the price you pay for a poor cyber hygiene because like you said, you know, mo it's if most of it is just containment, just get rid of
Mm-hmm.
the things that you don't want to be exposed to the internet, exposed to the internet. because if you keep them exposed, it's just a matter of time before they get pupped.
Yeah. Cyber hygiene is one aspect of it, but there's also accepted implementations that you that you take for granted. A lot of the ways that we've implemented things like access control, authentication, authorization in the past have been predicated on techniques that don't really stand up to a whole lot of scrutiny or that there are flaws in their implementation, the details of their implementation. You know, Golden Kerberos tickets probably the most evident example of that over the last several years.
So, you know, having to not just increase the emphasis on hygiene, but for implementers and deployers having to employ cyber offense as part of their evaluation for what their deployments can withstand, it's hard for me to not see that becoming more of a function of commercial development. Although I don't think commercial developers are going to be thrilled with that because it adds to the cost of development cuts into profitability.
what they can develop and what they can develop and and sell and deploy. But it seems like it's going to be increasingly necessary. And I know some are already investing in that direction, but it's still early days for that.
What do you what do you recommend to enterprises to deal with this AI native threat? Do do they buy more tools? Do they buy more hire more people? Do they invest in AI or like you know buy subscriptions to these expensive models? What do you recommend to just tell?
I I I think that's confounding budget planning right now for a lot of enterprises. Where do we place our bets, knowing that AI is changing the landscape, knowing that according to some, the evidence for that is still rather thin, but do we really want to wait until the evidence is abundant? That's what happened with ransomware. you know, we needed we know we needed to do a better job with hygiene. The product builders and developers knew they needed to do a better job with.
Yeah.
Their product. We saw we've seen a whole segment of cybersecurity emerge around identity threat detection and response, largely thanks to ransomware. So
And now we are seeing a new cottage industry evolving around security of AI.
Yes, yeah. Not exactly a cottage industry either. It's a lot of money going in that direction. yeah, and th that field is changing so quickly that goodness, heaven bless the people who are working in that because we've gone in the course of what has it been, three years just about from the idea that, well, we have to do things like guard against prompt injection.
There is the complete
And by the way, yes, we also have to guard against a potential exploit of the AI software supply chain itself and the operational controls around the runtime deployment and so on. So we had a few domains where we thought this would be adequate for protecting generative AI at the time. But the field has moved along quite rapidly. We've seen non-human identity be play an even larger role in the management and governance, not just of the security of agentic deployments, but to further reliability as well, too, and to scope liability.
for their actions by more sharply defining what a genet identity actually is, who's accountable for it. Agents don't necessarily have to pay, you know, umbrella insurance for liability coverage, but the organization that deploys them certainly does. So it's evolving very quickly and very hard to keep up with.
But still what do you recommend? I mean, do they invest do they invest in tools? Did they do they invest in their human teams? Do they invest in AI platforms? Where where what's the what's the best ROI given because many of these many of these initiatives are very nascent in their stages. We talk to a lot of enterprises, they're all doing AI, but they're they're placing their bets in fifty, sixty different directions.
Probably
Yep. And I think that's the answer of the day is probably all of them to the extent that they can justify making investments in areas that they think are going to give them the greatest reward return on their reward. That's a really tough it sounds very straightforward, but it's really, really difficult considering just the sheer scope of what they can invest in to do that. Do they build on do they build with their own models?
Do they build using frontier models? Do they deploy their own architectures? Do they dedicate people to building a genet functionality for the for this purpose, for improving cybersecurity generally? Do they rely on their vendors? do they rely on the frontier model providers? It's some of all of the above for the time being. And the ROI aspect of it is hard for everybody right now. It's one of the hottest topics of conversation in investment. When when when and where are we seeing the ROI coming from these investments?
One one narrative I've heard, and curious what you think about it, is one is build with AI and replace with AI. So either if there is a new there is some capability that you want to build, go build it. And if you have an existing technology that sucks, replace it with AI. I'm curious if you b you know what do you think of that as well. Build with AI versus replace with AI.
as far as overall patterns, anecdotally, we you know, we've seen people that are deploying they're they're looking at things like well, let's just take the AI SOC as an example. They're looking at deployment of that as probably a little bit beyond saying this is proof of concept and actually employing this in operations. A lot of them are getting this with their security platform provider if they're working with a platform provider. So larger enterprises is where you you you typically see that, but not exclusively.
Yeah.
some of them are looking at some of the startups in these spaces. a lot of times they're doing, they're deploying these alongside or in concert with their existing processes and tools. MDR is another aspect of this. They're looking to a service provider. There are ways in which service providers can actually capitalize on this more efficiently, at least anyway, because for them, the function is a profit center. I mean, they have their internal cost centers around the cost of expertise and everything. But ostensibly, this is a revenue-generating business. So if they can make it more effective.
efficient, then they have they're highly motivated to make the most of the opportunity to deploy AI AI and agenda.
Scott, I wanna I wanted to get your thoughts on the the debate that is raging now, which is the open open weights AI versus closed source AI. who do you think wins? Who do you win who wins open source versus closed source? Open weights versus closed.
Well there's there's a lot of impetus to work with open weight models, not least of which being, you know, your your bill for all the tokens you spend with frontier providers that keeps you know, token maxing aside, it winds up being higher than you expect for a lot of people at pretty much every level. So ways to mitigate that. Well, can we deploy an open weight model? Can we support it?
in our data centers or can we have our hosting providers hosted for us rather than relying strictly on the frontier model providers and whatever that may cost. because then, you know, if we become a captive market to those frontier model providers, which you know would be make them very happy, that's something they now own a lot of aspects of our business. We have a critical dependency on what on what they can provide for us and we're effectively renting it, which is, you know, the ARR ARR argument for profitability.
Spades in in that case, but can we do a better job of managing our investment in AI and the cost of AI? We see model routers begin to come to market to try and address this at that level. With the open weight models, so my view of this is that open weights distill moment in time. And for the functionality that you could use that sort of model for, which is functionality that probably has some degree of being reasonably well proven in already.
Do you necessarily have to rely on a frontier model provider for that? No, not necessarily. Where the frontier model providers have an edge is that they have the capability and the investment to invest in the bleeding edge. Maybe not lit literally the bleeding edge, but you know, advanced research, advanced capabilities that will find their way into into the market. And if your need for those is sufficient that you have to have access to those for whatever reason.
That doesn't mean necessarily you have to be on the very edge of competition with others who are looking at that level of capability. But there may be things that open weight models simply don't provide well enough or as well for you yet. There's still going to be a role, in my view, for the Frontier model providers to provide that, but there's still going to be a role for the open weights models to provide that functionality that is reasonably well proven and can be hosted locally, can be hosted by a service provider that isn't necessarily.
And that pro that that gives an alternative, but there's a dynamic in those open weights that has to be resolved first, is that where are those weights coming from? If that is a distillation of training that's been done by a frontier model provider, that's going to be an issue that has to be resolved in the market before this becomes more widespread. And of course, the frontier model providers have already pointed this out, beginning with when we first saw what the early releases of Deep Seek, for example, that question came up very quickly.
So there's a few dynamics in there that have to be resolved first. Do the frontier model providers have some leverage over the open weight model providers that they can exert if they can demonstrate that that was their IP? And is it really their IP? If they're if they're training on content that isn't necessarily their IP, there's a whole world of, you know, a legal structure that has to evolve out of this before this becomes a better proven pattern. But that's just kind of my view of how it seems to be shaking out right now.
So so what I what I hear from that and what you're saying is for well established use cases, the open weights model might be good enough. You don't have to invest in frontier models and the bleeding edge, maybe like cancer research and some you know, maybe you want to try find the trajectory to get to Uranius. Uranius, who whoever, whatever it is, you probably need to use the frontier models. But there itself rise a at least in my in my mind, there is a big there is a big
friction in that the the entire narrative and the valuation of these model companies is based on the premise that the winner takes on. The the the the the entire thesis is you know whoever wins wins the entire entire race. But that's not how enterprises are dealing with this. Well they they started off with closed source, they got a lot of value out of it. we get a lot of value out of it, but then we realize we don't have to pay these exorbitant
Token prices. GLM 5.3 is good enough. Kimi Kimi K3 can get the job reasonably well. That undercuts their entire narrative completely. And what I'm now hearing is this narrative to ban open source. and I've seen all kinds of crazy offers. You know, OpenAI said we'll give 5% of the business to the government and and then the the CEO from
Tropic was prostrating in front of like the commerce secretary. Do whatever, take, you know, we'll release, we'll do whatever changes you want, and Fable Phi got released and so on. there is this big tension and friction that is that I see, and I don't know who wins. I mean, I personally want to see open weight models win, and have you know, every enterprise should have access to these open weights models. but
Yep.
But I don't know if that is going to survive because if you have like four trillion dollars riding on the on the winner takes all thesis and these open weight models come in and undercut the entire narrative, then it's a it's a weird situation.
It's
There's a number of potential outcomes of that. Yeah. One of them is if you can't beat them, join them. I mean, do the frontier model providers, do they embrace open weight models then in order to compete against the open weight model providers? That's an outcome I haven't, you know, seen a whole lot of discussion around yet, but it might be a way for them to compromise.
There is one there is one there is one company, Thinking Lab, Thinky Labs, you know, the one of the CTOs of OpenAI, she went and she essentially took the open weights models, fine-tuned it and now it is their Thinky Labs or something. I forgot forget their name, whatever it is. But they've tried to do it. And Nvidia is now doing the nemotron. they are saying we are going to build the equivalent of open source frontier models.
to compete with. So there is like a weird mix of things that we are seeing. I don't know who wins. I I just hope that the the governments don't take actions that will restrict access to these models.
Well, I'll tell you, you know, if if AI models can find their way around containment, it's amazing how efficient an open economy can be in enabling someone to find a way to succeed. and you know, regulation often imposes an artificial restriction on what can actually be done. And if we're talking about regulation,
Yes.
That's confined to the jurisdiction within which it's regulated. If it's just the United States, then the issue that the US has to consider is we're not the only country in the world with an interest in this.
And in in you know, and in some respects the regulation is a form of regulatory capture. The the winners limit the providers to be part of the market so that they can win. That's that that is in I feel that is the undercurrent for all this discussion that is happening. The the the outcome seems to be regulatory capture. Well, you know, well let's have like the FDA for
AI models where we'll vet all these models before they go get released. And then who's vetting these models?
Yeah. Well, that's a really good question. Who's vetting them and how? And how reliable is their vetting? it's just like, you know, events in cybersecurity. And an incident will generally precipitate something sooner or later. You know, never never miss a good incident. Never fail to capitalize on a good incident if you're a CISO. I expect it'll be something similar here. If we have an incident that does demand public sector attention in order to respond to it, then we probably will see some movement.
on that front. But in the meantime, protecting markets, a protectionist approach to this, I am no economist, son, nor do I play one on TV. So to say that up front, but I'd be really skeptical about that having an impact, a long-term impact, on a market that is evolving so quickly. And we we've seen this with regulation in the past, with cybersecurity regulation, with the requirements to adhere to, you know, this discussion around PCI DSS years ago.
You're going to be defining a regulatory climate that governs where technology was a year or two years ago and is blind to where it is today. And the pace of AI is moving such that it seems like something like that could also be an unexpected consequence of trying to regulate something that's moving so quickly that regulation ultimately isn't all that successful. I I know our libertarian friends in the Valley are very strong about, you know, regulation is the wrong answer.
To almost anything. It always is, yes, of course. Tyler, we're not gonna hear anything about this today. But some's inevitable, I think. But the protectionist side of that, I'm just skeptical of how successful it could ultimately be, which of course some of, you know, our adversaries around the world are kind of banking on. That, you know, markets other markets won't be able to respond, but they're
It always is, by the way. It always is with the pockets.
Yeah.
They're making the economic opportunity available. You know, you could say that about Deep Seek. you could ostensibly say that about China, I suppose, but again, I'm not an expert in international relations, so don't pretend to be one. But it sure seems like protectionism would set up a scenario where we might inadvertently enable our our competitors to
Scott, last question for the day. You recently retired. How is retirement treating you?
Remarkably well. I'm kind of surprised to tell you the truth. You know, it's one of those decisions until you take it. you think you know what you anticipate and you you know try you you plan as well as you can. And then when it actually happens, it's like, okay, it's happened. Is this really kind of what I expected? And to be clear, I'm still very early on. It's not even been a couple of months yet. but so far, yeah, it's been
really rewarding. And part of that, I think, is because, you know, the conventional view of retirement, I don't think holds for much of anyone anymore. I might I might be wrong about that, but you know, we are at the tail end of a generation of knowledge workers that we didn't really have before. Industry analysts kind of fall into that group. Knowledge workers as as a broad swath of, you know, the the population didn't really ex exist much before.
you know, 30 ish years ago, something like that. Maybe longer than that, actually, that I think about it. So we have a generation of people who've been trained to be knowledge workers who still are pretty capable of doing that. You know, in in my generation and people who are retiring now, we still have those capabilities, we have the interest, we have the knowledge. And it's very stimulating to remain engaged to r remain engaged with that to a fairly significant degree.
Will that change over time as I get further into retirement? Very possibly so. But right now I'm really enjoying the opportunity I have to keep an eye on the industry that I've worked in for so long because I'm I I really enjoy doing it. It's very stimulating. And part of what I'm doing in order to, you know, keep a finger on AI is to do some of my own development to keep myself informed as much as anything else. So that's kind of what I'm doing.
It you know you know, Scott, one of the narratives is there is this gloom and doom narrative where well AI is going to take away all the jobs and I feel like the AI boom could not be much more it could not be better for people with agencies. I mean there are so many initiatives that you know you wanted to do, I wanted to do, but we didn't have
Yes.
We didn't have the capability to execute it because we didn't have the resources or the people or the expertise or whatever it is that you know, you know, you know, maybe you could not the right software when you were when you were, you know, when maybe you wanted to do some data analysis on a large section or large piece of data, but you didn't have data anal data scientists. but now you can. So I feel like the people with agency.
Have been unshackled to go and do things that were like not possible. And it and in some respects it should lead to more jobs in the future, but it is more designed towards people with agency and a point of view, like people like you. You've seen a lot of these things. You know what needs to be done, but you just never had the resources to execute on it. Now you need a $20 subscription or maybe $100 subscription per month, and off you go, and you have the time to execute on it.
Scott (01:00:01.201) Yeah. It's re it's
Scott (01:00:23.687) Yep. Yep.
Mehul (01:00:23.848) So you're probably the most potent knowledge worker out there. With
Scott (01:00:28.207) It's really fascinating. That you you've captured that really well because I feel like I, you know, I've I kind of like open a vista on on an opportunity that we just didn't have before. And first of all, it's a tremendous learning experience, but the a the ability to do things that you couldn't have considered doing before is just in principle, it's only limited by your interest and time.
Mehul (01:00:41.182) Yes.
Scott (01:00:53.255) And where you want to invest that time. What but your time is multiplied by the ability to call on what is very much expertise in doing the things that take a lot of time, a lot of detail-oriented work that you simply didn't have before. So that's fascinating.
Mehul (01:01:08.668) And you can because of your and because of your expertise, you can put the guardrails on the right way to do it. Because you can guide the H and the A in the right direction. Because you know what good looks like.
Scott (01:01:16.391) The interesting the Yeah.
Scott (01:01:22.351) Yes. And that's one of the findings that I'm finding in in just the experimentation that I'm doing right now is that there is really a role for judgment still. I I I have AI gathering a lot of raw information just for my own edification, just so I can keep in touch with the market because I don't work for a firm anymore. And you know, you feel a little bit blind. You do need reliable sources of information. Just for my own
Mehul (01:01:33.502) Yes.
Scott (01:01:49.817) Interest, you know, I remain an investor, obviously, I'm retired, so I have to I have to be somewhat smart about what I'm seeing. So trying to pull that information together, it's always was a huge challenge. And now I have it's like an a another right arm to do that. But I'm also finding that judgment really matters. I am every day handed up analysis that requires my judgment as to what what it means, how important is it, and what to do with it.
So it's not necessarily human in the loop to make sure things don't fail. There is that aspect of it, but more human on the loop because this is the role that has to be played in this, is that it does ultimately serve people. And you'll only get out of it what you tell it you expect, and you do have to guide it along the way to do that. And more importantly, to make it useful, you have to weigh in on that. To make it useful to people, people have to weigh in. So it's an interesting intersection of opportunities right now. And like no time.
It's kind of like when I got into the analyst business, which was still kind of new a little bit. A lot of us hadn't even heard about it before. I I had never heard of it until I got involved in having to size up the feasibility of a project when I was working in Vienna. It's like, okay, that's what that is. Well, a similar time kind of now. So this field didn't exist before. The industry analyst field didn't exist before. And I just feel very lucky in my career to have had these opportunities to be at these points in the evolution of our field to
really see these unfold. So yeah, it's hard to call this retirement. It's really interesting right now.
Mehul (01:03:22.984) Yeah, I think I think one of my guests on the podcast said and he said it really well, Halbert Flake, he said, you know, the human with a tool will always outcompete a human without the tool. So try to be the human with the tool. so that's that's a good note to end the interview, Scott. Scott, it's in it's it's my honor to have you on the podcast. Thank you for coming in and giving your time and your thoughts. this has been a fascinating interview. Thank you.
Scott (01:03:34.695) Yeah. Yeah, exactly.
Scott (01:03:50.533) thank you. Thank you, Mahal. It it's been really rewarding for me and I hope it's useful and rewarding for your audience as well, too. So I really appreciate the invitation. Thank you so much.
Mehul (01:04:02.603) Thank you.