The Physicist’s Pathway to Cybersecurity Mastery

Speaker 1: 0:00

How's it going, brian? It's really good to finally have you on. I'm really excited for a conversation today. I think it's going to be. I think it'll be an interesting one.

Speaker 2: 0:08

Yeah, thanks, thanks, joe, it's good to be here. Thanks for having me on.

Speaker 1: 0:12

Yeah, definitely so, brian. You know, why don't we start with how you got into IT? Maybe what you specialized in before that Did you start somewhere else, or were you always, you know, it focused throughout your whole life and you kind of just went into it, right?

Speaker 2: 0:30

Yeah, no, it was a bit of a weird journey. I think pretty much a lot of people who get into security have a weird journey, but for me I'm a fallen physicist, wow. So I started off doing going to TC, berkeley and studying physics, and so I have a bachelor's in physics from Berkeley, and then found that most of the things I was doing, being kind of naturally drawn to, are more related to computers, and so I ended up going back and getting into the grad program, the CS grad program at Berkeley, and went and studied there and then ended up as a professor of CS at Cornell for five years and left Cornell and started a company. Well, it's a bit of a different story. I joined a startup down in Austin that out of the ash called NetBank and out of the ashes net clients grew tipping point. So net clients was actually building these, these informational appliances. Think of it as an iPad they were building in 1999. So it was. It was a flat panel display, didn't have touchscreen in at that version, but it was hooked up over dial up line had an embedded processor. We ran our embedded OS. We built the operating system, built all the software that ran on. It was all custom software had big buttons on it so that you can, you know, access them Right. And that company went public right on the week of the height of the NASDAQ in the bubble, back in the 2000 bubble. We went out on Monday and Thursday was peak and then it was kind of like run like a lot of the dot coms at that point. And so what happened was the it was. It was not profitable and the only way to get it profitable was to do another offering. But the markets had crashed so there was no realistic way to do a secondary offering. So the board got together and said, well, we can run it into the ground, we can sell it all off and give the money back to the investors, or we can sell it all off, take what we have and go do something different. So they do chose door number three and they put literally eight of us in the room and said you have $53 million, you have 53 engineers and $70 million, go do something. And out of that grew tipping point. So it's kind of the anti startup normally have this idea and no people and no money. So here we had people and money and no idea. So we, we, we, we started tipping point and that was a company that built intrusion prevention systems, and so that's how I got into IT security, because I had a background to do networking background systems. Other people in that. That startup group had some backgrounds in security and some background in other other parts. And so we we decided to originally built this box that was going to be sort of a unified threat appliance and it was going to have a firewall and an IDS and a vulnerability scanner in it. Use the things to to reflect each other. And so what ended up happening was we looked at it and said, well, what's the IDX? Did text something? Want to block it in the firewall? No, just block it. You feed it's in the software, just drop the packet on the floor. And so we figured out how to build that thing and make it reliable and make it go online. So that was that was tipping point and build a great group of security pros there and that became what we call the tipping point mafia. They're kind of everywhere. It's been really fun watching all the different careers.

Speaker 1: 4:26

So, you know, take me back to when you got that degree in physics and then you made the switch to CS. You know computer science, how, what was that thought process? Like you know, why did you make that switch? Was it? Was it the right opportunity? Right, like kind of right opportunity came up and and you took a hold of it and you went with it. Was it difficult for you to pick up the material? Was it easy for me? Physics, back in back in my bachelors I took physics. It was like a basic, you know, physics 105 or whatever, and I would ace the labs, I mean completely ace the labs. But then we get to the test and I failed every single exam after the curve, which is really saying something. I mean that's, that's impressive all in its own right there. But it was just such a, I don't know, it was abstract to me, it was a. It was a huge word problem and I'm terrible at word problems and so, like God forbid, you throw me a paragraph with some math in it and ask me to put stuff together. I just can't do it, yeah.

Speaker 2: 5:39

Well, for me it was actually the exact opposite. I was terrible in lab but I did really well in the theory thing and I wanted to be kind of a theoretical physicist actually at the time when I went in. And so going through that I there were two things going on. One is my older brother was actually a grad student in the architecture department and so when I showed up as a freshman, he knew some people who were looking for someone that could help them set up this, this lab that they had there, which was a boundary layer wind tunnel lab, basically wind tunnel and they were trying to instrument it with and they needed a combination of instrumentation and computer programming. And so I had done enough that I could could do that with them, and so I was. I was doing a lot of programming and just kind of drawn to doing that, and in the physics I kind of liked a lot of the. I liked going to seminars, I liked a lot of the discussion, but I really didn't like doing physics. I found out like knowing physics, I didn't like doing physics, and so when I got out, as a as an undergrad, I didn't really want to to go into physics, but I didn't know what I wanted to do, and so I spent a couple years just you know, continuing to do the programming work in the architecture department. That got me into a bunch of other programming jobs, which eventually led me into the CS department as a programmer there, which is how I got into grad school, but as a side effect of that. but during that time, I mean, I was doing all sorts of things. I was teaching hang gliding, so it's just doing random things. It was a lot of fun.

Speaker 1: 7:25

Yeah, I mean it sounds like you're not afraid to try new things and potentially not be good at it. I mean there's no way that. I guess there's no way that you could have known like, hey, I like theoretical physics, I like the theoretical problems that I'll encounter right, and you excel at that. But then when you get into the practical side of it, you struggle on that end. You couldn't have foreseen that, but you still stuck it out, and which really says something, because, I mean, a degree is typically four years long and classes, especially in physics, are not push-over classes. I switched my major to criminal justice, which is a complete pushover compared to anything in science, thankfully Well yeah, I just got.

Speaker 2: 8:19

I began to realize that the other people in the physics that I was going to be competing against were just a hell of a lot smarter than I was. I just got my butt kicked. In general relativity I was just by the other guys. I think I barely got a lopie or something in that and I'd been used to getting good grades, but I think it was more just what you're drawn to. I just kept on coming back and finding that I liked doing the programming and I just naturally gravitated towards it. It wasn't forcing myself to do anything, it was just. The problems were interesting, they were tractable, they were fun little puzzles to solve, and physics is about problem solving and lots of things are about problem solving. And IT and a lot of IT at least is about problem solving. So it's kind of the same genre. I found that when I was over in the CS department there was all these conversations going on that I couldn't understand the words they were using. They said oh yeah, you just use a scatter gather vector and insert it into an implicit heap and then secondary two level hash tables solve the problem, something I don't know. It was just mumbo jumbo, so it kind of got back into CS, largely because of just wanting to understand the words.

Speaker 1: 9:45

Yeah, it's interesting. It's like you have to really enjoy puzzles and troubleshooting and get used to failing in IT overall, like even literally this week. I worked for a large automotive manufacturer and I had just something very simple and I'm a fully certified AWS guy. All I had to do was deploy an EC2 and log into it Run five commands once I log into it. And that was it. Pretty simple. You would think it's extraordinarily simple. It sounds easy. Yeah, and I spent three days deploying this EC2, and I mean, it was mistake after mistake, after mistake, and I reached out to an engineer that was more senior on the team because I'm newer at this company, and he's like oh, you're solving the problem that's in front of you. There's a whole other puzzle that you have never even heard of that now we have to go solve. And I'm like what? This is an EC2. It's AWS, like this should just work. He goes no, there's a lot more going on that you don't know about. And I'm sitting here like I'm the stupidest person at this company right now. I can't even do my basic functionality of my job. Like what am I?

Speaker 2: 11:14

doing. Well, that's one of the things that I've seen just over the years, watching IT kind of grow and develop. Because when I started we were relatively things were honestly just kind of simple. There were C programs, you compiled them, there wasn't that much going on. And now we've built layers upon layers upon layers of abstraction and stuff and you kind of need it to run. So all the stuff going on the cloud and when it works, great it works. And then when it does sense, there is just so many things that could have gone wrong and so many of them are completely opaque. Just it's hard to even know where Look. So you end up developing just a lot of experience and saying oh, I bet I know what that is, type things.

Speaker 1: 12:01

Yeah, there was probably 20 or 25 things that I had to check that I knew about, and then my other engineer gets on the call and he goes oh no, there's 15 other things that we need to look at right now.

Speaker 2: 12:15

This is crazy. We were trying to get this one just the other day. We were trying to get this machine to launch in a certain availability zone within a region in AWS, and so it was USE 1A that we had to get it to launch it to, because that's where the disk that we wanted to mount was, and we had the launch template set up and it was just ignoring it. It would come up in whatever zone it wanted to, and so we'd check it and have a crash and check it, shut it down and after five or six tries it would randomly get the right one. We finally found out that the thing that was wrong is we hadn't set the subnet explicitly in the launch template, seemingly not documented anywhere that you have to do this Such a smooth thing. And when it does now it comes up in the right zone. But it's random stuff like that, yeah. And so many of that you don't even know what. I think we're getting more and more things where we don't even know exactly what's wrong. We just find other ways to skin the cap to solve it.

Speaker 1: 13:21

Right, you mentioned when you were at that startup company and the board came together and they had three options. It sounded like it kind of sounded like the option that they went with was more like an incubator. That's the feel that I got, because they gave you a bunch of money, a bunch of engineers and said figure it out, create something new. Is that what you? Is that how you felt it was at the time as well? And how did you overcome that creativity part? Because I feel like, even being prompted with that situation, there's a creativity block right there. It's like, oh no, now I actually have to be creative, I have to come up with something new, or whatever it might be.

Speaker 2: 14:07

I feel like that could be an issue. Well, in that case, what we did is we split up into four groups of two and each of the group was responsible for coming up with one business plan a week, and so we kind of cranked through them and we must have gone through like 100 different plans before we landed on the right one. Wow, we thought and it was kind of a combination of looking at the businesses where we knew them well enough that we thought we could succeed, as well as what the talents of the team were and kind of what our assets were. So it was kind of intersecting all those and taking an idea and then just kind of running it to ground, and it probably took like 10, 12 weeks to, you know, sort it out and figure out the right one. It was a pressure cooker, that was a real pressure situation, but it worked out.

Speaker 1: 15:07

Do you do you think that there is Potentially some unique skills that you learned during that time that you're now utilizing today?

Speaker 2: 15:17

I yeah. So I kind of came into that as having been a you know CS professor and you know all these other things. So I didn't know much about the business side and I didn't know much about a lot of the other Aspects of running a startup. Because that first startup that was doing the iPad type things I came in as, just like you know, essentially the lead architect to help build out the system, but in tipping point you know as much more, being one of the founders essentially coming up with the idea. And then I Got to do kind of every job in the company practically Felt like at least I was. I ran one of the engineering teams for a while. I Ran, I wrote, I was product manager for a while I was essentially out in the field Working with sales, the sales teams, for a while I was, so I got I got a really broad exposure to do all of the different aspects of it and that helped a lot when we went to start the, the next company and the next company after that. Now I'm on, I'm on my third company now, but they've all been in the cyber security space and I think that the but it's it's such an interesting and deep field Because partly, just like all things in computing, it's constantly changing right, the technology Changes and aspects of a change so new solutions become possible that they just weren't there before. But it's got this added dynamic of a kind of a cat in mouse game between what the attackers are trying to do, when, what, how we try to defend them, and it's that the second company that we we sold. We had a second company was a company called click security. For tipping point got bought by three com, which ultimately became part of, got sold off into turn micro. The second company we did was something called click security and that was back in 2009, essentially an early day XDR. So we called it something different because the term XDR had a big point, but it was. It was in that same realm and that got sold to a company called alert logic down in Houston and I got really to spend a lot of time watching. We became the front end to their sock. It was managed security service and so we got to watch how they how kind of Frontline security analysts in the sock use this stuff day in and day out and that's a. That is such a treadmill job. You know, just pick up an alert, spend 15 minutes trying, and, you know, trying to figure out if this thing's real or not, move on to the next and just because they're just constantly in triage and it's exhausting and it's pretty repetitive and there's a really high burnout rate in that particular, that particular job. So it's been was really interesting just watching all these different aspects of the field and trying to figure out how to solve it better.

Speaker 1: 18:17

Yeah, that is, that's extremely true with. With the sock, like it's the burnout rate on that team alone, I is it's insane. I've never, I've never seen something like it before, and when I worked for a credit bureau previously, you know they had their own separate sock that even people in the security team couldn't get into the sock, like they didn't have access to get through the door, and I mean it was. It was intense. You know, those guys they do not mess around and they're they're constantly going for their entire Shift or whatever might be right. I mean it's just, it's nonstop. It, like you said, it's alert after alert after alert, investigating things, doing deep dives into things. I mean they, they told me about something that was Launching in my browser that I I had no clue was even there. I mean I'm a security engineer, right, so I should know what's going on on my computer. And they like confront to me because they're like there must be like a trojan on your computer or something like that that we haven't seen before. And they looked and they like pointed out you know the binary and everything that was running. And I'm like, dude, you guys are on a totally different planet.

Speaker 2: 19:33

Well, I remember just even talking to a lot of the security professionals, just as In the very my various roles at the companies, I would, I'd get these, you get these war stories about that attacks, some of Really smart and sophisticated it's and you kind of think about, hmm, I'm not sure how I would defend against that. Like one of them. There was a guy who told me about this, this Backdoor that they had planted into a browser where on rent and random, seemingly random intervals, it would go out and fetch data from this website and and pull it down. And it was just a regular looking website. They had also compromised the website and what they did is they put in the comment stream of the HTML that commit the they command and control instructions. It was a pseudo random number generator that would figure out when it would randomly go out to the website and they had the same random number generator running on the website. So about three minutes before they would modify the website with the instruction, put it in the comment section, it would get downloaded and then it would get deleted off the website after Got got things, so you can never see it. It's kind of like, how did you find that? And it was all being drifted and geez.

Speaker 1: 20:53

Like, even if an alert was triggered on that and you start investigating it, by the time you even start to investigate it it's already gone.

Speaker 2: 21:01

Yeah, yeah. So I kind of realize. This is kind of the thing that I ultimately realized a lot about this is that a lot of the way we've been doing this this is Kind of upside down and backwards, because what we do is we we get an alert and then what we do is we try to develop context for that alert. So we go because An alert coming in is like some Listening to a wiretap or something, and you hear the word assassinate and bomb and president, or something like that, and you're trying to figure out is this a plot, or is this a bunch of academics reviewing a you know historical paper or what's? What's the context? You can't really understand what that means until you establish some context. And so what this the sap guys are doing and if you step back, is really taking that alert and trying to develop enough context around it to say what is the attacker trying to do, or is this even an attack, or is there some mitigating factor? What kind of what's going on? That's that's that's sort of their job. So If you can automatically provide that context and this is sort of the idea of what I've been working on recently is try to build up the context for everything that's going on in the system. Then when you get first off, you can use that to detect a bunch of stuff. But then also when you have a detection, since you have the context already automatically built up, you can have a lot of rules that just say, oh, in that context this alert doesn't mean anything, and so you can really tune out the pulse positives. That way, and because you can tune out the pulse positives, that way, you can turn up the signal because you're not getting drowned in false positives. And so the combination of those means you can turn up the signal so high that it's almost impossible for an attacker to get in and wander around. You're no longer trying to detect malware or something like that, you're just trying to detect almost anything weird going on and then using the context to filter that out and the context to group them together and do a lot of that work that the SOC analyst does in triage kind of automatically. So you end up with it's a pretty neat system and it makes the SOC analyst job a hell of a lot more fun and it just leads to a lot more efficient, better, cheaper, faster, stronger security etc. So it's been 20 years trying to solve this problem.

Speaker 1: 23:40

Yeah, you know, whenever I demo product or anything like that or POC it and if I'm getting so many alerts that I don't know what to pay attention to, it's actually just qualified immediately from my POC. Because if I can't figure out in you know a couple, maybe one or two minutes of if I need to pay attention to it or not, then I know for sure I'm going to be spending my entire day in a tool that's generating nothing but a bunch of garbage alerts that I don't need. That has been the case one too many times in my career. Now that I'm beyond the alert assessment part and I'm on the, you know more of leading engineering teams and whatnot, yeah, now it's about kind of lifting that burden for my SOC and from everyone else saying like, yeah, you don't need to spend your time on these alerts, you can go do other things.

Speaker 2: 24:38

There's better ways to solve this problem. I think a lot of people get one of these tools and they say, oh, this is really cool. And then after a while they realize this thing's like my Christmas puppy. You know, it's things a lot of work. You know, take care of this. Or you know, babies are like that. Actually, babies are a lot of fun.

Speaker 1: 25:01

I understand you're a new father relatively yeah, yeah, almost six months now Cool.

Speaker 2: 25:08

Is this your first year? Yep, that's very cool. It's a journey.

Speaker 1: 25:15

Yeah, it's been a lot of fun, a lot of hands-on learning, I guess, and like finding out that I don't need sleep that bad, I guess.

Speaker 2: 25:27

It's a sleep deprivation experiment. Someone once said babies are all joy and no fun, and sometimes it feels like that for the first year and then it's funny how you look back on it and it's really neat. It's also a neat time because it's one of those kind of transition periods.

Speaker 1: 25:47

Yeah, yeah, she was born in March. And I mean the first two months. Like I don't even remember like anything that happened in those first two months, because, like I was talking to someone A couple of weeks ago at work and they said, oh, don't you remember this thing that happened in April. You know, like it was going to affect the criteria on this other thing. And I was like, dude, I don't remember April existing. Like my memory doesn't kick back in until the middle of May. Okay, that's where my brain's at right now. And it's insane because you know previously, right, like if I got four hours of sleep in a night, like I'm exhausted, I'm beat up, I need to take a nap during the day, whatever it might be. Now, if I get four hours of sleep, I'm like, oh, I'm well rested. I'm actually ready for the day.

Speaker 2: 26:41

Kids give you perspective. My oldest, my first kid, was born on September 8th 2001. So the first day home from the hospital was the night of September 10th. So I wake up in the morning on September 11th and turn on the news and it was, you know, in the sleep deprivation state that you're in, and it was very Alice in Wonderland. That was a very strange experience.

Speaker 1: 27:11

Yeah, I bet man that's a wild time to be born. You know, I forget like how long ago that actually was, because I was in fifth or sixth grade at the time. You know, I think fifth grade and that was just such an absurd moment.

Speaker 2: 27:32

Yeah, it was. It was 22 years ago now and almost exactly 22 years yeah.

Speaker 1: 27:39

But anyway it was.

Speaker 2: 27:41

You know, we, we, we muddled through that and we muddled through a lot of things, yeah, yeah, it seems like we're just like getting through the next thing.

Speaker 1: 27:50

One thing comes up. You just got to get through it, you know, one day at a time, but you know to. To circle back, I guess, a little bit. You know your current company is called Spiderbat, yes, sir, and I saw that it looks like it is focused around protecting on runtime, application security right or detection at the runtime, which is really it's an interesting place to be and, coming from someone that is not a developer, that's always an area where it's kind of like it's like it's the expertise that I'm not a fan of, right, like I need someone else to dive into it. But from just looking at you know, the website and whatnot, right, looking at the product a bit, it looks like a pretty easy platform to kind of dive into and learn it and understand what it's like. Was that, was that a core principle of when you were designing this product, that it had to be something that you know virtually anyone could pick up and figure out what's going on?

Speaker 2: 29:01

That's definitely. One aspect of it is that you know I came out of these, this set of experience, from building iPads basically to to building things that the SOC uses and ever since the beginning I've been very much into kind of user interfaces building very easy to use user interfaces. In fact, that's what I was studying at grad school in at Berkeley was user interface design. So that's always been very near and dear to my heart and it's such an important aspect of it and so many of the tools are bad, so you kind of want to build something beautiful, yeah. But I think that the core idea of the runtime security, if you think about it, is we're building runtime security for for Linux systems, in particular for Kubernetes. So let me back up just a second and kind of explain what that means. And I think about the world of security I mean it's such a broad thing because it goes all the way encryption and governance policies and things like that but broadly speaking you can break it up into left of boom and right of boom. So left of boom are all the preventative things trying to harden your world up, trying to prevent things from bad things from happening, that's. That's the left of boom stuff. The right of boom is something, something they got in anyway or something bad happened. How fast can you detect it? How fast can you respond? How, how effectively can you, can you get in there? And as an industry we suck at it, honestly, so we're pretty terrible. The average time to detect, I think from from the time when you kind of go back to these breaches, it's three months from the initial break To when they're actually detected. If it's a, if it's a real breach, it's 93 days is the call time. So it's it's kind of crazy. And then, because they're all over your network, the investigation time is really long and because then there's such a mess that the delousing time is really long. So it's like a year and a half. And, yeah, exactly. So the the right, the idea of right of boom is you want to catch them as close to right of boom as possible, because then it's little damage is done and it's relatively easy to clean it up and so on. And then what you discover as things go right of boom should inform the preventative side. So the preventative side tries to prevent boom from happening in the right of boom. You learn from those mistakes and say this is what I fix up. But the way that the problem with the left of boom is, our systems are so complicated, the attack surface is so huge that in prevention is almost perfect. Prevention is like just impossible. It's like having the entire continental US border and trying to prevent all possible ways of anyone from ever entering the country through any through underground tunnels or boats or on any shoreline, on some remote shoreline or whatever. It's kind of that's not going to happen. You're not going to be able to, you know, prevent everything. So you want to have something where you can focus. The most likely area is what people are gonna, you know what, where to best spend your prevention but then have really good detection. And so the same thing happens in in cybersecurity. I could try to patch absolutely Every vulnerability in my product, for example. That's been kind of the the latest way, that latest wave is the ship left. But the problem is is that only covers some of the things, right? What about all the unknown vulnerabilities in the product? They could be in libraries that could be in your code. Who's doing all the research to find the unknown vulnerabilities in your code? Because all the release are just bugs. And do you ship code code? That hat is a hundred percent bug free. Do you never find bugs at runtime? It's. It's a lot like saying that what I want to do is make my stuff so perfect that I won't monitor any of the systems running for crashes or reboots or out of memory errors or something like that, because Because I'm perfect and it's, it's doesn't work that. Instead, what you do is you build a good monitoring system, good testing, testing, monitoring systems, and you try to you, you detect where it, what's going on, and use that. If you can detect it really fast, you can recover from it, mitigate the problem before any damage is done, and then you use that to focus where you want to, where you want to spend your hardening efforts, and it's just a hell of a lot more cost-effective, more interesting, and you get to spend more time developing Features and things that benefit people rather than just trying to protect all, spending all your money and time trying to protect.

Speaker 1: 34:14

Yeah, that's that vulnerability management. It's almost like a. It's like a legacy process or mentality, right where, in the beginning of my career, I had a boss that if it didn't have a CVE attached to it, he didn't care about it, he didn't want to hear about it, he didn't want to know about it, like none of it mattered, you know. And I had to sit him down and say, like you know, can we both agree that, like me, gaining root without putting in a single password On this server is a bad idea, right, like that's what I want that. And he agreed to that. Then I said, okay, well, there's this thing that doesn't have a CVE and I just used it right in front of them and I got root on the server that I should have been able to get root on. And he's like, oh, how'd you do that? I'm like there's no CVE, I don't know what you're talking about. You know it was like kind of open up his eyes right to like, oh there's, there's more threats out there than then what I've been told before, what I've been used to. And it was that mentality shift, though, that it took to actually Like open someone's eyes and say like no, there's a whole lot more. Then just these CVE's. Even now, you know, even now I I encounter it where people are caring about the CVE's, you know, way too much. It's like, yeah, the known stuff I'm not concerned about, like MacaFee can catch that I'm not concerned about it. You know, not saying that we use MacaFee or anything which we don't, thankfully, but you know like the lower end endpoint detection response platforms can catch that stuff. It's the stuff that I don't know about that worries me.

Speaker 2: 36:02

Exactly, exactly. And so the way that if you look at, if you look at the attacker in the pre attack stage, you've got to defend against everything that they could possibly do In the post attack stage. They're on our turf, so they have to not make any mistakes to avoid getting detected. And so they. You know they'll do fileless malware and or file less attacks and living off the land to type of tax. But even that's really detectable and hindsight. And that's where the context comes, because if you see, if you look at what attackers do after they get in the system, it's kind of like I've been teleported into this room. That's a dark room and I've somewhere in the Pentagon and I don't know where I am. In their traps they were what do, what do I do? And so I might start looking around. Maybe you know light, a match or something and try to cautiously look around and then be really cautious about trying to move around. But very first, things I'm doing are just trying to orient myself. So they'll try to figure out what user am I running as, what privileges do I have? What are the other systems around? They do this and and do that in very subtle ways. But all that work is something that an ordinary program would never do, an ordinary user would never do, and so if you see a bunch of that type of activity linked together, it's almost certainly that someone's broken in it's. It's very, very suspicious activity. The problem we have is we can't when we get, we can't alert on all that stuff and have humans do it, because it happens too often. I actually knew a guy. There was a project at Google that you know tried to do anomaly detection, detecting these, these kind of weird things happening, and the problem is they did a really good job of doing anomaly detection but they were flooding the sock because at Google, scale anomalies happen all the time. One in a billion things is like a thousand a day. It's just the scale kills you, and so what you need is to be able to chain together those anomalies, to put them together into a bundle. In order to do that, you need enough context to be able to say these things are all related together. That's the anomaly side of it. The other side of it is being able to say characterize what is, what is normal, what is expected behavior for the system, so all, and then saying if it ever drifts from that, alert on it. So for example, if I have a container image, it runs three or four programs, it talks to three or four services, east, west, at north, south, and I can cattle, I can watch it and I can catalog all that behavior and after that I can say and I can build that into human profile, human readable profile. And after I put that in I can say if it ever does anything different than that, you know that then it's something bad. So now an attacker gets into that and they start trying to just like that lighter and look around a little bit. Well, it's all gonna be stuff that that thing is never done before and so that's gonna. That's the. That's drift could be called drift detection, but it's also kind of this really targeted anomaly detection that isn't generic, but it's really kind of trained on what, what you're, your, how your program works, and you integrate that into the development life cycle and now a developer can look at that thing and say, yeah, that's how my program is, all that's expected, and so you can get this loop that's closed and it's fricking almost impossible to break out of that that thing. As soon as you start doing anything, you can react to it and say that that's definitely bad, should it? You know, kill the pod, kill the process, kill the container, whatever you have.

Speaker 1: 39:44

And we're actually really interesting, and that's just right, a boom. Hmm, so is there a learning period where you know you deploy it in the environment and then is it learning, you know, for 90 days, right, or whatever it might be, where it's saying this is normal, this is expected behavior, then, based on everything that we've seen everywhere else, this is also normal and this isn't, you know, is that learning phase there?

Speaker 2: 40:13

It's not exactly it's. It's similar to that, but not quite so what. What you do, what spider bat does, just real briefly, is you deploy this, this agent, on on a node Kubernetes node or just a regular VM and it starts recording everything that's going on the node, every process, every network connection, every file access, and links them together. So that I know that this process launched this process which created a listening socket, which accepted a connection, which then opened this thing, which made a connection over to this other process on this other machine, which accepted the connection and then downloaded this file, wrote it in as a crown job. Three days later that crown job executed, and so now I can chase together this causal chain of what caused that crown job to execute. So that you can build that graph and you build that automatically. Once I'm watching, it's the mother of all root cause analysis. How did this thing get started? Why is this thing trying, you know, transferring data over here? And, oh, bob installed that script two days ago. There's a typo in the script, that's sort of that. That's doing doing stuff. So, as a mother of all root cause analysis. But then you can also look at that from a different lens and say, let me take an image run and say, for this run in this container of this image running, build out what it does, and then another and another, and you just do that on your development system. So now I know, after after 50 or 100 runs of the developer running all the tests and running it in their integration environment, I can merge those all together to create a merged baseline that a human can read and the human, the developer, can look at that and say, yeah, that these are the processes runs, these are the uses they run at, these are the executables, these are the connections each one makes, and you can bound it, outbound and generalize it as needed and then say that that is, that's my profile. Now you put that back in and then every time a deviation comes in, open it up as a pull request and get up. They modify the. Your thing is drifted. This looks like what's changed. Either accept the pull request or you know it's something bad has happened. But the developers in the loop and they know how the program works. So it's not this poor guy in the sock trying to reverse engineer what's a developer had. The developer is able to say this is the way it works. Once you have that, then that becomes something you can use in the staging environment, the production environment, in the same way. So it takes maybe about a week or two of learning period. But it's not learning an image, a particular instance of an image. It's learning what your program, what each program does, and then after that it's very stable. It takes very little effort to maintain it and it becomes really hard to break out of it Just because you know almost as soon as you get to get a foothold in there you're going to get caught.

Speaker 1: 43:22

Yeah, it's really fascinating because you know you're kind of shifting left in the perspective of who's responding to it, you know, and who's like kind of on the hook for it first. at least that's been a real headache that I have personally experienced and that I know others are going through, because you know, it always gets pushed back on the security guy right to figure out what's actually going on, when in reality we didn't write the code right, we didn't create this application or whatever's going on, like we have no clue what is going on there in that 10,000 lines of code or 100,000 lines of code, like who knows right, and so it always puts us into a very difficult situation of where, you know, we don't know what's going on, right, because we haven't been a part of it and it's our responsibility to determine it and, like, figure it out. You know all in 15 minutes, all in 20 minutes, you know which? I mean that's a really fascinating way of approaching this problem. Was that also a perspective or a thought that you had when you were creating? This is, like you know, let's go to the source, right, and I would actually know what this is doing.

Speaker 2: 44:43

It was partly that and partly it was based on kind of changing the conversation. So when I was a tipping point, we built this box that was an inline network inline box. Conceptually you cut in network wire, you plug both ends of the box and it's a bump in the wire but filters out the text. Okay, and one of our customers, a very large company, said that it completely changed the way they acquired other companies. So their old process was they would buy, they would like buying. A company. Week literally was such. It was a huge company and whenever they acquired a new company the networking team would have to go in before they could connect the network. So there's a lot of pressure to connect the networks together to start realizing the value of business. But there's a security risks of just blindly connecting them up. So they used to have to go in and do this kind of full audit of every system they had and review all their security plans and stuff. And it was just this deep dive exam, that no one. It was a painful conversation. It's like having the IRS audit type feeling you know, it's just no one, no one wants it and the IRS guys are not happy about it either. No one's happy in this. And so instead, what they do is they just put the tipping point box in there between them and connect them together and now they could say, okay, this machine is infected with this, this machine is infected with this, this machine. And so they go back to that same group with this very actionable. These things are problems in your environment that you can fix. And it was a much better conversation. So it's just kind of flipped the script and the same thing is happening here. You know, if we go, if we try to solve the vulnerability side or the way a lot of security groups work, they go to the development organization and try to embed themselves in and get them to develop a lot of process and review CVEs and for scanning all their coons. Tell me why this vulnerability that you know is in your code is can't be exploited, and they're like I don't even know one uses that library, so I don't, I don't really need to worry about that. And so you go this kind of a painful conversation. Instead you go to them and say, look, I'm seeing your application behave exactly this way and they look at and say, yeah, yeah, that's expected that. What the heck is that? And as a developer, they want to know about it because now, all of a sudden, it's a puzzle, it's an interesting thing, and then they find out more about how the how the code or some library that they're using actually works, which is is usually interesting, and if it's anything bad, they can catch it immediately and they're not reacting to it in a fire drill with the security team after some breaches happened, which is no one likes. So it's changing this security process from a bunch of things no one likes to something that that's kind of, frankly, more fun and easier, better, better conversations.

Speaker 1: 47:45

That's interesting. You know, brian, unfortunately I think that we're coming to the end of our time here, but I feel like I could talk to you for another hour, you know. So that just obviously means that you'll have to come back on some time and we'll talk more.

Speaker 2: 48:02

Absolutely. Anytime I'm joining the conversation, it's these things. It's always fun, awesome.

Speaker 1: 48:09

Well, brian, you know before I let you go, how about you tell my audience? You know where they could find you, where they could find your company spider bat, and you know all that good information if they wanted to learn more.

Speaker 2: 48:20

Yeah, so the name spider bat. Actually we're based in Austin. We're actually fully work. We're a virtual company, but that the founders based out of Austin, and Austin has bats. They have this largest colony of Mexican free tail bats in North America. About a million bats live under the Congress having a bridge.

Speaker 1: 48:41

It's kind of neat.

Speaker 2: 48:42

If you're down there, you can go out at night on the shore and the bats fly out of, starting about 30 minutes before sunset and it's just waves of bats coming out of it. It's really cool. So there's type of back called a spider bat that eats each spiders and so we're founding the company. You know, we thought that was just a neat name and kind of kind of cute. This was before COVID, when bats became a little less popular, and so we're going to open up the account and the guys said you know what type of business you're in? Oh, cybersecurity. So you were at spider bat with the Y, spy D or, and so that's how we the name got got turned into spider bat with a Y, but anyway, so that's, that's the name of the company, spider batcom and that with a Y, so you can be able to find it and I'm on there. I'm just Brian at spider batcom or B Smith at spider batcom, either way, both work and so you can find me there. And we have you, I'm on LinkedIn, I'm on, I'm on all the usual. So they can, you can, you can find me and I'm interested in hearing you know from anyone. I enjoy talking, as you can tell. I enjoy talking about cybersecurity and kind of most things interesting in both technology and physics. So it's still like physics, I still, but it's a hobby.

Speaker 1: 50:07

Not a job. Yeah, I can't imagine that being a job. But that's just me, I mean, someone that failed physics. Well, yeah, I mean, that's fantastic. You know, and Brian, I really appreciate you coming on and I'll definitely put you know all of your links in the description of the episode. So if anyone wants to, you know reach out or check it out. They absolutely can. But thanks everyone for listening. I hope you enjoyed this episode.