Curious about the real history of artificial intelligence and how it has woven itself into the fabric of modern life? Join us as Erick Galinkin returns to share his insights on the evolution of AI, from its early conceptual stages to its present-day applications like self-driving cars. We promise you'll walk away with a deep understanding of the various levels of autonomous driving and the enormous strides AI has made, surpassing even the most ambitious expectations of the past. This is not just a technical conversation; it's a philosophical journey questioning AI's origins and contemplating its future.
Discover the transformative role of massively parallel processing in AI, especially within computer vision. Learn how CUDA, initially designed for computer graphics, has become indispensable for deep learning by efficiently handling complex computations. We break down neural networks and activation functions, explaining how frameworks like TensorFlow and PyTorch leverage specialized hardware to achieve remarkable performance improvements. If you've ever wondered how deep learning mimics human neural behavior or how AI-specific hardware is optimized, this segment will be invaluable.
In the latter part of our episode, we tackle the intricate relationship between AI and cybersecurity. Hear about the challenges of training machine learning models to detect malware and the dual-use nature of AI models that can serve both defensive and offensive purposes. We shed light on the complexities of securing AI systems, emphasizing the need for specialized risk management strategies distinct from traditional cloud security. From tools like Garak to frameworks like Nemo Guardrails, we explore various solutions to secure large language models and ensure they operate safely within an organization. This episode will arm you with the knowledge to understand and mitigate the risks associated with deploying AI technologies in your own projects.
https://github.com/leondz/garak
https://github.com/nvidia/nemo-guardrails
Follow the Podcast on Social Media!
Instagram: https://www.instagram.com/secunfpodcast/
Twitter: https://twitter.com/SecUnfPodcast
Patreon: https://www.patreon.com/SecurityUnfilteredPodcast
YouTube: https://www.youtube.com/@securityunfilteredpodcast
TikTok: Not today China! Not today
Speaker 1: How's it going, eric?
00:00:01
It's a real pleasure having you back on the podcast.
00:00:05
You know, the first one kind of like cracked my mind open, so
00:00:09
to speak, with AI.
00:00:11
You know, I think when we were first talking, openai had come
00:00:16
out with their version 3, right, and you know, now they're on
00:00:20
version 4, with 5 quickly coming out.
00:00:23
I think is what it is.
00:00:24
It seems like ai is and it's like it's it's in its infancy.
00:00:32
You know, in terms of technological evolution I would
00:00:35
say it's like in its infancy, but the impact that it's having
00:00:42
is growing significantly and it's rapidly changing.
00:00:45
Would you agree with that?
00:00:48
Speaker 2: Yes and no.
00:00:49
So very happy to be back.
00:00:51
I've seen that you've done a couple episodes on AI since I
00:00:54
was first on, thrilled to see that.
00:00:56
I think what I would disagree with is that AI is in its
00:01:02
infancy, right.
00:01:02
Like AI is a pretty old field of computer science.
00:01:05
I mean we've had artificial intelligence in some form or
00:01:09
fashion for 70 years, 60 years, depending on where you want to
00:01:14
start counting.
00:01:15
But we have seen this pretty explosive growth in large
00:01:19
language models and applications of like transformer-based
00:01:25
architectures, right.
00:01:26
So, without getting too in the weeds, this particular type of
00:01:30
neural network, this transformer-based architecture,
00:01:34
has seen a ton of growth in computer vision, natural
00:01:37
language processing, right, all of these things.
00:01:39
So I think we've definitely seen a lot of growth and
00:01:43
evolution and adoption.
00:01:44
I think adoption is the big thing, right.
00:01:47
Like we've had AI.
00:01:49
I mean you look at like anomaly detection, right, like good
00:01:53
old-fashioned anomaly detection, all the way back to like PID
00:01:57
controllers, right In like electrical engineering, because
00:02:00
you know, can we use functionally as anomaly
00:02:01
detectors, you know, can we use functionally as anomaly
00:02:03
detectors.
00:02:03
You know, those things we've had forever.
00:02:07
It's the fact that it's so on the surface now, right, rather
00:02:13
than living deep in a process where it's like yeah, I don't
00:02:16
know, the anomaly detector told, accustomed to dealing with
00:02:26
right it's.
00:02:26
You give it text, the way that you would give a person, you
00:02:30
give it an image, right, and I think that's kind of the big
00:02:33
difference and the big evolution especially in the last, like
00:02:41
three to five years.
00:02:42
Speaker 1: Well, you know, I feel like when I say it's in its
00:02:44
infancy, I'm thinking of, you know, seeing where this is going
00:02:48
right.
00:02:48
What's that end state?
00:02:50
What's that end phase look like compared to right now?
00:02:54
You know, and right now that, like, that end phase or that end
00:02:58
state is going to make this look like an infant, it's, um, I
00:03:07
, I feel like it could be that advanced.
00:03:07
I guess that's where, um, you know, I was getting that idea,
00:03:09
but absolutely I I've.
00:03:10
I've actually read a lot, a lot of different opinions on, like,
00:03:15
when it started and how it started and things like that.
00:03:18
And you know, sometimes it's hard to even like, put that
00:03:22
together right because we're so used to this.
00:03:25
You know, I guess, ai advancement, uh, that it's hard
00:03:31
for me in some cases to say, oh yeah, it started 70 years ago by
00:03:35
this thing that, you know, is trivial today.
00:03:39
So it's, it's it's hard to like that dot together, I guess.
00:03:44
Speaker 2: Yeah, I think that's true, and this is where the
00:03:48
science and philosophy start to meet.
00:03:50
So I think two of the things you said make me feel that way,
00:03:55
and, of course, the first is when did it start?
00:03:57
Does it start all the way back with polynomial regression,
00:04:05
right, where you have an algorithm for performing
00:04:07
polynomial regression?
00:04:08
Is that blemish to you?
00:04:10
I don't remember Something you can look up, it's fine, right,
00:04:13
but you can go back hundreds of years if you really want to.
00:04:16
And then, on the other hand, right, like modern machine
00:04:19
learning, do you start with computer vision, do you sort of
00:04:22
acknowledge that, or do you start with vision?
00:04:24
Do you start with Outlook set, or do you start with you know
00:04:30
where do you start?
00:04:30
I think that's where it starts to get into the philosophy.
00:04:32
And the other place where I think it starts to get into
00:04:34
philosophy is like what is the end state?
00:04:35
Because there are, I think, these varying schools of thought
00:04:39
about okay, so we have this, you know, exponential growth, or
00:04:43
this, you or this super linear growth, or whatever.
00:04:45
Is that sustainable?
00:04:49
And for how long Is there a period where it's going to level
00:04:53
off?
00:04:53
Because I mean, I've been doing this long enough to remember
00:04:57
gosh back in 2012, when people were saying like, oh yeah, like
00:05:02
computer vision is practically solved and we'll have full
00:05:06
self-driving cars within five years.
00:05:08
And here we are, in you know 2024 and you know, say what you
00:05:13
will about Tesla, we can leave that whole conversation aside.
00:05:16
But even even that isn't full system.
00:05:19
Five self-driving Right, it's just advanced lane control and
00:05:25
speed control.
00:05:26
Speaker 1: Um isn't that considered to be like, uh, like
00:05:30
I think I read it was like level 3.5 or something like that.
00:05:33
I mean, I I'm not really even sure what the levels are.
00:05:36
If, if you don't mind, would you be able to go over it real
00:05:40
quick, like just a quick gloss over it yeah, so this is this is
00:05:43
not my ultimate area of expertise, but essentially level
00:05:46
one is like cruise control right.
00:05:50
Speaker 2: Level two, uh, operates on two axes.
00:05:53
So that is the adaptive cruise control right, like you know
00:05:58
even my, you know, toyota corolla has that where it's like
00:06:02
there's a car in front of you so we're going to slow down,
00:06:05
there's a hospital in front of you, so we're going to slow down
00:06:07
, even though cruise control is set to 55 or whatever.
00:06:10
And then the lane management.
00:06:19
So that's forward and back and left and right, that two axes
00:06:23
when it says okay, I know where the lanes are and I'm going to
00:06:24
keep you in the lanes right.
00:06:25
And then level three and four are blurry to me.
00:06:27
But level five is like no human intervention whatsoever,
00:06:34
general purpose anywhere, right, can go anywhere, can do
00:06:37
anything.
00:06:39
Speaker 1: If.
00:06:39
Speaker 2: I remember correctly, level four is that it is able
00:06:42
to drive under any conditions within a restricted environment.
00:06:45
So if you think about, like Waymo's robo-taxis in San
00:06:49
Francisco, those are, I believe, level 4 systems, you know, but
00:06:54
those also use, right, a lot more than computer vision.
00:06:57
They're still using LiDAR, they're still using all of these
00:06:59
other things, so it's not something that is nearly as
00:07:04
advanced as we kind of hoped it would be right, these
00:07:09
self-driving cars that use computer vision systems, they
00:07:14
are still struggling with things , like you know.
00:07:17
I mean, they struggle with object detection, right, like
00:07:19
they aren't even exceptional at object detections and they're
00:07:24
even worse at object detection when, like, it's raining a
00:07:27
little bit or it's foggy, or you know it's late at night or
00:07:32
there are, you know, streetlights are out or whatever
00:07:34
.
00:07:34
All of these things are just limitations of the system, which
00:07:40
isn't to downplay how incredible it is that it even
00:07:42
exists, right, like that's cool, that's amazing.
00:07:46
But there are limitations to these systems and I think that
00:07:49
if you look at the progress in computer vision, we did see a
00:07:53
huge spike after, like AlexNet and all that, and then over the
00:07:58
last several years, probably since the vision transformers
00:08:02
were the last, I think like really big advancement, and if
00:08:06
you have listeners who are deep into computer vision, they can
00:08:09
write me nasty emails or LinkedIn messages and tell me
00:08:12
how wrong I am, but those systems haven't really continued
00:08:18
to grow at that rate.
00:08:19
There's all this huge explosion and then it's a S-curve right a
00:08:25
it's a sigmoid function.
00:08:26
It's which is something that we in our official intelligence
00:08:29
should be really familiar with right, you have that flat line
00:08:33
and then exponential growth and then a flat line right.
00:08:35
So the question really is like, if it is an s-curve right and
00:08:41
not exponential growth forever which I think is a tough gesture
00:08:47
to justify, right, that's a tough assumption to make.
00:08:49
The question really becomes where on that S-curve are we
00:08:53
right?
00:08:53
Are we toward the bottom or are we toward the top?
00:08:55
And that's something that is hard to know until we really
00:08:59
experience the state where it's like okay, yeah, this thing came
00:09:04
out and it was trained on 10 times as much data with 400
00:09:08
times as much compute and it's only like 20% better, like okay,
00:09:12
well, now I think we're hitting the point where we're at the
00:09:15
top of that curve.
00:09:18
Speaker 1: Yeah, you know it's.
00:09:19
It's like that saying right is hindsight is 2020.
00:09:24
Right, right.
00:09:25
You don't realize how easy things were, right, For instance
00:09:29
.
00:09:29
You know, when you're a kid, when you're a teenager, all you
00:09:32
want to do is become an adult, right, you get to drive, you can
00:09:38
travel, you can do whatever you want, you make your own
00:09:40
schedule, whatever it might be right, and you become an adult
00:09:43
and you're like man.
00:09:44
I wish I was a kid again, you know, like, like I didn't have a
00:09:49
mortgage, I didn't have to worry about paying a car note or
00:09:52
anything like that.
00:09:52
You know, student loans were not a thing.
00:09:54
It wasn't even a thought in my mind.
00:09:58
Speaker 2: Trade places with my seven-year-old.
00:09:59
I would do it in a heartbeat.
00:10:00
You know you can work, you can do all the things.
00:10:08
You can.
00:10:08
Go to the grocery store, I'll just like hang out, go to school
00:10:10
, have recess, play on my ipad.
00:10:11
That's yeah, I'm about that.
00:10:12
I would absolutely yeah, yeah, absolutely.
00:10:15
Speaker 1: You know, predicting where ai is going is extremely
00:10:21
difficult.
00:10:21
What I, you know, what I, what I, I guess, what I still kind of
00:10:26
fail to understand.
00:10:27
I mean, this podcast, right, with this question alone, could
00:10:30
probably go for days, right, but what I, what I failed to
00:10:34
understand is the evolution of the AI chip, right?
00:10:40
So that's really interesting to me because it's like I guess
00:10:46
one of my questions would be at what point in time did someone
00:10:51
determine whether it's NVIDIA, apple, whoever right?
00:10:53
I don't know who determined it, but at what point did someone
00:10:57
determine, okay, we need a dedicated chip, a dedicated
00:11:00
hardware module just for AI, dedicated hardware module just
00:11:08
for AI.
00:11:08
And oh, we're going to build this whole you know ecosystem
00:11:09
like NVIDIA, for instance, as CPUs that are just for AI.
00:11:11
You know, I'm not trying to.
00:11:13
This podcast isn't sponsored by NVIDIA in any way.
00:11:17
You only work at NVIDIA.
00:11:19
But again, your thoughts aren't NVIDIA's thoughts or anything
00:11:23
like that.
00:11:23
We're just speculating on information that's out there.
00:11:28
Speaker 2: Yeah, so I mean okay.
00:11:29
So I'll say the original usage of GPUs for deep learning, right
00:11:40
?
00:11:40
Because, again, when we talk about AI, like AI is this whole
00:11:42
big thing, but I think it has become shorthand for deep
00:11:43
learning, right?
00:11:43
Because, again, when we talk about AI, like AI is this whole
00:11:45
big thing, but I think it has become shorthand for deep
00:11:48
learning, right?
00:11:49
Obviously, ai includes things like expert systems, traditional
00:11:53
statistical machine learning.
00:11:55
you know, you know like logic programming, which is my
00:11:58
personal favorite because it's a form of generative AI that's
00:12:02
based on good, old-fashioned AI, not the modern deep learning
00:12:06
paradigm.
00:12:07
Speaker 1: I think it's really cool.
00:12:08
I think it's neat.
00:12:09
Speaker 2: But anyway, I'll go back to this.
00:12:11
You go back and you see that CUDA, the Compute Unified and
00:12:16
Fuzz architecture on NVIDIA GPUs , has always been this massively
00:12:23
parallel processing framework.
00:12:24
You have your CPUs, your general-purpose CPUs, you have
00:12:29
your RISC CPUs, you have all of these different CPU
00:12:31
architectures and instruction sets and whatever, and what CUDA
00:12:35
kind of did was look at computer graphics as this
00:12:41
massively parallel computation and specifically computer vision
00:12:45
has always dealt with matrices, right, it deals with sensors,
00:12:49
really, right, the RGB channels, and then you have your image
00:12:54
which is like n, by, and so you have an n by c, right, your
00:12:59
length, width and then the depth of the tensor.
00:13:02
Those are how you produce computer graphics.
00:13:06
It turns out that in deep learning you're dealing with the
00:13:11
same sort of thing, where you have these matrices, right,
00:13:14
you're doing these massively parallel matrix multiplications
00:13:18
across these different channels and then balancing them forward.
00:13:24
Gtus were just kind of well-suited to the task it is,
00:13:28
honestly, producing computer graphics and doing deep learning
00:13:32
is under the hood.
00:13:34
The math behind it is a very similar type of math, right,
00:13:37
it's massively parallel matrix multiplication.
00:13:40
You know, the element of CUDA led to CUDNN, which all kinds of
00:13:45
frameworks have been, to CUDNN, which all kinds of frameworks
00:13:46
have been built on top of that.
00:13:47
Right, you have your Torch and your Kerrix and your TensorFlow
00:13:52
and Theano if you're an old head or Chainer, if you've been
00:13:56
doing this for a hundred years and more modern frameworks like
00:14:00
Jaxx, right, and they just take advantage of that hardware
00:14:02
capitalism.
00:14:03
So that's how these sort of chips have become so ubiquitous
00:14:08
in AI.
00:14:10
And then, when it comes to the more AI-specific hardware, right
00:14:14
, this high performance computing that's not really a
00:14:20
consumer-grade graphics processing unit that happens to
00:14:23
be good at this thing, but are more specifically designed for
00:14:27
it.
00:14:27
It's just kind of looking at okay, well, where are those
00:14:30
differences between the parallel matrix multiplications you're
00:14:34
doing in computer graphics and the ones that you're doing in
00:14:36
deep learning?
00:14:37
And then how can we enhance the chip to be better at deep
00:14:41
learning than it is at computer graphics?
00:14:43
Right, they'll still render images, right?
00:14:46
If you want to plug into your you know Cortex A6000's display
00:14:52
port, you can.
00:14:53
I don't know why you would do that, but you certainly could.
00:14:57
It's just that it's going to be more performant tasks like deep
00:15:02
learning, because that's kind of what it's designed for.
00:15:07
Speaker 1: That's interesting.
00:15:08
So can we touch on what deep learning is?
00:15:11
Right?
00:15:11
Because, from an uneducated perspective of AI right, like
00:15:16
you know, I'm getting my PhD in securing satellites and
00:15:19
preparing them for quantum communications right, and it's
00:15:23
so very close to ai that I actually have to read the white
00:15:27
papers on ai and all this other stuff, right.
00:15:31
So I'm still under the impression that deep learning is
00:15:35
like hook it up to the internet and let it go learn for a
00:15:38
couple years, right.
00:15:39
What is?
00:15:40
What is deep learning?
00:15:42
Speaker 2: so at level right, like the simplest version of
00:15:46
deep learning is.
00:15:47
You think about like, put yourself in like I don't know
00:15:51
10th grade, right, go all the way back to grade school and
00:15:54
remember you're, y equals nx plus b.
00:15:56
So you've got your y and you've got your x and you're trying to
00:16:00
find the slope of the y-intercept.
00:16:01
Now you have to do that with matrices, right, where, instead
00:16:07
of finding just the number or the fraction for the slope and
00:16:10
the number or the fraction for the y-intercept, you have to do
00:16:14
this with a matrix, right?
00:16:15
And for anybody who doesn't know what a matrix is, it's like
00:16:19
think about an exercise sheet, right.
00:16:22
You have a certain number of cells and each one of those
00:16:24
cells has the weights in it.
00:16:26
What you're trying to do is take your collection of Xs and
00:16:31
Ys and find the best and B, so that you have the least amount
00:16:36
of error, so that if you plug in an arbitrary X, which is going
00:16:40
to be a vector, since you're vacuuming the trustees, you're
00:16:43
going to get a y that's close to it.
00:16:45
And then the other thing that it does on top of that x plus b,
00:16:51
it wraps it in a non-linearity.
00:16:55
So non-linearity is sort of what it implies it's a function
00:17:01
that isn't linear.
00:17:02
So some common ones are like the value, which is just the max
00:17:07
of the output, and zero.
00:17:10
So if it's zero or less than zero, if it's not a positive
00:17:15
output, then you get zero.
00:17:18
Otherwise you get the output of the function.
00:17:20
Your mx plus b Sigmoid is another one, which is basically
00:17:24
an exponential function that maps it between zero and one.
00:17:27
Right, there's a whole bunch of different non-linearities.
00:17:30
If you google activation function, uh, you'll find
00:17:35
thousands of them, right, like there's.
00:17:37
There's a whole literature just on activation functions, uh,
00:17:40
but at the end of the day, that is what kind of gives it this
00:17:43
spiking.
00:17:44
All of this is inspired by human biology, right?
00:17:48
So the neuron activation on or off right is one or zero, and so
00:17:54
you get that spiking through these activation functions.
00:17:58
So that's a single neuron, right.
00:18:00
It's that y equals mx plus b and a non-linearity on top of it
00:18:05
.
00:18:05
In a neural network in deep learning, you have many of these
00:18:09
neurons that are interconnected .
00:18:12
What makes it deep is that you have multiple layers.
00:18:15
So the input to the next layer of the network is the output of
00:18:20
the previous layer and you basically stack these on top of
00:18:22
each other and then get your output of the previous layer and
00:18:23
you basically stack these on top of each other and then get
00:18:24
your output.
00:18:24
There's all kinds of fun stuff in there, right when, like you
00:18:29
know, that describes that.
00:18:30
It's a good old-fashioned multilayer perceptron or, like
00:18:35
you know, dense neural layer, but there are transformer blocks
00:18:38
, there's recurrent blocks, there's all kinds of fun stuff
00:18:41
you can do inside of those neurons, but essentially, like
00:18:46
it all boils down to that interprets language and provides
00:18:51
responses based on the broad knowledge that it has.
00:19:11
Speaker 1: Are there limitations to what you're able to teach it
00:19:15
in terms of?
00:19:16
I've already given it all the information that I have
00:19:19
available, right?
00:19:20
What's the next evolution?
00:19:23
To teach it more, to make it smarter, right?
00:19:25
What does that look like?
00:19:27
Speaker 2: uh, if I had the answer to that question, I would
00:19:30
, uh, I don't.
00:19:32
I don't know if I would have time to be on a podcast at all.
00:19:35
I wish I had the answer to that question, but that is.
00:19:38
That's a real problem that we're running into right is like
00:19:40
we've basically exhausted the world's greatest repository of
00:19:46
knowledge.
00:19:46
The people training these large-scale models, whether it's
00:19:51
OpenAI or Anthropic or whoever, they've collected basically
00:19:57
everything that's out there, and that is a huge limitation.
00:20:02
Once you've trained it on all the stuff, how do you get it to
00:20:05
be better?
00:20:06
And there's some applications of synthetic data.
00:20:09
Right, where you have, these models generate new data.
00:20:13
That carries its something called mode collapse.
00:20:17
So basically it starts.
00:20:18
But let me back up what these models are really doing.
00:20:30
Right, whether they're the instruction-tuned GPT-4 that
00:20:34
you're using with chat GPT or they're more of a base LLM,
00:20:40
they're essentially doing next token prediction.
00:20:43
And think of a token as a word.
00:20:44
Right, it's not exactly, but close enough analogy.
00:20:47
It's trying to predict the next word.
00:20:50
So the trouble with synthetic data is that you give it this
00:20:55
input, you get this output.
00:20:56
That's synthetic and it's generating the next word by
00:21:00
picking the most probable, subject to some stochasticity,
00:21:06
some randomness.
00:21:06
Right, it's picking the most probable next output.
00:21:10
So if it starts generating, do you want to go to the next first
00:21:15
value and be like store?
00:21:17
Do you want to go to the mall?
00:21:19
Do you want to go right somewhere you might want to go?
00:21:21
And as you continue to train this thing on data that it
00:21:26
generates, you're raising the probability because it's seeing
00:21:30
these things more often.
00:21:31
You're raising the probability of each one of these tokens and
00:21:35
so, essentially, you're taking this beautiful heterogeneous
00:21:40
probability distribution uh, this distribution over
00:21:44
distributions and you're slowly collapsing it into a single
00:21:50
point, right, where, if you keep outputting the most probable
00:21:54
tokens and then training on the most probable tokens, it's just
00:21:58
going to continue reinforcing itself until it can't do all
00:22:03
that general stuff right?
00:22:05
There's a new paper on this that came out a couple of years
00:22:09
ago about training on synthetic data causes.
00:22:12
Catastrophic forgetting is what they call it, and you know this
00:22:17
is something that, again, we take.
00:22:19
Now for language, which is the hot thing right now.
00:22:21
You know that's what I work in, right, it's like securing large
00:22:24
language models.
00:22:25
But if you look back at computer vision like again if
00:22:29
you've been doing this a while you look at generative
00:22:32
adversarial networks, gans.
00:22:33
This was a well-known phenomenon in GANs.
00:22:37
Right, this is something that people who trained GANs were
00:22:40
super familiar with is you had to watch out for mode collapse
00:22:43
because you're generating something, you're getting an
00:22:46
error and then you're casting that back in as your next step
00:22:51
in your training phase and if you're not careful, your gam
00:22:54
will just collapse to this single best possible output and
00:23:00
it loses its ability to generalize wow, that is really
00:23:06
fascinating how it would basically self-reaffirm and push
00:23:15
more importance on a certain topic or language or whatever it
00:23:18
might be, and then it just works out all the other stuff
00:23:23
from its system.
00:23:24
Speaker 1: It's like, well, all of that stuff isn't important.
00:23:26
I'm going to put all my processing power towards this
00:23:28
thing, right.
00:23:30
That seems to be really important right now.
00:23:32
It seems like you know that could be really helpful for some
00:23:37
use cases.
00:23:37
You know, like thinking of like medicine, for instance.
00:23:40
Or you know, engineering, a new type of structure, whatever it
00:23:44
might be right, like you could think of a lot of critical
00:23:46
services where reinforcement would benefit in some ways to
00:23:53
focus it, you know, on a certain topic or a certain situation.
00:23:57
So I never thought of how reinforcement of these things
00:24:02
would force it to, you know, collapse in on that one topic.
00:24:06
That's really interesting.
00:24:08
Speaker 2: Yeah, it is really interesting and, like, synthetic
00:24:11
data is not all bad.
00:24:12
It's just that if you use too much of it, it can be a problem.
00:24:16
But it's also like super important to use synthetic data
00:24:20
in certain cases where the data is rare, right, where you don't
00:24:26
have a lot of data, because, again, these models are trained
00:24:29
on huge, huge amounts of data.
00:24:32
It's just like incongruent, hensible amounts of data.
00:24:35
So if you're trying to train something in, say, let the cyber
00:24:38
security go in, right?
00:24:39
Uh, you want to train on logs?
00:24:41
Okay, well, you have your logs, but what if you need more logs?
00:24:46
Like, do you just wait?
00:24:47
But what if you need more logs?
00:24:48
Do you just wait a year to get more logs?
00:24:50
Well, maybe you can feed your logs into a thing that allows
00:24:54
you to generate things that look like your logs and now you can
00:24:58
say, okay, these are normal logs , this is what our system looks
00:25:02
like.
00:25:03
We're assuming we're uncompromised, which is a heavy
00:25:06
assumption, but put that to the side, right?
00:25:08
So this is what the system looks like under normal
00:25:10
operation.
00:25:11
Now you have enough data to start training your thing, to do
00:25:16
that anomaly detection right, because maybe otherwise you
00:25:20
don't it's easy to come across malicious traffic, right, you
00:25:26
can generate basically infinite amounts of malicious stuff.
00:25:29
You just pop it in a sandbox and dump the logs, right, but
00:25:33
it's a lot harder to generate things that are both benign and
00:25:39
look normal right.
00:25:41
So not just benign and nobody's using it, but benign and people
00:25:46
are interacting with the system .
00:25:47
Speaker 1: Benign and people are interacting with the system.
00:25:50
Do you think?
00:25:51
I mean, this is obviously purely, you know, hypothetical.
00:25:55
Do you think that cloud providers are doing that
00:26:02
somewhere somehow?
00:26:02
Because I mean, think of the vast amount of the logs, just
00:26:06
the logs, not even talking about the data itself or anything
00:26:09
else like that, the attacks that you know are launched at them
00:26:12
and everything else like that.
00:26:13
Right, you know, if you're training it on data like that,
00:26:18
it wouldn't be very intelligent to not use that data and train
00:26:22
an ai with it.
00:26:23
Now you have you know in security terms.
00:26:24
Now you have you know.
00:26:25
Know in security terms.
00:26:26
Now you have, you know, the greatest SIM that ever existed.
00:26:29
Right, there's nothing that would compete with that.
00:26:31
Like ever, do you think that that is a route that cloud
00:26:37
providers could take?
00:26:41
Speaker 2: Two things.
00:26:41
One is is it a route that cloud providers could take?
00:26:45
Maybe?
00:26:46
So I'm extremely not a lawyer.
00:26:50
I'm as much not a lawyer as you can possibly be, but you know,
00:26:57
the agreements between the CSP and their customers would kind
00:27:02
of predicate the level of access to logs that they have.
00:27:04
Right, they might have access to infrastructure logs, but they
00:27:08
may not have authorization to access to infrastructure logs.
00:27:12
But they may not have authorization to access your
00:27:16
NetFlow logs or your WAF logs.
00:27:18
They may not be able to see that legitimately.
00:27:21
I'm sure that if somebody were determined to, they'd probably
00:27:25
have the capability, probably have capabilities, but like that
00:27:32
would be kind of I think that is the sort of initiative that
00:27:33
would be hence from more by lawyers than by technical
00:27:34
capability.
00:27:35
Right, because the CSPs you think about Microsoft, google,
00:27:38
amazon, the three big ones they're also companies that are
00:27:42
training and putting out huge language models and doing really
00:27:46
incredible research in artificial intelligence.
00:27:48
Right, microsoft and Google in particular?
00:27:51
Right, have been doing phenomenal AI research and
00:27:55
security research for quite some time and, like I said, people
00:28:01
occasionally cuckoo on Microsoft's security posture.
00:28:04
Right, like they have gotten a lot better about being
00:28:07
forthcoming with security information and publishing
00:28:10
security research.
00:28:10
You know, shout out to my Microsoft friends Now, would
00:28:15
that lead to a sim to end all sims?
00:28:19
Right, the one sim to rule them all?
00:28:21
I think it depends on how optimistic you are about AI
00:28:26
alone.
00:28:26
Right, because you could use that for incredible you know sim
00:28:31
rule development and you could definitely have a world-class
00:28:36
sim based off of that data.
00:28:38
But a lot of, like the really good sim rules, handwritten
00:28:43
rules that are written by, you know, security experts right,
00:28:47
they're things that are known to be bad, because one of the
00:28:52
things with security data is that there's a lot of noise,
00:28:59
there's a lot of weirdness and there's a lot of things that, to
00:29:05
an ignorant party, might seem really important, that don't end
00:29:11
up actually mattering and should not be used as a factor.
00:29:16
I'll give you a concrete example where, once upon a time, I was
00:29:20
working with a team and we were trying to train a malware
00:29:23
classifier.
00:29:23
So we had this giant corpus of malware, huge corpus of malware,
00:29:27
and we took all of these features, we featurized the
00:29:31
thing and we trained this machine learning system.
00:29:35
It was a tree-based architecture, a decision
00:29:37
tree-based architecture.
00:29:38
This is just a few years ago and it had this great
00:29:43
performance.
00:29:44
It did a great job detecting malicious samples False algebra
00:29:50
wasn't too bad.
00:29:51
And then we looked at the components that were considered
00:29:56
most important, determining whether or not something was
00:29:58
malicious.
00:29:59
Right, the most highly weighted components, kind of the top of
00:30:02
the tree.
00:30:02
And item number one was the compiler.
00:30:05
It was a compiler.
00:30:09
If it was compiled with Borland Delphi, it's malicious.
00:30:12
I don't know if that's true.
00:30:16
I don't know if that's right.
00:30:17
Item number two the second most important feature.
00:30:20
So if both of these things were true, it was malware.
00:30:22
All the time was if it was compiled in Russian or Chinese.
00:30:28
So the language of the XTUAL right and what's the mistake?
00:30:34
As a malware analyst, those two things are functionally
00:30:39
meaningless in determining whether or not something's
00:30:42
malicious and it was just an artifact of where our samples
00:30:46
came from, right.
00:30:47
So you know, when you're optimizing for those things and
00:30:52
you're doing it in a way that doesn't account for expert
00:30:54
knowledge, where you don't have people who are reverse engineers
00:30:58
, who are malware analysts.
00:30:59
Looking at this, you might think , if you're, you know, a lay
00:31:02
person like well, look, the accuracy is good, the F1 score
00:31:06
is good, like we're happy with this thing.
00:31:08
And then you look at the features that matter to it and
00:31:13
it's like you know, does it, does it import?
00:31:16
Uh, wsoc2 right?
00:31:18
Does it do?
00:31:18
Networking is like the seventh most important feature on that
00:31:22
list.
00:31:22
It's like I don't know man.
00:31:24
To me that is like one of the more important features for me
00:31:29
to look at.
00:31:30
Does it modify or break keys?
00:31:31
These sorts of things are going to matter.
00:31:34
Whether it's Chinese or Russian doesn't matter.
00:31:38
These are the sorts of things that would make it hard to just
00:31:42
take a bunch of data trade the world's best sin and call it a
00:31:45
day.
00:31:45
You still need that security expertise to write the good
00:31:50
rules to correlate things.
00:31:55
Speaker 1: That is man.
00:31:57
That's really fascinating.
00:31:59
I feel like we could literally talk on that for hours.
00:32:02
You know, in security, like you said, the attributes what the
00:32:08
malware is actually doing.
00:32:09
What the code is actually doing is far more important than its
00:32:12
origin.
00:32:13
Once you figure out what it's doing and you kind of attribute
00:32:18
okay, this is typically from China, this is typically from
00:32:22
Russia, this is typically from America, right, that's a
00:32:26
possibility as well, and I wonder what kind of result and I
00:32:30
mean, this is again purely hypothetical, right I wonder
00:32:33
what kind of result you would have if you trained a model on
00:32:36
only written malware and then you had a separate one that was
00:32:40
trained on other, you know other nation states that are playing
00:32:44
in the space, and then you use that to somehow inform some
00:32:48
other higher tier model that when it looks at a piece of
00:32:52
malware, it dissects what it's doing, it tells you where it's
00:32:56
from all that sort of stuff, rather than you know, because
00:33:00
right now it's a manual process, right?
00:33:02
You throw it into a reverse engineering tool like ito pro,
00:33:07
right, and you start going through it and God forbid, they
00:33:09
compile, you know, different modules of the code with
00:33:12
different languages and things like that, right, because it
00:33:14
just extends your reverse engineering process
00:33:18
significantly.
00:33:19
But I wonder if that is a very viable thing.
00:33:23
And then you could even use that model to write more malware
00:33:27
.
00:33:27
Right, you could use that model from the malicious side to to
00:33:31
use it to create like stuxnet version 5.
00:33:34
Right that that that is undetectable?
00:33:38
Speaker 2: well, that's the.
00:33:39
That's the thing with a lot of the machine learning systems,
00:33:43
right, a lot of these, these ai systems is that as a side effect
00:33:48
of training a model that's good at defense, you kind of get a
00:33:54
model that's good at offense.
00:33:56
Right, you can invert the output.
00:33:58
That's something that we're seeing.
00:33:59
You know.
00:34:00
You look at more and more models and like one of the big
00:34:03
things that people are talking about and concerned about is
00:34:05
phishing, and I think that's valid about and concerned about
00:34:06
is phishing, and I think that's valid.
00:34:15
The thing about phishing is I don't freak out about how LLMs
00:34:17
enable phishing, because I have seen people fall for some of the
00:34:19
most obvious phishing emails ever written, right.
00:34:21
I've seen people respond to the literal.
00:34:26
I am from IT.
00:34:28
We're resetting passwords.
00:34:29
Can you please email me the password?
00:34:32
But I have seen that in my real life.
00:34:35
Things like do LLMs enhance the ability to scale?
00:34:41
Does it let attackers be lazier ?
00:34:44
For sure, but when you train a large language model that's
00:34:48
designed to help anybody write an email, you know if I go to
00:34:52
chat to you after this your perplexity or whatever and say,
00:34:55
like, write an email to Joe Self thanking for having me as a
00:34:59
guest on his podcast.
00:35:00
Well, it's not as an ability to write a phishing email, because
00:35:08
a well-written email is going to be a well-written email
00:35:12
regardless.
00:35:13
It's just a matter of whether you know you include it or not.
00:35:20
Right, and so I mean that's why a lot of the user education
00:35:22
around phishing, I think, is kind of dated right as I look
00:35:26
for misspellings, look for you know these things.
00:35:29
And I mean you know there are phishing dated right.
00:35:31
It's like look for misspellings , look for you know these things
00:35:32
right.
00:35:33
And I mean you know there are phishing templates, right, you
00:35:34
can go on whatever the you know current version of exploitin is
00:35:40
breach forum or whatever, right, who cares?
00:35:42
Go on one of those forums and there are phishing templates.
00:35:45
Those are well-written, legible , you can have them in any
00:35:49
language you want, great, good.
00:35:52
So I mean, does a large language model really increase the
00:35:57
threat land state?
00:35:58
Not necessarily.
00:36:00
But does that mean that it's a dual-use technology that we need
00:36:05
to be aware of?
00:36:05
Yes, right, and so when you have coding assistants that can
00:36:08
help you write any code, can those be used to be aware of?
00:36:09
Yes, right, and so when you have coding assistants that can
00:36:10
help you write any code, can those be used to write malicious
00:36:13
code, probably.
00:36:15
And I think this gets into the big conversation around model
00:36:18
alignment how do we put safety into models?
00:36:21
How do we put safeguards into models?
00:36:23
And I think that that's a good first layer.
00:36:28
But I don't think that alignment in and of itself is a
00:36:32
solvable problem.
00:36:33
I don't think it's a tractable problem.
00:36:35
Right To encode these things into the model weights.
00:36:40
At the end of the day, these are just aisles of linear algebra.
00:36:44
They don't know anything, they don't think, they don't have
00:36:49
values.
00:36:50
Right, we can't necessarily encode that.
00:36:52
But having systems around it that validate the input with
00:36:58
respect to some policy and the output of the model with respect
00:37:01
to some policy, I think that's where we can start making
00:37:05
meaningful gains.
00:37:06
Start making meaningful gains Because with these, these
00:37:09
water's language models, one of the big things that a lot of
00:37:12
people don't realize about them is a key difference between
00:37:17
these AI systems and the old-fashioned computing systems
00:37:23
is in the old-fashioned computing systems, we have a
00:37:25
data plane and a control plane, right, and they're separated.
00:37:29
And so if you can do things like see something happening on the
00:37:30
data plane and a control plane, right and they're separated, and
00:37:31
so you can do things like see something happening on the data
00:37:33
plane and say, okay, don't opt in this IP address anymore,
00:37:37
right, you're not allowed to go there.
00:37:38
And you can manage that at the control plane layer to affect
00:37:41
what happens on the data plane.
00:37:43
In LLMs, the data plane and the control plane are the same
00:37:47
thing.
00:37:47
There's only one place to put anyone.
00:37:49
So your system instruction right, your thing that says,
00:37:52
like you are an unbiased, friendly, helpful language model
00:37:56
who does X, great, that goes into the exact same input space
00:38:02
as your arbitrary user input.
00:38:04
And there's very few mechanisms and certainly no provably
00:38:10
mathematics, no provably secure way to differentiate them and to
00:38:17
have the model treat them differently.
00:38:19
It just treats it all as one big input.
00:38:21
And those are the sorts of things that make the problem of
00:38:24
securing these systems really hard.
00:38:24
Yeah, you know, makes the problem of securing these
00:38:26
systems really hard.
00:38:30
Speaker 1: Yeah, so do you think that this is going to turn into
00:38:35
a situation with cloud, like what cloud security is currently
00:38:41
evolving into?
00:38:42
Right, you have cloud security at such a scale now, right
00:38:46
Across so many different domains , that it can't be handled by
00:38:51
one person.
00:38:51
Right, it can't be handled by, you know, one person per cloud.
00:38:55
Even right, you really need to start having, like a cloud IAM
00:39:00
team.
00:39:00
You really need to have, like a cloud data security team.
00:39:03
You know people that specialize in it.
00:39:06
Do you think that that is also the route that AI will be going
00:39:12
and, if so, what do you think those specialties would be?
00:39:18
Speaker 2: So I don't think it will go the route that cloud is
00:39:20
going and how it gets managed, I think depends on how the
00:39:25
community right, like the broad security community and, even
00:39:28
more broadly, the risk community , the broad security community
00:39:30
and, even more broadly, the risk community.
00:39:33
If we look bigger than InfoSec risks, a lot of AI risks.
00:39:39
A lot of the trouble that you're going to get into with AI
00:39:43
today is falling all the way to your PCer and your IR teams and
00:39:48
those poor people who have to deal with the way since your
00:39:50
PCER and your IR teams, and like those poor people who have to
00:39:51
deal with the fact that your job said something kind of nasty
00:39:56
right, or said something that wasn't truly part of your
00:40:02
corporate policy, right.
00:40:03
You have the Air Canada instance where it said you know
00:40:07
something about like a refund policy, offered somebody a
00:40:10
refund or whatever, and that wasn't their policy, and a court
00:40:15
made them uphold what their model said right, like is that
00:40:21
something that your PCER should be dealing with?
00:40:23
Probably not.
00:40:25
Probably not right.
00:40:27
There are certainly potential security issues, but what I've
00:40:31
found in AI security, in the security of AI systems, right,
00:40:38
rather than applications of AI to general security problems, is
00:40:41
a lot of it just comes back to like AppSec.
00:40:44
We have a peripheral elemented generation system, a RAT.
00:40:51
It's a language model that talks to a database and maybe we
00:40:57
did a bad job of seeing what should be in the database or we
00:41:02
let the language model read anything in that database and it
00:41:06
should have user base user.
00:41:08
You should have role-based access controls but, like that's
00:41:14
annoying to do.
00:41:15
You have to make the user authenticate a lot and then you
00:41:18
have to make it, you know, take a token and pass that token into
00:41:21
the database.
00:41:21
That's annoying, so we're not going to do that.
00:41:24
Like, okay, well, that's, your problem is like it's bad app set
00:41:27
practices right.
00:41:30
A lot of it comes back to that and I think you know if AI's
00:41:35
surety ends up being AI risk management and it's that big
00:41:40
broad umbrella where, like, bias becomes part of it or, you know
00:41:45
, hallucination or whatever general risk, then then yeah,
00:41:48
there's no way one person could possibly do that.
00:41:51
If you're publishing, you know, if you're a model provider, if
00:41:55
you're hosting a model that end users interact with in ways you
00:41:59
can't predict, but if we stop it down to security, you know, I
00:42:03
think it's maybe not quite as complex, because a lot of this
00:42:09
is just architecture and it's like you have this thing that
00:42:12
takes English language input and generates SQL queries.
00:42:16
It's like, okay, well, maybe you should do some output
00:42:18
validation.
00:42:19
Don't send arbitrary SQL to your database.
00:42:22
We know this.
00:42:23
You know a lot of these things are familiar to us.
00:42:25
Things are familiar to us, but when they get abstracted away
00:42:30
through this large language model, we kind of have trouble
00:42:33
seeing the trees for the forest, as it were.
00:42:36
We get stuck on just seeing okay, this is some big thing
00:42:39
that has an English language interface and talks to our
00:42:42
database.
00:42:43
Who's talking to it?
00:42:46
Do we pass their user role through as a habitability to
00:42:50
read the whole database?
00:42:51
Who's talking to it?
00:42:52
Do we pass their user role through?
00:42:54
Does it have the ability to read the whole database?
00:42:56
And you can't really trust the language models output.
00:42:58
I think it goes back to that sort of stuff right, taint
00:42:59
tracking, right Stuff that we've noted about, which is like if
00:43:02
you talk to an AI system and it has the capability to go
00:43:07
retrieve data from the internet and then process that data from
00:43:11
the internet, then it shouldn't be allowed to take its output
00:43:16
from the internet and interact with privileged systems, right?
00:43:20
You wouldn't let a random internet server do things on
00:43:26
your systems right.
00:43:27
These are the sort of things that we need to be cognizant of,
00:43:30
so I don't know that it's necessarily going to get to that
00:43:34
point, but we have our baby steps, which is like good app,
00:43:38
stack practices, and then we can start getting into the broader.
00:43:41
How do we do AI risk management for organizations?
00:43:45
And I think that's a much bigger and harder problem that
00:43:48
can't be solved by a single person.
00:43:53
So, it's, it's uh it's really fascinating.
00:44:00
Speaker 1: You know this entire new, new area right that just
00:44:04
kind of, you know, spurred up into everyone's, you know,
00:44:09
forefront of their mind right now.
00:44:11
Do you know roughly how large the lom is like, how large the
00:44:17
biggest lM is like, how large the biggest LLM is right now?
00:44:23
Speaker 2: In terms of like data size.
00:44:25
Speaker 1: I mean, I don't even know how to.
00:44:27
Speaker 2: So so most people believe that the largest LLMs on
00:44:33
the planet are either TP4 or Clon 3, opus Like those seem to
00:44:39
be the largest, most capable models, or Cloud 3.
00:44:40
Speaker 1: Opus.
00:44:40
Speaker 2: Those seem to be the largest, most capable models.
00:44:41
Cloud 3 Opus is, by the way, really cool.
00:44:45
I am not paid by Anthropic, I just think it's cool.
00:44:48
It's the first model in a while .
00:44:50
That's really surprised me with its capabilities.
00:44:52
But they have not neither Anthropic or OpenAI has said how
00:44:57
many parameters it has, how many tokens it's been trained on
00:45:01
, right, All of that is information that they haven't
00:45:04
released to the public.
00:45:05
So we have some pretty large open weight models, right, some
00:45:11
people say open source, but if the data is not public I'll get
00:45:16
off my set of thoughts, right, these open weight models.
00:45:19
And so we do have some pretty large ones out there, right?
00:45:21
I mean, croc is quite a big one .
00:45:23
Mixtrol is a pretty big mixture of experts, but all that sounds
00:45:28
good.
00:45:29
You have a couple of like 340 billion parameter models out
00:45:33
there.
00:45:33
I think there are even some meaningfully larger ones that
00:45:35
have been other weights released , right?
00:45:37
Speaker 1: Yeah, some meaningfully larger ones that
00:45:38
have been other ways for least right, yeah, so that's I ask
00:45:43
because the parameters is a hard thing, I think, for people to
00:45:48
understand the size of the data.
00:45:50
Do you know what that roughly would translate into in like
00:45:55
petabytes or even exabytes at this point?
00:45:59
Speaker 2: So I think the parameters of the model and the
00:46:02
training data are kind of distinct things, right?
00:46:05
So the parameters of the model basically just say how much RAM
00:46:10
it's going to use, right?
00:46:11
How much VRAM or how much whatever, right?
00:46:14
If you want to run it on CPU because you're an insane person,
00:46:18
that's fine, I've had it if you want a 40 billion parameter
00:46:22
model on your CPU with, you know , a terabyte of RAM, I guess.
00:46:27
Right, but you know these models .
00:46:31
The parameter count basis is how much compute is used, right,
00:46:35
how much RAM and GPU if you run GPU, cpu, whatever, right, the
00:46:40
training data.
00:46:41
On the other hand, there's a paper from a couple of years ago
00:46:47
, the Chinchilla paper on Google , that actually suggests that
00:46:52
there's like an optimal number of tokens to train a model on,
00:46:56
based on the number of parameters.
00:46:57
Right, like most of these are following that chinchilla model.
00:47:01
Um, because you don't want a model that's over trained or
00:47:05
under trained, right?
00:47:06
I mean, hypothetically, you can train a 500 billion parameter
00:47:09
model on no data.
00:47:11
It's not going to be any good, but you could do it right now.
00:47:17
Looking at that, I mean we're talking gosh.
00:47:21
I mean, if you have like eight trillion tokens, yeah, you're
00:47:25
talking like several petabytes of data it's an insane amount of
00:47:31
data, right?
00:47:33
Speaker 1: where do you even start with securing it?
00:47:36
What's what's?
00:47:37
I feel like it like it's.
00:47:38
You know, obvious what the best practices are for normal
00:47:43
security, even cloud security, right?
00:47:45
Are those best practices translating into LLMs or is
00:47:50
there something else you know that you're starting to
00:47:53
potentially even adjust the best practices for?
00:47:56
Speaker 2: Yeah.
00:47:57
So I would say that, like your security best practices, all
00:48:00
your security best practices still yeah.
00:48:00
So I would say that your security best practices, all
00:48:01
your security best practices, still hold right, and you still
00:48:04
want your defense in death.
00:48:06
You still want your role-based access controls.
00:48:08
You still want to make sure you're doing input validation,
00:48:12
all this stuff Right, no question about that.
00:48:15
Now, I think, if we separate the open AIs and anthropics and
00:48:22
Googles and metas of the world and let them worry about how
00:48:28
they protect their training data and model ways from being
00:48:32
manipulated, set that to the side for the moment and think
00:48:36
about somebody whose organization is trying to deploy
00:48:39
an LLM powered application.
00:48:41
Right, they want to have some customer facing chatbot, or they
00:48:44
want to integrate some natural language interface into their
00:48:47
product.
00:48:48
Whatever I will say, you inadvertently set me up to plug
00:48:54
the two open source projects I work on Right.
00:48:57
One of those is Garak.
00:48:59
So Garak is like Metasploit for LLMs, right?
00:49:04
So it's got a collection of a bunch of different known ways to
00:49:09
get library models to say bad things and you can kind of pick
00:49:14
and choose what you're concerned about, right?
00:49:16
So if you're concerned that your chatbot is full of jinkies
00:49:19
and tossing output right, you're concerned about that sort of
00:49:22
thing Then you can send these prompts it's like a command line
00:49:26
interface, right, and it'll do all the work for you to take
00:49:29
these prompts, fang them against your model and give you a
00:49:31
report.
00:49:33
So then you can get kind of a sense of like, okay, well, where
00:49:40
are the risks associated with this, right?
00:49:41
Or if you're concerned about, you know, package hallucinations
00:49:43
, which is a thing that is of a non-zero amount of concern to
00:49:46
people, right, like with all the NPM and PyPy package hijacking
00:49:51
that, you see, right, if this thing makes up a package name
00:49:56
and you, okay, sure, import this thing and somebody has seen
00:50:00
that hallucination and put that package into PyPy and it's a
00:50:04
malicious package, well, that fessed off as bad as bad day for
00:50:09
you.
00:50:09
So you know, you can test for that sort of stuff.
00:50:14
There are a couple of different ways to deal with that.
00:50:18
If you're framing or fine-tuning your own language
00:50:22
model, you can try and fine-tune that out or props tune it out
00:50:27
or modify your system prompt or whatever right Like that might
00:50:31
work.
00:50:32
There's also these LLM firewalls right, I think Cloud Flarecoat
00:50:36
went out.
00:50:37
There's a bunch of companies that have them.
00:50:39
Right, I think Cloud Flarecoat went out.
00:50:39
There's a bunch of companies that have them right, and those
00:50:42
will kind of look for and detect known quote-unquote jailbreaks,
00:50:49
right, so like known attacks against these models, the sort
00:50:53
of stuff that we have in Dirac.
00:50:54
Right, that's fine and that stuff is done.
00:50:58
Some of it's quite good.
00:51:00
The other thing is the other open source project that I work
00:51:04
on, which is called Nemo Guardrails, and so Guardrails is
00:51:08
more of like a dialogue management framework, so you can
00:51:13
write these really specific configs or these you know,
00:51:18
essentially write through3L files.
00:51:21
But this colang is the language, this abstraction that we deal
00:51:25
with.
00:51:25
It's like a domain-specific language that lets you say, okay
00:51:29
, well, you should only talk about, you know, scheduling,
00:51:34
vehicle maintenance, and if you're asked about anything
00:51:37
other than vehicle maintenance, don't do it Right.
00:51:40
And it provides this external check on the model so that when
00:51:44
somebody asks a question that's not about vehicle maintenance,
00:51:48
you're not depending on the model, which has no separation
00:51:51
between the control plane and the data plane.
00:51:53
To parse that.
00:51:54
It's kind of saying, okay, well , this isn't about that, so
00:51:58
we're going to tell the model to do something, right, we're
00:51:59
going to tell the model to do something.
00:52:01
We're going to get in between you and the model and you can
00:52:06
make those as complicated as you want.
00:52:09
There's a bunch of other stuff in there.
00:52:10
It's a really powerful framework.
00:52:13
But these are the sorts of things that you can also put
00:52:23
around your models, whether they are something you're deploying
00:52:25
yourself, right, whether you've downloaded Lama 3 and you're
00:52:26
using that as your basis and you're self-hosting it in some
00:52:28
cloud service provider, or if you're using, like, an open AI
00:52:31
or a Coheer or Anthropic on the backend.
00:52:34
Yeah, there are a couple of different ways to both test
00:52:40
these things Garag, pyrit, p-y-r-i-t, because Microsoft is
00:52:47
another framework like this.
00:52:48
And then on the other side, you have your LLM firewalls.
00:52:54
You have your Nemo guardrails.
00:52:56
You have there are alternatives to Harpreils, I think they're
00:52:59
all called guardrails in some form or fashion.
00:53:01
Amazon has their own guardrails , but you know, nemo guardrails
00:53:06
is the one that I know best because it's the one that I work
00:53:08
on.
00:53:08
So there are definitely ways to start thinking about the
00:53:13
AI-specific stuff.
00:53:16
It just gets back to like, what do you care about?
00:53:20
What is what is the?
00:53:22
What is the threat model?
00:53:23
You have to figure out that, right?
00:53:25
So you don't care that your customer service chat sometimes,
00:53:29
like, says awful things to people.
00:53:32
Uh, you don't care that it's toxic, right, like it's funny,
00:53:35
it's like the dick's last resort chat, greatbot, great, cool,
00:53:39
fine.
00:53:39
But maybe your legal team doesn't want to expose
00:53:46
themselves to the liability that comes with having that happen,
00:53:50
and in that case, you have ways to test it and ways to try and
00:53:56
mitigate it.
00:53:57
And, of course, there's no fancy for you, right, like a
00:54:00
sufficiently determined attacker will always find a bypass.
00:54:03
These things aren't foolproof, but they can certainly make it a
00:54:07
lot better.
00:54:11
Speaker 1: Yeah, you can make it , you know, reasonably difficult
00:54:13
to where it's like okay, if and or when they still get in, it's
00:54:19
like I couldn't have done any more.
00:54:21
You know what do you expect, right?
00:54:23
Well, you know, eric, honestly, you know same thing as the
00:54:28
first time around, right?
00:54:29
I feel like this conversation would go another two, three
00:54:32
hours, right?
00:54:32
So I guess that just means I got to have you back on a couple
00:54:35
more times.
00:54:38
Speaker 2: It's always a pleasure, as always.
00:54:41
Speaker 1: Yeah, it's such a fascinating topic and of course
00:54:45
you know it's at the forefront of everyone's minds now.
00:54:48
You know, I think my wife uses ChatJPT to write more emails
00:54:53
than she actually writes at this point.
00:54:55
But you know, eric, before I let you go, how about you tell
00:54:58
my audience?
00:54:59
You know where they?
00:54:59
Before I let you go, how about you tell my audience?
00:54:59
You know where they could find you if they wanted to reach out
00:55:01
to you, and maybe even what those open source projects are
00:55:05
that you're working on?
00:55:06
Speaker 2: Yeah, sure, so I am on almost no social media.
00:55:11
You can feel free to find me on LinkedIn.
00:55:13
I'm the only Eric Galenkin, if assuming my name is in show
00:55:17
notes, you can just copy and paste that and it's probably me
00:55:22
wherever you find me.
00:55:23
Yeah, feel free to drop me a message there.
00:55:26
You can email me Whatever.
00:55:29
I'm an easy person to find.
00:55:30
It's a perk of being one of one .
00:55:34
And then those two open source projects are both on GitHub.
00:55:36
Please feel free to check them out and feel free to join the
00:55:39
direct Discord.
00:55:40
That is actually a place where you can find me and pester me
00:55:42
directly, and that's githubcom slash, nvidia.
00:55:46
Slash nemo, n-e-m-o.
00:55:49
Dash guardrails, all one word.
00:55:51
And then githubcom, slash.
00:55:53
It's currently under liangdz, hasn't been moved to an nvidia
00:55:57
repo yet.
00:55:57
Slash, garak, g-a-r-a-k.
00:56:00
Yeah, by all means, feel free to reach out.
00:56:04
Love to chat.
00:56:07
Speaker 1: Awesome.
00:56:07
Well, thanks, eric, and I hope everyone listening enjoyed this
00:56:12
episode.
00:56:12
Bye everyone.
Related Episodes
From Military Emergency Response To Cyber Security Guru - Brad LaPorte
Send us a textBrad's journey from sci-fi enthusiast to cybersecurity expert is an unconventional path filled with unexpected twists and valuable insights. Hear firsthand how his initial pursuit of engineering took a dramatic turn following 9/11, leading him to the military and into the Signal Corps,...
Balancing Speed and Security in the Tech World with Idan Plotnik CEO of Apiiro
Send us a textJourney into the world of cybersecurity with Idan Plotnik, a true pioneer in the field, as he revisits the path that led him to become a leading figure in tech innovation. Starting with a childhood fascination for computers, Idan advanced to play a pivotal role in the Israeli cyber sec...
Embracing the Suck, Lessons From #NASA In #Cybersecurity
Send us a textIn this engaging conversation, Robert Vescio shares his unique journey from horticulture to cybersecurity, emphasizing the importance of economics in understanding cyber risk. The discussion highlights the value of learning from mistakes, the need for transparency in cyber risk managem...