WEBVTT 00:00.000 --> 00:21.000 So, let's start the river everyone and very happy to be here at this station. 00:21.000 --> 00:24.000 Is it more clear? 00:25.000 --> 00:28.000 So, hello everyone. 00:28.000 --> 00:31.000 It's a big pleasure to be in this station. 00:31.000 --> 00:34.000 It's my first time, as a speaker. 00:34.000 --> 00:39.000 So, I am very impressed to see many people in front of me. 00:39.000 --> 00:47.000 I know that a child but I will try to do my best to give them a more clear answer. 00:47.000 --> 00:53.000 So, this evening we will talk about AI security. 00:53.000 --> 00:56.000 So, we will link to the topics. 00:56.000 --> 00:58.000 AI security. 00:58.000 --> 01:10.000 In order to explain how to an idea that a future against friction in a machine learning system. 01:13.000 --> 01:16.000 I do two things. 01:16.000 --> 01:18.000 So, just an introduction. 01:18.000 --> 01:20.000 My name is Samuel Desso. 01:20.000 --> 01:23.000 I am an IT consultant and trainer. 01:23.000 --> 01:28.000 I am the founder of the society, Irritate, which is best in the front of friends. 01:28.000 --> 01:30.000 Here's a Belgian frontier. 01:30.000 --> 01:35.000 Our job is to implement, we improve monitoring on the service, 01:35.000 --> 01:37.000 on critical infrastructure. 01:37.000 --> 01:44.000 IT, we are not politically, we work on AI. 01:45.000 --> 01:50.000 As we are using a company, we have an example from our team. 01:50.000 --> 01:57.000 We have a company, a partner, a partner, a team, excuse me. 01:57.000 --> 02:01.000 We've picked our metrics, which is a monitoring company. 02:01.000 --> 02:06.000 And now we are talking about this later. 02:06.000 --> 02:11.000 So, we will talk about what we are talking about today. 02:11.000 --> 02:15.000 We will talk about monitoring on AI. 02:15.000 --> 02:19.000 We will focus on security. 02:19.000 --> 02:21.000 So, we will see the problem. 02:21.000 --> 02:25.000 Why machine learning needs a different security? 02:25.000 --> 02:28.000 Before making a deep dive on how can we structure, 02:28.000 --> 02:34.000 and can we develop and approach to develop, to mobilize, 02:34.000 --> 02:39.000 our monitoring on a machine learning attacks. 02:40.000 --> 02:45.000 We will see, after the strikes from a lad, I have a bit. 02:45.000 --> 02:50.000 I will make a demo. 02:50.000 --> 02:57.000 And before seeing more technical on architecture, I will speak. 02:58.000 --> 03:03.000 So, the core question. 03:03.000 --> 03:08.000 Generally, traditional security monitors. 03:08.000 --> 03:12.000 We will register, you can monitor network traffic, 03:12.000 --> 03:15.000 system lock, and so on. 03:15.000 --> 03:21.000 But the core question, we will monitor the AI layer itself. 03:27.000 --> 03:29.000 Why is this question? 03:29.000 --> 03:32.000 Because there is a fact. 03:32.000 --> 03:36.000 You are AI-modest in this particular attacks surface. 03:36.000 --> 03:41.000 On the AI list, several types of attacks. 03:41.000 --> 03:46.000 I can list, for example, the proven injection. 03:46.000 --> 03:51.000 The data design, the algorithm input, model extraction. 03:52.000 --> 03:55.000 It is a very well known attack. 03:55.000 --> 04:03.000 But first, more of an list. 04:03.000 --> 04:09.000 On the AI, I can do the demo after. 04:09.000 --> 04:12.000 So, we have seen new surface attacks. 04:12.000 --> 04:16.000 But the core question, when machine learning system, 04:16.000 --> 04:18.000 are fundamentally different. 04:18.000 --> 04:22.000 So, for me, there is a key point. 04:22.000 --> 04:26.000 The first data is a code. 04:26.000 --> 04:32.000 In traditional software, you can use it with your code. 04:32.000 --> 04:36.000 To understand, there is a better machine learning. 04:36.000 --> 04:43.000 Tuning data has its own logic, so when you inject poison data, 04:44.000 --> 04:51.000 you have to confirm the model with a new code. 04:51.000 --> 04:57.000 The second part is the logic, which is more opaque. 04:57.000 --> 05:01.000 Because you can step through a debugger with a billion of parameters. 05:01.000 --> 05:03.000 It's a very important part. 05:03.000 --> 05:10.000 So, there is no clear execution point. 05:10.000 --> 05:11.000 So, that's all. 05:11.000 --> 05:15.000 Understanding why you model a method of decision, 05:15.000 --> 05:18.000 which is actually a problem yourself. 05:18.000 --> 05:26.000 On the last one, it's a margin of behavior. 05:26.000 --> 05:31.000 Oh, it's a bit more easy for all forms. 05:31.000 --> 05:37.000 So, why I told you about security, the AI, no. 05:37.000 --> 05:40.000 I have put the context. 05:40.000 --> 05:45.000 On the one hand, we will see the fluid landscape in 2026. 05:45.000 --> 05:53.000 The two-panduation is the first one, is the worst element of the token. 05:53.000 --> 06:02.000 We have also more of a six thousand attack technique in the meter at last. 06:02.000 --> 06:05.000 On the other hand, 2026, more than the expression. 06:05.000 --> 06:09.000 So, as a jbreak on the x. 06:09.000 --> 06:14.000 So, it's not a small problem, so it should be. 06:14.000 --> 06:20.000 It is only to become a bigger problem. 06:20.000 --> 06:28.000 So, now, as I have put the context, as a constant part, 06:28.000 --> 06:34.000 I will continue to organize our model from work in a machine learning system. 06:34.000 --> 06:39.000 So, you have four questions to answer. 06:39.000 --> 06:44.000 The answer is the theory vector on the signals. 06:44.000 --> 06:49.000 You have to answer the question to us. 06:49.000 --> 06:51.000 What are we protecting? 06:51.000 --> 06:53.000 This is the model IP. 06:53.000 --> 06:56.000 Is it a training that that privacy? 06:56.000 --> 07:01.000 Or, in particular, is the first part. 07:01.000 --> 07:06.000 In the second part, you have to focus on the other theory. 07:06.000 --> 07:10.000 We will text, is it an external IP address? 07:10.000 --> 07:13.000 Is it, also malicious inside there? 07:13.000 --> 07:16.000 Or, compromise the vendor? 07:16.000 --> 07:21.000 The further part of this framework is the vector. 07:21.000 --> 07:29.000 Or, do the attack for IPI access or training pipelines or the supply chain? 07:29.000 --> 07:33.000 On the last, it's more of the signals. 07:33.000 --> 07:34.000 What detects attacks? 07:34.000 --> 07:35.000 Is it metrics? 07:35.000 --> 07:36.000 Is it log? 07:36.000 --> 07:40.000 Is it traces specific to machine learnings? 07:40.000 --> 07:46.000 I just have a look at the time, okay? 07:46.000 --> 07:53.000 So, we will not make a little dive on the detection of the patterns. 07:53.000 --> 07:56.000 We have three patterns. 07:56.000 --> 07:58.000 The result is in two detection. 07:58.000 --> 08:01.000 The model will be very much including. 08:01.000 --> 08:04.000 On the LSDC security. 08:04.000 --> 08:10.000 So, for each pattern, we have detection signal. 08:10.000 --> 08:15.000 We have metrics, which are based on the vector metrics. 08:15.000 --> 08:16.000 We have a lot of tools. 08:16.000 --> 08:19.000 We need this executioner quiz. 08:19.000 --> 08:23.000 But, no benefit is the demo. 08:23.000 --> 08:34.000 I don't think I will share the results at the end of this talk. 08:34.000 --> 08:35.000 Yes. 08:35.000 --> 08:36.000 Okay. 08:36.000 --> 08:40.000 So, the first, we have another failure in the detection. 08:41.000 --> 08:45.000 The goal is to detect and put craft to fully on the model. 08:45.000 --> 08:48.000 Why they are putting a normal tool to human? 08:48.000 --> 08:54.000 So, the key insight is the sign and put of the signature. 08:54.000 --> 08:57.000 These are some of the icons. 08:57.000 --> 09:00.000 On the icon, the icon, reconstruits your error. 09:00.000 --> 09:04.000 So, that's why we need one for this. 09:04.000 --> 09:07.000 We have four detection signal. 09:08.000 --> 09:11.000 Everything, in which we're concerned, we're concerned about error. 09:11.000 --> 09:14.000 We're instability for prediction. 09:14.000 --> 09:16.000 I will ask you this. 09:16.000 --> 09:19.000 Also, if you do a space in distance, 09:19.000 --> 09:20.000 I'm sorry. 09:20.000 --> 09:23.000 Yeah, this is a demand. 09:23.000 --> 09:30.000 For example, I have, you will see a point where you need tricks. 09:30.000 --> 09:32.000 That's an example. 09:33.000 --> 09:36.000 Or you can look at the position. 09:36.000 --> 09:40.000 The second one, which is the model, the behavior monitoring. 09:40.000 --> 09:45.000 So, in this case, the goal is to detect the positioning on the model, 09:45.000 --> 09:49.000 set by monitoring behavior over time. 09:49.000 --> 09:54.000 So, we have four parts, the prediction is to the distribution route. 09:54.000 --> 09:58.000 We have, with your metrics. 09:59.000 --> 10:02.000 The query patterns, the situation patterns change, 10:02.000 --> 10:07.000 or the performance revision. 10:07.000 --> 10:10.000 So, another one example of a rule. 10:10.000 --> 10:16.000 You can deal with a problem query. 10:16.000 --> 10:21.000 So, the one, it's a dynamic specific security monitoring. 10:21.000 --> 10:23.000 We have to, so, I will discuss the detection 10:23.000 --> 10:26.000 and the detection menu, the direct point injection. 10:26.000 --> 10:29.000 For example, you set to your AI. 10:29.000 --> 10:33.000 If you know, for the pilot's instruction, or whatever, 10:33.000 --> 10:35.000 you have indirect injection. 10:35.000 --> 10:40.000 So, you can put a machine to control the rich with the document. 10:40.000 --> 10:45.000 You are jail-breaking on a data system-point situation. 10:45.000 --> 10:52.000 So, like for the, you have an example of metrics, you can export. 10:52.000 --> 11:05.000 So, I have, so, yeah, I have an example of a description of a, 11:05.000 --> 11:09.000 as a framework of organization of metrics. 11:09.000 --> 11:13.000 How can you build an internal tuning? 11:13.000 --> 11:18.000 So, I have a build a run, which has, in the first run, 11:18.000 --> 11:23.000 a build a critical, a goal to run, to test, 11:23.000 --> 11:27.000 or you can test, you can put your monitoring. 11:27.000 --> 11:33.000 So, I have a build, this stack, with, for tools, 11:33.000 --> 11:39.000 the monitoring, which I have focused on metric on the editing. 11:39.000 --> 11:43.000 We refer to the visualization, and I can, 11:43.000 --> 11:47.000 which is also looking for the regulation of log. 11:47.000 --> 11:51.000 On the open telemetry, only for tracing. 11:51.000 --> 11:56.000 We stack as, for, of course, it's, no vendor looking. 11:56.000 --> 12:00.000 We use less resources. 12:00.000 --> 12:04.000 It is prone to a compacting. 12:04.000 --> 12:09.000 An example of a structure, a looking, with a key. 12:09.000 --> 12:13.000 You have also a look shimmer, with, from machine learning security. 12:13.000 --> 12:17.000 We use the application of queries. 12:17.000 --> 12:20.000 But I will, for, dashboard. 12:20.000 --> 12:23.000 So, in terms of integration, we already take clear. 12:23.000 --> 12:27.000 So, you have this model, you have a machine learning, 12:27.000 --> 12:29.000 you have a model or, you have, you have, 12:29.000 --> 12:34.000 an, for, your pipeline, you have also your expertise. 12:34.000 --> 12:38.000 You have a, on, after, you have, you have so, 12:38.000 --> 12:51.880 So, as we use the open source, we have the chance to have something flexible. 12:51.880 --> 12:57.320 You can also building your customer machine learning security expertise. 12:57.320 --> 13:08.320 So, in this slide, you have an example of an exporter, which is a basic problem with this. 13:08.320 --> 13:17.320 So, now, I have finished with the theoretical part on the open source of the figure. 13:17.320 --> 13:24.320 So, you can do a quick demonstration. 13:24.320 --> 13:42.320 So, you can do a quick demonstration on the open source of the figure. 13:42.320 --> 14:02.320 So, the connection is very slow, but in routing for, I will have, also over. 14:02.320 --> 14:10.320 So, actually, it will be difficult to make the move due to the connection. 14:10.320 --> 14:18.320 So, I will want you to make a live demo, but whatever, I will explain a quick thing. 14:18.320 --> 14:27.320 In this slide, you can make some tests at attack with several levels of security. 14:27.320 --> 14:41.320 So, in this slide, you have three attacks. 14:41.320 --> 14:48.320 And first, I will put model extraction on the problem injection. 14:48.320 --> 15:04.320 So, I have a few, before you will be following this one, I can catch all of this here on this address. 15:04.320 --> 15:11.320 And in addition, obviously, I will show you some fit model from work. 15:11.320 --> 15:19.320 You come, you have some model of primitive selection on the circuit relation. 15:19.320 --> 15:25.320 But what I have present a quick thing, it's a natural reduction. 15:25.320 --> 15:32.320 So, more concentration of the problematic of security on your LLM application. 15:32.320 --> 15:44.320 I think that we would come often, but to be honest, both of this approach, there is some limitation on consideration. 15:44.320 --> 15:56.320 The first one, it's not a cyber bullet, because the pattern I have present, I have this one, detect non-attack signals. 15:56.320 --> 16:04.320 But, if you see everything which concerned, they were there on several techniques and many of the detection insurers. 16:04.320 --> 16:11.320 So, that's alright, this is a difference in depth, not a complete solution. 16:11.320 --> 16:24.320 The second limitation is, if you have a separate requirement, you can switch on the error detection means a training, a training, a auto encoder. 16:24.320 --> 16:28.320 On baseline, there's a decrolibating parameter. 16:28.320 --> 16:33.320 On the other side, that's one, which is a particular pattern. 16:33.320 --> 16:40.320 When you do a monitoring, it is a risk of a further positive. 16:41.320 --> 16:47.320 So, I have a finish with presentation. 16:47.320 --> 17:00.320 So, in conclusion, it is a natural reduction, I have made a natural reduction of a risk topic, which is not so easy to understand for the first time. 17:00.320 --> 17:04.320 I have given some keys on some repairs. 17:04.320 --> 17:08.320 In a one sentence, you have an error mode, it is an attack surface. 17:08.320 --> 17:12.320 So, you have to monitor it like one. 17:12.320 --> 17:27.320 On short question, if you want to take it further, to either we have a plan webinar on a master class. 17:28.320 --> 17:33.320 So, in March 20, 26, we will organize a webinar about this topic. 17:33.320 --> 17:39.320 We will make a dive with demo in life with some attack. 17:39.320 --> 17:49.320 In order to see how you can organize and can do protect your LLN based on the open source monitoring tools. 17:49.320 --> 18:04.320 On April 20, 26, we will organize a master class, but we will focus on an LL1 monitoring activity. 18:04.320 --> 18:12.320 So, we will talk about secreting aspect. 18:12.320 --> 18:16.320 So, thank you, Raymond, for your attention. 18:17.320 --> 18:24.320 So, we've got plenty of time. 18:24.320 --> 18:28.320 So, do we have some questions before we will leave? 18:28.320 --> 18:30.320 Riser hands. 18:35.320 --> 18:37.320 Yeah, thanks for the talk. 18:37.320 --> 18:43.320 So, you mentioned the starting with the high force positive, and then lower it down. 18:43.320 --> 18:45.320 What happens in between? 18:45.320 --> 18:55.320 Do you use the legitimate out of distribution data to fine-tune? 18:55.320 --> 19:00.320 Yes, how do you manage to lower down the force positive? 19:00.320 --> 19:03.320 What happens in between? 19:03.320 --> 19:09.320 Do you use the legitimate out of distribution data to fine-tune? 19:09.320 --> 19:14.320 Yes, how do you manage to lower down the force positive? 19:14.320 --> 19:21.320 Monitoring is an improvement in your model, but it's not the perfect model. 19:21.320 --> 19:26.320 With your utility, you can improve your model. 19:26.320 --> 19:31.320 When you are the first positive, you can improve a choice. 19:31.320 --> 19:35.320 I talked about monitoring, but you have also a possibility. 19:35.320 --> 19:41.320 So, you can improve your model on reduce the part of a false positive. 19:41.320 --> 19:44.320 You can avoid it or whatever you do. 19:44.320 --> 19:53.320 It is a, you can monitor high-tune in a multi-system, you can monitor it, but that's the same thing. 19:53.320 --> 19:58.320 Do we have some other questions in the room? 19:58.320 --> 20:00.320 So, if not, thank you for your presentation. 20:00.320 --> 20:04.320 Thank you for your questions and see you on the next talk.