WEBVTT 00:00.000 --> 00:09.280 Hello everyone, my name is Alex Mikhalitsyn, I'm working for containers for a few years already, 00:09.280 --> 00:16.640 also in the Xe containers project and also I'm doing some checkpoint restoring user space stuff, 00:17.840 --> 00:26.480 little obscure now, and today I'm going to talk about the second listeners and bringing support for 00:26.480 --> 00:36.720 nested listeners. So first of all, just a really quick crash course in the second, 00:37.600 --> 00:44.480 so we have two different modes, we have a second set strict mode, which only allows a few 00:44.480 --> 00:53.040 secals, not very useful for containers applications, because we need much more. So today we're going 00:53.040 --> 01:00.160 to focus on the second one, which is a filter mode where users space can load arbitrary classical 01:00.160 --> 01:10.560 BPF program that can judge if this particular Cisco should be allowed or not, and as an input, 01:10.560 --> 01:18.560 this program receives a COMP data struct that contains a Cisco number, architecture, arguments, 01:18.880 --> 01:24.400 instruction pointers, so the program can decide if that's the only information that program can get, 01:24.400 --> 01:30.880 and based on that Josh if Cisco should be allowed or not, and then at the end program should 01:30.880 --> 01:38.880 return and result, that is basically decision, it can be allowed to adjust where this is called to be 01:38.880 --> 01:46.640 executed or log Cisco, and at the same time allow it to be executed for trace, or what is really 01:46.640 --> 01:55.360 important and interesting for us is user 95 thing, which means that we're going to discuss what 01:55.360 --> 02:05.360 this means, and in a filter mode, how another important thing to understand is that in the kernel 02:05.360 --> 02:13.040 second filters are represented by struct second filter, which is basically, which is a stable structure, 02:13.040 --> 02:21.280 so only if you fields are actually can be changed, it's rev counters basically, all the rest 02:21.280 --> 02:27.280 of is static, so once you load the BPF program into the kernel, it's stable, you cannot replace it, 02:27.280 --> 02:34.720 even the privileged user cannot modify the filters after they allowed it, and also filters are 02:34.720 --> 02:40.960 inherited, and filters can be staked, so once you for the process you can load another filter, 02:40.960 --> 02:46.240 or even within the single process, you can load a bunch of, you can you can call the second 02:46.240 --> 02:50.960 Cisco a few times, and load different if you different BPF filters, there is no problem, 02:50.960 --> 02:59.040 they will just be naturally staked and executed in a right order, so as I mentioned the ice, 02:59.040 --> 03:04.320 there is an entry, so we can always go up the tree to, that's what we do when we actually 03:04.320 --> 03:15.040 like figuring out if this is called, what we're going to do is this is called right, and so how, 03:16.240 --> 03:22.960 actually I mentioned that, right, so how kernel traverses through the tree, so we always start from 03:22.960 --> 03:30.400 the current test struct, find the last filter, and then we just go up and execute every single 03:30.720 --> 03:37.440 classical BPF program, see the check the result, and the logic is that we should find the 03:37.440 --> 03:48.720 strictest result in the tree, and this will be the final decision that kernel will enforce on the 03:48.720 --> 03:56.240 user space, right, and today we're going to, for this talk, we're going to assume that the final result 03:56.320 --> 04:01.920 is interesting for us, which means user 95 because this is the case that covers the listeners, 04:03.520 --> 04:11.520 so how, how this user 95 works, so from the user space perspective, user space, when loading the 04:11.520 --> 04:17.200 filter program should set a special flag, called second filter flag new listener, 04:18.080 --> 04:23.280 which means that user space basically wants kernel to allocate a special file descriptor, 04:23.840 --> 04:29.760 that SIPOMC is called will return, and we can use this for the SIPOMC to gather with ioccal 04:29.760 --> 04:36.080 SIPOMC to receive notifications or, and also send a reply to notifications, so basically, 04:37.600 --> 04:44.400 it's a combination of two things, so you, you load the B classical BPF program that in some circumstances 04:45.280 --> 04:56.480 returns user 95 result, then kernel puts, during this SIPOMC, kernel puts the current process 04:56.480 --> 05:02.880 into the SIPOMC, and then sends a notification to the user space, user space reads that notification 05:02.880 --> 05:10.960 from the file descriptor, and once it user space judges and decides what to do with this SIPOMC, 05:10.960 --> 05:15.360 user space can actually implement the SIPOMC basically in user space, or do something, 05:15.360 --> 05:20.160 really stupid, like, faking a SIPOMC basically, right? It's sometimes very useful, 05:20.880 --> 05:25.920 and then once the user space is ready with the answer, it can just send a reply notification 05:25.920 --> 05:32.240 and process continues, right? And also there are some super cool features, like you can actually 05:32.240 --> 05:37.680 even emulate the SIPOMC that are modifying the photo sticker table, because there is a feature for that too, 05:38.240 --> 05:45.040 and another last button, please, the very important thing, is that user space can also also 05:45.040 --> 05:52.000 decide to do a continue, which means that the original SIPOMC will be executed, 05:52.000 --> 06:00.320 because in classical, initially when the user notified your edit, the SIPOMC logic was basically replaced, 06:00.960 --> 06:06.960 right? And this is really important, this continued to think is really important for 06:06.960 --> 06:15.600 today talk. So, you might ask a question, like when they really need that. So, in like C project, 06:15.600 --> 06:23.440 for example, we heavily rely on that, and some cases, for example, it's a well-known thing in 06:23.440 --> 06:28.240 the kernel that things like MKNOS just call are not really allowed in the user namespace, 06:28.240 --> 06:35.120 there are reasons for that. It's, we try a few times to change this, but there are some security 06:35.120 --> 06:41.440 concerns around. So, basically, if you want to use the fully secure containers, which are, of course, 06:41.440 --> 06:48.000 containers using user namespace, then you kind of limit it to do MKNOS. So, even, you cannot even 06:48.000 --> 06:53.840 create a dev now device notice, either container with that, right? So, we need some way to do this, 06:53.920 --> 07:00.720 to circumvent this limitation, right? And also, there are some other cases, like, for example, 07:00.720 --> 07:08.160 if you want some, there are some in-system containers, we're doing some virtualization from things 07:08.160 --> 07:14.320 like progress, women's forething, or, or, or pro-captime, and others, and sometimes users 07:14.320 --> 07:20.640 pays may also obtain the same information using, for example, C-synthosis call, and if we also 07:20.640 --> 07:26.240 want, sometimes we want to emulate that, because we want to basically fool the user space inside 07:26.240 --> 07:34.400 the container, and tell the virtualized view on the memory, and get the memory limits from the 07:34.400 --> 07:41.120 C group instead of reporting the host information, right? There are also some other use cases 07:41.120 --> 07:45.760 when people doing interesting things like running Android inside the container, that's something 07:45.760 --> 07:50.640 actually like, it's, this, this, this problem grown from that, quite a case, because the 07:50.640 --> 07:56.320 guys from Unbox Project approached me, and actually told me a story about what they do, and 07:56.320 --> 08:06.080 the solution was kind of very natural to do, and instead listeners. And what's the problem? 08:06.080 --> 08:12.160 The current limitation of, or, with listeners is that you can only get one listener. So, 08:12.160 --> 08:17.520 why have you no limitation about how many second filters you might have? Actually, there is a limitation. 08:17.520 --> 08:22.720 It's, it's, it's, it's, it's, it actually measures number of BPM instructions, so there is a, 08:22.720 --> 08:27.600 there is a, there is a limit for that, but this kind of, not, not that, I cannot say, like, it's 32 levels, 08:27.600 --> 08:32.400 or, or, sometimes, it's basically measured in number of, it depends on the sizes of the BPM 08:32.400 --> 08:40.160 programs involved, right? But in general, there is no nature, like, explicit limit of 08:40.160 --> 08:49.200 nesting, so we, you can nest, BPM, BPM, C-confield, but if you, once you register, once you 08:49.200 --> 08:55.360 obtain a, listener file descriptor, and if you try to install another one, you will get a 08:55.360 --> 09:02.400 BZ error from the kernel, because there is a specific check for that, and, I mean, when I started 09:02.400 --> 09:09.120 looking into this, my first, maybe, like, for, for 15 minutes, my impression was that it's, 09:09.120 --> 09:14.160 it's kind of easy to get rid of this limitation, but then I looked closely, and I noticed 09:14.160 --> 09:19.280 that there is a feature that was added a bit later, it wasn't there from the beginning, 09:19.280 --> 09:24.480 which allows to continue to score, which, as I mentioned earlier, is about, basically, 09:26.400 --> 09:32.640 letting kernel note that we just want to, actually, let the original C-scal logic to be executed, 09:33.600 --> 09:40.960 and this thing actually is a problem in this case, so we need to, we need to be really careful 09:40.960 --> 09:46.960 because otherwise, we, we, we can buy in this feature, we've nested listeners, we can easily 09:46.960 --> 09:54.640 make the way to, actually, bypass a secont, which is not good, right? So the proper way, 09:54.640 --> 09:59.120 what, what is the proper way, right? We just need to iterate over the filters three, as usual, 09:59.200 --> 10:10.080 we need to figure out what is the, let's say, a most restrictive decision, right? And if it's 10:10.080 --> 10:18.080 95, we need to go up the tree, basically, again, but now in this turn, we need to go through those 10:18.080 --> 10:26.880 folks who return 95 and ask every single, right? So if, for example, if the last filter 10:26.960 --> 10:31.840 tells you that you should continue to score, you cannot just go and continue, you need to go up the 10:31.840 --> 10:40.000 tree, ask the, ask the next one, if the next one tells you that, for example, we need to return an error, 10:40.000 --> 10:45.120 we just need to return an error, right? We cannot just allow to, this thing to go further. 10:45.280 --> 10:56.000 So, and while carefully, so, basically, the patch was already really quick, but then I started to, 10:56.000 --> 11:04.560 to validating myself and shaking like, if I'm not introducing something dangerous, and I was actually 11:04.560 --> 11:11.840 learning the entire code, checking different corner cases, and actually, the comp has amazing, 11:12.480 --> 11:19.920 amazingly covered with case of tests. So, there are already a few of the self tests that were born 11:19.920 --> 11:28.640 from a real bug reports and tricky use cases, so I was carefully checking everything, and so I came up with 11:28.640 --> 11:36.800 some sort of abilities that were really interesting. And so, the problem one, let's start from this 11:36.800 --> 11:42.800 simple chunk of code that you, unfortunately, have to remember, not, not fully, of course, but 11:42.800 --> 11:49.440 idea is simple, so we have a VPN program that has only one instruction that returns a 95, 11:49.440 --> 11:54.720 so the basicly this program always tells, please 95 years by. So, basically, for every, 11:54.720 --> 12:00.960 no matter which this call was called, this thing will say 95, let's see, right? And the second thing 12:00.960 --> 12:07.520 is not really interesting, it's just like we just connecting these filters to the socket program, 12:08.640 --> 12:15.040 because the VPN is a classical VPN, so, and then we use this P variable everywhere on the other 12:15.040 --> 12:20.240 slides. So, sorry for that, but it's, there is no other way really to show this. So, and, 12:20.240 --> 12:28.880 but I will explain everything. So, we have two columns, so on the left, we have an example of the 12:28.880 --> 12:36.320 how we can call this CACOMP CISCOL, and also, to us, the kernel to allocate a listener, and, 12:36.320 --> 12:41.760 as you can see, the last argument is this pointer to this P structure. So, basically, we just 12:41.760 --> 12:47.200 asking the kernel to load this program that always returns 25, and allocate a listener for us, 12:47.200 --> 12:52.880 and return that listener as a file descriptor, right? So, what I do, and basically stupid thing, 12:52.880 --> 12:57.760 I immediately closing file descriptor, that's okay, right? I can do this, it's legitimate, right? 12:57.760 --> 13:06.160 On the, on the right, a colon, we have another call for CACOMP, and now we just loading the same 13:06.160 --> 13:14.320 BPS program, but I'm, I'm, I'm, I'm, I'm it the, uh, uh, listener, uh, flag, which means that kernel 13:14.320 --> 13:19.680 won't allocate a listener, but the program still will return 25, right? So, what usually kernel 13:19.680 --> 13:24.880 doing this case, if, if it sees that, if kernel sees that there is no notification of the, but, 13:25.120 --> 13:31.600 uh, program returns 25, kernel will return ENOCs, right? Because kernel will see that, 13:31.600 --> 13:36.640 there is nobody who listens for notifications, and just, it cannot allow us this call to 13:36.640 --> 13:44.480 continue just ENOCs, right? Um, so, uh, the tricky point is that, actually, the code on the left, 13:44.480 --> 13:50.800 and code on the, on the right, produces precisely the same state in the kernel, which means that 13:50.880 --> 13:56.720 there is no difference, right? But you might wonder, like, how, in a way? It, it is how it works, 13:56.720 --> 14:01.520 uh, because, uh, once we close the for the sector, all the, uh, struck fellow, 14:01.520 --> 14:05.200 busy getting, uh, released, all the, all the, all the, all the, all the, all the, all the, connect 14:05.200 --> 14:10.640 stuff, oh, also getting released, and also, in a second filter structure, we have a pointer, 14:11.840 --> 14:18.640 uh, basically, to this notification, which also gets nullified, and basically, we get to the 14:18.720 --> 14:25.360 same state as, we would just, from the beginning, uh, load this program without allocating a filter, 14:26.400 --> 14:34.400 allocating a listener, sorry. Um, let's then consider that the next bunch of snippets, right? On 14:34.400 --> 14:41.920 the left, we, we are trying to, uh, load the program via a listener, and then we trying to load 14:41.920 --> 14:46.480 the second one with listener, and we obviously get a busy, because of this limitation for 14:46.560 --> 14:53.360 the next thing, right? Uh, then on the second column, we, what we do, basically, it's very 14:53.360 --> 14:58.560 natural if you think about the previous slide, right? Because once we, uh, as we learned from the previous 14:58.560 --> 15:03.280 slide, once you close the for the sector, you, you getting the same state, as you would not 15:03.280 --> 15:07.440 really allocate the listener, so basically the second thing, what it does it allocates, 15:07.440 --> 15:13.120 listener, then closes it, then allocates a listener again, and this is fine. There is no real 15:13.200 --> 15:19.200 nesting, right? Because the listener was close, so it's only one level. And the third column, 15:19.200 --> 15:26.640 uh, it's just normal, normal scenario when we, uh, when we, uh, allocating the, uh, one listener 15:26.640 --> 15:32.320 explicitly and, uh, loading the program without this. And so, basically, the second and third column 15:32.320 --> 15:40.800 are the same from the kernel perspective. And, uh, so how it works, I basically explain it, so 15:40.800 --> 15:47.120 and everybody in this room, you know, a lot about kernel and picture. So, uh, yeah, basically, 15:47.120 --> 15:54.320 just, uh, we, we load the filter. For example, the, uh, what your process, uh, somehow mediates the 15:54.320 --> 16:00.160 Cisco's and stuff, right? And the process that is running under the second restrictions, 16:00.160 --> 16:04.960 in this particular scenario, I'm considering it gets pizza score, as an example, but it doesn't 16:05.040 --> 16:11.200 really matter, right? So, uh, process, let's say, cause a get pizza slips, then, uh, kernel sees that 16:11.200 --> 16:17.600 there is no 35G because we haven't asked the kernel to allocate one, right? And then the, uh, kernel, 16:17.600 --> 16:23.680 I see that there is no way to send an notification and the return signal sees. Uh, next example 16:23.680 --> 16:29.120 is that we, we, we, we, we, we, we, loading, uh, we actually asking the kernel to allocate a listener 16:29.760 --> 16:36.720 and before that, we are also loading the program without this here. So, then tricky, tricky thing is 16:36.720 --> 16:46.800 that first, uh, the step number one on the left column has no effect, right? And, uh, I was, uh, showing 16:46.800 --> 16:52.880 this, uh, uh, uh, uh, presentation and we have discussed, we have a great discussion on this 16:52.960 --> 17:02.400 bumper's conference this year. And, uh, I've gotten very, uh, good, uh, input from Alexa and Alexa, 17:02.400 --> 17:09.440 Soraya basically mentioned that it feels like a bug. So, I was initially very careful, maybe too 17:09.440 --> 17:15.840 careful, because I, because I have a feeling that this is a bit weird and looks like a bug, but at 17:15.920 --> 17:23.600 the same time, I was kind of afraid to change the user space. And, but Alexa, when pointed 17:23.600 --> 17:31.040 and a key scoop who is maintainer, um, of second, uh, they actually agreed that this, this sounds 17:31.040 --> 17:37.200 like a bug. So, the, the plan is to, to maybe change this behavior because probably nobody really 17:37.360 --> 17:45.520 relies on this, hopefully. Uh, so next problem, uh, it's really, I mean, um, uh, it's, it's actually 17:45.520 --> 17:49.760 solved, like it's, it's the opposite performance, right? Because the second one was executed on a hot 17:49.760 --> 17:58.000 path in a Cisco, and we cannot really modify the main loop where we going through the filters. 17:58.000 --> 18:04.320 So, once we start going, uh, in, in my current approach, I basically have two different iterations. 18:04.400 --> 18:09.680 So, first time, I iterate quickly through the all the BFF programs up, figure out, figure out 18:09.680 --> 18:16.560 the result, and once I know the results is 95, I go again, but now loop will be shorter, because I 18:16.560 --> 18:28.880 only need to go through the, through those folks who, um, reported, uh, 95, right? Uh, next question 18:28.960 --> 18:36.800 is, uh, the semantics, because in, in the current approach, I've sent to LKML is to introduce, um, 18:36.800 --> 18:45.040 explicit flag, that, uh, that, that user must set to allow listeners to be installed afterwards. 18:45.040 --> 18:52.480 So, basically, it means that, if you go with that way, it means that Lips a comb, uh, other, 18:52.480 --> 18:58.400 a Lips a comb users. I mean, Lips a comb should be modified to probably by default, just allow 18:58.480 --> 19:04.400 listeners, nice listeners, because there is no real security issues them, uh, they are just 19:04.400 --> 19:09.920 saved by design, right? So, uh, because of this, that we check every sink in the tree, uh, 19:09.920 --> 19:15.760 or, you know, also we need to modify every single project that relies on sick, 19:15.760 --> 19:20.640 on listening to right now, but doesn't use Lips a comb, because so we basically need to write 19:20.640 --> 19:26.000 this flag everywhere. So, and when, when we think about this, it feels like, okay, maybe we should 19:26.160 --> 19:31.840 just drop this flag really and just by default allow, nice to go. But there are some concerns 19:31.840 --> 19:36.560 around that. So, we have, uh, actually, I will, in the presentation that we'll be adding to LKML. 19:36.560 --> 19:42.560 So, please, if, if anybody interested, you can join the discussion, but yeah, it's, it's, it's not that 19:42.560 --> 19:51.280 trivial. Uh, another worry we're interesting, uh, use case was reported by Andre Wagen, who is, uh, 19:51.360 --> 19:57.760 working on Javizer in Google, and he told me a very interesting story about how they were using 19:57.760 --> 20:04.400 a sick comb, um, because the, in Javizer is basically application kernel, right? So, and they have 20:04.400 --> 20:09.680 a few different backgrounds, uh, one is petrase based, another one is sick comb based, and another one 20:09.680 --> 20:18.880 is kidney and based. So, uh, I'm very pointed out that, uh, some of, uh, sick comb semantics 20:18.960 --> 20:26.160 makes it not very efficient, uh, for Javizer needs, and we can consider changing some of those, 20:26.160 --> 20:36.960 or maybe extending, uh, specifically, um, it is about, uh, notifications, and about how we, um, 20:38.960 --> 20:45.200 about, uh, about the, the fact that when we traverse the tree, we're trying to find the most 20:45.360 --> 20:54.160 restrictive result, and, and once you find it, we, uh, it, it can't be that, for example, 20:54.720 --> 20:59.360 on the first level, you, you, you, let's say you uploaded a VPN program that says deny, right? 20:59.760 --> 21:06.240 But, uh, then you forked the process, allowed it another filter that says 95, right? So, what we'll 21:06.240 --> 21:12.960 happen now is kernel will say deny. Notifications won't be delivered, because as, as, as somebody 21:13.040 --> 21:18.480 up the tree says deny, we shouldn't even send an notification. So, and the point was that, 21:18.480 --> 21:25.920 okay, but if you have notifications, um, and if you want, if you want going to use continue, 21:26.560 --> 21:30.960 then it's safe, actually, to deliver an notification, and let it use this way to do something, right? 21:31.600 --> 21:37.200 Because the original, this is called logic will, will never be executed anyways, right? So, 21:37.520 --> 21:41.920 it's kind of hard, because it probably will require to introducing another flag, 21:43.200 --> 21:49.600 thanks, another flag for, uh, for notifications, because I need to, I need to, to dive into 21:49.600 --> 21:55.040 even more details about how this stuff, forks to explain, but I have some, I had some idea about how to 21:55.040 --> 22:00.880 do this, but it's, it's a, it's probably a matter of different, um, work, um, associated with this, 22:00.960 --> 22:07.440 connected, but it'll, it'll, it'll, it'll be different. And yeah, here, actually, like, showing my, 22:07.440 --> 22:14.000 my, really brief idea about, that we can introduce another flag, that, uh, because, actually, this, 22:15.520 --> 22:22.160 decisions, uh, decisions that program returns, they actually just numbers, and they are basically, 22:22.880 --> 22:26.560 in the kernel, if you look in the logic, you will just see that the basic number, 22:26.560 --> 22:32.800 comparison, right? So we even iterate over the decisions, we just compare them, uh, as numbers, 22:33.760 --> 22:40.880 and, uh, they have sorted, so basically, uh, the, um, least number means, uh, more restrictive 22:40.880 --> 22:49.760 result, and so what we can do is, user interfaces, um, um, between, um, trap, which means, uh, to 22:49.760 --> 22:55.120 generate a signal, basically, and allow, and we can introduce another one, uh, that will be 22:55.200 --> 23:01.440 between trap and Q, that will make it, uh, more restrictive that trap, so it will allow this 23:01.440 --> 23:07.200 combination of trap of returning trap and returning can identify. So, but again, it's, it's, it's, 23:07.200 --> 23:17.680 it's probably something that should be handled separately. Uh, so yeah, as I mentioned, I, uh, I, 23:18.560 --> 23:27.840 as a keyscook suggested, we should, uh, really fix this semantico problem, I describe a problem, 23:27.840 --> 23:33.360 as a problem number one, as a separate patch set, probably get it merged, and then I can 23:33.360 --> 23:41.040 re-spend a series with nested listener support. This is my plan right now. And yeah, that's basically 23:41.600 --> 23:48.000 it, uh, also thanks, or many great people who were involved and, and left the comments and 23:48.000 --> 23:52.640 reviewed the stuff. And yeah, that's it. Thanks for coming from me at all. 23:58.640 --> 24:02.720 All right, any questions? Oh, comments. I'm going to go with comments. 24:02.720 --> 24:07.120 Thank you. Uh, I was just a full, we should talk about later, because it sounds crazy to me, 24:07.120 --> 24:11.920 but I was going to say for, for the, for the previous thing, um, do you know what the reason is? 24:11.920 --> 24:16.560 Why, pausing a filter with the user notified that, the, the can return user notify? Why, 24:16.560 --> 24:20.080 second doesn't refuse that if you don't pause the listener flag, because the, because the, 24:20.080 --> 24:25.440 feel, the filter verify could check it, but it, it doesn't. Do, do, do you know why? That's 24:25.440 --> 24:30.400 the case, because it seems a little bit funny. I'm sorry, can you? Oh, so wait, so the, the example, 24:30.400 --> 24:33.920 we had at the very beginning, where you have a filter that returns user, like, uh, user, 24:33.920 --> 24:39.280 notify return, um, second will happily load that filter, even if you don't pause the listener flag. 24:39.280 --> 24:43.440 Yeah. But second, like the typical could check and deny the load. I mean, with, with way past that, 24:43.440 --> 24:46.480 we can't fix that now, but do you know, I need a why, why that's the case, because it sounds, 24:46.480 --> 24:50.080 it's a little bit funny if you think about it. But yeah, it's, it's funny, but as I said, 24:50.080 --> 24:55.040 how, how can you, the BFF program, a lot of time check that, you can't, because you can't, 24:55.040 --> 24:57.840 there's no indirect references. It's, like, it's literally, you just check what the return 24:57.840 --> 25:01.680 goes, like you can totally do it, the verify, because CBPF doesn't have, uh, like, 25:02.320 --> 25:04.720 because every return is in immediate return. There's no, there's no, there's no, there's no, 25:04.720 --> 25:08.560 yeah, and the sense of basically, you're saying that this specific case, we could cover, 25:08.560 --> 25:13.120 which is a very fair, right? Yeah, but, I mean, it doesn't be cashing. We could do what it's like, 25:19.120 --> 25:22.000 so, the only, I have a much more, a general question, 25:23.040 --> 25:28.400 when I hear your thoughts on this, like, isn't it, like, long way down the road, kind of a conceptual, 25:28.400 --> 25:33.760 I mean, intercepting system calls just to maintain our workloads inside of the container, 25:33.760 --> 25:39.120 be transparent, like, not have to worry about, for example, consuming the educated APIs, 25:39.120 --> 25:47.680 isn't that in the long run, not going to work, like, if you have complex system call machinery, 25:47.680 --> 25:53.760 that keeps state across system calls, and you can do it with the second listener, right? Unless 25:53.760 --> 25:58.320 you switch to something like, or combine it with EMPF where you have maps and stuff and so on, 25:58.320 --> 26:03.360 like, I'm just, I use to think this is, like, a really, really good idea, and it's just, for some extent, 26:03.360 --> 26:09.360 this is a really, really good idea, but I, I think it's way better if, like, we push people to 26:09.360 --> 26:15.440 what's like, you need to consume dedicated APIs for this. If you want to mount down privileged stuff, 26:15.440 --> 26:20.000 then probably you need to, like, do it via a, a simple IPC protocol. 26:20.960 --> 26:25.280 Yeah, I mean, which, like, if I get this working, I notice this is not, this is not as pleasant, 26:25.280 --> 26:28.880 because, like, if you have legacy workloads inside of the container, obviously, this won't work, 26:28.880 --> 26:34.080 necessarily, but in the long run, I think this is a much more feasible way that we need to go. 26:36.080 --> 26:39.280 Yeah, I mean, it's kind of the same point we had yesterday with the, they would leave resource, 26:39.280 --> 26:44.880 kind of discussion, I'm like, we can, like, this is info intercepts and effectively is that, 26:44.880 --> 26:48.800 what, like, we intercepts is info right now, because the canal doesn't really give us a right 26:48.800 --> 26:53.680 idea, the right solution would be a proper user space library that everyone uses instead of 26:53.680 --> 26:58.640 messing with the canal's text output. But it's going to take a decade plus for people to actually 26:58.640 --> 27:02.480 adopt, that's the issue. So, like, it's kind of, we need something to deal with the legacy stuff 27:02.480 --> 27:06.400 we have today, and then find something better to do in the future at the same time. 27:08.400 --> 27:10.400 All right, thank you.