WEBVTT 00:00.000 --> 00:13.000 Let's welcome Ramon and Ben, who will tell us about precision landing with PX4 and Ros2. 00:13.000 --> 00:15.000 Hello, hello, you can hear me? 00:15.000 --> 00:16.000 Yes? 00:16.000 --> 00:17.000 Okay. 00:17.000 --> 00:19.000 Yes, the applause please. 00:19.000 --> 00:20.000 Thank you. 00:20.000 --> 00:26.000 We're technical difficulties. 00:26.000 --> 00:29.000 Sorry for the presentation. 00:29.000 --> 00:31.000 Wi-Fi is what it is. 00:31.000 --> 00:33.000 But thank you for being here today. 00:33.000 --> 00:37.000 I know it's a packed room for those who are waiting outside to see us. 00:37.000 --> 00:38.000 Thank you. 00:38.000 --> 00:45.000 Yeah, so we're going to talk about drones and how to land precisely. 00:45.000 --> 00:49.000 Let me see if I can start moving this. 00:49.000 --> 00:50.000 Okay. 00:50.000 --> 00:54.000 So let me introduce my co-speaker Benjamin. 00:55.000 --> 00:56.000 Ben. 00:56.000 --> 00:57.000 Ben. 00:57.000 --> 00:58.000 He's a PX4 maintainer. 00:58.000 --> 01:01.000 He's a maintainer for the Ros2 interface in PX4. 01:01.000 --> 01:03.000 Do you want to? 01:03.000 --> 01:04.000 Hello, everybody. 01:04.000 --> 01:05.000 Yes. 01:05.000 --> 01:06.000 I'm the emino-potsam. 01:06.000 --> 01:08.000 Ben, if you want to. 01:09.000 --> 01:12.000 I have four years of experience in validation. 01:12.000 --> 01:14.000 Like a validation. 01:14.000 --> 01:15.000 No. 01:15.000 --> 01:17.000 God knows navigation control. 01:17.000 --> 01:20.000 For a robotics, I'm the exhumantainer. 01:20.000 --> 01:24.000 Last couple of years have been working in genesis in the line of education. 01:24.000 --> 01:27.000 These are days for AOV calls. 01:27.000 --> 01:28.000 Yeah. 01:28.000 --> 01:29.000 We're going to continue the engineering. 01:29.000 --> 01:31.000 I hope to talk a little. 01:31.000 --> 01:33.000 If I could use the body. 01:33.000 --> 01:34.000 Okay. 01:34.000 --> 01:36.000 And I'm Raymond Roche. 01:36.000 --> 01:38.000 I'm the general manager for Drunker Foundation. 01:38.000 --> 01:41.000 I'm employed by the Linux Foundation to work in open source robotics. 01:41.000 --> 01:44.000 I've been working in drones for the last decade. 01:44.000 --> 01:45.000 A little bit more than that. 01:45.000 --> 01:52.000 And the co-lead together became really my wire of the IRO robotics work group or community work group. 01:52.000 --> 01:57.000 However, whatever the name is right now for the Ross IRO robotics group. 01:57.000 --> 02:02.000 And I'm also leading another project within the within the Linux Foundation called Space Race Linux, 02:02.000 --> 02:05.000 which will build a job to distribution for space devices. 02:05.000 --> 02:09.000 We're interested in space, come talk to me later. 02:09.000 --> 02:10.000 All right. 02:10.000 --> 02:14.000 So brief intro for context for those of you that don't know what this is. 02:14.000 --> 02:15.000 This is Drunker. 02:15.000 --> 02:18.000 We have an autonomy stack. 02:18.000 --> 02:20.000 We've been around for since 2014. 02:20.000 --> 02:23.000 The projects themselves since 2008 or so. 02:23.000 --> 02:26.000 So it's been a while now. 02:26.000 --> 02:30.000 Some numbers, some contributions to put things into perspective. 02:30.000 --> 02:35.000 Last year, we managed to have almost 18,000 contributions. 02:35.000 --> 02:40.000 300,300 pool requests distributed across 100 repositories. 02:40.000 --> 02:45.000 More than 1400 contributors around 220 organizations. 02:45.000 --> 02:48.000 And, like I said, yeah, a lot of repositories. 02:48.000 --> 02:51.000 So it's not just me and my basement. 02:51.000 --> 02:54.000 Companies depend on us to build products. 02:54.000 --> 03:00.000 Fly controllers, which are the real time operating system. 03:00.000 --> 03:04.000 And the embedded controller, which piece for runs on top of this, 03:04.000 --> 03:06.000 piece for runs on top of notexartos. 03:06.000 --> 03:10.000 For these of you that don't know notexartos is a post-excompliant 03:10.000 --> 03:13.000 Artos by the Apache Foundation. 03:13.000 --> 03:17.000 And there's multiple companies building drones from swarming drones 03:17.000 --> 03:20.000 to underwater vehicles and all types of drones. 03:20.000 --> 03:24.000 All right, so I'm assuming you have some context of PX4. 03:24.000 --> 03:26.000 So I'm going to skip the basic introduction. 03:26.000 --> 03:30.000 However, we're going to talk about some basic concepts that you're going to need to 03:30.000 --> 03:32.000 be able to do precision landing with PX4. 03:32.000 --> 03:36.000 And then, stop me if I get something wrong here. 03:36.000 --> 03:38.000 So let's talk about middleware. 03:38.000 --> 03:41.000 Inside PX4, there's a middleware. 03:41.000 --> 03:44.000 So PX4 is not just like a single process running inside the artos. 03:44.000 --> 03:48.000 It's a collection of applications running inside the artos. 03:48.000 --> 03:51.000 As you can see here, and sorry for the people and the screen, 03:51.000 --> 03:53.000 I'm going to have to move. 03:53.000 --> 03:57.000 You got different modules, all of them think of them like apps. 03:57.000 --> 04:01.000 And some of them need to talk to each other because they have valuable information 04:01.000 --> 04:02.000 that they need to share. 04:02.000 --> 04:05.000 So for this, we divide something that's called micro-uart, 04:05.000 --> 04:07.000 which is all middleware. 04:07.000 --> 04:11.000 It's that we got topics, messages, modules, very standard stuff 04:11.000 --> 04:12.000 for a middleware. 04:12.000 --> 04:14.000 It's run in the embedded controller. 04:14.000 --> 04:16.000 Why does this matter to you? 04:16.000 --> 04:21.000 Because we extended this middleware to rawst to through different 04:21.000 --> 04:23.000 I'll tell you in the next slide. 04:23.000 --> 04:25.000 But basically, we extend all of these messages. 04:25.000 --> 04:28.000 Well, some of these messages, the most important ones to you, 04:28.000 --> 04:31.000 so that you can use rawst to be able to modify PX4. 04:31.000 --> 04:34.000 What it actually looks like is something like this. 04:34.000 --> 04:37.000 It's crazy if we have hundreds of modules. 04:37.000 --> 04:41.000 So it's a lot of messages, a lot of things going on in there. 04:41.000 --> 04:44.000 Next concept that it's important to know, it's flight modes. 04:44.000 --> 04:47.000 Whenever you think of a drone that is flying somewhere, 04:47.000 --> 04:50.000 like the one that's in front of you right there, 04:50.000 --> 04:52.000 the very first thing you think is like, 04:52.000 --> 04:55.000 I'm going to control a drone with my RC, right? 04:55.000 --> 04:59.000 Well, you really want to control it with raw power in the RC. 04:59.000 --> 05:02.000 You will have to have the sticks on you and control all the time. 05:02.000 --> 05:06.000 But there's also guided modes that help you alleviate some of those constraints. 05:06.000 --> 05:09.000 We got manual stabilized. 05:09.000 --> 05:11.000 The pilot has the direct control, 05:11.000 --> 05:14.000 but the autopilot stabilizes the inputs position, 05:14.000 --> 05:17.000 which it holds, like it says, the position, 05:17.000 --> 05:19.000 and the pilot commands the velocity. 05:19.000 --> 05:20.000 Have mission. 05:20.000 --> 05:23.000 This is one that is fully autonomous. 05:23.000 --> 05:26.000 You take off, the round goes to some different waypoints 05:26.000 --> 05:28.000 and does mission that you predefined, 05:28.000 --> 05:31.000 and then we have another example, return to landing, 05:31.000 --> 05:33.000 something that you go, you fly, 05:33.000 --> 05:35.000 and then you activate the switch or a mode, 05:35.000 --> 05:38.000 and then it comes back to certain positions that you define. 05:38.000 --> 05:40.000 This is already inside the embedded controller. 05:40.000 --> 05:44.000 There's nothing noble in here. 05:44.000 --> 05:51.000 But we recently added a new feature called external flight modes, 05:51.000 --> 05:55.000 but allows you to do is basically go beyond the flight controller, 05:55.000 --> 05:59.000 for when you do not want to modify the explorer inside the module, 05:59.000 --> 06:05.000 inside the flight controller, inside Ross, in Linux lands. 06:05.000 --> 06:08.000 You connect, and you actually have access to all of the middle 06:08.000 --> 06:11.000 messages like I said, and we have a really nice system that allows you 06:11.000 --> 06:14.000 to define flight modes, similar to what the ones that are defined 06:14.000 --> 06:17.000 in the flight controller, and then it's an orchestration 06:17.000 --> 06:21.000 that allows you to predefined certain movements, certain sets, 06:21.000 --> 06:26.000 and you can match and choose between the internal and external flight modes. 06:26.000 --> 06:29.000 I didn't skip anything in there, right? 06:29.000 --> 06:30.000 Yeah, okay. 06:30.000 --> 06:33.000 I promise you the boring part is going to be over soon. 06:33.000 --> 06:35.000 Now, let's talk about how do we actually move 06:35.000 --> 06:37.000 for middleware to Ross? 06:37.000 --> 06:39.000 So we're using actually the DDS for this. 06:39.000 --> 06:42.000 Micro-XRCDDS is the Ross to Bridge. 06:42.000 --> 06:45.000 We take all of the topics from New York, 06:45.000 --> 06:50.000 and when we have an a macro-XRCDS client running inside PX4, 06:50.000 --> 06:53.000 and then we assume that you have on the mission computer, 06:53.000 --> 06:57.000 somewhere else in Linux land, an agent, we connect both of them, 06:57.000 --> 07:00.000 and then we expose our middleware messages to Ross. 07:00.000 --> 07:03.000 Now, this is the raw messages from PX4. 07:03.000 --> 07:06.000 So you have no context of what's going on in PX4. 07:06.000 --> 07:09.000 You have to go to the source code to be able to know what messages 07:09.000 --> 07:11.000 to use and how to modify the behavior. 07:11.000 --> 07:14.000 Otherwise, you'll be shooting a blanks into the wall, 07:14.000 --> 07:17.000 without knowing what's happening. 07:17.000 --> 07:20.000 We also support CNO for this, 07:20.000 --> 07:22.000 so we have not only have a DDS, 07:22.000 --> 07:24.000 support we also support CNO. 07:24.000 --> 07:28.000 We say it is experimental, not because it's a fragile, 07:28.000 --> 07:29.000 it's because it's you. 07:29.000 --> 07:32.000 We haven't tested it as much, but it is available, 07:32.000 --> 07:36.000 and it's fully supported. 07:36.000 --> 07:37.000 All right. 07:37.000 --> 07:39.000 You want to take over the raw screen interface? 07:39.000 --> 07:40.000 Yes. 07:40.000 --> 07:41.000 Is your other maintainer? 07:41.000 --> 07:42.000 No. 07:42.000 --> 07:43.000 That one. 07:43.000 --> 07:44.000 Yes. 07:44.000 --> 07:48.000 So for making easier for user to interact with Ross Chu 07:48.000 --> 07:53.000 and not to have to subscribe to the internal PX4 topic, 07:53.000 --> 07:57.000 I'll tell you on the company that is deeply involved 07:57.000 --> 07:59.000 in this sort of development as developed 07:59.000 --> 08:02.000 the Ross Chu interface library. 08:02.000 --> 08:06.000 So this library abstracts most of the hinders, 08:06.000 --> 08:09.000 inert PX4 components, 08:09.000 --> 08:13.000 and with a simple API that you can just hold 08:13.000 --> 08:15.000 from your host tuners, 08:15.000 --> 08:19.000 let you particularly explore 08:19.000 --> 08:22.000 the interfaces, the control and navigation, 08:22.000 --> 08:25.000 and the mission interface. 08:25.000 --> 08:29.000 We've seen a bit of set of data 08:29.000 --> 08:31.000 and how we can actually do that. 08:31.000 --> 08:33.000 But in particular, control just set 08:33.000 --> 08:35.000 to create your custom modes, 08:35.000 --> 08:38.000 the custom mode will have activation, 08:38.000 --> 08:40.000 growth routines, update routines, 08:40.000 --> 08:42.000 and combination routines. 08:42.000 --> 08:45.000 And the nice part about it is that 08:45.000 --> 08:48.000 when you have your mission planner, 08:48.000 --> 08:50.000 you're not a mission planner, 08:50.000 --> 08:52.000 you're going to do a station. 08:52.000 --> 08:54.000 So the software normally, you have on the ground, 08:54.000 --> 08:57.000 that controls the domain, make sure everything is going well. 08:57.000 --> 09:00.000 Your new mode will actually appear there, 09:00.000 --> 09:03.000 so you can trigger it when you want. 09:03.000 --> 09:04.000 Of course, you can do it from the last show, 09:04.000 --> 09:07.000 which will move also from other part of the system. 09:07.000 --> 09:09.000 On top of the control part, 09:09.000 --> 09:11.000 you can also have the navigation part, 09:11.000 --> 09:13.000 because indoor navigation, 09:13.000 --> 09:14.000 all the navigation without GPS, 09:14.000 --> 09:17.000 you always have to provide new data 09:17.000 --> 09:19.000 to PX4 position data, 09:19.000 --> 09:22.000 can come from Islam, 09:22.000 --> 09:24.000 map matching, 09:24.000 --> 09:25.000 any sort of feature. 09:25.000 --> 09:28.000 So you keep the heavy intense part on the show, 09:28.000 --> 09:30.000 and just then send a data to PX4, 09:30.000 --> 09:31.000 because it's really happy. 09:31.000 --> 09:34.000 It's easy to get to use all of that. 09:34.000 --> 09:35.000 And you can also do, like, 09:35.000 --> 09:39.000 mission with more, even more high level part of it. 09:39.000 --> 09:41.000 For the position landing example, 09:41.000 --> 09:42.000 we'll see data, 09:42.000 --> 09:46.000 we'll use just the 09:47.000 --> 09:48.000 water navigation, 09:48.000 --> 09:50.000 meant to see control part, 09:50.000 --> 09:51.000 using the control part, 09:51.000 --> 09:55.000 to drive the drone where we actually want it. 09:59.000 --> 10:01.000 Simulation gazebo, 10:01.000 --> 10:06.000 this was put a lot of effort in using, 10:06.000 --> 10:09.000 been directly integrated into gazebo, 10:09.000 --> 10:11.000 from gazebo class, 10:11.000 --> 10:14.000 to a new gazebo. 10:15.000 --> 10:17.000 We do it by passing. 10:17.000 --> 10:19.000 Last show, 10:19.000 --> 10:21.000 we pick so itself in, 10:21.000 --> 10:24.000 because the user, 10:24.000 --> 10:26.000 user transport, 10:26.000 --> 10:28.000 so we have our motor plugin, 10:28.000 --> 10:29.000 sensor plugin, 10:29.000 --> 10:30.000 some of them are, 10:30.000 --> 10:32.000 we actually defined them, 10:32.000 --> 10:34.000 combined and linked to gazebo, 10:34.000 --> 10:37.000 like optical flow plugin. 10:37.000 --> 10:39.000 For example, 10:39.000 --> 10:43.000 and this allows us to just run simulation without having, 10:43.000 --> 10:44.000 actually, 10:44.000 --> 10:47.000 the extra dependency of, 10:47.000 --> 10:48.000 from, 10:48.000 --> 10:49.000 from our shoe. 10:49.000 --> 10:50.000 Of course, 10:50.000 --> 10:51.000 if you also want to connect, 10:51.000 --> 10:52.000 our shoe, 10:52.000 --> 10:54.000 then you can use the as a CDS agent, 10:54.000 --> 10:56.000 or the design interface, 10:56.000 --> 10:59.000 and then you have access to all of your 10:59.000 --> 11:00.000 washrooms. 11:00.000 --> 11:01.000 Thank you. 11:01.000 --> 11:04.000 So I'm guessing the video is not going to play. 11:04.000 --> 11:05.000 Hmm. 11:05.000 --> 11:08.000 That's why. 11:08.000 --> 11:10.000 This video just shows you the takeoff. 11:10.000 --> 11:12.000 Okay, it actually works. 11:13.000 --> 11:14.000 So, 11:14.000 --> 11:15.000 if you've got, 11:15.000 --> 11:16.000 if you saw there, 11:16.000 --> 11:18.000 I moved as a slide, 11:18.000 --> 11:19.000 triggered a takeoff, 11:19.000 --> 11:20.000 this is the, 11:20.000 --> 11:22.000 the contestation, 11:22.000 --> 11:24.000 on this, 11:24.000 --> 11:26.000 side here, 11:26.000 --> 11:27.000 on, 11:27.000 --> 11:28.000 down there, 11:28.000 --> 11:29.000 it triggered, 11:29.000 --> 11:30.000 actually, 11:30.000 --> 11:31.000 this was, 11:31.000 --> 11:33.000 where the pixel simulator was running, 11:33.000 --> 11:34.000 oh, 11:34.000 --> 11:35.000 okay, 11:35.000 --> 11:37.000 and it received a takeoff, 11:37.000 --> 11:38.000 in simulation, 11:38.000 --> 11:40.000 a date takeoff, 11:41.000 --> 11:43.000 and the raw shoe interface, 11:43.000 --> 11:46.000 now we show them as this box of panel. 11:46.000 --> 11:48.000 So we have the raw shoe interface, 11:48.000 --> 11:50.000 connection running, 11:50.000 --> 11:52.000 just a couple of packages to do, 11:52.000 --> 11:54.000 causing information, 11:54.000 --> 11:56.000 and then, 11:56.000 --> 11:58.000 a box of 3D panels showing that 11:58.000 --> 12:00.000 they don't take off, 12:00.000 --> 12:01.000 we expect to, 12:01.000 --> 12:04.000 our predefined map play. 12:04.000 --> 12:05.000 No, 12:05.000 --> 12:06.000 so nothing novel here, 12:06.000 --> 12:08.000 just showing you how the stack works, 12:08.000 --> 12:09.000 and everything is connected. 12:10.000 --> 12:12.000 So click on the side part, 12:12.000 --> 12:13.000 yes? 12:13.000 --> 12:15.000 All right, 12:15.000 --> 12:17.000 so let's talk about px for control, 12:17.000 --> 12:19.000 with raw shoe. 12:19.000 --> 12:23.000 So how do you actually control the fly controller? 12:23.000 --> 12:26.000 Why do we want to control the fly controller from boss? 12:26.000 --> 12:29.000 So we can enable more advanced autonomous behavior, 12:29.000 --> 12:30.000 remember, 12:30.000 --> 12:31.000 running in the embedded controller, 12:31.000 --> 12:33.000 you're a bit constrained on resources, 12:33.000 --> 12:35.000 so moving to Linux land, 12:35.000 --> 12:36.000 you'll be able to, 12:36.000 --> 12:38.000 use other type of hardware that is more, 12:38.000 --> 12:39.000 performance, 12:39.000 --> 12:40.000 you can integrate with raw shoe, 12:40.000 --> 12:41.000 raw shoes as I'm huge, 12:41.000 --> 12:42.000 ecosystem as you're all aware, 12:42.000 --> 12:44.000 you can test new control strategies, 12:44.000 --> 12:45.000 you can go beyond what's inside px force, 12:45.000 --> 12:46.000 you can modify the behavior, 12:46.000 --> 12:47.000 but there's some risk in here, 12:47.000 --> 12:48.000 you have, 12:48.000 --> 12:50.000 you can blue some of the fail saves, 12:50.000 --> 12:51.000 Px4 has a lot of fail saves, 12:51.000 --> 12:53.000 embedded inside of it, 12:53.000 --> 12:56.000 so if you are only using the middleware inside 12:56.000 --> 12:57.000 Ross, 12:57.000 --> 12:59.000 you can blue some of the fail saves, 12:59.000 --> 13:01.000 Px4 has a lot of fail saves, 13:01.000 --> 13:02.000 embedded inside of it, 13:02.000 --> 13:05.000 so if you are only using the middleware inside Ross, 13:06.000 --> 13:08.000 without using the API, 13:08.000 --> 13:11.000 you might lose some of those protections in the fail saves, 13:11.000 --> 13:13.000 if it's unstable, 13:13.000 --> 13:14.000 unstable flight, 13:14.000 --> 13:16.000 if set points are wrong or the rate of, 13:16.000 --> 13:18.000 delivering those is too slow, 13:18.000 --> 13:21.000 so direct control obviously means that you're, 13:21.000 --> 13:24.000 directly responsible for the vehicle behavior, 13:24.000 --> 13:25.000 and yes, 13:25.000 --> 13:27.000 please test in simulation first, 13:27.000 --> 13:29.000 but, 13:29.000 --> 13:30.000 okay, 13:30.000 --> 13:32.000 so when you want to take on this, 13:32.000 --> 13:34.000 so how do we actually control Ross 2, 13:34.000 --> 13:35.000 we have 2 modes, 13:35.000 --> 13:37.000 the old one was off board, 13:37.000 --> 13:38.000 data approach, 13:38.000 --> 13:40.000 we no longer recommend you do, 13:40.000 --> 13:41.000 but it, 13:41.000 --> 13:43.000 because it actually exposes a lot of control, 13:43.000 --> 13:45.000 and it can be a bit dangerous, 13:45.000 --> 13:48.000 so we have made a new API for this. 13:48.000 --> 13:49.000 Yeah, exactly, 13:49.000 --> 13:51.000 so that's all, 13:51.000 --> 13:53.000 thanks for the second one, 13:53.000 --> 13:54.000 the older approach, 13:54.000 --> 13:56.000 maybe you are familiar with marbling, 13:56.000 --> 13:58.000 so all the approach was meant to be used, 13:58.000 --> 13:59.000 by marbling, 13:59.000 --> 14:02.000 and we are the internal of Px4 components, 14:02.000 --> 14:03.000 to do the transition, 14:03.000 --> 14:04.000 so it was by safe, 14:04.000 --> 14:06.000 the moment we will move that 14:06.000 --> 14:07.000 and expose everything to Ross 2, 14:07.000 --> 14:08.000 then it becomes, 14:08.000 --> 14:10.000 machines here, 14:10.000 --> 14:12.000 extremely powerful, 14:12.000 --> 14:13.000 if you know what you're doing, 14:13.000 --> 14:14.000 but at the same time, 14:14.000 --> 14:15.000 by, 14:15.000 --> 14:17.000 by the dangers, 14:17.000 --> 14:19.000 so here they come, 14:19.000 --> 14:21.000 the two depicts for Ross 2 interface, 14:21.000 --> 14:22.000 the custom mode, 14:22.000 --> 14:23.000 so, 14:23.000 --> 14:24.000 you, 14:24.000 --> 14:26.000 when you are working on it, 14:26.000 --> 14:28.000 it just exposes the, 14:28.000 --> 14:29.000 the mode ways, 14:29.000 --> 14:31.000 the mode ways will interact with, 14:32.000 --> 14:33.000 with the bridge to Px4, 14:33.000 --> 14:34.000 again, 14:34.000 --> 14:35.000 DDS or, 14:35.000 --> 14:36.000 or Zeno, 14:36.000 --> 14:37.000 it's suppose, 14:37.000 --> 14:38.000 compare behavior, 14:38.000 --> 14:39.000 autonomous logic, 14:39.000 --> 14:40.000 it's, 14:40.000 --> 14:43.000 let you control Px4 in many different ways, 14:43.000 --> 14:46.000 because Px4 force runs its own control inside, 14:46.000 --> 14:47.000 from, 14:47.000 --> 14:48.000 rate, 14:48.000 --> 14:49.000 attitudes, 14:49.000 --> 14:50.000 stabilization, 14:50.000 --> 14:51.000 velocity position, 14:51.000 --> 14:52.000 emission, 14:52.000 --> 14:53.000 so you decide, 14:53.000 --> 14:54.000 how deep do you want to go, 14:54.000 --> 14:57.000 if you want to control the single propeller, 14:57.000 --> 14:58.000 find by me, 14:58.000 --> 15:01.000 make sure you're publishing that one kilo or so, 15:01.000 --> 15:03.000 and the communication is safe, 15:03.000 --> 15:07.000 and, 15:07.000 --> 15:08.000 and you have access to the tail safe, 15:08.000 --> 15:09.000 and the life cycle, 15:09.000 --> 15:10.000 yes, 15:10.000 --> 15:11.000 of the vehicle, 15:11.000 --> 15:12.000 basically, 15:12.000 --> 15:13.000 that's, 15:13.000 --> 15:14.000 that's, 15:14.000 --> 15:15.000 that's a bit, 15:15.000 --> 15:17.000 the interface, 15:17.000 --> 15:19.000 exposes shoe, 15:19.000 --> 15:21.000 to different entities. 15:21.000 --> 15:22.000 One is the mode, 15:22.000 --> 15:23.000 to mention, 15:23.000 --> 15:24.000 to use a bit for, 15:24.000 --> 15:25.000 so they are, 15:25.000 --> 15:26.000 just, 15:26.000 --> 15:27.000 enabling, 15:27.000 --> 15:28.000 so the internal, 15:28.000 --> 15:29.000 px4 modes, 15:29.000 --> 15:31.000 you can switch to them, 15:31.000 --> 15:33.000 you can define, 15:33.000 --> 15:34.000 starting, 15:34.000 --> 15:35.000 running, 15:35.000 --> 15:37.000 stopping condition, 15:37.000 --> 15:38.000 and it's, 15:38.000 --> 15:39.000 basically, 15:39.000 --> 15:40.000 the brain, 15:40.000 --> 15:41.000 control, 15:41.000 --> 15:42.000 the behavior of the dog, 15:42.000 --> 15:43.000 it has, 15:43.000 --> 15:44.000 in that moment, 15:44.000 --> 15:45.000 one mode, 15:45.000 --> 15:46.000 we see later, 15:46.000 --> 15:48.000 we'll be the procedural ending mode, 15:48.000 --> 15:49.000 but you could do, 15:49.000 --> 15:50.000 we have, 15:50.000 --> 15:51.000 like, a precision, 15:51.000 --> 15:52.000 we point, 15:52.000 --> 15:55.000 following, 15:55.000 --> 15:56.000 like, 15:56.000 --> 15:57.000 multiple modes together, 15:57.000 --> 15:58.000 and, 15:58.000 --> 15:59.000 and, 15:59.000 --> 16:01.000 specific logic to translate, 16:01.000 --> 16:02.000 one of them, 16:02.000 --> 16:03.000 okay, 16:03.000 --> 16:04.000 you can create your own, 16:04.000 --> 16:06.000 find a machine, 16:06.000 --> 16:07.000 or, 16:07.000 --> 16:08.000 or similar, 16:08.000 --> 16:09.000 or you can use the, 16:09.000 --> 16:10.000 the, 16:10.000 --> 16:11.000 the, 16:11.000 --> 16:12.000 the, 16:12.000 --> 16:13.000 the, 16:13.000 --> 16:14.000 the, 16:14.000 --> 16:15.000 the, 16:15.000 --> 16:16.000 the, 16:16.000 --> 16:17.000 the, 16:17.000 --> 16:18.000 the, 16:18.000 --> 16:19.000 the, 16:19.000 --> 16:20.000 the, 16:20.000 --> 16:21.000 the, 16:21.000 --> 16:22.000 the, 16:22.000 --> 16:23.000 the, 16:24.000 --> 16:25.000 like, 16:25.000 --> 16:26.000 and, 16:26.000 --> 16:27.000 the, 16:27.000 --> 16:29.000 actually cognitive condition. 16:29.000 --> 16:31.000 So they comes onto the mission flow, 16:31.000 --> 16:32.000 of a, 16:32.000 --> 16:33.000 of, 16:33.000 --> 16:34.000 of your location, 16:34.000 --> 16:35.000 Um, 16:35.000 --> 16:37.000 Yeah, 16:37.000 --> 16:38.000 we should skip forward, 16:38.000 --> 16:39.000 because we're running at the time. 16:39.000 --> 16:41.000 these example, 16:41.000 --> 16:42.000 but, 16:42.000 --> 16:44.000 we see the nice, 16:44.000 --> 16:45.000 Demulator. 16:45.000 --> 16:46.000 Yeah, 16:46.000 --> 16:48.000 owania build 16:48.000 --> 16:49.000 for, 16:49.000 --> 16:50.000 so let's talk about, 16:50.000 --> 16:51.000 perception quickly, 16:51.000 --> 16:52.000 before we jump into the actual, 16:52.000 --> 16:53.000 Okay, let's skip. 16:53.000 --> 16:54.000 Yeah, we can skip. 16:54.000 --> 16:59.000 So this is just a primer and a loop of markers, just in case, I guess we can skip. 16:59.000 --> 17:04.480 All right, this is the pipeline that we're going to follow in the example now, so I cannot 17:04.480 --> 17:05.480 read from here. 17:05.480 --> 17:06.480 Can you read? 17:06.480 --> 17:07.480 Yeah, I can read it. 17:07.480 --> 17:15.440 So, Ding Azebo, we have our own, this demo, down facing camera, so we bridge that with 17:15.440 --> 17:22.160 us, this is the bridge over to us, that we have our last shooting order, the bullet, our 17:22.160 --> 17:29.920 contract, that just simple, open, civil function, it might process, macro detection, and 17:29.920 --> 17:30.920 distortion. 17:30.920 --> 17:36.680 Yeah, for the second mission, we have the pose, we can skip out again over to us network 17:36.680 --> 17:45.400 and maybe also publish just a frame, and you might be attached, a marker on top of it. 17:45.400 --> 17:53.120 All right, so now the group is finally, 20 minutes to get here, yes, we can do it, so, all right, 17:53.120 --> 17:59.360 so we're going to be landing on a Rucom marker, so what do we actually need to do here? 17:59.360 --> 18:04.240 We're obviously we need a camera, we need to be able to look at the Rucom marker, and then 18:04.240 --> 18:08.200 find the position, relative to the drone, and then find the drone in top of it, and 18:08.200 --> 18:09.920 then find to land it, right? 18:09.920 --> 18:14.120 So that's what's basically going to happen. 18:14.120 --> 18:15.120 Do you want to take this one? 18:15.120 --> 18:20.680 So just a bit of a call unit on its form, of course, the Rucom detector, our Rucotlacker 18:20.680 --> 18:28.120 doesn't know anything about the Ruconis, so it's having the Rucom target, the Rucom position, 18:28.120 --> 18:32.720 this part, which we respect for the frame of the drone, sorry, to the frame of the camera, 18:32.720 --> 18:35.760 of the frame of the camera, so we need to do a bit of conversation, we actually want 18:35.760 --> 18:43.200 the target in the world frame, for this example, we are assuming that the fix for to GPS 18:43.200 --> 18:49.800 or anything similar as it knows where it is, it has an option position, therefore, to call 18:49.800 --> 18:55.200 into a form, we can actually get the position of the target in some global reference, or 18:55.200 --> 18:57.480 local, depends on how you consider your map. 18:57.480 --> 19:01.240 Yes, and we'll see at the code example of the actual formula. 19:01.240 --> 19:06.200 So we have some sealed code, because the actual code doesn't fit in the slides, so we 19:06.200 --> 19:10.840 defined the class, called PrecisionLand, and we are extending the mode base that we were 19:10.840 --> 19:15.240 talking about in the API, and this is basically the signature, we have a state, we're 19:15.240 --> 19:19.720 going to switch states from searching to approaching, to descending, to finished, and then 19:19.720 --> 19:23.720 we have an update set point class that is actually going to be switching the stage for 19:23.720 --> 19:24.720 us. 19:24.720 --> 19:30.160 Again, the search is going to help you fly to find the Rucomarker, or approach here is basically 19:30.160 --> 19:35.840 going to spiral around an area to try to find an Rucomarker, once we find it, we're 19:35.840 --> 19:40.800 going to position on top of it, like I said before, and then we're going to try to land. 19:40.800 --> 19:45.640 All right, so yeah, you take this one and I'll take the next one. 19:45.640 --> 19:46.640 Sure. 19:46.640 --> 19:54.280 So for this position landing mode, we form the Rucomarker, we are only interested in the target 19:54.280 --> 20:00.440 calls, probably by the camera tracker, Rucomarker, and also we actually go dive to Rucomarker 20:00.440 --> 20:06.280 in this case, just to get this topic, it's called the vehicle land detected, is one of 20:06.280 --> 20:11.320 the internal effects for topic messages, and it contains information, if the pixel 20:11.320 --> 20:17.280 of things is landed all in air, we're going to use it to detect when we are fully, fully 20:17.280 --> 20:20.280 on the ground, and terminate our mode. 20:20.280 --> 20:25.000 From the Rucomarker interface library standard, we get a trajectory set point, we'll use 20:25.000 --> 20:30.120 this subject to send a set point to click for, vehicle attitude to do the, the, the 20:30.120 --> 20:37.120 information, and vehicle local position, vehicle completion, and vehicle attitude to do the 20:37.120 --> 20:41.160 frequency-concephome and the PID for the position approach. 20:41.160 --> 20:47.160 All right, so now the optics, say, point, let's talk about the search, so we're going to 20:47.160 --> 20:51.320 have a search waypoint, and we're going to update the position of the drone, based on 20:51.320 --> 20:55.960 this search point here, we're not putting in how we're doing this spiral, imagine something 20:55.960 --> 20:59.280 is feeding this spiral, and it's giving you the search waypoint, and then we're sending 20:59.280 --> 21:03.880 that to the API, we'll be able to update position to PX4, and then there's a state machine 21:03.880 --> 21:07.280 inside search, it says if the attack is visible, finally, then you're going to switch 21:07.280 --> 21:11.920 to your approach, and I promise you the code is available later, you can download everything, 21:11.920 --> 21:13.920 it's real code, not serial code. 21:13.920 --> 21:19.040 In the approach, it's very simple, you get the target, which is getting the tag position, 21:19.040 --> 21:23.440 and the approach altitude, which is predifying by you, and then you get the trajectory set 21:23.440 --> 21:27.680 point again, and you say, okay, here's the local marker, again, just send this position 21:27.680 --> 21:32.680 back to PX4, PX4 will position itself into this altitude, they do their fine, and then 21:32.680 --> 21:37.080 you transition to the next one, now this is some magic here, so you get the velocity, and 21:37.080 --> 21:42.680 you have a PID controller here to actually control the velocity, let's jump into that, Ben. 21:42.680 --> 21:43.680 Yeah. 21:43.680 --> 21:45.680 All right, do you want to take this one? 21:45.680 --> 21:51.240 Yes, so of course, we would have just said the typical, so, just stay above the 21:51.320 --> 21:57.640 U.C. marker, and use the internal PX4, but you want to allow the user to actually 21:57.640 --> 22:04.600 resume them to have a better tracking of that specific position in X, Y, so we just implement 22:04.600 --> 22:12.800 a simple PID controller, so computer velocity, proportional gain, inter-ideal, computer velocity, 22:12.800 --> 22:18.480 integral gain, make sure you don't want to go 1,000 meters per second, because you set 22:18.480 --> 22:26.880 the game wrong, and return it, at the point, directly update, we call it a velocity, the API 22:26.880 --> 22:34.000 is, as a kind of distinguish, we can tell it position of velocity over the level, you prefer, 22:34.000 --> 22:39.840 so in this case, we update it well, with velocity, and if at some point we are landed, 22:39.840 --> 22:45.400 we just, we, we, we, we terminate the modern, we go back to it. 22:45.480 --> 22:52.840 Let's show the video now, then, this was just a simple flow, sequence flow, but why don't 22:52.840 --> 22:53.840 you show it like that? 22:53.840 --> 23:07.080 Two minutes, no rush, like them all, time constraint, you can do it, nice, so the 23:07.160 --> 23:12.880 level simulation, apologies for the zoom, yeah, this is, this is supposed to be an 23:12.880 --> 23:20.440 aruko marker on the floor, we attach the X4 to it, we just run in bash script on the other 23:20.440 --> 23:27.920 on this screen, trust me, it will appear, I don't know, yeah, this is up here, I just 23:28.160 --> 23:36.640 want to zoom in a little bit, yeah, then you go, I see a run, yes, this is done, perfect, so now 23:36.640 --> 23:45.280 it's always running and we can actually connect two gans on the floor to it, why 23:45.280 --> 23:52.720 mean I thought yes, I do have time, yes, plenty of time, hoping to question of absolute, so 23:52.800 --> 23:57.040 you don't go to it, we actually connect to our beautiful, what is the last one, what 23:57.040 --> 24:03.840 is the last one, yeah, so we are connected, why to fly, you position mode, why in position 24:03.840 --> 24:09.520 mode, because I actually have a joystick connected to it, so I can fly it manually and 24:09.520 --> 24:19.200 I will do that if I find my, in a minute, yeah, let me go back here, lunching just a bit of 24:20.000 --> 24:25.680 auxiliary mode, what are you doing there, lunching mode, ordering your food dinner, 24:27.360 --> 24:32.400 okay now they lunch another lunch another lunch, I can actually connect to the fox 24:32.400 --> 24:40.400 hello, that's true, give you please fox food, you know, it already failed, it's freedom fox 24:40.480 --> 24:47.520 or fox bloodslave, oh, fox bloodslave failed, oh no, no, it didn't, it didn't, it didn't, yeah, it didn't, 24:48.320 --> 24:50.880 it didn't, it didn't, it didn't, it didn't, it didn't, it didn't, don't worry, 24:54.240 --> 24:59.680 okay show the fox blood then, yeah, I want to show them the fasma, you got me, you got 16 seconds, 25:01.120 --> 25:05.440 16 seconds, why don't you just lunch the drone and said okay, and I just lunch the one without fox 25:05.440 --> 25:14.880 then nine, five, five, five, five, and no question at all, so let me know this, 25:24.880 --> 25:29.920 there we go, oh, you can take off and just use it to see if you take off, 25:30.880 --> 25:44.320 yeah, you can go up, okay, now I'm not doing this a bit out of the photo, I'm at four meters 25:44.320 --> 25:49.840 down there, you can see six meters with too much oil, you know, then if you have questions, 25:49.840 --> 25:53.280 I'm in a stand next to Sefer and then you can come and ask, 25:53.280 --> 25:57.440 but now it's a moment of delaying the thing, so if I go to position here, 25:57.440 --> 26:02.320 you can see the battery resistance, position a custom, this is the custom mode that is running 26:02.320 --> 26:12.240 rush, just click it, moving down, and then, yeah, and then fine set and then it goes in and 26:12.240 --> 26:24.560 lands, yeah, again, but plus again, most importantly, there's a code here, if you want to see the 26:24.560 --> 26:28.160 code, you can scan this in this repository, thank you. 26:28.160 --> 26:45.840 Yeah, oh, I think I think, thank you again for the speaker.