WEBVTT 00:00.000 --> 00:14.000 OK, hello everyone. Welcome to this talk about automated short-term trend planning in OSRD. 00:14.000 --> 00:19.280 Well, talk what is the problem? How we solved it? And then how we went from a nice proof of 00:19.280 --> 00:26.480 concept into actual production. First, some context OSRD stands for Open Source Railway Designer. 00:26.480 --> 00:32.480 It's an open source tool that can be used to edit railway infrastructure and when all kinds of 00:32.480 --> 00:38.720 simulations. It's open source primarily to encourage cost-conferior collaboration with such 00:38.720 --> 00:43.840 and things like that. Today, we'll talk about one of its features, which is trend planning. 00:45.520 --> 00:51.920 But what is trend planning? Well, let's say a freight company wants a train to go from 00:51.920 --> 00:59.520 A to B at some point tomorrow. And with our railway infrastructure manager, and with one 00:59.520 --> 01:04.480 in charge of actually scheduling or the train and fighting some warm for that train, 01:05.600 --> 01:09.440 the problem being, of course, that's all much trends have already been scheduled. 01:10.160 --> 01:15.360 Quite a lot in fact, we import about 10,000 to 50,000 every day. 01:15.520 --> 01:24.000 Here's some words. We cannot delay any scheduled trends. We have two actors. It's a 10, 01:24.000 --> 01:30.160 10, 10, 10, 10. We don't live. In fact, we cannot provide an unrealistic path. What I mean by that 01:30.160 --> 01:35.760 is let's say the trend has a pretty weak engine, very heavy load, and the short-term short-est 01:35.760 --> 01:43.920 path would go through mountains. It may be a difficult or even impossible. But we can at 01:44.000 --> 01:49.760 details, we don't have to take the shortest path, or even just when as fast as we can. 01:50.880 --> 01:56.240 We can change the departure time. We can change the stops at the extra step. 01:58.080 --> 02:05.920 We end up with a pretty complex search space. In this case, this is a bit south of Lyon, I think. 02:06.800 --> 02:10.480 On the horizontal axis, we have the time vertical axis distance. 02:11.120 --> 02:18.880 Every thing is an available because of another trend. In this case, we did find the solution 02:18.880 --> 02:24.320 in blue, where we did manage to squeeze between every trend, and somehow get through the destination. 02:25.360 --> 02:30.400 So this is the problem with facing, which is not easy. 02:32.640 --> 02:37.520 So I'll talk about a bit about the solution. In general, I won't go too much into the details as 02:37.520 --> 02:45.120 I'll take way too long. But still explaining the general principles is helpful at this whole context. 02:47.360 --> 02:53.760 General ID is that we build one big decision tree. So we take the initial 02:56.400 --> 03:02.160 state of the trend. Like we started the departure location with speed at zero and departure time 03:03.120 --> 03:13.520 and a few other stuff. And then for every state, we see wherever we can go from there and then just keep iterating to build one big graph. 03:14.400 --> 03:21.520 So we first do that on space. So let's say we train our way right there. And then we take every possible path. 03:23.840 --> 03:29.840 We do the same on space. So basically, on one given path, we have lots of decision to take, 03:29.840 --> 03:37.360 which mostly comes up to do we pass before or after that or the trend. So we build the next 03:37.360 --> 03:42.720 draw decision tree, basically, on top of that. And once we have that, we can just 03:43.360 --> 03:51.760 pass finding algorithm in that case and A star, where we minimize travel time and we build a 03:51.840 --> 04:03.920 hewistic to evaluate the best path first. So on the hewistic itself, we realize that what we 04:03.920 --> 04:10.480 takes a lot of time is the combination of space on time. But if we just go through space itself, 04:10.480 --> 04:16.080 it takes actually not around the time. So we go through the entire infrastructure starting from the arrival 04:16.960 --> 04:22.480 and compute the time it would take at best, ignoring every conflict, ignoring every thing, 04:22.480 --> 04:27.360 to go from there. So in this case, to insert a middle path, we take two hours and it would, 04:27.360 --> 04:32.640 we first go through there. And only one, we take half an hour of extra delays, we would start 04:32.640 --> 04:41.040 considering the top path in the bottom one. And so that's the general idea. And it works actually 04:41.040 --> 04:49.440 pretty well. And so I have a short video just to show what we're talking about. And so we have a path 04:49.440 --> 04:54.800 here. It's not the most realistic path, I just wanted to see a draw a little bit. We are starting 04:54.800 --> 05:02.880 at the mess and going towards the end of the day. And just to see how it works. And then doing the 05:02.880 --> 05:08.240 search, we actually see here where it's searching. So we see that it first tried to go through 05:08.240 --> 05:12.080 emerge, then couldn't get past there, maybe there's some work on the tracks or something like that. 05:12.080 --> 05:17.520 I'll try some other paths, try to go through the bottom and just seem to find some paths here. 05:18.480 --> 05:26.800 We see it's probably a lot just evaluate the Paris area. And then on the time axis, 05:26.880 --> 05:39.760 and then the path. And then this is a different path that I just wanted to showcase a few things 05:39.760 --> 05:49.760 here. Hold on, one to first. And there's a few cases like one here where we actually like this 05:49.760 --> 05:54.960 rain is very slow compared to the other one on the time table. So we need to get overtaken by the 05:54.960 --> 06:00.400 schedule train. And we do that by stopping on the next track, which shows us a space way of 06:00.400 --> 06:11.280 conflicts. We wait to start. It's pretty cool. And the water it works. And now back to the slides. 06:16.080 --> 06:24.160 So now we have something that works fine. On data we control, we use a control and we control 06:24.160 --> 06:30.560 basically everything. From there, how do we actually be useful? How do we face actual users, 06:30.560 --> 06:36.160 actual data, and all that? And in this section, I'll mostly talk about things that are specific 06:36.160 --> 06:41.440 to essence here, but I assume they'll sound familiar to people. And we did learn some sounds 06:41.440 --> 06:46.160 stuff that we applied to. Yes, something cool, something like that, so far. 06:46.160 --> 07:01.040 So first question is how do we actually interact into the existing workflows? Because 07:01.040 --> 07:07.600 clients can already come to see essence here and be like okay, one this rain to go in a few days. 07:08.000 --> 07:15.840 And there are people in charge of doing that. They are called tentable planners. And the way 07:15.840 --> 07:23.680 generally works is that the clients come with a demand like okay one this rain tomorrow. And then 07:23.680 --> 07:32.720 after some time they can get an answer, which sometimes is just not possible. Sometimes even a few days 07:33.680 --> 07:39.120 to explain the particular ways this is low. There's one team to investigate a bit, one team, 07:39.120 --> 07:43.760 a region. So when a train goes to several regions, there's a lot of people that needs to go back 07:43.760 --> 07:51.920 and forth with its team and all that. So what do we want to do? We want to keep this workflow 07:51.920 --> 07:57.840 with clients and technical planners, but we want to make OSD available to clients. So as they 07:57.840 --> 08:04.080 can send a request and see immediately within a few minutes at most. If it's possible, if we can 08:04.080 --> 08:11.200 find a path or if we can't. And if we do, we generate a PDF that contains all the data 08:11.200 --> 08:15.920 about the past we found and that PDF can be forwarded to that other planners saying hey, 08:15.920 --> 08:24.640 this looks like a reasonable solution. So how do we get there? Well, so this proof of 08:24.640 --> 08:30.080 concept, we have it for a few years, but we had to work quite a lot to make it work first. 08:31.280 --> 08:38.720 First topic was data, of course it's always data. Because we need to know that all the information 08:38.720 --> 08:45.600 tracks, the infrastructure itself, so it tracks the signals, the data on the tracks, the slopes, 08:45.600 --> 08:50.480 all kind of stuff. And we also need to know all the trains that are scheduled to run. 08:50.560 --> 08:57.840 And you think that SNCF would know precisely where the tracks were on them and where the trains 08:57.840 --> 09:07.280 are supposed to run, right? So anyway, we had to focus on one specific area and make that 09:08.560 --> 09:14.240 reasonably good enough to use. At first, when we first released it, it was the hour and the 09:14.240 --> 09:18.480 wet, what's currently open is there are a lot of black, so I don't know if it's really visible, 09:18.480 --> 09:25.280 but it's basically that arc there. So we had to work a lot on the data. We had to fix a few 09:25.280 --> 09:32.720 same manually. We had to guess which data could enough quality to use. Sometimes we had to guess 09:32.720 --> 09:38.480 the data, sometimes we had to use open straight map, which is about better qualities than SNCF. 09:38.480 --> 09:46.800 So we had data, we opened it to a few users, that was about a year ago, and they could use it. 09:47.840 --> 09:54.240 So we had the first feedback, that was starting to become interesting. What we were 09:54.240 --> 09:59.200 as is that, so this workflow is optional, they could still talk to the entire planners directly. 10:00.240 --> 10:05.440 And it only works on one small part of France, so what they did is that they did not use it. 10:05.520 --> 10:13.520 I mean, why would they learn a new workflow? But they quickly figured out that they could use it as 10:13.520 --> 10:21.040 some kind of first initial check as in. They want their trend to go to one in like three days. 10:21.040 --> 10:27.040 They don't want to wait one whole day to know if that one has a chance to one. So their first 10:27.120 --> 10:34.560 goes through OSRD, enter their requirements, and then immediately see if it's possible if there's 10:34.560 --> 10:42.800 a chance it would work or not. So that only works if we are optimistic, but at that point we 10:42.800 --> 10:49.840 very much pessimistic. So we did have to fix a few things. Mostly because of our decision tree 10:49.840 --> 10:56.480 approach, anytime we find a solution, it means we know it works because we have a full simulation 10:56.480 --> 11:04.000 that avoids every conflicts. So by nature we are on the optimistic side, but if we need some special 11:04.000 --> 11:10.640 tweaks to get a solution, then we will not consider it. So we had to implement all of those tricks 11:10.640 --> 11:15.840 basically and integrate them into the decision tree. Mostly specifically adding extra stops, 11:15.840 --> 11:21.840 having the trend stop on the side to be over taken by other trends. But we did implement that 11:21.840 --> 11:27.360 so it works fine. And then users actually started using it a bit more and we had feedbacks from 11:27.360 --> 11:33.120 the time-to-roll planners. The one who checked the generated files and actually tried to insert 11:33.120 --> 11:39.280 that trend into the time-to-roll. And I had quite a lot of feedback, which was also quite interesting. 11:41.600 --> 11:47.680 First thing they had was specific data issues. For example, they would say hey the trend you 11:47.680 --> 11:52.160 submitted there, it's actually conflicted with that other trend at that specific time. 11:52.880 --> 11:58.640 Which is extremely valuable. We would know either, we did not properly import that trend or we 11:58.640 --> 12:04.640 had some bugging. Both options are possible. And at first we could not do anything about it, 12:05.200 --> 12:11.440 because we actually did not save enough context, did not log it enough stuff. So the 12:11.440 --> 12:17.040 article is that simulation from weeks ago, we did not even have the time-to-roll anymore. So we could 12:17.040 --> 12:22.160 not know. Since then we worked quite a lot to import the logging, the context saving and everything. 12:22.160 --> 12:28.320 So now we can actually directly see on successful simulation. We can directly access the whole 12:29.440 --> 12:34.800 spacetime chart, which showed earlier. So we know precisely where every trend is and if one is missing, 12:34.880 --> 12:41.840 we would know. Some other thing we learned, they don't focus on the same kind of data. 12:43.520 --> 12:47.920 What I mean by that, that we mostly focus on the data that's necessary to one simulation 12:49.040 --> 12:55.840 and well to have to just find solution to squeeze the trend into the time-to-roll. 12:56.880 --> 13:03.840 And we have lots of other kinds of data, which I would call metadata in this case. So things like 13:04.800 --> 13:09.680 let's say labels on the tracks, we just say, hey, this trend goes to that point at that time. 13:11.760 --> 13:17.280 And some of these points are actually, well, we do have some errors in that data. 13:18.560 --> 13:22.240 And we're like, okay, that's where the trend goes through there. There's a track, it works, there's no 13:22.240 --> 13:29.040 conflicts. But the time-to-roll planners, they would be like, hey, this trend goes from A to B instead of 13:29.040 --> 13:35.840 A to B to C, there's something wrong. And we try to convince them, then just ignore it, it's fine. 13:35.840 --> 13:43.360 But they were like, no, that's not usual. So we did have to fix a lot more data than just the 13:43.360 --> 13:48.720 straight minimum for when the simulations. And what we've learned is that if we don't do that, 13:48.720 --> 13:55.360 then we just lose some credibility. Like, this is a requirement. So I had that trust, actually, 13:55.360 --> 14:04.720 I was up. Last thing we learned there is sometimes even communicating about why our solution is 14:04.720 --> 14:10.320 wrong can be quite difficult. So I'll give one specific example of that. At some point, one of 14:10.320 --> 14:16.080 the feedback was that, hey, your trend is willing to fast in this specific section. And what we do, 14:16.080 --> 14:23.600 so we want simulation for every single pass fragment. So if our trend is willing to fast, 14:23.680 --> 14:29.520 it can be possible, it would mean that we haven't properly imported some speed limit or something 14:29.520 --> 14:36.560 like that. So we try to talk with them, what's missing, where the speed limit and stuff like that. 14:36.560 --> 14:40.960 And like, no, I don't know, I don't care about the speed limit. All I know is that all the trends 14:40.960 --> 14:48.240 that go through there at that time, they won't slow us and you do. Okay. So there's likely something 14:48.880 --> 14:54.000 wrong. But actually, if you're going to get what it is, it's actually very difficult. 14:54.000 --> 14:58.320 They don't really work the same way as we do. They don't win simulations. They mostly just 14:58.320 --> 15:04.640 compare the trends to what's around them. So we learn that communication, we're important, 15:04.640 --> 15:12.400 we're difficult, as always. And to what we've developed is some numbers. We don't receive 15:12.400 --> 15:16.720 a lot of requests, mostly because there's not a lot of users, especially since we don't have 15:16.880 --> 15:20.720 the entirety of fans yet. We still get a few dozens of requests per day. 15:22.320 --> 15:28.480 Also, there's only a tiny amount that gets actually forwarded to an entire planners. We suspect 15:28.480 --> 15:33.440 they just don't bother with that. They know the simulation works and so they just go with it. 15:35.680 --> 15:41.200 Out of the ones, we actually, that actually get up to the entire planners. We have certain 15:41.440 --> 15:47.120 person for a positive, 6% for a snagative, which is not perfect, but considering all the 15:47.120 --> 15:53.360 data issues, it's good enough. All the identified errors do come from data issues. We haven't 15:53.360 --> 16:00.240 identified any bugs yet. I mean, sometimes it crashes altogether, but then that's it for a better. 16:02.080 --> 16:09.040 Confitation times, it's generally about a few dozens seconds. It can get a lot longer since we 16:09.120 --> 16:12.240 basically boot for a single problem, but most of the time it's pretty fast. 16:14.800 --> 16:19.920 To conclude, what we've learned to summarize, the technical approach itself is pretty 16:19.920 --> 16:28.000 sound. It works fine. Generally speaking, if we get quality data, which is as always the most 16:28.000 --> 16:34.960 difficult part, especially on this amount of data, like it's really massive, we need to automate 16:35.200 --> 16:42.320 a lot of it, so we can't manually check that makes sense. We've learned that there's a lot of 16:42.320 --> 16:49.520 people who need to convince, which can be difficult, especially in some times, that try it, 16:49.520 --> 16:56.320 see something that we consider minor and they're like, not using it. Which is fair. 16:56.320 --> 17:05.120 Anyway, so it can work on any infrastructure as long as the data. We're always looking for 17:05.120 --> 17:11.440 collaborators, feedbacks and, of course, if any other infrastructure manager would be interested 17:11.440 --> 17:18.080 in trying it, we would be very interested. Yeah, that's it. You have any questions? 17:26.320 --> 17:44.800 Do you do you, like, get from these people, like, questions? Hello, Stations Masterpiece, 17:44.800 --> 17:50.880 a fellow who's gotten very new tools, and then you have a website somewhere, where you can 17:50.960 --> 17:57.280 model yourself. You're in person correct problems, and some information, if you don't know who 17:57.280 --> 18:04.080 you use this from a sticky note, you say, an inspiration that you don't find, but you end up 18:04.080 --> 18:13.520 to the model, and then basically multiply your hand as one thousand. And it's quite easy to 18:13.760 --> 18:20.400 do. Yeah, so that's an interesting question. The question is about people in the 18:20.400 --> 18:26.480 stations that know their infrastructure, and if they could use the website on a bit, basically. 18:28.640 --> 18:37.200 So, yeah, there are people in charge of their stations. I do, to be very honest, I don't know 18:37.200 --> 18:44.880 how they interact jointly with the 10-year-old planners. From what I've heard, so what I've heard, 18:47.360 --> 18:53.040 10-year-old planners need to know on which track each train is staying at the station. Some 18:53.040 --> 19:01.120 stations have, like, modern tools, so they can know that. The most remote train station, it's 19:01.120 --> 19:05.760 basically a note pad, and people wipe it down, so when you need to know on which track is that 19:05.760 --> 19:13.520 train you call the station. So this is basically impossible to automate, so this is currently out of scope 19:14.160 --> 19:22.240 for RSD, which is an issue, pity big one. But it could be improved, I think it would cost quite a 19:22.240 --> 19:27.360 lot of money, and that the reason is not currently being done. They could use RSD, but then 19:28.240 --> 19:34.080 learning how to use it, I think it's just not that job, and it would be hard to convince them 19:34.160 --> 19:40.400 that it's useful. They would benefit from it, but convincing them, that's always a hard part. 19:43.440 --> 19:49.120 We'll talk about the 10-year-old planners. We'll be very few of them are actually forward 19:49.120 --> 20:00.640 that are visual, and what, why is that? We suspect they just don't. So they do send a request, 20:00.720 --> 20:05.920 they get a response, and then they can fill the form, the usual form, the one that takes 20:05.920 --> 20:12.960 it before, to request a new train, and they may or may not attach the PDF file, they just don't. 20:13.680 --> 20:17.680 We need to talk about them, that it would be helpful to do so. 20:19.120 --> 20:26.000 At least at what we suspect, we really need to talk to them to figure that out, we don't know 20:26.000 --> 20:28.000 what it says, we won't.