WEBVTT 00:00.000 --> 00:08.640 This next talk looks at mentorship in the AI area. 00:08.640 --> 00:14.440 If newcomers can get senior level answers instantly, what's going to shift for human mentors? 00:14.440 --> 00:16.920 What gets easier and what's going to get lost? 00:16.920 --> 00:21.600 Please welcome Abigail Kebnerk Mays for a thoughtful look at what still needs to be human 00:21.600 --> 00:23.600 as tools change around us. 00:23.680 --> 00:33.040 Thank you all so much and huge thanks to organizers in oil and thank you for doing the 00:33.040 --> 00:34.640 time keeping and for the chocolate. 00:34.640 --> 00:37.400 It's a really good chocolate everyone in the room. 00:37.400 --> 00:42.960 Everyone on the string you're missing out, you should have come to foster next year. 00:42.960 --> 00:48.120 But before I start, I did want to say thanks to all these people that sent examples, 00:48.120 --> 00:53.280 I had conversations with me sent insights and there was a lot more just sharing examples 00:53.280 --> 00:54.280 online. 00:54.280 --> 00:59.000 It really influences talk so just so you know it's not just my thinking it's so many 00:59.000 --> 01:00.000 other people. 01:00.000 --> 01:03.720 So I'm going to paint and we'll picture for you. 01:03.720 --> 01:08.440 This polished PR lens in your inbox, it looks amazing at first glance. 01:08.440 --> 01:11.160 You start looking into it, things look a little bit off. 01:11.160 --> 01:15.760 Next you know 45 minutes later, you've gone through the whole thing, you've crafted a really 01:15.840 --> 01:22.320 empathetic response, we'll try to be a little bit critical and you get back something that 01:22.320 --> 01:26.480 makes it clear they don't understand the code at all or you realize you're just debugging 01:26.480 --> 01:30.320 someone else's chat to be session or they ghost. 01:30.320 --> 01:36.720 So that's the synthetic senior, something that looks senior at first glance but that understanding 01:36.720 --> 01:40.240 that makes it a real contribution is missing. 01:40.240 --> 01:44.000 So this is a problem we're seeing across the ecosystem, things that you may know, till 01:44.000 --> 01:50.560 you are closed there, pull requests because of this problem and we see groups like Curl 01:50.560 --> 01:56.160 turning up hacker one, no turning on the signal for hacker one and the volume of contributions 01:56.160 --> 02:00.240 is going up but the signal to noise ratio is really going down. 02:00.240 --> 02:05.280 So old signals are relied on which is like clean code, fast turnaround to that complexity 02:05.280 --> 02:10.240 that used to mean that someone had the right understanding that they spent time in your project. 02:10.960 --> 02:14.560 But now I can generate that all in seconds so that signal doesn't mean as much anymore. 02:15.680 --> 02:20.320 So platforms including GitHub, where I work, are working on long-term systemic 02:20.320 --> 02:23.600 solutions so we just publish this RFC I'm not going to talk too much about GitHub don't worry 02:24.400 --> 02:29.360 but we are interested in hearing your thoughts on this and this like what can platforms do to help 02:29.360 --> 02:34.400 with this problem but I do think communities need strategies they can use right now so I'm going to 02:34.480 --> 02:38.640 focus more on what you can do now but platforms aren't thinking. 02:42.240 --> 02:47.680 So this is why I think it's really urgent because mentorship is how fast can you scale? 02:48.080 --> 02:51.040 How many people here had someone help them get into open source? 02:52.560 --> 02:54.000 Yeah, it's nearly all of us. 02:55.040 --> 03:01.280 So mentorship's why we're here at Phastam it's how movements grow and why so many people do open source 03:02.240 --> 03:06.960 so when you mentor someone you're not just growing one contributor you're multiplying yourself 03:06.960 --> 03:11.120 so that one person learns how to onboard others and then they teach those to do the same. 03:11.680 --> 03:17.520 So I like to use this table where it's like you could just train a thousand people a year 03:17.520 --> 03:22.480 in some sort of broadcast session and after the first year yeah you have like almost a thousand 03:22.480 --> 03:26.880 more people than someone doing mentorship but after five years you're at five thousand while with 03:26.880 --> 03:31.520 mentorship if you're mentoring two people every six months and then they go on to do the same 03:32.080 --> 03:38.480 by year five you're at 59,000 in 49 it's never that clean but you really blow it out of the 03:38.480 --> 03:45.440 water and this is why I care so much about mentorship and why this is how we make a difference in the 03:45.440 --> 03:50.640 world but the problem is a lot of maintainers are just burning out trying to mentor everyone 03:50.640 --> 03:56.720 that's sending a pull request so if we lose yeah we don't want to do that we don't want to 03:56.720 --> 04:05.120 lose this multiplier effect entirely and projects like T.L.D.R. T.L.D.R. are closing their pull request and 04:05.120 --> 04:12.720 we're we're at risk of stopping this knowledge and position to next generation so we can 04:12.720 --> 04:17.520 abandon mentorship now especially with this grain of open source we need to be strategic about 04:17.520 --> 04:23.280 where we do spend our effort and how we do find the right people to mentor so I have been thinking 04:23.280 --> 04:28.720 about mentorship selection for years along before AI made this really urgent right now and looking 04:28.720 --> 04:33.840 at what's working across projects I see a few things that maintainers are filtering for I've mentioned 04:33.840 --> 04:39.440 some of these things before but I didn't have like a pithy way to say them so I tried three 04:39.440 --> 04:45.280 seas hopefully people remember this one it's a looking for comprehension context and continuity 04:46.080 --> 04:51.600 so comprehension do they understand the problem well enough to propose this change 04:52.240 --> 04:57.120 context they give me what I need to review this well and then continuity do they keep coming 04:57.120 --> 05:01.120 back so I'm going to go through each of these with some examples of how I'm seeing projects 05:01.680 --> 05:09.600 like find real signal for this see so first comprehension do they understand the problem well enough 05:10.000 --> 05:16.560 to propose this change so some projects are actually starting to test comprehension before 05:16.560 --> 05:21.600 any co get submitted um in the last few weeks is actually really interesting projects like 05:21.600 --> 05:27.440 OpenAI's Codex and Google's Gemini CLI they require contributors to discuss an issue before opening 05:27.440 --> 05:33.760 a PR that's all within the last couple weeks super interesting so this comprehension check happens 05:33.760 --> 05:40.160 in that conversation beforehand before any codes written you also see on the other end so maintainers 05:40.160 --> 05:43.920 asking follow up questions to test comprehension this one doesn't always work as well so it's 05:43.920 --> 05:49.680 super easy to copy paste between your AI but I think this is where things like in person code 05:49.680 --> 05:54.320 sprints and hackathons really shine where you can talk to someone and check where their comprehension 05:54.320 --> 05:59.200 level is it's okay if someone doesn't comprehend the whole project and not expecting that but just 05:59.200 --> 06:04.400 make sure that they're not committing something that's above their level of comprehension and then 06:04.400 --> 06:09.600 like as you work with them they can commit more and more so I believe there are some code sprints 06:09.600 --> 06:14.880 happening here at Boston and being here in person that's a great way to sort of do that comprehension 06:14.880 --> 06:22.880 check okay next see as context do they give me what I need to review well so comprehension was 06:22.880 --> 06:30.320 about their understanding but this is about your ability to do your job as a reviewer so if you're 06:30.320 --> 06:35.280 requiring that issue beforehand do they link to that issue to explain the trade-offs considered 06:35.280 --> 06:41.760 do they disclose that they use AI so that last one's becoming a little bit more prevalent so 06:41.760 --> 06:46.960 roost has a simple policy with three principles human oversight attribution transparency 06:47.760 --> 06:54.080 the processing foundation added a checkbox to use AI if so how for door landed a lightweight disclosure 06:54.080 --> 06:59.680 policy after months of community discussion and I think that's helpful for review or see you know 06:59.680 --> 07:05.440 how empathetic you have to be about the code sometimes and how much of a teaching moment this 07:05.440 --> 07:10.800 has to be or you're just telling an AI to change a little thing differently and then the last 07:10.800 --> 07:15.440 point I think is really interesting we're seeing more people use agents agents that MD files 07:16.000 --> 07:22.560 and this is instructions for AI coding agents so if someone releases an agent on your repository 07:22.560 --> 07:28.640 this file tells them like hey follow our contribution guidelines do these things so it offsets 07:28.640 --> 07:34.640 this need for gathering contacts to the contributor or their tools so um yes I could learn 07:34.640 --> 07:39.040 it's been doing that goose and processing are all all have this agent set on B file 07:41.360 --> 07:47.440 and the last one continuity so this is the one that I think is the mentorship filter you don't need 07:47.440 --> 07:52.560 continuity from every contributor that comes to your project drive by contributions can be really helpful 07:53.520 --> 07:58.560 but I think you should limit who you mentor or who you give those like teachable feedback 07:58.560 --> 08:04.160 reviews to to the people who start coming back or they start interacting like really nicely 08:04.160 --> 08:10.000 with you in that pull request so your level of mentorship can scale up if it's their first pull 08:10.000 --> 08:15.760 request you're having great conversation yes start crafting those empathetic reviews we're so they 08:15.760 --> 08:21.120 can start to learn they start coming back maybe offered a pair with them on something keep coming 08:21.920 --> 08:27.360 back invite them to an event give them conduct access so this this is the one where I think you 08:27.360 --> 08:32.080 can sort of scale your mentorship with and just don't mentor people that only show the first 08:32.080 --> 08:38.880 to comprehension and context you can that's the barrier for when you review something but then 08:38.880 --> 08:43.360 when you start to mentor someone is when they keep coming back and show that they have the interest 08:43.360 --> 08:52.640 and the time to contribute all right so comprehension and context get you merged and then 08:52.640 --> 08:59.840 continuity gets you mentored I was trying to be pitty again be memorable it might be more 08:59.840 --> 09:06.240 comprehension context get you reviewed not necessarily merged yet but continuity will get you mentored 09:06.240 --> 09:12.320 if you show all three of them so how does this look like in practice little flow chart so 09:12.320 --> 09:18.880 PR lands let's say you have this implemented where you have new guidelines that add signal for 09:18.880 --> 09:24.960 that comprehension that context do they follow those guidelines if no you can just close it guilt free 09:24.960 --> 09:30.880 don't spend effort on it if they do have that comprehension context give it a review it doesn't 09:30.880 --> 09:35.680 have to be an empathetic review yet but if they start coming back in that review they come back with 09:35.680 --> 09:40.000 another polar quest that's when you can start to really consider mentorship and really investing 09:40.000 --> 09:45.920 your time there so this is how we can yeah keep that and keep that multiplier effect and keep 09:45.920 --> 09:50.880 that mentorship scaling without burning yourself out mentoring every single PR that comes through 09:52.480 --> 10:00.320 so much time left I'm almost done we're gonna have questions okay so three season your project 10:00.320 --> 10:06.000 when you go home from Boston I want you to pick one of these seas and implement it so if you 10:06.000 --> 10:11.360 want to check for comprehension maybe it's requiring that issue before PR maybe it's just hosting 10:11.360 --> 10:15.440 something person so you can just talk to people and understand that comprehension level beforehand 10:16.560 --> 10:22.880 if it's context maybe add that AI disclosure at that agent's dot MD few other like anything that 10:22.880 --> 10:27.920 you think you need to review well add that to your guidelines just to offset that burden of trying 10:28.000 --> 10:32.240 to figure out how to review this and then there's like this watch who comes back 10:35.520 --> 10:40.400 so here's a list of resources if you want to keep the conversation going get 10:40.400 --> 10:45.440 up actually hosting an unconference story we do have a few spots left so you can register there 10:45.440 --> 10:51.440 at that top link there is that RFC that I shared earlier if you want to weigh in on how the 10:51.440 --> 10:59.680 product itself is helping with this issue and then all the examples I listed from this newly implemented 10:59.680 --> 11:07.440 issue before PR and like the agent MD policies and things like that so that's what I have thanks 11:07.520 --> 11:09.440 everyone 11:15.440 --> 11:22.960 all right we can claim your time for questions yeah yes yellow shirt first then blue shirt 11:24.480 --> 11:35.040 yeah is that yellow okay can you use the microphone yeah for the stream is that good 11:35.360 --> 11:42.720 I wondered if you thought maybe that people might seek out mentorship less because they're 11:42.720 --> 11:47.600 getting some of the things you get from mentorship from using AI and if that might be a problem 11:47.600 --> 11:51.920 how to address it oh yeah I hadn't really thought it yeah people are getting some of the things 11:51.920 --> 11:58.080 you get from mentorship using AI but I still think that people mentorship is what makes it like a 11:58.080 --> 12:04.560 multiplier and what gets people really into the movement and excited to keep going I do think people 12:04.560 --> 12:10.640 still need that kind of handholding even if the AI can do some of it I think it doesn't really 12:10.640 --> 12:15.680 replace a human I don't know if that's yeah that was a very deep answer but that's my 12:15.680 --> 12:22.160 initial thoughts on that right now yeah thanks for their question so say you have a mature project 12:22.160 --> 12:26.800 and you're imbalanced maybe you have not enough mentors because senior people have turned out 12:27.440 --> 12:32.880 or you have a lack of people that are worth mentoring maybe that are not showing 12:34.880 --> 12:40.880 like they're they're not able to meet your three seas right right what do you recommend to help 12:40.880 --> 12:48.720 bring larger projects into a better balance without sufficient people on either side yeah to get 12:48.720 --> 12:54.080 more mentors I think you need to start mentoring people which is not always the answer people want to 12:54.640 --> 12:59.600 but when you mentor someone like them they can become a mentor so just start off with finding 12:59.600 --> 13:05.280 like a couple of people that are almost there I think and then that will scale quicker than you think 13:08.000 --> 13:09.680 I didn't 13:09.680 --> 13:15.360 question actually I wanted to ask to show the previous slide the previous slide 13:16.560 --> 13:19.520 this one oh yeah yeah of course 13:19.600 --> 13:27.520 I have one yeah I have some kind of template for the agents MD or something is there any effort 13:27.520 --> 13:33.520 around that there's a website agent study MD that where it talks about the spec and what to do there 13:33.520 --> 13:37.760 I can't remember there's a template I don't know if anyone in the audience knows but there are a few 13:37.760 --> 13:44.720 examples I linked to the examples from from roost and processing so you can use that as a template 13:44.880 --> 13:46.720 over there at the end there 13:56.000 --> 14:02.080 Hi thank you yeah this is a great talk I'm kind of interested in how you think about incentives 14:02.080 --> 14:06.320 because I think a lot of the way that I've seen contributions is that people are really motivated to 14:06.320 --> 14:11.680 put in a pull request because they get their name listed in contributions but I guess I'm seeing 14:11.680 --> 14:16.320 now that like most of the value is in the discussion and like oh there's so thankful for someone 14:16.320 --> 14:20.960 that files an issue that's like human written and really well thought out and then like the problem is 14:20.960 --> 14:26.560 almost solved so I'm curious if you think about like how do we move incentives to have people 14:26.560 --> 14:31.760 contribute in the issue of discussion instead of the pull request yeah actually that might be a good 14:31.760 --> 14:36.720 thing to respond to that RFC I'll go back here and just like help platform so you know that 14:36.720 --> 14:41.280 because right now you only get the green square once your pull request is merged in so I have put 14:41.280 --> 14:44.720 it a lot of people they can you get a green square for like filing a really good issue 14:46.000 --> 14:51.600 I think that would help with that incentive problem I'm not sure yeah that's in the platform side 14:51.600 --> 14:56.080 for people side I think celebrating that kind of stuff as a community can really help 14:57.200 --> 15:01.280 but it's a little harder to implement if the platform's not really supporting what you're trying to do 15:01.920 --> 15:09.120 yeah thanks so hi thanks for the talk I work for a good lab and we have kind of a bit of a similar 15:09.200 --> 15:15.120 problem and trying to find as as the question was like incentives for a mentorship or kind of going 15:15.120 --> 15:23.680 to that other level we implemented badging and levels and you brought up with a good question for 15:23.680 --> 15:29.440 me is like how could we put mentorship as some kind of a requirement to level up to maybe level four 15:29.440 --> 15:34.480 five and then how do you think about certifications maybe give that back to people's LinkedIn like 15:34.560 --> 15:40.000 all of that stuff is that something that you're also thinking of here to kind of build this 15:40.000 --> 15:47.440 automation this machine instead of relying on people's goodwill yeah I know there is some work 15:47.440 --> 15:54.640 with like skilling and in GitHub skills and tying that to certificates I actually haven't seen a 15:54.640 --> 16:00.400 discussion as part of this RFC where it's focused a little bit more like what do we do with all of the 16:00.480 --> 16:10.480 PR coming in but that's a good that's a good thought yeah thank you yes yeah I don't know if you're 16:10.480 --> 16:16.400 familiar with the eight principles of Elino Ostrom for managing the comments okay I think it could 16:16.400 --> 16:21.440 be relevant here because it's a digital come on basically an open source community and one of the 16:21.440 --> 16:26.080 thing that this shows is that there is a need to actually create a membrane I want the community to 16:26.080 --> 16:32.400 protect from all of those AI issues one quick question is would it be possible for example for 16:32.400 --> 16:39.840 a committee to say that hey you need to be a sponsor or part of a white list to be able to file a 16:39.840 --> 16:44.240 put request in the first place yeah thanks thank you to create an incentive to us of the other 16:44.240 --> 16:50.640 issue which is funding how are we funding the comments and funding open source I have heard that before 16:51.440 --> 16:56.720 and that's a good thing like talking I don't think it's in the discussion in that issue yet 16:56.720 --> 17:01.040 but I have heard people say that like hey maybe if someone's only the certain tier sponsored 17:01.040 --> 17:05.920 then they're allowed to open a pull request I do think that can be tricky with the definition of open 17:05.920 --> 17:12.000 source or like definition of open contribution but it is a contribution I'm hearing so there might be 17:12.000 --> 17:18.560 something there just jumping jumping back a little bit to the question on what motivates people to 17:18.560 --> 17:24.960 mentor as well because I'm a triple developer and we give issue credits for anybody who opens 17:24.960 --> 17:30.640 an issue anybody who improves on the description of the issue anybody who views anybody who writes 17:30.640 --> 17:37.440 the writes the actual code who reviews the patch who's organizing a community event and who 17:37.440 --> 17:45.280 mentors so we have community credits for much much more than the actual code afterwards and on our 17:46.240 --> 17:52.240 user pages we also have a way of showing who has mentioned me and you're basically find a kind of 17:52.240 --> 17:58.080 tree of people to say oh this person that mentioned me was spent up by so and so and so and so 17:58.080 --> 18:05.360 so you also make that that community mentoring much much more visible not just the code 18:06.080 --> 18:10.240 yeah I love that and it's no surprise that your bulls fix this problem already 18:10.320 --> 18:17.360 Dries are as close as career ideas but thank you for that we have time for more questions 18:21.120 --> 18:27.600 so my question would be for a major project which is 20 years old how do you start with 18:28.960 --> 18:36.240 contributing guidelines so currently we have one or two programmers mostly doing all the stuff 18:36.240 --> 18:44.240 yeah now a new people want to come in but how to place all those rules set on them they are 18:44.240 --> 18:51.120 mostly technical and just in the code and you don't want to detail everything in the code 18:51.120 --> 18:57.280 when talking about pull requests and issues so how to get that started for project which was 18:57.280 --> 19:02.320 more closed development before yeah yeah that's when I'd recommend definitely having contributing 19:02.320 --> 19:06.800 guidelines there's a lot of templates online that you can take a look at I think open source guides 19:06.800 --> 19:12.640 has a nice getting started but definitely working with that close community to figure out what 19:12.640 --> 19:17.920 your contribution guidelines are because you you have so many unwritten rules amongst yourselves I'm 19:17.920 --> 19:21.440 sure just making sure they're written down and everyone agrees on them I think that's a great first 19:21.440 --> 19:27.200 start I cannot something the Linux Foundation and CNCF in particular they have these kind of 19:27.200 --> 19:31.600 templating around governance how you can build all that the competition for example in contribution 19:31.600 --> 19:37.040 guide so you can add all these stuff inside and you can just reuse it from some of the projects like 19:37.040 --> 19:43.280 Kubernetes for example yeah I'll have to such a good job with templates that's great cool 19:43.280 --> 19:49.760 right we're on time so thank you very much once again thank you all