Thank you for coming to our talk. I am David Binyar's Eek. This is Jan Cherney. We're
both students, master students at the Charles University Faculty of Mathematics and Physics,
which also does computer science, although it's not in the name. And we are the main
trainers, both code and content wise, of Mathis Binyar, our student Binyar that we built.
Thank you very much for introduction, David. Our faculty has a really long-running problem
that finding information about anything was actually really hard. Everything was really fragmented,
and not just the source of information wise. Our school itself is really fragmented. We have
buildings all over the park. We have multiple fields of study. We can study computer science, physics,
linguistics, mathematics. Every building has its own organization and systems they use.
And it's really easy to get lost in it. And so, the attempt of erosion in the past was multiple
wikis, which mostly ended unmentant or broken or full of spam, forums, some social media groups,
which ended up being too chaotic to actually search with bad signal-to-nose ratio,
and more spam that any content it could be useful. Then this code service pop-up. There was many of
them, but they are very hard to archive or search to past. And there is a tons of personal
and organizational websites which are very often outdated or contain only
incomplete information. So, our vision for solution would be to finally make one central place,
where you can find the information or find where you can find the information. Everyone can
contribute it and even anonymously content is organized and categorized. So, it's easy to navigate
and search. It would be very versatile and universal. So, you can put text in it, media files,
PDFs, study materials, photos and anything we need. And everything will be free and open source.
So, everyone can contribute even into the code of the solution itself. And that's how
Matriz wikis was born. This is the front page and how it currently looks like. As you can see,
there are four, there are four main categories, which we'll check later. Currently, some statistics,
we started by importing a lot of content from other dead platforms and replacing them.
There are currently, which generated more than 10,000 items.
Now, about 950 pages were edited from last year and are currently in the
not archives by the living part of the content which is heavily edited by students and
university adjacent people. And since start of this year, 140 items were edited since.
This is the categorization we've finally went through because it's in check-out translated.
This one is about school, this and how it works. This one is about studies, this one about student life
and this is about general tips. School contains information about bureaucracy, how it's organized,
campus information, information about buildings, where you can find it, the hierarchy of power,
what you need to ask for and stuff like that. It's very useful for freshmen who can
diversity and try to orient in the system of university. The study, which is the biggest and most
used category, where you can find information for specific subjects in school and study materials,
past exams, tips, what you need, but even more meta tips like how to set up your latex.
So it's good for your physical reports and other tips like what content management system to use,
knowledge management system to use, how to set up your editor and stuff like that.
Then there is a student life which is mostly to snapshot of our culture, which is very strong
strong. And we have very strong sense of community and long history. So this one is something
that contains it and archives and preserves it and there you can find information from other
organizations from part of the school and what is happening where and the general tips, which are
much more general. There are tech tips that you can do on your dorm, how to set up any tips,
what to choose for Linux distributions and stuff like that. First content as I already said was
important. We've imported from mainstream sources, which we all replaced all WK, which was
Media WK maintained by students, but it died and got a really large problem with bots.
So it became basically unusable. There was another project called Student Googlebook, which was
another media WK, but in an administration of specific group of people. So it wasn't able to be
publicly edited by all students. It was very selective on what it contained. And forum, which was
all page BB forum, which was in its last years mostly bots talking to each other, trying to
sell each other something. What do we do with moderation? Our faculty has a really strong sense of
community and it's a very high trust environment. We allow anonymous edits and basically we have
only three rules of all the WK. User common sense, every article must have exactly one
H1 heading and links to pages should have depth of one. And theoretically it's moderated,
but we never had any malicious actor. And all we did was maybe contact someone who used
macros in a bad way, how it's not intended. But that's basically it. So one of morals, which you can
take from a lecture, is if you are scared of moderation, if you are working in a specific community,
and there is some level of trust, you do not need to worry about it. It actually works really great.
And now I will give space to David to tell more about the software side of Matthus Vicki.
Thank you, thank you. We decided to replace the media WK system that the old WK used,
because it was a really old version, and we didn't want to move forward with that. And we
got made a few decision to make a list of requirements for the new WK. And we expected
there being like tens of different WK projects that one would fit. Well, we need something that
used Markdown because students already know Markdown from GitHub, from GitLab, from various other
applications that faculty uses that do use GitLab, Markdown. We wanted something that could be
extended with custom macros. So we could create plugins that would work to add maybe reusable
components, or even something that could talk to a different server and pool data from it,
without having to manually modify the source code of the parser or something like that.
We wanted multiple input and input formats joined by a common ASD. You may know this sort of
happens by design from Pandok, and we used this to import the other WKs.
And we wanted something that could use the university cast system for authentication, because the
secret of our moderation is that we only allow people with a university account
access to the WK or write access to the WK. If anyone else wants it, you have to write an email
and be nice to us and explain why they want the access. So that's there have been very few cases
of our happening, so it's not a very big workload for us. And the last requirement we wanted
something that didn't require JavaScript for viewing. Sadly, after going through these requirements,
we were left with only two WK software options. Moinmoin and X WK, and the X WK being
really large and written in Java, and it's own scripting language. Partly, we picked Moinmoin
because it was an impiphon and much smaller and more comprehensible for us as the primary
maintainers of the WK. Moinmoin used to be very popular, but then came Python 3 and Moinmoin
version 2 and V2 is still in perpetual beta or even alpha. I think it works mostly pretty well,
but it's a minority choice currently. Well, we didn't want to really modify it too much. We just
wanted to add our own theme to make a bit of a brand statement with the WK to make it look like
something new and a few features here and there may be a bug fix or two. It worked well in our
testing instance, but once we imported the old WK with its tens of thousands of revisions at
first before we cut down all the spam and other bad stuff, it became really slow, like multiple
seconds per a simple page load and tens of seconds for more demanding pages. The problem here
is that Moinmoin 2 and also Moin 1 use the file system for storing everything, data, metadata,
everything. Moinmoin 2 added an index using the WK library and the index
WK is mostly a full text indexing library, so it was used for a search function, but Moinmoin
1 used it for everything, including things like looking up articles by ID or looking up users.
Somewhere along the line, WK, which is now unmaintained, became really slow, because I can find
benchmarks from maybe 2012, which say on data sets bigger than ours, it's really fast,
but currently or rather two or three years ago when we debug this, it was slower than
grep on read and rebuilt almost the whole index on right, so we clearly needed to either fix it or
replace it, fixing it didn't get us anywhere or attempt things to fix it rather, so we decided to
move everything over its very postgres database. First, as just a secondary store, as a secondary
index, replacing the WK and now replacing the whole storage layer of the WK, replacing it with
SQL alchemy and type entities, and after three years of work on this, request now take 50
milliseconds and on a week, you saw the size of our wiki, and I don't think it's small, I think it
deserves the 50 milliseconds. We didn't just do this, the two of us, I think our greatest success
is actually creating and organizing these student meetups, these student hackathons, you could say,
where we got last time I think it was like eight people programming roughly, as a pretty big amount
of people working on the wiki and first converting it to the postgres storage, and then later adding
various functions and making it more user-friendly, we got pizza. This was partly organized by
our sort of partner group, which officially controls the wiki and then hands over the
control to the both of us, but they both are pizza, so thank you, Spot Matrizak.
We didn't just change the storage, of course, that's a big thing, and I think moving from
untyped dict to typed classes, holding the main data in the wiki is a big improvement,
but we also added typing, basically, everywhere, because it's a big help when onboarding other people.
I think those eight or ten people who went to the hackathons couldn't have done what they did
if the wiki had no type hints as they had at the start. Postgres being a proper transactional
database also really helps with integrity issues. We previously had at least once a month, I'd say,
an issue with index, the secondary index, and the primary file system storage de-synced
through something which we were not always able to trace back to its source problem.
Now that storage and indexing is fully done in the same postgres database, we haven't had such
problem yet. We read it, the permission system, it's now simpler and simpler and more powerful,
somewhat at the expense of user-configurability, and I mean, like, a user, not as in the admin,
but the person with an account on the wiki. We put in a more powerful mark on parser with macro
support, because previously Moindmore had macro support, but only using its own custom markup
language. Now, Markdown has full-fact macro support, and we made various UI tweaks to our theme
and to Moindmore and as a whole to make it a bit more friendly to new users who don't have the
wiki training necessary to maybe know how to add attachments when attachments don't really exist.
We also removed a lot of un-maintained features, and we consider that a feature, not a bug,
because those features mostly didn't work, and they made everything much more complicated.
So, during the process of converting a rewriting to postgres, we asked a lot of them
and the wiki is now more maintainable as a result. We're still fixing something,
like, Moindmore and had a great suite of automated tests, most of those are no broken,
because we just haven't had the time to rewrite them with the new code-based in mind,
maybe one day. We have a few hopes for the future. We've just a week ago about
we submitted our fork to the Moindmore developers, which they are still very diligently developing
MoindMind version 2. With the help that they could use, if not the helping,
then at least some of the parts of the code that we refactored. I also like to reuse some of their
improvements that they've made in the three years since we've worked. It'll take a lot of work,
but maybe we can make it happen. I think we've demonstrated that using a free and open source software
in this kind of setting is a reality and not just using, but actually improving it. Without having
to do any complicated things like pay for a maintenance contract or have great support from
some internally at Mathvis. I think the two of us managed it on the side of most official
processes. I think you don't really need moderation if what you're doing is giving select people
access, and that's somewhat of a trueism, but it turns out that select people can mean a whole
university, so thousands upon thousands of people, and we hope that our software will be useful
for other applications as well. Right now, it probably has a few words in that regard. We don't
really test it outside Mathvis Wiki and our development setups, our local development setups,
but over the like 10 people who've come to the HECA phones and contributed to the Wiki,
almost every one of them has found some sort of bug in our onboarding documentation and scripts,
so hopefully all the bugs have been ironed out and you can mostly problem free run your own
instance of our Wiki. Thank you very much for your attention. We hope that you check our code
and you check the website and we hope that we were able to inspire you to do try something like
this because it seems like a big project but it's now here and we are very glad that we didn't
get scared and we actually tried it. These are our contacts. You feel free to reach for us if you have
any questions after if you get any after the questions you can ask here right now. So thank you again
for your attention.
So onboarding process for new contributors on the content side, I presume.
That's something we have documentation, maybe I can, we have a few pages on the Wiki which show
examples of the macros we use, examples of markdown itself, examples of how pages look and we
sort of hope people figure it out by example and if they don't we tell them what they're doing wrong.
Ons are here keeps a tight watch on every page modification every day who looks at the global
history. So that's we haven't figured out anything great or awesome is just
examples and hard work and plus we got advertisement through posters and social media posts by faculty.
I think we had of from all time editors in a hundred it's one hundred 36 editors of all time
with most of them did only small amount and there is a smaller, much more active group which
there's like 80 people who've made five or more edits and 10 people who've made a hundred or
more edits.
How did the faculty react to the fact that apparently the exclamation paper side is not sufficient
to know that you have to make a decision of your own?
Well how did the faculty react that their information is insufficient?
They didn't I think math is decentralized enough that there is no definitive answer to that
question but I think everybody in the faculty they know that sort of distribution of information
on math is I wouldn't call it a four point it's very much it's anarchy so we have some central
system which should contain loads of things and has the potential to contain loads of things but
it's sort of ossified it hasn't been properly developed for I don't know 10 years it's on
life support and all the new system systems have so far failed or not been delivered so
information is on people's personal pages hosted in on various servers in various
buildings or elsewhere or on personal domains or I think they they all know that it's a bit of a mess
Thank you