All right, next time we're going to have Luca talking to us about a reproducible
XFS file systems.
Hi, everyone, I'm Luca, today we will talk about reproducible file system with XFS.
A little thing about me, I'm a software engineering Schengard.
I know an open source enthusiast.
You can find me everywhere with 89 Luca 89, that's my mail.
Why are we interested in reproducible file systems?
We build stuff and we want that stuff to be very viable by the end user.
The user has access to the source code, so they should be able to reproduce our own
artifacts and be sure that we are not missing stuff up.
So it's also useful for us for debugging, for auditability and so on.
So it's very important and working in Schengard with for our product of VMs.
We wanted to be transfer for our users and have reproducible images for them.
How do we use one usually approach, publicizing the file system, MKFS, that file system,
you mount it, you copy stuff inside and you amount it, but this is actually very much
not reproducible.
We have some caveats, we need root privileges to do mounts and we need a kernel for
that, which is not, I mean, it's very important.
Time stamps, obviously vary, you write different times, you have different time stamps and
we have different random seeds that lead to different Inaut generation and so many moving
parts.
My first attempt was using a limb effect time, if someone doesn't know what it is, it's
a library where you basically use an LD-prilode attack, let's say, it will, in Ijak, most
of the random functions like get time of day, get random, get from 32 and stuff like
that.
So, if we try to use a limb effect time and create an empty MKFS file system, these actually
works, we have an empty file system which is reproducible, but in the moment that we mounted
to populate it, this will break reproducibility, even if we use no time, no time, no
this stuff.
There are many reasons for that.
First of all, LD-prilode, he's only on user space and mount is kernel space, so that won't
work simply.
Also there is the creation time, a set by the kernel, a lot by the ellipse, so you cannot
intercept that.
Also XFFFFF is a very complex file system, it has allocation groups which has some
a risk to spread the load, as parallel and I think I owe, which is non-deterministic
because it's not.
That non-deterministic order of allocation changes the B plus 3 shape, so that's also
not reproducible, so I went out and checked on reproducible.org, there is a system
images section that shows you how some devices them are actually B can be reproducible,
like all the XV family, Squishafz, EFS, and the idos.
The common part is that you don't mount it to populate it, you populate it while you do
it.
So let's do it.
I went, the plan was first, let's work on the hardest part, which is
populating the directory while we do MKFS.
Then we can do the deterministic parts of block that fix the time and fix the other
random points and then attest because we always have to attest.
First step was populating.
I went out and searched and actually XFS has a way to
already had a way to populate a file system, which is called
Protophile.
It's a very, very old specification, which basically is like a text file that
represents the file system in our text manner.
And as you can see, you have a reference to the original file that we
want to copy only for regular files.
All the other files like Siem links, char devices,
I don't know, block devices and stuff like that are actually
created and they do not refer to the original file.
So we have limited extended attribute supports because we
lose them for directories, for Siem links and stuff like that.
We don't have actually control on timestamps because we
could copy the timestamps only for regular files.
All the other files are set to current time, which we can control,
but we lose the information from the original source.
At the beginning, I proposed to change this, but this was rejected because
it's a very old and let's say stable.
Yeah, and specification.
So they suggested me.
We should actually create a populate from directory functionality.
I tried, so basically what it does is we use the
prototype infrastructure that was already there and just branch out
in the case that we are pointing to a directory to populate from
instead of five to populate from.
So it will be a recursively traverse all that are actually in
that source directory and then if it's a directory recurs,
if it's a file created, copying from the source,
timestamps, which are the modification time is what we usually
care about.
Access time usually is just nice, so by default it's not copied and
set to current time.
Creation time is set always to current time.
We will control current time afterwards.
So yeah, usually in the reproducible builds world,
access time is stripped anyway, so by default it's not preserved.
All the time stamps we need attributes.
One caveat was that when we forgot files, this was pretty easy.
There were already lots of function pre-made to do that.
For simple links, sockets and feeforce, you cannot actually use the
regular FGAT attributes because we will not open them regularly.
We use them with all parts and we know follow.
So we don't have loops, so we have to handle the bed
file descriptor error in full back to L get X attributes and so on,
because simple links and habits extended attributes that are important to
and if we just skip them, we might not have a function of file system
actually.
The other part was tracking card links from the source directory in the
destination file system.
The implementation is added concentrate more on correctness than
performance, so it's linear array.
It's a static global array and then we just registered all the various
card links and refer to that with some logic for the growth of the array.
Actually, I didn't find it a problem performance-wise because we are so
MKFS is IO bounded and it's sequential, no parallelism and I tested it
on my flat pack folder which is full of hard links and it behave good.
Anyway, this is maybe for the future we can improve that and do it
and ask for something like that.
We have populated the file system.
Now we have to add the determinism to it.
So we can try again with leapfack time and this actually works.
But we want to get away from leapfack time because first of all,
we add deeper load type of things is not ideal.
It doesn't always work, static linking is a thing.
This doesn't follow the reproducible build standards for source data
So, and basically we will depend on external tooling
instead of just doing MKFS, right?
So I went away ahead and implement the source data epoch thing.
So we get current fixed time.
So if source data epoch is specified in its valid number,
because it's a unique timestamp actually,
so seconds from 1970s.
Instead of get time of day, we just return the fixed value.
This is used for creation time, change time for I notes.
And the other part is having the get from them part.
I was suggested to do a very simple thing about just,
is it deterministic mode and return a fixed value instead of just passing the value
that you want to be return it?
Because anyway, get random in MKFS time.
It's only used for the I note generation number.
And at MKFS time, you don't reuse it.
I note, so it's not a problem if every I note has the same generation number.
So this doesn't break stuff.
And then we have the testing.
So in the XFS tests, I added a test where we create a five system three times in a row
and they should always have the same harsh, very simple test.
The file system, sorry, the source directory is generated with FFS stress.
So it has all type of files, links and so on.
So it should be quite representative.
We can see it in action.
So we can have a very basic XFS file system that is probably populated from our root of FFS directory.
We can preserve access time with eight-time equals one.
Maybe it can be useful for someone.
We can have an actually reproducible file system,
an empty file system without using livefake time by using the environment variables for source data book.
And the deterministic set, so this works.
Be mindful, we still need to fix the new idea of the file system because it's random else.
But yeah, that works.
And we can do a little test with make OFI to create root of FFS for debion and create multiple root of, sorry, images for it.
So this is a script that just uses MK OSI to create our root of FFS, create an image file,
and then MKFS it and populated.
So if we build it the first time, this will go and, sorry, create the root of FFS, which is the slow part actually.
And after this, it will automatically, you will see the familiar output of MKFS, which is one.
And it automatically also populated the file system.
You can see the root of FFS, we save it in another name and delete the root of FFS and regenerate it again.
This will rebuild the root of FFS directory, which is reproducible because MK OSI supports reproducible builds.
After that, it will create a new image file and a new MKFS command as you can see.
If we file the images, they are both valid FFS file season and both of them have the same hash.
This is very useful for us.
We use it to build our images.
So we know when an image actually change it or not, before maybe shipping them to clouds and stuff like that,
because many other steps are not reproducible like converting our root, this to, I don't know, VHD, VNDK for bio-file format, cloud format.
So we know if we have to fire up our CI or not from this, for example.
So it's important for distributions.
Maybe also for embedded system, this can be integrated with the octo build root.
It integrated with MK OSI as you see as you saw.
It's important for security for salsa for compliance.
You need reproducible builds and for testing.
These are the links to the patches, the first two are the MKFS patches, the test patches, and then the documentation or reproducible builds.
Thank you.
The question would be, do you keep some artifacts of what was the environment you used to generate the image?
Because if you want to, do you keep metadata files or artifacts that traces the software you used on the version you used to obtain a particular file system?
For the MK OSI part.
So this one is just for the demo.
In Changga we use an upcode to build our root of the images, which is also the tool that we build for our container images.
And that one is also fully reproducible, and we have a very historic part of how it was composed.
It's rendered, we call it, it's rendered, so it fits the packages versions and it's pinned to snapshots of the repositories.
All right.
Thanks a lot.
We're going to set the out of time.
Thank you.