WEBVTT

00:00.000 --> 00:12.000
Yeah, welcome to this talk about WebOTC in WebKit with Distremor.

00:12.000 --> 00:18.000
I'll give brief lecture updates and the plans we have for the future.

00:18.000 --> 00:20.000
So, hi, I'm Philip Normal.

00:20.000 --> 00:24.000
I work at Tigalia and since 2009 already.

00:24.000 --> 00:29.000
I'm Distremor developer and WebKit's reviewer.

00:29.000 --> 00:32.000
So, I maintain code both in WebKit related with media,

00:32.000 --> 00:35.000
and also in Distremor as well.

00:35.000 --> 00:38.000
When I need to fix something in WebKit,

00:38.000 --> 00:40.000
something needs to be done also in Distremor.

00:40.000 --> 00:42.000
So, we have to work in both projects.

00:42.000 --> 00:46.000
I was involved in WebOTC for a long time, actually,

00:46.000 --> 00:48.000
for almost ten years already.

00:48.000 --> 00:51.000
I don't know if you guys remember, but the open WebOTC project

00:51.000 --> 00:56.000
was kind of the beginning of WebOTC in WebKit.

00:56.000 --> 00:59.000
So, up to the brief introduction,

00:59.000 --> 01:02.000
I'll talk about what the current stake is.

01:02.000 --> 01:06.000
Of WebOTC, the ongoing work, the things that we are working on,

01:06.000 --> 01:09.000
and the things we plan for the future.

01:09.000 --> 01:13.000
Brief reminder of WebKit, I guess you are all familiar about that,

01:13.000 --> 01:15.000
but just a recap.

01:15.000 --> 01:19.000
Web engine that was started by Apple, based on KGML,

01:19.000 --> 01:23.000
in Long time ago, now already, in early 2000.

01:23.000 --> 01:25.000
And it's pouring safari.

01:25.000 --> 01:28.000
Lots of deployments are saw in a bit of platforms,

01:28.000 --> 01:33.000
like set of boxes, as the WP port,

01:33.000 --> 01:36.000
which is a specialized port for embedded platforms.

01:36.000 --> 01:40.000
And also on Linux desktop, we have the WebKit GTK,

01:40.000 --> 01:43.000
and one of the main users is the GNOME web,

01:43.000 --> 01:45.000
also known as Epifany.

01:46.000 --> 01:48.000
Reminder about the architecture,

01:48.000 --> 01:50.000
because it's quite important.

01:50.000 --> 01:54.000
We have the four processes, the UI,

01:54.000 --> 01:56.000
where you create the WebView,

01:56.000 --> 01:59.000
and where all the rendering is happening.

01:59.000 --> 02:02.000
And then we have the web process, where all the DOM is passed,

02:02.000 --> 02:04.000
just could execute it.

02:04.000 --> 02:06.000
And that process is really sandboxed,

02:06.000 --> 02:09.000
quite important for the future.

02:09.000 --> 02:13.000
And we also end on media there right now.

02:13.000 --> 02:17.000
And then for all the work related stuff,

02:17.000 --> 02:20.000
like receiving instant invites, you have no process.

02:20.000 --> 02:22.000
And then for WebGL and Graphics related,

02:22.000 --> 02:26.000
we have a GPU process, where some media stuff

02:26.000 --> 02:29.000
is happening for the Apple ports.

02:29.000 --> 02:32.000
Just the WebRTC, so it's two things,

02:32.000 --> 02:35.000
just to my plugin, and the library also.

02:35.000 --> 02:39.000
And the plugin is making use of the library.

02:39.000 --> 02:41.000
So in your application, you create,

02:41.000 --> 02:44.000
when you have your pipeline, I would show an example later,

02:44.000 --> 02:46.000
you have an integral WebRTC bin,

02:46.000 --> 02:47.000
that will under kind of,

02:47.000 --> 02:51.000
lots of things for you related with WebRTC.

02:51.000 --> 02:55.000
And this is an abstraction of the RTC peer connection project

02:55.000 --> 02:58.000
that is defining in W3C spec.

02:58.000 --> 03:02.000
And it has signals and properties that you can use.

03:02.000 --> 03:04.000
It's quite convenient.

03:04.000 --> 03:08.000
It's almost the one-to-one mapping with the WebRTC spec, kind of.

03:09.000 --> 03:13.000
And then GST WebRTC, which is a library,

03:13.000 --> 03:16.000
which defines various subjects that are used in the plugin,

03:16.000 --> 03:19.000
like a transverse, we see, where I stand there.

03:19.000 --> 03:23.000
And then you have another thing called the Ice Backend,

03:23.000 --> 03:24.000
which is used for transport,

03:24.000 --> 03:28.000
also used by WebRTC bin for the ice candidates,

03:28.000 --> 03:30.000
some link and all the things.

03:30.000 --> 03:33.000
And we have default implementation,

03:33.000 --> 03:36.000
based on live-nice currently in GSTrum.

03:37.000 --> 03:41.000
So for capturing, we have quite a few pipelines,

03:41.000 --> 03:42.000
and that's one of them.

03:42.000 --> 03:45.000
So we have to capture the camera,

03:45.000 --> 03:47.000
all the screen casting.

03:47.000 --> 03:49.000
We have a pipeline there,

03:49.000 --> 03:55.000
that is using the source elements from digital.

03:55.000 --> 03:57.000
Like if the Web Process is not sandbox,

03:57.000 --> 03:59.000
we can use GST device monitor.

03:59.000 --> 04:02.000
Otherwise, we have to use the camera portal.

04:02.000 --> 04:04.000
So if you use WebQGTK,

04:04.000 --> 04:07.000
which is a new, we have the camera portal as support now

04:07.000 --> 04:09.000
in 162825.

04:09.000 --> 04:10.000
And then for screen casting,

04:10.000 --> 04:12.000
we don't really have a choice.

04:12.000 --> 04:14.000
We have to use the pipeline,

04:14.000 --> 04:16.000
I use through the desktop portal,

04:16.000 --> 04:19.000
like all the other browsers are doing.

04:19.000 --> 04:21.000
And we use the pipeline,

04:21.000 --> 04:24.000
we are explaining for that in the capture pipeline.

04:24.000 --> 04:30.000
Now, media stream is quite important.

04:30.000 --> 04:34.000
So that's how you interact with,

04:34.000 --> 04:37.000
how we bridged media stream,

04:37.000 --> 04:39.000
with the streamer using a source element,

04:39.000 --> 04:41.000
called media stream source.

04:41.000 --> 04:45.000
And it's used in many components in WebKit.

04:45.000 --> 04:48.000
So for capture, we both use Canvas.

04:48.000 --> 04:51.000
Anything you need to interact with the media stream,

04:51.000 --> 04:54.000
you have to use that element for.

04:54.000 --> 04:59.000
And it's also used for incoming and outgoing media stream handling.

04:59.000 --> 05:02.000
And if you're familiar with WebWatch's spec,

05:02.000 --> 05:06.000
it's kind of mapping with just the pads there

05:06.000 --> 05:08.000
with the media stream track.

05:08.000 --> 05:14.000
And then I have five view of,

05:14.000 --> 05:16.000
because the pipeline is really huge.

05:16.000 --> 05:18.000
So I simplify things quite a lot.

05:18.000 --> 05:21.000
So here you have an example where data flows from the left to right.

05:21.000 --> 05:25.000
You have a review and video source element on the left.

05:25.000 --> 05:27.000
And then I simulated, like,

05:28.000 --> 05:32.000
where the video is included twice at different resolution,

05:32.000 --> 05:33.000
for instance.

05:33.000 --> 05:39.000
And everything goes to the WebWatchCB element.

05:39.000 --> 05:41.000
And then on the incoming side,

05:41.000 --> 05:44.000
we have one track.

06:57.000 --> 07:02.000
And then on the next,

07:02.000 --> 07:05.000
we have one track.

07:05.000 --> 07:07.000
And then on the left.

07:07.000 --> 07:09.000
We have one track.

07:09.000 --> 07:12.000
And then on the right.

07:12.000 --> 07:14.000
And then on the left.

07:14.000 --> 07:17.000
And then we have one track.

07:17.000 --> 07:20.000
And then on the right.

07:20.000 --> 07:24.000
So you have to leave,

07:24.000 --> 07:26.000
You

07:54.000 --> 07:56.000
You

08:24.000 --> 08:26.000
You

08:54.000 --> 08:56.000
You

09:24.000 --> 09:26.000
You

09:54.000 --> 09:56.000
You

10:24.000 --> 10:26.000
You

10:26.000 --> 10:28.000
You

10:54.000 --> 10:58.000
You

11:24.000 --> 11:26.000
You

11:26.000 --> 11:28.000
You

11:28.000 --> 11:30.000
You

11:30.000 --> 11:32.000
You

11:32.000 --> 11:34.000
You

11:34.000 --> 11:36.000
You

11:36.000 --> 11:40.000
You

11:40.000 --> 11:44.000
You

11:44.000 --> 11:50.000
You want to know more about this term and all the good stuff in the audio video room.

11:50.000 --> 11:52.000
Thank you.

11:58.000 --> 12:00.000
Any question?

12:00.000 --> 12:12.000
Do you know about working and re-opensors available in limitations of the web

12:12.000 --> 12:14.000
RTC for screen sharing?

12:14.000 --> 12:18.000
There was one from the photo-resting file.

12:18.000 --> 12:26.000
The question is about screen sharing on the implementation open source

12:26.000 --> 12:32.000
So if you are on a Linux desktop, you can do that already.

12:32.000 --> 12:38.000
If you work it, you can use the display media and that will use pipe wire.

12:38.000 --> 12:40.000
But it really depends on the shell you use.

12:40.000 --> 12:44.000
If you use a dome, it should work also on KD.

12:44.000 --> 12:48.000
But if you are like an uncommon window manager,

12:48.000 --> 12:54.000
you need to have all the plumbing done at the window manager level to have that working.

12:54.000 --> 12:56.000
You need to check it.

12:56.000 --> 12:58.000
It depends.

13:06.000 --> 13:08.000
Thank you.