Short talk on video-streaming in linux

This commit is contained in:
Nemo 2020-07-28 23:45:08 +05:30
parent 3c3ed1c654
commit c19fa916ca
2 changed files with 392 additions and 0 deletions

194
v4l.html Normal file

File diff suppressed because one or more lines are too long

198
v4l.md Normal file
View File

@ -0,0 +1,198 @@
# everything i learned about streaming video in linux
\-nemo
---
# demo
---
# let's talk about X11
```
+-------------------------+
| input devices |
| keyboard | mouse |
+-------^---+-^-----------+
| |
| | +------------------------+
+-v-----v-------+-------->+ |
| | | |
| X Server +-------->+ |
| | | |
+--+---------+--+-------->+ Screen |
^ ^ | |
| | | |
| | +------------------------+
v v
+-------+---+ +--+-------+
| browser | | |
| X client | | xterm |
| | | |
+-----------+ +----------+
```
---
# x11 - displays and screens
- You "select" a display with the `:$DISPLAY:SCREEN` syntax.
- So the first screen on your main display server would be :0.0
- You can set the display an application launches with DISPLAY environment variable
`export DISPLAY=:0.1 leafpad`
---
# Xvfb
>Xvfb or X virtual framebuffer is a display server implementing the X11 display server protocol. In contrast to other display servers, Xvfb performs all graphical operations in virtual memory without showing any screen output.
>However, no output is shown. This virtual server does not require the computer it is running on to have any kind of graphics adapter, a screen or any input device.
---
# `xwd`
> xwd - dump an image of an X window
```
xwd -root image.xwd
convert image.xwd image.png
```
---
# `ffmpeg`
can "grab" X11 windows
```
ffmpeg -f x11grab -framerate 10 -i :1.0 \
-vaapi_device /dev/dri/renderD128 \
-vf 'format=nv12,hwupload,scale_vaapi=w=854:h=480' \
-c:v h264_vaapi -qp:v 26 -bf 0 -tune zerolatency -f mpegts \
udp://localhost:12345
```
---
# video devices
- `/dev/video*`
```
v4l2-ctl --device /dev/video2 --all
Driver Info:
Driver name : uvcvideo
Card type : HD Pro Webcam C920
Bus info : usb-0000:00:14.0-4
Driver version : 5.7.9
Capabilities : 0x84a00001
Video Capture
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04a00000
Metadata Capture
Streaming
Extended Pix Format
Priority: 2
Format Metadata Capture:
Sample Format : 'UVCH' (UVC Payload Header Metadata)
Buffer Size : 1024
```
---
# v4l2loopback
> v4l2loopback - a kernel module to create V4L2 loopback devices
>this module allows you to create "virtual video devices" normal (v4l2) applications will read these devices as if they were ordinary video devices, but the video will not be read from e.g. a capture card but instead it is generated by another application. this allows you for instance to apply some nifty video effects on your Skype video... it also allows some more serious things (e.g. I've been using it to add streaming capabilities to an application by the means of hooking GStreamer into the loopback devices).
---
# create new webcams!
```
modprobe v4l2loopback devices=4
```
`/dev/video3,4,5,6,7`
---
# ffmpeg can "write" to v4l
```
ffmpeg -f x11grab \
-framerate 4 \
-s 854x480 \
-i :1.0 \
-pix_fmt yuv420p \
-f v4l2 /dev/video3
```
`ffmpeg -pix_fmts`
---
# aside: MJPEG over HTTP
`ffmpeg -i ... -f mjpg -o file.jpg`
```
Content-Type: multipart/x-mixed-replace; boundary="JPGBoundary"
--JPGBoundary
Content-Type: image/jpeg
<jpeg bytes>
--JPGBoundary
Content-Type: image/jpeg
<jpeg bytes>
--JPGBoundary
... and so on ...
--JPGBoundary--
```
---
# MJPEG streaming is simple
- `mjpeg-server` (uses `ffmpeg -f mjpg`)
- `ustreamer` (only supports `/dev/video*`)
---
```
+---------------------+ +---------------------------+
| | | |
| Xvfb Virtual +------> | ffmpeg -f x11grab |
+---------+-----------+ | -o /de^/video3 |
^ +---------------------------+
+------+----------+ |
| X Server +<---+ +---------+-----------+
+---+-------------+ | | /dev/video3 |
^ | | |-???->USTREAMER->Viewers ???
+---+---+ | +-------+-------------+
| baba | | |
| is | +-----+---------+ | +-----------+
| you | | | +-->+ Zoom |
| | | xdotool | | |
+-------+ +---------------+ +-----------+
```
---
# security
YOLO!
Pitch: RFCs We Love!