4.7 KiB
everything i learned about streaming video in linux
-nemo
demo
let's talk about X11
+-------------------------+
| input devices |
| keyboard | mouse |
+-------^---+-^-----------+
| |
| | +------------------------+
+-v-----v-------+-------->+ |
| | | |
| X Server +-------->+ |
| | | |
+--+---------+--+-------->+ Screen |
^ ^ | |
| | | |
| | +------------------------+
v v
+-------+---+ +--+-------+
| browser | | |
| X client | | xterm |
| | | |
+-----------+ +----------+
x11 - displays and screens
-
You "select" a display with the
:$DISPLAY:SCREENsyntax. -
So the first screen on your main display server would be :0.0
-
You can set the display an application launches with DISPLAY environment variable
export DISPLAY=:0.1 leafpad
Xvfb
Xvfb or X virtual framebuffer is a display server implementing the X11 display server protocol. In contrast to other display servers, Xvfb performs all graphical operations in virtual memory without showing any screen output.
However, no output is shown. This virtual server does not require the computer it is running on to have any kind of graphics adapter, a screen or any input device.
xwd
xwd - dump an image of an X window
xwd -root image.xwd
convert image.xwd image.png
ffmpeg
can "grab" X11 windows
ffmpeg -f x11grab -framerate 10 -i :1.0 \
-vaapi_device /dev/dri/renderD128 \
-vf 'format=nv12,hwupload,scale_vaapi=w=854:h=480' \
-c:v h264_vaapi -qp:v 26 -bf 0 -tune zerolatency -f mpegts \
udp://localhost:12345
video devices
/dev/video*
v4l2-ctl --device /dev/video2 --all
Driver Info:
Driver name : uvcvideo
Card type : HD Pro Webcam C920
Bus info : usb-0000:00:14.0-4
Driver version : 5.7.9
Capabilities : 0x84a00001
Video Capture
Metadata Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04a00000
Metadata Capture
Streaming
Extended Pix Format
Priority: 2
Format Metadata Capture:
Sample Format : 'UVCH' (UVC Payload Header Metadata)
Buffer Size : 1024
v4l2loopback
v4l2loopback - a kernel module to create V4L2 loopback devices
this module allows you to create "virtual video devices" normal (v4l2) applications will read these devices as if they were ordinary video devices, but the video will not be read from e.g. a capture card but instead it is generated by another application. this allows you for instance to apply some nifty video effects on your Skype video... it also allows some more serious things (e.g. I've been using it to add streaming capabilities to an application by the means of hooking GStreamer into the loopback devices).
create new webcams!
modprobe v4l2loopback devices=4
/dev/video3,4,5,6,7
ffmpeg can "write" to v4l
ffmpeg -f x11grab \
-framerate 4 \
-s 854x480 \
-i :1.0 \
-pix_fmt yuv420p \
-f v4l2 /dev/video3
ffmpeg -pix_fmts
aside: MJPEG over HTTP
ffmpeg -i ... -f mjpg -o file.jpg
Content-Type: multipart/x-mixed-replace; boundary="JPGBoundary"
--JPGBoundary
Content-Type: image/jpeg
<jpeg bytes>
--JPGBoundary
Content-Type: image/jpeg
<jpeg bytes>
--JPGBoundary
... and so on ...
--JPGBoundary--
MJPEG streaming is simple
mjpeg-server(usesffmpeg -f mjpg)ustreamer(only supports/dev/video*)
+---------------------+ +---------------------------+
| | | |
| Xvfb Virtual +------> | ffmpeg -f x11grab |
+---------+-----------+ | -o /de^/video3 |
^ +---------------------------+
+------+----------+ |
| X Server +<---+ +---------+-----------+
+---+-------------+ | | /dev/video3 |
^ | | |-???->USTREAMER->Viewers ???
+---+---+ | +-------+-------------+
| baba | | |
| is | +-----+---------+ | +-----------+
| you | | | +-->+ Zoom |
| | | xdotool | | |
+-------+ +---------------+ +-----------+
security
YOLO!
Pitch: RFCs We Love!