diff --git a/v4l.html b/v4l.html new file mode 100644 index 0000000..7d3b76a --- /dev/null +++ b/v4l.html @@ -0,0 +1,194 @@ +
+

everything i learned about streaming video in linux

+

-nemo

+
+
+

demo

+
+
+

let's talk about X11

+
+-------------------------+
+|    input devices        |
+|  keyboard | mouse       |
++-------^---+-^-----------+
+        |     |
+        |     |                 +------------------------+
+      +-v-----v-------+-------->+                        |
+      |               |         |                        |
+      |   X Server    +-------->+                        |
+      |               |         |                        |
+      +--+---------+--+-------->+      Screen            |
+         ^         ^            |                        |
+         |         |            |                        |
+         |         |            +------------------------+
+         v         v
+ +-------+---+  +--+-------+
+ | browser   |  |          |
+ | X client  |  | xterm    |
+ |           |  |          |
+ +-----------+  +----------+
+
+
+
+

x11 - displays and screens

+
    +
  • +

    You "select" a display with the :$DISPLAY:SCREEN syntax.

    +
    • +
  • +

    So the first screen on your main display server would be :0.0

    +
    • +
  • +

    You can set the display an application launches with DISPLAY environment variable

    +
    • +
+

export DISPLAY=:0.1 leafpad

+
+
+

Xvfb

+
+

Xvfb or X virtual framebuffer is a display server implementing the X11 display server protocol. In contrast to other display servers, Xvfb performs all graphical operations in virtual memory without showing any screen output.

+
+
+

However, no output is shown. This virtual server does not require the computer it is running on to have any kind of graphics adapter, a screen or any input device.

+
+
+
+

xwd

+
+

xwd - dump an image of an X window

+
+
xwd -root image.xwd
+convert image.xwd image.png
+
+
+
+

ffmpeg

+

can "grab" X11 windows

+
ffmpeg -f x11grab -framerate 10 -i :1.0 \
+  -vaapi_device /dev/dri/renderD128 \
+  -vf 'format=nv12,hwupload,scale_vaapi=w=854:h=480' \
+  -c:v h264_vaapi -qp:v 26 -bf 0 -tune zerolatency -f mpegts \
+  udp://localhost:12345
+
+
+
+

video devices

+
    +
  • /dev/video*
    • +
+
v4l2-ctl  --device /dev/video2  --all
+Driver Info:
+  Driver name      : uvcvideo
+  Card type        : HD Pro Webcam C920
+  Bus info         : usb-0000:00:14.0-4
+  Driver version   : 5.7.9
+  Capabilities     : 0x84a00001
+    Video Capture
+    Metadata Capture
+    Streaming
+    Extended Pix Format
+    Device Capabilities
+  Device Caps      : 0x04a00000
+    Metadata Capture
+    Streaming
+    Extended Pix Format
+Priority: 2
+Format Metadata Capture:
+  Sample Format   : 'UVCH' (UVC Payload Header Metadata)
+  Buffer Size     : 1024
+
+
+
+

v4l2loopback

+
+

v4l2loopback - a kernel module to create V4L2 loopback devices

+
+
+

this module allows you to create "virtual video devices" normal (v4l2) applications will read these devices as if they were ordinary video devices, but the video will not be read from e.g. a capture card but instead it is generated by another application. this allows you for instance to apply some nifty video effects on your Skype video... it also allows some more serious things (e.g. I've been using it to add streaming capabilities to an application by the means of hooking GStreamer into the loopback devices).

+
+
+
+

create new webcams!

+
modprobe v4l2loopback devices=4
+
+

/dev/video3,4,5,6,7

+
+
+

ffmpeg can "write" to v4l

+
ffmpeg -f x11grab \
+       -framerate 4 \
+       -s 854x480 \
+       -i :1.0 \
+       -pix_fmt yuv420p \
+       -f v4l2 /dev/video3
+
+

ffmpeg -pix_fmts

+
+
+

aside: MJPEG over HTTP

+

ffmpeg -i ... -f mjpg -o file.jpg

+
Content-Type: multipart/x-mixed-replace; boundary="JPGBoundary"
+
+--JPGBoundary
+Content-Type: image/jpeg
+
+<jpeg bytes>
+--JPGBoundary
+Content-Type: image/jpeg
+
+<jpeg bytes>
+--JPGBoundary
+... and so on ...
+--JPGBoundary--
+
+
+
+

MJPEG streaming is simple

+
    +
  • mjpeg-server (uses ffmpeg -f mjpg)
    • +
  • ustreamer (only supports /dev/video*)
    • +
+
+
+
+---------------------+        +---------------------------+
+|                     |        |                           |
+|     Xvfb Virtual    +------> |      ffmpeg -f x11grab    |
++---------+-----------+        |     -o /de^/video3        |
+          ^                    +---------------------------+
+   +------+----------+                     |
+   |  X Server       +<---+      +---------+-----------+
+   +---+-------------+    |      |  /dev/video3        |
+       ^                  |      |                     |-???->USTREAMER->Viewers ???
+   +---+---+              |      +-------+-------------+
+   | baba  |              |              |
+   | is    |        +-----+---------+    |   +-----------+
+   | you   |        |               |    +-->+  Zoom     |
+   |       |        |  xdotool      |        |           |
+   +-------+        +---------------+        +-----------+
+
+
+
+

security

+

YOLO!

+

Pitch: RFCs We Love!

+
+
\ No newline at end of file diff --git a/v4l.md b/v4l.md new file mode 100644 index 0000000..dbfdd3b --- /dev/null +++ b/v4l.md @@ -0,0 +1,198 @@ +# everything i learned about streaming video in linux +\-nemo + +--- + +# demo + +--- + +# let's talk about X11 + +``` ++-------------------------+ +| input devices | +| keyboard | mouse | ++-------^---+-^-----------+ + | | + | | +------------------------+ + +-v-----v-------+-------->+ | + | | | | + | X Server +-------->+ | + | | | | + +--+---------+--+-------->+ Screen | + ^ ^ | | + | | | | + | | +------------------------+ + v v + +-------+---+ +--+-------+ + | browser | | | + | X client | | xterm | + | | | | + +-----------+ +----------+ +``` + +--- + +# x11 - displays and screens + +- You "select" a display with the `:$DISPLAY:SCREEN` syntax. + +- So the first screen on your main display server would be :0.0 + +- You can set the display an application launches with DISPLAY environment variable + +`export DISPLAY=:0.1 leafpad` + +--- + + +# Xvfb + +>Xvfb or X virtual framebuffer is a display server implementing the X11 display server protocol. In contrast to other display servers, Xvfb performs all graphical operations in virtual memory without showing any screen output. + +>However, no output is shown. This virtual server does not require the computer it is running on to have any kind of graphics adapter, a screen or any input device. + +--- + +# `xwd` + +> xwd - dump an image of an X window + +``` +xwd -root image.xwd +convert image.xwd image.png +``` + +--- + +# `ffmpeg` + +can "grab" X11 windows + +``` +ffmpeg -f x11grab -framerate 10 -i :1.0 \ + -vaapi_device /dev/dri/renderD128 \ + -vf 'format=nv12,hwupload,scale_vaapi=w=854:h=480' \ + -c:v h264_vaapi -qp:v 26 -bf 0 -tune zerolatency -f mpegts \ + udp://localhost:12345 +``` + +--- + +# video devices + +- `/dev/video*` + +``` +v4l2-ctl --device /dev/video2 --all +Driver Info: + Driver name : uvcvideo + Card type : HD Pro Webcam C920 + Bus info : usb-0000:00:14.0-4 + Driver version : 5.7.9 + Capabilities : 0x84a00001 + Video Capture + Metadata Capture + Streaming + Extended Pix Format + Device Capabilities + Device Caps : 0x04a00000 + Metadata Capture + Streaming + Extended Pix Format +Priority: 2 +Format Metadata Capture: + Sample Format : 'UVCH' (UVC Payload Header Metadata) + Buffer Size : 1024 +``` + +--- + +# v4l2loopback + +> v4l2loopback - a kernel module to create V4L2 loopback devices + +>this module allows you to create "virtual video devices" normal (v4l2) applications will read these devices as if they were ordinary video devices, but the video will not be read from e.g. a capture card but instead it is generated by another application. this allows you for instance to apply some nifty video effects on your Skype video... it also allows some more serious things (e.g. I've been using it to add streaming capabilities to an application by the means of hooking GStreamer into the loopback devices). + +--- + +# create new webcams! + +``` +modprobe v4l2loopback devices=4 +``` + +`/dev/video3,4,5,6,7` + +--- + +# ffmpeg can "write" to v4l + +``` +ffmpeg -f x11grab \ + -framerate 4 \ + -s 854x480 \ + -i :1.0 \ + -pix_fmt yuv420p \ + -f v4l2 /dev/video3 +``` + +`ffmpeg -pix_fmts` + +--- + +# aside: MJPEG over HTTP + +`ffmpeg -i ... -f mjpg -o file.jpg` + +``` +Content-Type: multipart/x-mixed-replace; boundary="JPGBoundary" + +--JPGBoundary +Content-Type: image/jpeg + + +--JPGBoundary +Content-Type: image/jpeg + + +--JPGBoundary +... and so on ... +--JPGBoundary-- +``` + +--- + +# MJPEG streaming is simple + +- `mjpeg-server` (uses `ffmpeg -f mjpg`) +- `ustreamer` (only supports `/dev/video*`) + +--- + +``` ++---------------------+ +---------------------------+ +| | | | +| Xvfb Virtual +------> | ffmpeg -f x11grab | ++---------+-----------+ | -o /de^/video3 | + ^ +---------------------------+ + +------+----------+ | + | X Server +<---+ +---------+-----------+ + +---+-------------+ | | /dev/video3 | + ^ | | |-???->USTREAMER->Viewers ??? + +---+---+ | +-------+-------------+ + | baba | | | + | is | +-----+---------+ | +-----------+ + | you | | | +-->+ Zoom | + | | | xdotool | | | + +-------+ +---------------+ +-----------+ +``` + +--- + +# security + +YOLO! + +Pitch: RFCs We Love! \ No newline at end of file