Low latency (0.4s) H.264 livestreaming from Theta V WiFi to html5 clients with RTSP plug-in, ffmpeg and Janus Gateway on raspberry Pi

, , ,
1

Hello,

After many tries on Theta V, I achieved to get a H.264 livestreaming on a Raspberry Pi for multiples html5 clients with a 0.4s latency.
It uses almost all realtime protocols available and powerfull softwares just to bypass USB livestreaming witch is not available on linux !

How it works:

  • Theta V with RTSP plug-in connected in client mode (should work in AP mode too, 5GHz WiFi for better bandwith). RTSP with Device WebAPI Plug-in also works.
  • ffmpeg (running on a Raspberry Pi) gets the RTSP stream and send it in rtp to Janus Gateway.
  • Janus Gateway (running on a Raspberry Pi) transforms rtp stream and distributes it to html5 clients (tested on both FireFox and Chrome) with webRTC.

How to do it:

theta-rpi: {
    type = "rtp"
    id = 1
    description = "Theta"
    audio = false
    video = true
    videoport = 8080
    videopt = 96
    videortpmap = "H264/90000"
    videofmtp = "profile-level-id=42e028;packetization-mode=1"
}

Debug level to 0 in janus.jcfg for better Cpu usage

  • Install fmpeg on the Pi (already installed with full raspbian) and run it:

ffmpeg -i rtsp://192.168.1.48:8554/live?resolution=1920x960 -an -c:v copy -flags global_header -bsf dump_extra -f rtp rtp://192.168.1.28:8080

192.168.1.48 is the Theta IP address,192.168.1.28 is the Raspberry Pi address
You can use vlc instead but it add latency and when i change the buffer size (1000ms by defauft) it is not stable.

vlc -vvv rtsp://192.168.1.48:8554/live?resolution=3840x1920 --network-caching=1000 --sout “#rtp{dst=192.168.1.28,port=8080}”

  • Install RTSP plug-in and Theta V, start Theta V, wait for Wifi connection, start the plug-in

Pros:

  • It works !
  • low latency => 330 ms
    My solution to test latency: screen capture with chronometer
    latency
    latency2068×893 425 KB
  • Multiple clients
  • low cpu usage (it works on a Pi Zero W), example in 1920x960 with 2 clients
    top
    top1016×164 9.77 KB
  • 4K available
  • Endless streaming if theta V is connected with external power (tested in fullHD)
  • Bonus : after the plug-in is started, you can connect theta in USB and download files from it at the same time

Cons:

  • Low bitrate, 30fps/2Mbps in 1920x960, 18 fps/3Mbps in 4K
    => more powerfull Pi needed for 4K
  • Equirectangular format (need three.js for better view)
    => use RTSP with Device WebAPI Plug-in for a standard view or VR view
  • Some freezing

Next:

  • Add audio (not needed for my goal)
  • Rebuild my three.js implementation (it consumes lot of cpu on the client side)
  • See if gophoto2 works (I need powerOff function only available in USB mode) and how to start the plug-in even if the theta is connected in USB (for now i need to disconnect Theta from USB and start the plug-in with the wifi API)
  • Use ffmpeg to get both streaming and record on the Pi (easy task)
  • Help needed for a new plug-in with all Theta functionnalities and better bitrate

All this work to see wonderfull images from space, NO just sewer pipes :slight_smile:

Hugues

4 Likes