The LiveView is MotionJPEG, which is a set of JPEG images. It has limitations with framerate and resolution, but many people are using it. Search for Amelia drone on this forum or FOX SEWER ROVER. If you can’t find it, let me know.
Suggest you look at this cool Janus Gateway solution by the author of FOX SEWER ROVER.
Camera API for plug-in is here:
I don’t think you can stream in unstitched dual-fisheye, but I can’t fully remember.
Both the Amelia Drone and FOX SEWER ROVER authors researched low-latency solutions, so I think those are good starting points for your research.
I believe the Amelia Drone solution by @Jake_Kenin is using A-Frame for the headset navigation. A-Frame can handle the headset orientation and controller movements for different headsets.