Ricoh ThetaX USB Streaming Gyro Lag

Hello all,

We are intending to use the Ricoh Theta X for teleoperating our robot ( as well as full autonomous operations ).
We noticed that when we rotated our robot on the spot with the Ricoh Theta X attached to the top of it, the camera seems to “lag behind” the actual movement of the robot.

Here is a video link to what I am describing:

https://dconstructgroup-my.sharepoint.com/:v:/g/personal/jiayi_chong_dconstruct_ai/EbbxfdqXMVdPijk3DXEfidYBGHmP50b6M55Hu7X_71k-tg?e=wxYsud

Is there any option in the API to turn this off and remove the lag? On further investigation, we postulate it has something to do with the on board gyro compensating for the motion. We do not get this gyro lag when we use the Live Preview ( WiFi Motion JPEG streaming ). However, we will like to use the USB streaming instead since it gives us much better resolution. Hence, is there anyway for us to Turn OFF the gyro ( or at least prevent it from causing this lag ).

Thanks,
Jiayi

I didn’t think the gyroscope sensor would affect the live stream. If you change the orientation of the camera in livestream, does the stream automatically adjust? For example, if you hold the camera sideways, does the stream look upright?

If you hold a stopwatch up to the camera, is the latency around 350ms?

I don’t think there is a way to change how the gyroscope sensor is interacting with the USB cable live stream. As far as I know, the live stream does not use the gyroscope data, so I’m not sure how it would impact the stream.

Hello Craig,

Thank you for your reply.

Please see the video below for what I am describing:
https://dconstructgroup-my.sharepoint.com/:v:/g/personal/chowyi_lee_dconstruct_ai/ET52GutM_EVFtkwZcybERgYBz9T74Tfm_RJkv51rWRlTNw?e=1WAZbD

Notice the camera lags behind the actual orientation of where we are pointing. It then attempts to almost catch up/rotate into the final actual orientation after a short while. This is done using standard USB streaming. You should be able to easily replicate this issue with your OBS example on Windows.

We will like to get rid or turn off this lag for USB streaming.

Thank you

I can’t seem to replicate this. See my video below.

Hello @craig ,

Thank you for trying it out and posting the video. We are on a short break this weekend but will get back to you next week on this ( and probably try to post another diagnostic video if we can ).

To add more information while we are on our short break:

"Here are some results from our testing this week.
When we pick up the Ricoh X from a different orientation, ( as in turn ON the device while it is lying flat on a table, then flip it physically to a vertical pose ), the lag when rotated on the y-axis ( assuming y is UP ) is gone. But the lag is there if we rotate it along the x-axis ( assuming x is HORIZONTAL ).

It seems that on starting the Ricoh, the gyro captures the gravity axis and only rotation about the gravity axis is throttled."

This lag is not a latency issue ( the latency is fine ), it’s an issue with the Gyro ( or something similar ) making the virtual orientation of the camera lag behind and rotate in/catch up with the physical pose/orientation in which the Ricoh is pointing at. The actual frames ( latency ) are updated fine.

Cheers,
Jiayi

I’m not sure if the gyro data is being used in the stream.

The only thing I can think of is the stitching of the live stream. However, I haven’t noticed this problem.

Have a nice holiday!

Can you test a RICOH THETA Z1? Maybe buy it from a vendor with a return policy?

See my test in the video below. The X is behaving differently from the Z1 and V in my tests.

Hi @craig, I work with @jychong.

it seems like the theta X always has zenith correction on while the Z1 does not. is that by design? I tried playing with this setting theta-api-specs/top_bottom_correction.md at main · ricohapi/theta-api-specs · GitHub but I can’t seem to turn off zenith correction on the theta x

and, for what it’s worth, I didn’t notice gyro lag issue while testing with firmware 1.30, but I can’t verify if it’s a regression in 1.40 since I can’t downgrade the firmware

@craig Hello, thank you for your video, it served as a great visual diagnostic of the current problem. Is there any chance of us downgrading the firmware to 1.30 to see if this problem still exists?

Thanks

There is no way to downgrade the firmware.

@jcasman do you have an X with firmware 1.30? What about the unit you just got back yesterday?

I don’t think the zenith correct can be adjusted in the live stream.

I’ll check this later this morning and post here again, but I believe it is running 1.40.0.

@jcasman please watch the video and discuss with the people you know at RICOH.

Main point is that the behavior of the X model on rotation around the Y axis is different from the V and Z1 models.

Maybe there is a setting that can be adjusted?

OK, I’m able to bring this up with a RICOH contact today. We may be able to get some feedback.

if you have time, test the theta x on Mac/Windows and compare to Z1. rotate on the y axis of the camera.

The Y axis rotation I am referring to is the center diagram.

image

The actual quality of the stream looks better than the Z1 on the camera I’m using. There’s a better balance of the light between the two lenses.

THETA X stream with +1 EV compensation and auto white balance

THETA X stream with +.07 EV compensation and light bulb color fluorescent light.

light on each lens looks good.

fw 1.41

confirmed that behavior is the same with fw 1.41

1 Like

@jychong and @zhen_ling_tsai , Happy New Year.

Thank you for all the tests. The gyro lag you are referring to is from the TopBottomCorrection in the live stream of the THETA X. At the current time there is no way to disable the TopBottomCorrection of the THETA X during the live stream.

I do not have any information as to whether or not a feature to disable topBottomCorrection in the THETA X live stream will be added in the future.

To continue your product development, can you use the Z1?

Hello @craig ,

Happy new year and thank you for the reply. We tested out a Z1 at a local shop this week and confirmed that issue doesn’t show up on the Z1.

We will switch our development over to the Z1 but I hope that Ricoh will not add some firmware update to force this feature onto the Z1 moving forwards in the future. If this happens, the Z1 will also become unusable for our purposes.

Are my concerns unwarranted or is this something we need to worry about? Also it will be great if this TopBottomCorrection feature can be disabled for the X as well in an upcoming firmware update. Any robotcs/drone company using the USB streaming feature for their purposes will be impacted quite a bit.

Thanks

Unfortunately, I do not have any information on future firmware plans from RICOH.

I am sending community reports to people I know at RICOH and advocating for your use case.

@craig Thank you for your quick reply, much appreciated. The TopBottomCorrection issue is most certainly a deal breaker for us unfortunately. I really hope this issue can be addressed.

Cheers

@jychong , to move forward, you just need the Z1 to continue to behave the same way it is now, right? There’s no other reason to require the X, right?

Do you need some feature on the X that the Z1 doesn’t have? For example, you want the LCD panel on the X, or you want the removable battery? Or, maybe you want the 11K images, or 8K low-framerate video?

I don’t work for RICOH. However, RICOH does provide sponsorship of this forum and I have the opportunity to report back on community activities, use cases and product requirements.

replying in @jychong’s stead:

  • the LCD is convienient but not a necessity
  • the removeable battery only helps if continuous operations exceed 4 hrs, we don’t expect that to happen yet.
  • 11K res captures is not a use case currently, but we will not be surprised if our own customers request this in future.

We have yet to test extensively with the Z1 yet, outside of features exclusive to the X, we expect to have similar asks with regards to latency in livestreaming (ref:THETA X reducing latency for livestreaming - #20 by craig), and the ability to install specific firmware versions