Telepresence Robot Uses RICOH THETA

Found some information about a Japanese company working on a telepresence robot that uses a RICOH THETA for its visual awareness.

Tweet from Sho Nakanose (@sho0315), the 29 year old engineer CEO founder of MacroSpace in Tokyo, from Dec 19, 2016.

Currently developing a tele-existence robot (a VR robot that can be operated remotely with your own movements by making full use of VR/robotics/communication technology) - Finished prototype No. 3. Have partially implemented haptics (tactile feedback) for using both arms at the same time, moving forward and back, left and right

It lists the technology used in building the robot as:

Language: HTML/CSS, Javascript, Java, C, C++, C#, PHP
Database: MySQL
Library: Three.js, A-Frame
Tools: Unity
Network: Photon, Serial, UDP
PaaS: AWS, Heroku, milkcocoa
Others: NodeJS, Socket.io, GitHub, Arduino

Original tweet in Japanese here.

And a newer tweet from today (Feb 15, 2017)

We developed Humanoid Telepresence Robot Unit No.4.
The human and the robot are synchronized via the internet.

Original tweet in Japanese and English here.

MacroSpace Founder’s blog (in Japanese) here. The blog’s subtitle says they are developing telepresence prosthetic limbs called “GITAI” and a specialized operating system called “GITAI OS.”

The Portfolio section of the blog includes many more videos.

2 Likes

Did you find out how he’s stitching the dual-fisheye live stream into equirectangular? Maybe ask him on Twitter if he’s using Unity?

It looks like the viewer inside the headset can display the image of the THETA as well as the virtualized hand of the user in real-time

Can you ask him if he wants to demo his system at our next meetup? He might be flying out to the SF Bay anyway. We could do the Japanese translation if he needs it.