RICOH THETA SC2 livePreview MotionJPEG Single Frame Extraction

craig · September 14, 2021, 6:41pm

This is the fourth article on motionJPEG, livePreview with RICOH THETA cameras.

Similarities Between SC2 and V/Z1

The data format appears to be the same. I’m using the same logic and algorithm for all three camera models.

differences in request/response behavior between SC2 and V/Z1

The main difference is that for the SC2, I am using the http Dart package to get the stream. For the V/Z1, I am using HttpClient, which is part of dart:io. The problems I encountered are likely not going to impact people using Kotlin or Swift.

I’m using Dart and there may be a problem with the Dart implementation of the http client. However, the technology is stable enough for testing. I do get errors on my debug console, but the JPEG frames look to be intact.

Code to Extract Single Frame with SC2

import 'dart:async';
import 'dart:convert';
import 'dart:io';

import 'package:http/http.dart' as http;

/// only works on SC2.  Does not work with Z1 or V
void main() async {
  http.Client client = http.Client();
  Uri url = Uri.parse('http://192.168.1.1/osc/commands/execute');

  var request = http.Request('POST', url);
  Map<String, String> bodyMap = {"name": "camera.getLivePreview"};

  request.body = jsonEncode(bodyMap);
  Map<String, String> headers = {
    'Content-Type': 'application/json; charset=UTF-8'
  };

  client.head(url, headers: headers);
  http.StreamedResponse response = await client.send(request);

  var startIndex = -1;
  var endIndex = -1;
  List<int> buf = [];
  File fileHandle = File('sc2_frame.jpg');
  bool cancelledSubscription = false;

  StreamSubscription? videoStream;

  videoStream = response.stream.listen((List<int> data) async {
    buf.addAll(data);
    for (var i = 0; i < data.length - 1; i++) {
      if (data[i] == 0xFF && data[i + 1] == 0xd8) {
        startIndex = i;
        print('found frame start');
      }
      if (data[i] == 0xff && data[i + 1] == 0xd9) {
        endIndex = buf.length;
        print('found frame end');
      }
    }

    // save frame
    if (!cancelledSubscription) {
      if (startIndex != -1 && endIndex != -1) {
        try {
          await fileHandle.writeAsBytes(buf.sublist(startIndex, endIndex));
          if (videoStream != null) {
            await videoStream.cancel();
            cancelledSubscription = true;
            client.close();
          }
        } catch (error) {
          print(error);
        }
      }
    }
  });
}

The problem I encounter is how to properly close the HTTP stream after the subscription is cancelled.

Running the test Code

Save the code snippet above to a file. In this example, my file is called, sc2_2_client_post.dart.

dart .\sc2_2_client_post.dart
found frame start
found frame end
found frame start
Unhandled exception:
Connection closed before full header was received
...

There are more errors, but I’m going forward with the tests for now.

The code snippet saves the first frame as sc2_frame.jpg.

The frame is 64KB in size with dimensions of 1024x512.

Attempts to change the stream to 640x320 failed at both 30fps and 8fps. I was also not able to change the stream to 1024x512 at 8fps.

The only resolution I could get working was 1024x512 @ 30fps.

Summary

The SC2 uses a standard MotionJPEG data format. The same code for the V/Z1 can be used for the SC2. People using Dart may need to try different HTTP libraries. I’m using HttpClient for V/Z1 and http for the SC2. Although I’m getting a client IO error, the test program appears to be usable as the http.Client does appear to be closing and freeing up resources on the camera.