RICOH THETA SC2 livePreview Multiple Frame Extraction

craig · September 16, 2021, 8:54pm

This demonstration shows how to test lmultiple livePreview frames from the SC2 at different intervals.

In the example above, I am extracting one frame every 3000 milliseconds.

Each frame is viewable as a 360 image. The first frame is shown below.

sc2_screenshot

The livePreview from the SC2 is coming at 30fps. I’m saving a frame every 90 frames. If I save 5 frames, the camera will have streamed approximately 450 frames. I’m using videos played on my development workstation as a way to test motion in the video stream frames that I capture.

In this next test, I captured frames every 2 seconds. I’m saving 1 frame for every 60 frames the camera sends. I saved 20 frames out of the 1,200 frames received.

Frame Data Size

Each frame is 70KB in size.

70KB per frame at 30 frames per second requires 2.1MB/sec over the 2.4GHz wi-fi.

1 minute of motionJPEG @ 30fps will require 126MB. 10 minutes will require 1.26GB.

Code for Tests

I’m using a Dart stopwatch to set the delays between saving frames.

import 'dart:async';
import 'dart:convert';
import 'dart:io';

import 'package:http/http.dart' as http;

/// number of frames to save for the test
const int framesToSave = 20;

/// 500 = 2fps
/// 0 = 30fps
/// 125 = 8fps
/// 1000 = 1fps
/// 2000 = 1 picture every 2 seconds
const int millisecondDelay = 2000;

/// only works on SC2.  Does not work with Z1 or V
/// discards first frame of test.  if the delay is 3000ms, the
/// test will start after 3 seconds
void main() async {
  bool skipFrame = true;

  http.Client client = http.Client();
  Uri url = Uri.parse('http://192.168.1.1/osc/commands/execute');

  var request = http.Request('POST', url);
  Map<String, String> bodyMap = {"name": "camera.getLivePreview"};

  request.body = jsonEncode(bodyMap);
  Map<String, String> headers = {
    'Content-Type': 'application/json; charset=UTF-8'
  };

  client.head(url, headers: headers);
  http.StreamedResponse response = await client.send(request);

  var startIndex = -1;
  var endIndex = -1;
  List<int> buf = [];

  List<File> listOfFiles = [];
  int frameCount = 0;

  Stopwatch stopwatch = Stopwatch();
  stopwatch.start();
  print('started stopwatch');

  for (var i = 0; i < framesToSave; i++) {
    listOfFiles.add(File('sc2_frame_$i.jpg'));
  }
  bool cancelledSubscription = false;

  StreamSubscription? videoStream;

  videoStream = response.stream.listen((List<int> data) async {
    buf.addAll(data);
    for (var i = 0; i < data.length - 1; i++) {
      if (data[i] == 0xFF && data[i + 1] == 0xd8) {
        startIndex = i;
        // print('found frame start');
      }
      if (data[i] == 0xff && data[i + 1] == 0xd9) {
        endIndex = buf.length;
        // print('found frame end');
      }
    }

    // save frame
    if (!cancelledSubscription) {
      if (startIndex != -1 && endIndex != -1) {
        try {
          if (frameCount < framesToSave + 1) {
            if (stopwatch.elapsedMilliseconds > millisecondDelay) {
              skipFrame = false;
              print(
                  'elapsed time (milliseconds): ${stopwatch.elapsedMilliseconds}');
              stopwatch.stop();
              stopwatch.reset();

              print('reset stopwatch');
              stopwatch.start();
            }

            if (!skipFrame) {
              if (frameCount != 0) {
                listOfFiles[frameCount - 1]
                    .writeAsBytes(buf.sublist(startIndex, endIndex));
                print('wrote frame $frameCount');
              }
              skipFrame = true;

              frameCount++;
            }
          } else {
            if (videoStream != null) {
              await videoStream.cancel();
              cancelledSubscription = true;
              client.close();
            }
          }
        } catch (error) {
          print(error);
        }
        startIndex = -1;
        endIndex = -1;
        buf = [];
      }
    }
  });
}

Next Steps with Tests

I’m having problems saving many frames if I set the delay to 10,000 milliseconds or 10 seconds. I also want to test the camera for continuous streaming for 20 minutes instead of a 40 second test.

Log of Additional Tests with Dart http package

September 17, 2021

Test #	delay	frames	result	note
1	5000ms	20	failed at frame 8	3 feet from computer
2	5000ms	20	failed at frame 9	camera standing on computer
3	4000ms	20	failed at frame 12
4	3000ms	20	failed at frame 17

The connection is stopping after roughly 100MB of data transfer.

The Z1 worked (with different code) for 9,000 frames, 5.6GB.

switched to dio

Transferred 6000 frames with no problem. I saved a frame every 30 frames. This is a big improvement for the SC2.