Creating a WebUI with Node Running Inside the Camera

Technology demo for developers. These techniques are not usable for a production system right now.

Access image files directly from filesystem on SDCard.

I’ll show you how to create a WebGUI for your THETA that allows you to control the camera through web browsers, including mobile browsers. You can either connect to the THETA using access point (AP) mode or client mode (CL) mode.

In AP mode, you can also display images directly from the camera onto a web page.

WebGUI Overview

I previously explained how to create a WebGUI using Java and Android Studio. In the previous example, I used NanoHTTPd, a Java library. Although the technique worked, I had some problems putting external files such as images into the page.

In the example below, the fish is an SVG file.

By using Node and express, people can use the vast JavaScript library and install the libraries with npm. Additionally, many people either know node/express already or want to learn about it. Putting node on your THETA is a great way to learn about the camera functions and have fun with Node.

Limitations of This Project

This project doesn’t start the node server automatically when the Termux plug-in starts. It’s a technology demonstration that requires you to start the node application with node index.js from the command line. You can access the Termux Linux command line inside the THETA with ssh or Vysor.

Prerequisites

  1. Set Up Termux in your THETA
  2. Use Termux Basic Commands on THETA

Code on GitHub

Steps

  1. Connect THETA to your Router with Client Mode
  2. Connect laptop to same router
  3. ssh into THETA (assuming you have set up ssh public key) or use Termux from the command line
  4. clone code example from GitHub
    5 . run npm install
  5. node index.js
  6. point browser to http://ip-address:3000

In this example, I’ll use Termux through Vysor in case people don’t have their ssh key set up on the THETA.

Make a directory for Development.

image

Install git

image

Clone the repo.

image

Install Node
image

run npm install

image

Get IP address if you don’t already know it. I’m getting it with ifconfig

image

My IP address is 192.168.43.41

Run node index.js

image

The server should start on port 3000

Use a web browser to connect to the IP address and port.

Example: 192.168.43.41:3000


Note that IP address changed in the examples because I moved from the shared workspace, Hana House in Palo Alto.

this is the result of clicking on “Get Info”

I am using Chrome with JSON Viewer Awesome extension.

This is the output of the ssh session.

Currently, the images will only appear in access point mode.

The fix is to set up a static directory with express.


app.use('/100RICOH', express.static('/sdcard/DCIM/100RICOH'));

With this modification, my camera images are available from /100RICOH/filename.JPG

You can list all files in a directory with fs.readdir


const fs = require('fs');

const app = express();

const ricohImageDir = "/sdcard/DCIM/100RICOH";
app.use('/100RICOH', express.static(ricohImageDir));

app.post("/listFilesCL", (req, res) => {
    fs.readdir(ricohImageDir, (err, items) => {
	console.log(items);
	res.send(items);
    })

});

image

The full example code listing is here:

Instead of listing the text name of the file. You can list the actual file.

In this example, I’ve set up an ejs template with a for loop to list all the images in the 100RICOH directory. I’m using bootstrap cards.

  <%      imageNames.forEach((imageName) => {   %>
  <div class="col-md-3">
    <a href="/view/<%= imageName %>">
    <div class="card shadow-sm"> 
      <img class="card-img-top"  width="90%" src="/100RICOH/<%= imageName %>" >
      <div class="card-body">
	<p class="card-title"> <%= imageName %> </p> 
      </div>
    </div>
    </a>
  </div>
<%      });   %>

file listing on GitHub

Display 360 Image

I’m displaying the 360 image with A-Frame.

<script src="/static/js/aframe.min.js"></script>
<a-scene>
    <a-sky src="/100RICOH/<%= imageName %>" ></a-sky>
</a-scene>

With that bit of code, you can display the full 360 image with navigation.

in public/js, I have the JavaScript libraries for a-frame, bootstrap, jquery, and popper.

The screenshot below is from Emacs terminal running directly on the THETA.

image

Dropdown menu

I modified the bootstrap album demo to provide a drop-down menu for information about the project. You can modify this to provide information on your pictures.

Internal Camera Image Processing

I intend to evaluate the following for in-camera image processing:

The first test I did was to create thumbnails for all the images on my Z1.

$ mogrify -format png -path thumbs -thumbnail 200x100 /sdcard/DCIM/100RICOH/*.JPG

This appears to have worked:

$ cd thumbs/
$ ls -l
total 476
-rw------- 1 u0_a56 u0_a56 43437 Dec 15 07:20 R0010097.png
-rw------- 1 u0_a56 u0_a56 43257 Dec 15 07:20 R0010098.png
-rw------- 1 u0_a56 u0_a56 42009 Dec 15 07:20 R0010099.png
-rw------- 1 u0_a56 u0_a56 42264 Dec 15 07:20 R0010100.png
-rw------- 1 u0_a56 u0_a56 44237 Dec 15 07:20 R0010101.png
-rw------- 1 u0_a56 u0_a56 41567 Dec 15 07:20 R0010102.png
-rw------- 1 u0_a56 u0_a56 43509 Dec 15 07:20 R0010103.png
-rw------- 1 u0_a56 u0_a56 41528 Dec 15 07:20 R0010105.png
-rw------- 1 u0_a56 u0_a56 41345 Dec 15 07:20 R0010106.png
-rw------- 1 u0_a56 u0_a56 33350 Dec 15 07:20 R0010107.png
-rw------- 1 u0_a56 u0_a56 44424 Dec 15 07:20 R0010108.png

Using the Ubuntu file manager, I can connect to the camera with sftp (camera is at 192.168.2.101:8022). I can then use my mouse to navigate to the appropriate directory and double click on the thumbnails.

So far, things are going smoothly.

I installed graphicsmagick, which is new to me.

$ apt install graphicsmagick

I also installed the node module gm.

$ npm install gm
npm WARN theta-node@1.0.0 No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.2 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.2: wanted {"os":"darwin","arch":"any"} (current: {"os":"android","arch":"arm64"})

+ gm@1.23.1
added 6 packages from 6 contributors and audited 383 packages in 6.455s
found 0 vulnerabilities

Testing Node GraphicsMagick

I created a standalone node application to test the gm module.

const fs = require('fs');
const gm = require('gm');

gm('/sdcard/DCIM/100RICOH/R0010097.JPG').resize(200, 100).noProfile().write('./thumbnail.png', function (err) {
    if (!err) console.log('done');
});

It worked.

$ node gm-test.js 
done
$ ls -l
total 40
-rw------- 1 u0_a56 u0_a56   204 Dec 15 07:41 gm-test.js
-rw------- 1 u0_a56 u0_a56 33950 Dec 15 07:41 thumbnail.png

Adding test to node. Screenshot below of my Emacs session running directly on RICOH THETA Z1.

it worked.

Push changes up to GitHub directly from camera.

$ git add *
The following paths are ignored by one of your .gitignore files:
node_modules
Use -f if you really want to add them.
$ git commit -m "added graphicsmagick test"
[master 3944854] added graphicsmagick test
 7 files changed, 71 insertions(+), 3 deletions(-)
 create mode 100644 experiment/gm-test.js
 create mode 100644 experiment/thumbnail.png
 create mode 100644 public/thumbs/thumb.png
$ git push

Dev Tip

In addition to Emacs, you can also use tmux to show multiple windows of the THETA. In the example below, there are 5 windows open on the THETA OS. Two windows are using Emacs and three are using tmux.

Process All Images on Camera

First step is to use a forEach loop to process all images on the camera. At this stage, I’m not checking if the thumbnail already exists.

The loop works.

image

With the 4kb images, the page loads much faster.

Reduce Image Quality and File Size

This keeps the image dimensions the same, but performs lossy compression.


// reduce file size by reducing quality

app.post('/reduce-quality', (req, res) => {

    let imageArray = [];
    fs.readdir(ricohImageDir, (err, items) => {
	items.forEach ((item) => {
	    if (isImage(item)) {
		imageArray.push(item);
		gm('/sdcard/DCIM/100RICOH/' + item)
		.quality(30)
		.noProfile()
		.write(__dirname + '/public/gallery/' + item, function(err) {
		    if (!err) console.log('wrote reduced image file size ' + item);
		});	    
	    }
	});
    });

});

Lossy Compression

image

Original

image

30% Quality, 750kB size, 6720x3360 dimensions, 100% scale

100% quality, 8.8MB size, 6720x3360 pixels, 100% scale

Oil Painting Style

inside of viewer

Oil Painting Code

main command

gm(filename).paint(30)

The integer 30 specifies the radius around the pixel to use the same color.

working code snippet on GitHub

app.post('/paint', (req, res) => {
    fs.readdir(ricohImageDir, (err, items) => {
	const item = items[items.length -1];
	gm('/sdcard/DCIM/100RICOH/' + item)
	    .paint(30)
		.write(__dirname + '/media/paint/' + item, function(err) {
		    if (!err) console.log('wrote paint ' + item)
		    else console.log(err);
		});	    
    });
});

Drawing Shapes on Image

In 360 viewer

app.post('/drawshapes', (req, res) => {
    fs.readdir(ricohImageDir, (err, items) => {

	const item = items[items.length -1];
	gm('/sdcard/DCIM/100RICOH/' + item)
	// rounded rectangle x0, y0, x1, y1, wc, hc
	    .drawRectangle(500, 500, 1000, 1000, 100)
	// circle x0, y0, x1, y1
	    .drawEllipse(2000, 500, 300, 300)
	    .stroke("red", 20)
	    .drawLine(2500, 500, 4000, 900)
	    .fill("#888")
	
		.noProfile()
		.write(__dirname + '/media/watermark/' + item, function(err) {
		    if (!err) console.log('wrote watermark ' + item)
		    else console.log(err);
		});	    
    });
});

monochrome

gm('./z1-original.JPG').monochrome().write('./monochrome.JPG', function (err) {
    if (!err) console.log('done');
});

Dystopian view on Google Photos

Fingerpaint Swirls

Rotating pixels 180 from center point

360 view

// swirl parameter is the degrees around the center point that the image rotates
app.post('/fingerpaint', (req, res) => {
    fs.readdir(ricohImageDir, (err, items) => {
	const item = items[items.length -1];
	gm('/sdcard/DCIM/100RICOH/' + item)
	    .swirl(180)
		.write(__dirname + '/media/paint/' + item, function(err) {
		    if (!err) console.log('wrote fingerpaint ' + item)
		    else console.log(err);
		});	    
    });
});

Charcoal

Viewed with Google Photos

Negative

Edge

Problems with Tests

Watermark Text Overlay Problem

This works from the command line, but not within node. I’m getting an error with the sub-process.

The following code works on the command line, but not within Express.

gm('./z1-original.JPG')
    .fontSize(200)
    .drawText(1000, 800, 
        "theta360.guide        theta360.guide        theta360.guide" )
    .drawText(1000, 1600, 
        "theta360.guide        theta360.guide        theta360.guide" )
    .drawText(1000, 2400, 
        "theta360.guide        theta360.guide        theta360.guide" )    
    .write('./textOverlay.jpg', function (err) {
        if (!err) console.log('done');
    });

Views in FSPViewer (note: this only worked from command line, not from inside of node. Still trying to fix this)

Next

Auto-start Node Application When Termux Plug-in Starts

3 Likes