This tutorial builds on the Node.js + Express, as well as the Getting Started tutorial on the three.js website. It also assumes that you have Node.js and Express installed already. Refer to the Node.js + Express for help on how to do that.
Test The Sample Code
- Clone the “Cardboard” repo from the Making-Things-Interactive github page with
git clone https://github.com/Making-Things-Interactive/Cardboard.git -
cdinto the directory -
Install all dependencies with
npm install -
Make sure your phone and computer are on the same wifi network (CMU Secure, CMU, or MediaLabLighting if you’re desperate)
-
Take note of your computer’s IP address (ifconfig or option + wifi icon on OSX)
-
Start the express server with
npm start -
Open a browser (Safari, preferably) on your phone and go to
<Computer-IP-Address>:3000- for example, if my computer’s IP address is
192.168.0.100, i would go to192.168.0.100:3000
- for example, if my computer’s IP address is
Walking Through the Code
VR Components
Just like in our intro to Three.js + Express, all of the VR-related javascript is held in public/javascripts/script.js
worldSphere
The worldsphere is essentially the background or your VR world. It is made by mapping an “equirectangular” image texture to the inside face of a sphere.
- We instantiate this object on line 13
var worldSphere - Then we setup the worldSphere starting on line 101:
|
1 2 3 4 5 6 7 8 9 10 11 |
worldSphere = new THREE.Mesh( // create new background sphere new THREE.SphereGeometry(450, 32, 32), // size it new THREE.MeshBasicMaterial({ // skin it map: THREE.ImageUtils.loadTexture('3D/textures/background.jpg'), side: THREE.DoubleSide }) ); worldSphere.rotation.set(0, Math.PI / 2, 0) // position it scene.add(worldSphere); // add it to the scene meshes.push(worldSphere); // add it to the list of meshes |
Previously, when creating an object, we created the geometry first, the material second, and then created an object called “cube” by combing those two. Here, we’re doing it all at once. we are basically saying mesh(geometry,material) However, both the geometry and material objects each take their own parameters.
THREE.SphereGeometry takes three parameters: radius, widthSegments, heightSegments. Checkout the documentation for an example.
THREE.MeshBasicMaterial takes one parameter, which is a JSON object. That JSON object in this case has two components:
– map which is the texture map
– side, which we set to DoubleSide, which means we can see both the back and front face of the mesh
Check out the documentation for MeshBasicMaterial
Now, let’s make some modifications:
- Comment out lines 122 and 124, this make it so we only create the worldSphere.
- Try changing the background image:
- Go to flickr, and search for “equirectangular” in the search bar up top.
- Under ‘Any License’, filter for just “All Creative Commons”
- Select an interesting image, and under the Download icon on its bottom right, pick the Large (2048 x 1024) resolution image and download it to
public/3D/textures - Change the path accordingly in
THREE.ImageUtils.loadTexture('3D/textures/xxx.jpg')
Loading Objects
On line 15, we introduce an array called selectableObjects. We’ll use this array to store objects in the scene that we can “select” with the VR cursor.
Starting on line 138, we have a function that loads all of our objects.
– To start, we instantiate a loader object from an external class called OBJMTLLoader()
– If you look in the project files, you’ll see that OBJMTLLoader.js is a separate file contained in the public/javascripts/threejs/ directory.
– Much of the functionality that you’ll see in the examples on three.js comes from extra class files like this. If you’ve downloaded the examples, you can often find these files in the examples/js folder.
|
1 2 3 4 5 |
function loadAssets() { var objMtlLoader = new THREE.OBJMTLLoader(); // create the OBJ loader |
- Next, we want to import some objects and corresponding materials. in the
public/3Ddirectory, you’ll find two.objfiles and two.mtlfiles- sublime, by default, does not show
.objfiles. you can change this in preferences
- sublime, by default, does not show
- with each object we import, we want to add it to our scene
scene.add(object), as well as our array of meshesmeshes.push(object). - We also want to give our object a filename
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
// UNSELECTABLE OBJs ========== var filenames = ["building"] // a list of all the filenames to load as unselectable OBJs for (var i = 0; i < filenames.length; i++) { objMtlLoader.load("3D/" + filenames[i] + ".obj", "3D/" + filenames[i] + ".mtl", function(object, url) { // load the OBJ and companion MTL scene.add(object); // add it to the scene meshes.push(object); // add it to the meshes list var filename = (url.split('.')[0]).split('/')[1] object.name = filename // add a property to the new object that is its filename }); } |
In the case of objects that we want to be “selectable”, we also add it to our selectableObjects array.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
// SELECTABLE OBJs ========== var filenames = ["stool"] // a list of all the filenames to load as selectable OBJs for (var i = 0; i < filenames.length; i++) { objMtlLoader.load("3D/" + filenames[i] + ".obj", "3D/" + filenames[i] + ".mtl", function(object, url) { // load the OBJ and companion MTL scene.add(object); // add it to the scene meshes.push(object); // add it to the meshes list selectableObjects.push(object); // add it to the selectable objects list var filename = (url.split('.')[0]).split('/')[1] object.name = filename // add a property to the new object that is its filename }); } } |
- Try to import your own
.objfile and make it selectable - You can find some free obj and mtl files from tf3dm.
- Download one, place the
.objand.mtlfiles intopublic/3D - Change the file name in code. (instead of “stool”, you would place the file name without extensions. avoid spaces in your name)
Sockets
To send information between the server and client in realtime, we will use Socket.IO. Our socket.IO system is mentioned in a few places in our system.
On the server side, we’ll be sending and receiving socket.io messages through the www file, found at bin/www.
– We’ve added socket.io down at the bottom of the file. Look at like 91 of www
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
var io = require('../app').io; io.on('connection', function(socket) { console.log('User connected'); socket.emit('message', 'Hello, user!'); socket.on('disconnect', function() { console.log("User disconnected"); }); }); |
Here, we instantiate an instance of socket.io var io = require('../app').io;. Now that we have the io object, we can use it to emit and receive messages. Here we will only be emitting messages.
In this example, when we receive a ‘connection’ message, we write a message to our sever console, and then send a message back to the client (browser) that says ‘Hello, user!’.
On the client side, we change two files; views/index.pug and our primary js file public/javascripts/script.js.
For Pug, have a look at index.pug. we’ve added some stuff under div#container
|
1 2 3 4 5 6 7 |
script(src="/socket.io/socket.io.js") script. var socket = io(); socket.on('message', function(data) { // log messages to the console console.log("from pug: " + data); }); |
Since we instantiate the socket instance this before we call script.js we can use the socket in any of the following js files, including our script.js
With our socket instantiated, we can start to use the data that we’re receiving from the server. At the bottom of script.js, starting on line 329, we have a callback function that exectutes anytime a message is received:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
socket.on('message', function(data) { // log messages to the console console.log(data); if(data == 'down'){ for (var i = 0; i < selectableObjects.length; i++){ selectableObjects[i].position.x+=15; } } if(data == 'hold'){ for (var i = 0; i < selectableObjects.length; i++){ selectableObjects[i].position.x-=15; } } }); |
In this example, any time we receive a ‘down’ or ‘hold’ message from the server, we change the position of all of the selectableObjects.
###Arduino
Since we are sending messages through the bin/www file, we will place all of our Arduino communication code here. For this example, we’ve chosen to use the ‘Johnny-Five’ library, which is a Node.js implementation of the familiar Firmata. In short, Johnny-Five allows us to read pins or write pins on the arduino without explicitely sending serial messages. This may not be the best option for your project, but is useful for a quick example.
Before running this, make sure you’ve uploaded StandardFirmataPlus to your arduino. You can do this in the Arduino IDE by going to File>Examples>Firmata>StandardFirmataPlus and uploading.
On our server code: First, we set up the arduino object:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
var five = require("johnny-five"), board, button; board = new five.Board(); board.on("ready", function() { // Create a new `button` hardware instance. // This example allows the button module to // create a completely default instance button = new five.Button(2); // Inject the `button` hardware into // the Repl instance's context; // allows direct command line access board.repl.inject({ button: button }); |
Now that we have a button object, we can send messages when certain events happen:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
// "down" the button is pressed button.on("down", function() { console.log("down"); io.emit('message', 'down'); }); // "hold" the button is pressed for specified time. // defaults to 500ms (1/2 second) // set button.on("hold", function() { console.log("hold"); io.emit('message', 'hold'); }); // "up" the button is released button.on("up", function() { console.log("up"); io.emit('message', 'up'); }); }); |
Here, we are sending a different socket.io message for up, down and hold events on the arduino. This section of code is pulled almost directly from the Johnny-Five documentation