JS – Using the Camera with getUserMedia

I just found out you can now use the camera and the microphone with JS. This all hinges on: navigator.getUserMedia. At this time it only has partial support and you’ll need to use a browser prefix for all of the popular browsers. You can check support here.

Note! You can’t use navigator.getUserMedia from a URL that begins with file://. This means you can’t use this command when testing your project from the desktop! You will need to create a local server on your computer, or upload your files to your web host and test them from there.

If you’re looking for a local web host try MAMP (Mac), or WAMP (Windos). These are simple and easy to use.

Since navigator.getUserMedia() is not implemented by all browsers (yet), you’ll need to check for the vendor specific function: webkitGetUserMedia, mozGetUserMedia, msGetUserMedia. THe block below assigns the vendor method to navigator.getUserMedia, after which you’ll use that name.

if (!navigator.getUserMedia) {
    navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia ||
        navigator.mozGetUserMedia || navigator.msGetUserMedia;
}

Next choose a media source, and set success, and error callbacks.Notice the first parameter for getUseMedia is an object with properties: audio, and video. Set the media type you wish to access to true.

if (navigator.getUserMedia) {
    navigator.getUserMedia({
        audio: true,
        video: true
    }, success, error);
} else {
    alert('getUserMedia not supported in this browser.');
}

What happens with the callbacks is up to you. For the error, you might as well log a message.

function error() { 
    console.log("Couldn't get a stream."); 
} 

On a success, what to do is a little more involved. The success method will receive a stream. The data in the stream will depend on the device.

function success(stream) { 
    // Here you have connected to a device, and are receiving a stream.
} 

The success handler returns a MediaStream object which represents a stream of audio and video data. How you handle this depends on what you want to do and what time of stream, you requested: audio, video, or both.

The MediaStream Object can be assigned to a DOM element. Which makes for an easy way to display a video stream. Audio on the other hand doesn’t really sense if you attach it to a an audio tag. Most often you probably will not want to play audio as you are recording it.

Image Capture example…

HTML


<div class="camera">
 <video id="video-still">Video stream not available.</video>
 <button id="startbutton">Take photo</button>
</div>

<canvas id="canvas">
</canvas>

<div class="output">
 <img id="photo" alt="The screen capture will appear in this box.">
</div>

JavaScript

// ** Capture a still image **
// This block of code is contained in a self invoked function. 

(function() {
    // Set a width and height for the video/image

    var width = 320; // We will scale the photo width to this
    var height = 0; // This will be computed based on the input stream

    // |streaming| indicates whether or not we're currently streaming
    // video from the camera. Obviously, we start at false.

    var streaming = false;

    // The various HTML elements we need to configure or control. These
    // will be set by the startup() function.

    var video = null;
    var canvas = null;
    var photo = null;
    var startbutton = null;

    // ****************************************************************
    // Start capturing video
    function startup() {
        // Define elements 
        video = document.getElementById('video-still');
        canvas = document.getElementById('canvas');
        photo = document.getElementById('photo');
        startbutton = document.getElementById('startbutton');
        
        // Check for vendor version of getUserMedia
        
        navigator.getMedia = (navigator.getUserMedia ||
            navigator.webkitGetUserMedia ||
            navigator.mozGetUserMedia ||
            navigator.msGetUserMedia);
        
        // Check for getUserMedia
        if (!navigator.getMedia) {
            // No user media exit
            console.log("Has get user media");
            return; 
        }
        
        // invoke getUserMedia to start a video stream. 
        navigator.getMedia({
            video: true,    // Get video
            audio: false    // No audio
        }, getMediaSuccess, getMediaError); // Pass a success, and error function
        
        // Media success function
        function getMediaSuccess(stream) {
            // Check for FireFox. 
            if (navigator.mozGetUserMedia) {
                video.mozSrcObject = stream; // Assign the stream to #video-still
            } else {
                var vendorURL = window.URL || window.webkitURL;
                video.src = vendorURL.createObjectURL(stream); // Assign the stream to #video-still
            }
            video.play(); // Tell #video-still to play
        }

        // This is invoked on a user media error. 
        function getMediaError(error) {
            console.log("An error occured! " + error);
        }

        // Add an event to the #video-still. The canplay event occurs when 
        // the video is ready to play. 
        video.addEventListener('canplay', function (event) {
            // Check the streaming flag. 
            if (!streaming) {
                // Not streaming.
                // Get the height of the video
                height = video.videoHeight / (video.videoWidth / width);

                // Firefox currently has a bug where the height can't be read from
                // the video, so we will make assumptions if this happens.

                if (isNaN(height)) {
                    height = width / (4 / 3);
                }
                
                // Set some attributes of the #video-still element
                video.setAttribute('width', width);
                video.setAttribute('height', height);
                canvas.setAttribute('width', width);
                canvas.setAttribute('height', height);
                // Set streaming to true
                streaming = true;
            }
        }, false);

        // Add click event to #startbutton
        startbutton.addEventListener('click', function (event) {
            event.preventDefault();
            takepicture(); // Take a picture
        }, false);

        clearphoto();
    }
    // End startup function
    // ****************************************************************

    
    // #canvas is used to hold a still image. Here the convas is cleared 
    // by filling with gray. 
    function clearphoto() {
        // Fill with a gray
        var context = canvas.getContext('2d');
        context.fillStyle = "#AAA";
        context.fillRect(0, 0, canvas.width, canvas.height);
        // Set the data url of #photo to the image on the canvas
        var data = canvas.toDataURL('image/png');
        photo.setAttribute('src', data);
    }

    // Capture a photo by fetching the current contents of the video
    // and drawing it into a canvas, then converting that to a PNG
    // format data URL. By drawing it on an offscreen canvas and then
    // drawing that to the screen, we can change its size and/or apply
    // other changes before drawing it.
    
    // Capture an image. 

    function takepicture() {
        var context = canvas.getContext('2d');
        if (width && height) {
            canvas.width = width;
            canvas.height = height;
            context.drawImage(video, 0, 0, width, height);

            var data = canvas.toDataURL('image/png');
            photo.setAttribute('src', data);
        } else {
            clearphoto();
        }
    }

    // Set up our event listener to run the startup process
    // once loading is complete.
    // This calls startup (above)
    window.addEventListener('load', startup, false);
})();

 

Famo.us – Scrollview

Famo.us

I really like the look of Famo.us, I just haven’t had the time to play with it much. I had some time today, and thought I should wait no longer! Having already done the tutorials on the Famo.us University pages, I thought I’d try and tackle some more practical problems.

Scrollview

When making mobile applications, the scrollview is ubiquitous. This is a UI widget that shows up everywhere and has so many uses. Luckily Famo.us has a an example in their Starter Kit. Download it from their home page.

The scrollview example is located in reference-examples/views/scrollview.

Take a look at the example itself. The code looks pretty simple, if you’ve gone through the University Tutorials, you should recognize many of the features. Open the example in a browser and test it. Just a set of colored rows with a single line of text. The scrollview doesn’t scroll with a click and drag. It does scroll with a swipe on the track pad. This doesn’t look like it’s doing anything, or is it? Try it in the iOS simulator, or on your mobile device. Here it scrolls with a swipe as expected.

Screen Shot 2014-09-19 at 12.10.43 PM

Take a look at the files in the example folder. There are really two files:

  • index.html
  • src/main.js

Looking at index.html you will see that there is not a whole lot there. There are a few links to js files, that are commented as “Shims for backwards compatibility.” I’ll guess these are not necessary for modern browsers. Then there is a section commented as “famous.” I’ll take a guess that this is the main famous stuff. This section has two style sheets and two js scripts. The first script loads famous from the famous CDN. While the second script loads “src/main.js”, which is the example js.

Take a look at main.js. Overall this is pretty short. The code is wrapped in a define() block. The function passed to define must be called when the rest of the famous and the DOM is loaded. Here require() is used to load modules of the famous code base.

  • Engine
  • Surface
  • Scrollview

After the requires, a context is created, mainContext. The context is the container that renders surfaces.

Next is a new Scrollview object is created.

Then a an empty array, surfaces, is defined. The array is passed to the scrollview object via sequenceFrom().

A note on scrollviews. In general these are set up to display items from a list, or array. The scrollview only displays the rows that are currently visible. If you have a list containing thousands of items, I imagine your iTunes library has thousands of items, your mobile computer does not have enough memory to display all of these as one giant list. Instead the the computer displays only the rows that are currently visible on the screen. I’m guessing Famous is using the same strategy. Hence it needs a list of surfaces to display. The scrollview will mange displaying these surfaces.

Next there is a for loop that builds surfaces and adds them to the surfaces array. Each surface is created with newSurface(). The content of a surface will be the text: “Surface: #”, where # is the number of the surface. The width is undefined, I’ll guess this allows the surface to be any width, and the height is 200 px. Next the surface is assigned properties. The background color, line height, and text align. These are standard CSS properties.

Each new surface was created in a local variable named temp. Next we’re calling the pipe() method and passing a reference to the scrollview. I’m not sure what this doing, I’ll have to revisit the Famous University.

Next push each new surface into the surfaces array.

Last, add the scrollview to the mainContext with add().

What next?

The best way to understand how these things work, and to develop ideas as to how you can use them for your own projects is to experiment! The best way to test these things is to start small. Making small changes and testing to observe the results.

As a start lets work with the content. Line 52 shows:

content: "Surface: " + (i + 1),

This seems to be the content of each row in the scrollview. Let’s play with this and test that assumption. Change the content, try making these changes:

  • Change the text
  • Add HTML tags to format the text

Next change the size of the rows. As is each row 200 px tall, note that the content has a line height 200 px which places the text in the vertical center, and also makes the text take up the whole height of the row.

Change the line height to something more smaller. Add a second line of content to each row.

Try out these properties:

  • fontFamily, fontSize, color
  • border, borderTop

Customize row content

The tableview is not very useful if you can’t customize the content of each row. In the example, rows are generated within a loop, and the only real content is the row number.

Make an array to mock up some more relevant content. The demo uses a loop:

for (var i = 0, temp; i < 40; i++) {
    ...
}

Try replacing this with an array. Define an array and then loop through the array:

var array = ["A", "B", "C", "D"]; // add a lot more, do the whole alphabet
for (var i = 0; i < array.length; i++) {
    ...
}

With this you should see the same rows, but only as many rows as you have elements in the array. Now display the content from the array. Find where the content property is set for the surface. Try this:

...
content: "Surface: " + array[i] + " " + (i + 1),
...

At this point you should see something like: “Surface: A 0” for each row. You can use HTML and CSS in here also. Imagine this:

...
content: "<strong>Surface: " + array[i] + "</strong> " + (i + 1),
...

Add a class name. You’ll need to style this class in your style sheet.

...
content: "<strong><span class='title'>Surface:</span> <span class='content'>" + array[i] + "</span></strong> <span class='count'>" + (i + 1) + "</span>",
...

Watch the quotes carefully. Attribute values must always be in quotes. Since the outer quote is a double “, the inner quotes need to be single ‘.

Adjust the height of the cell.

...
size: [undefined, 100], // Change the height to 100
...

The example adds a few properties like:

 properties: {
     backgroundColor: "hsl(" + (i * 360 / 40) + ", 100%, 50%)",
     lineHeight: "200px",
     textAlign: "center"
 }

These these generate CSS properties that are attached to the row element as inline styles. Prove this to yourself. Use the inspector to view one of the scrollview cell rows. You should see something like:

<div class="famous-surface" style="line-height: 200px; text-align: center; opacity: 0.999999; -webkit-transform-origin: 0% 0%; -webkit-transform: matrix3d(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 200, 0, 1); width: 400px; height: 200px; background-color: rgb(255, 38, 0);">Surface: 2</div>

This shows each row is displayed from a div with a famous class, and some style properties. Some of the properties are applied by famous, while others were added by us when the surface was created.

I think it’s probably better to move our styles to a style sheet. At this point it’s hard to say if this is best practice for Famous. In any regular HTML page it would definitely be best practice.

Note that the color value is generated for each row, the value is based on the row number. It’s this system that gives us the great rainbow effect. For styles that need to be generated on the fly based on data only available when the surface element is created, setting the properties here is a good idea. For general properties, I’m guessing it will better to use a style sheet.

 

 

 

 

 

 

WordPress – jQuery

WordPress will load jQuery automatically without you having to add the files yourself. I’ve seen a different methods, some of which do not always work. Here’s the method I use that seems to work reliably.

The following would be added to your functions.php file. If you don’t have a functions.php yet, create one, in your theme folder.

 

<?php 
function theme_init() {
	if (!is_admin()) {
		wp_enqueue_script('jquery');
	}
}
add_action('init', 'theme_init');

 

Note! There can be no characters, this include spaces, or extra lines, in front of the tag. This is good practice. Extra characters outside of the php tag will cause the server to create HTTP header and send it to the browser before WordPress is ready.

If you already have a funtions.php file in your theme, you can add the snippet above, without the <?php.

WordPress loads jQuery in “No Conflict” mode. This means that jQuery will be accessed through the variable jQuery rather than the $. To work with this easily, set up your document ready actions like this:

jQuery(function($){ // Notice the $ here!
	$("#box>div>div").eq(0).siblings().hide();
	$("#box>ul>li.slide-thumb").click(function(event){
		var index = $(this).index();
		$("#box>div>div.slides").fadeOut(400).eq(index).fadeIn(400)
	});
});

Notice here, that jQuery (note the uppercase Q!) replaces the $ outisde the ready function. While inside the function I’ve used the $. This is possible by including the $ as a parameter to the function inside jQuery(function($){}). Placing this variable here allows jQuery to assign itself to this name, so inside of the function you can use that variable in place of jQuery.

Famo.us

Famo.us is a new JS library that is a very cool and inspiring. From my cursory investigation I’ll present my thoughts on Famo.us.

The goal of Famo.us is to create a graphical engine that renders elements to the browser more efficiently than current techniques. At the present time browsers are designed to render documents, rather than graphical elements, this presents a lot of bottle necks. Famo.us converts the typical HTML document type content into graphical elements, and renders them to the browser using canvas and webGL. This evolves the process of creating web apps from an experience of wrestling with HTML/CSS/JS to make something that mimics an app-like appearance. To, something more akin to making real apps.

There is something very different and distinct about using Famo.us. Rather than creating a web page with it’s DOM, and styles, and relying on the browser to position content based on a long list of arcane (and, should I add twisted) rules, is very different from freely placing things on the screen using x, y, and z coordinates. With Famo.us you are creating graphical elements that live in a 3 dimensional space on the screen. These elements can be freely translated and transformed.

The bigger promise of Famo.us is to create templates for apps that anyone can apply to their content. Much in the way that themes are applied to WordPress content. Realizing this goal would create an explosion of HTML5 apps, and make good on real the promise of HTML5, which at the current time has not lived up to it’s hype. All of these templates would be using the Famo.us system to position, and animate elements close to native rendering speed.

Here are a few basic concepts. To display things with Famo.us you need a Context and a renderable. A context is an area where one or more renderables can be drawn. A renderable maps to a display element in your page. A page can contain more than one context, or the context can take up the entire page.

The basic renderable is called a Surface. A Surface can contain HTML content, and is styled with CSS rules. You can set styles via JS when creating a new surface. Surfaces then become the basic view elements containing your content that are rendered to a Context. Imagine buttons, text boxes, and images all as Surfaces moving around a 3d environment in the browser window. Neat!

Famo.us looks pretty cool. I’m going to call it a second generation JS tool kit. Like AngularJS, it moves beyond the previous tools and sets new paradigms and increases the performance of web apps. If you were a Flash developer, and long for the days when you could do all of the things people were doing with Flash in the browser, I feel that this is the tool that will get you excited again. I can definitely see showcase web sites, like FWA, popping up to show Famo.us work.

All of that’s pretty rosy so I’ll end with a word of caution. Browser based apps are still hampered by downloads. Every file asset must be downloaded and cached. Ever been browsing a page when the style sheet did not load? Browser based apps built with Famo.us will still be prone to these issues. On desktops with a fast connection, the issues are minor. On mobile devices with slower connections these issues will be more apparent.