Javascript voice control

Since Chrome 25 we have had access to the new Web Speech API which allows us to create web apps that can utilse voice to text or voice control with a microphone. I have been wanting to experiment with this for quite a while so I built simple example to using voice commands to control an e-learning module made with my e-learning framework. I recorded a video below demonstrating navigation through voice commands. After the video I will show show you how easy it is to set up this basic control.

The following link provides a tutorial on implementing a speech to text example: http://updates.html5rocks.com/2013/01/Voice-Driven-Web-Apps-Introduction-to-the-Web-Speech-API

To use speech recognition for voice commands, this is how I implemented it:

1. Create a new speech recognition object

1
var recognition = new webkitSpeechRecognition();

2. Make the object continuously check the microphone

1
recognition.continuous = true;

3. Set the language to use. By default it will use the document’s language

1
recognition.lang = "en-AU";

4. Start the speech recognition

1
recognition.start();

5. Get results on the ‘onresult’ event

1
2
3
4
5
6
7
8
9
10
11
recognition.onresult = function (e) {
    // loop through the results
    for (var i = e.resultIndex; i < e.results.length; ++i) {
        // only get the final results
        if (e.results[i].isFinal) {
            // trim any whitespace from result and pass to our command handler
            // note: I am using jQuery here to trim the string because my e-learning demo already had jQuery included
            runCommand($.trim((e.results[i][0].transcript).toLowerCase()));
        }
    }
};

6. Set up a function to handle the commands

1
2
3
4
5
6
7
8
9
10
11
12
13
14
function runCommand(command){

    switch (command) {
        case "alert" :
            alert("Hello");
            break;
        case "prompt" :
            prompt("Enter some text");
            break;
        case "confirm" :
            confirm("Confirm?");
            break;
    }
}

Creating a mathematical colour picker

The most common technique for creating a colour picker (at least in the ActionScript world) seems to use a colour wheel bitmap and then getting the colour of the pixel that was clicked on. I found that this was not a very accurate way to go about it and decided to build a colour picker that gave colours based on calculations in a HSV cube – Hue on the Z axis, Saturation on X axis, and Value on the Y axis.

Of course a 3D cube colour picker would be pretty hard to use, so what we end up with is a vertical hue slider, then an area that determines the saturation and value like so:

picker

READ MORE

Reverse engineering Captivate for HTML5 widgets

Adobe Captivate is pretty horrible, especially the HTML5 output. I feel like I’m banging my head against a wall every time I use it. The HTML5 output is riddled with bugs and most of the features simply don’t work. If there is one upside to the HTML5 publishing though (and there aren’t many), it’s the ability to access almost anything in the Captivate movie due to the openness of JavaScript and work around its frustrations. I thought I would document a couple issues I faced which could potentially help someone else out. If you’re trying to figure out how to create a HTML5 widget then see my post on Creating HTML5 widgets for Adobe Captivate 6/7.

Problem 1: Create a custom slide index widget

This is pretty much impossible in the Flash publish. I needed a widget that would automatically get all the slide labels and generate an index. After a bit of digging thanks to Chrome’s dev tools, I discovered that most slide information can be found in the ‘toc’ global object on main Captivate window.

Below is a snippet of code that can be used in the HTML5 widget to get all slide names:

1
2
3
4
5
6
7
8
9
10
11
var mainWindow = window.parent; // get captivate main window, widgets are loaded into an iframe

// get list of slides
var slides = mainWindow.toc.mainMovie.stage.slides;

// get slide labels
var i = 0, lgt = slides.length;
for (i; i < lgt; ++i){
  // log all slide labels to the console, you could add them to your custom menu
  console.log( mainWindow.toc.movieProperties[slides[i]].lb );
}

Problem 2: Setting variables doesn’t work properly

Another bug I found in Captivate HTML5 widgets was sometimes setting a variable on the varHandle object doesn’t trigger a change in the movie, such as the cpCmndGotoSlide variable, which tells the movie to go to a particular slide. I found the following snippet to work more reliably:

1
2
// use the main movie's executeAction to set the variable
mainWindow.toc.mainMovie.executeAction("cpCmndGotoSlide = 2"]); // go to slide 3

I hope someone else will find this useful.

Webcam to canvas or data URI with HTML5 and Javascript

This post has been sitting around unfinished since April so I thought I should finish it and get it out there.

In this post I am going to show how you can capture an image from a webcam via JavaScript and convert it to a Data URI which can then be saved in a database or displayed in an IMG tag.

Note: This is more of a proof of concept rather than a best practices example so the code is a bit messy and hacked together from various sources.

I have only tested this on Safari for iOS 6 and latest Chrome with a webcam. It may or may no work in other browsers.

View demo

All explanations for the the code and markup are in the code comments, so please read them for more detail.

READ MORE

Creating HTML5 widgets for Adobe Captivate 6/7

I decided to write this post due to the frustration and lack of information out there on how to make HTML5 widgets for Adobe Captivate. There is almost no documentation on this and the only useful article I could find was riddled with mistakes. My approach may not be the best but this should get you started.

First things first, you will need to make two versions of your widget – a SWF version and a HTML5 version. Even if you don’t plan on using the SWF version, it is required during authortime in Captivate and is where you can provide the edit mode GUI.

These two versions are packaged up into a .wdgt file which is just a zip file with the extension changed to wdgt.

READ MORE

Windows 8 start screen in HTML, CSS and Javascript

About 7 or 8 months ago I built a tile based launch page at work to launch learning modules which looked similar to the Windows 8 start screen. When I had spare time I kept building on and adding 3D animations to match it even closer to Windows 8. Then I kind of forgot about it until this week when I stumbled upon this article Creating Windows-8-like 3D animations with CSS3 and jQuery which provides a tutorial on creating a similar effect.

I had been meaning to do a post on my laucher when I first started making it but it never happened. So now that that article reminded me I thought I should post it. I am not going to provide any explanations of the code or tutorials but rather just show a demo (It’s HTML and javascript so you can just view the source anyway). I have ripped a bunch of LMS related code out of it and turned it into a web page launcher.

Click a tile to launch an ‘app’ (in this case a website). Hover you mouse over the top or bottom right corner for the ‘charms’ bar to close the site and return to the start screen. I have currently only tested it in Chrome and on an iPad and there isn’t a way to get to the charms bar on iPad yet.

Click to launch demo

Getting all element attribute values with jQuery

Here is a little snippet which I have found extremely useful especially when working with XML in Javascript. You can use the following to loop through all of an element’s attributes and store them in an object:

1
2
3
4
5
6
var element = $("#myElement"),
    attributes = {};

$.each(element.get(0).attributes, function(i, attrib){
    attributes[attrib.name] = attrib.value;
});

So for example if we have the following XML node:

1
<item id="item1" name="the first item" image="thumbnail1.png" />

After running the snippet above on the node we can now access the values on the attributes object we created like so:

1
2
3
console.log( attributes.id ) // item1
console.log( attributes.name ) // the first item
console.log( attributes.image ) // thumbnail1.png

I have found this very useful when I don’t want to hard code XML parsing and I don’t need to know all the values right away.

I hope this is useful for someone else!

AS3 / JS Constrain a value in one line

Here is a little snippet to constrain a value in a single line. It works by using two nested inline conditionals. The ‘else’ part of the first conditional is another conditional. It may not be the best approach but it works and is much easier and faster to write than two if-else statements.

In English it looks like this:
value = is value greater than max ? set to max, otherwise is value less than min ? set to min, otherwise leave it as is;

1
2
3
4
5
// for js - remove the type declarations (:Number)
var minValue:Number = 0;
var maxValue:Number = 100;

value = value > maxValue ? maxValue : (value < minValue ? minValue : value);

JS Transform Handles

The other day I was playing around with Mozilla’s Popcorn Maker and I had the idea of using HTML/JS/CSS to create transformable divs using transform handles and a bounding box. I did a quick search for any existing examples of HTML transform handles but i couldn’t find anything so I decided to give it a go myself. Below is a demo of what I came up with. It might not be the best solution but it works pretty well. I even implemented rotation but I had some issues with the scaling when the element was rotated. The demo has the rotation disabled but you can get the source by view the source of the frame. All JS, and CSS is in the one HTML file. Enabling the rotation is as simple as removing display none from the rotation handles CSS style.

The main reason I wanted to create transformable DIVs is for the next version of my Module Builder. I had experimented with transforming HTML via the AIR app but I think doing it directly in the HTML page.

Adding console.log() to AIR’s HTMLLoader

Here is a little code snipped I thought I’d post for future reference and may help someone else out.

I was using the HTMLLoader class in Adobe AIR to load content and I came across an error where my Javascript was calling console.log(). In the AIR HTMLLoader environment this function does not exist so I had to come up with a way for it to stop throwing errors.

The solution I came up with was to create the console object and add the log function. That way I could trace out any log info or do anything else with it if I need to. So here is the snipped to add console.log to the HTMLLoader.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/* creating the loader object */
var ldr:HTMLLoader = new HTMLLoader();
ldr.load(new URLRequest('path/to/html/file.html'));
ldr.addEventListener(Event.COMPLETE, onLoaded);
addChild(ldr);

/* when load is complete, do the magic */
function onLoaded(e:Event):void
{
    /* Create console object on the window */
    ldr.window.console = {};
    /* add the log function to the console object, this function could do anything with the log data but here I am just tracing it */
    ldr.window.console.log = function(msg){trace(msg)};
   
}

Easy.