You’ve read plenty about HTML5 <audio> and HTML5 <video>. But there’s a third media option to explore: HTML <canvas>. Molly Holzschlag explains what it does, and what developers and designers should watch out for.
While much ado has been made about HTML5 media, most of that has had to do with audio and video. Whether it’s been issues related to compression patents or managing backward compatibility in an ever-forward moving Web, the greater conversation has been around HTML5 <audio> and <video>.
Along with these two mission-critical advancements has come a third media option, the <canvas> element and its associated HTML5 API. While <canvas> has proved to be very popular in certain circles – particularly in gaming (Figure 1) or other highly visual interactive applications – it also poses unusual challenges with which we must work.
Figure 1. Biolab Disaster by Dominic Szablewski: A canvas game written to demonstrate the speed and complexity available via HTML5.
Rendering Modes
To gain a better understanding about how <canvas> works, technically speaking, it’s good to grasp the difference between vector and bitmap graphics. Many readers are familiar with the concepts, but for those who might not be, here’s a short, basic definition of each so as to better gain insight into what <canvas> does – and doesn’t – do.
- Vector graphics: The vector format relies on mathematical algorithms, allowing for re-sizable files with no visible image loss. Common vector technologies on the Web include Flash and Scalable Vector Graphics (SVG).
- Bitmap: Bitmaps are exactly what you’d expect them to be – a pixel-based format. They are typically found in conventional Web graphics such as GIFs, JPEGs, and PNGs.
In the past, these formats helped us create an interesting visual Web, but with the exception of Flash, none have a means for users to interact directly with the format in complex ways.
Retained Mode: Conventional Web Media
Within the graphic rendering of browsers, the list of objects used in your media design is retained in the renderer, and those objects are displayed on the screen in according to the conventional media’s means of managing such issues as x/y positions and the object’s transparency.
Retained mode has worked well with vector-based technologies such as Flash and SVG, but <canvas> can offer more control to the developer in certain circumstances.
Immediate Mode: <canvas> Based Media
Working with <canvas> uses a different rendering mode, referred to as Immediate Mode. As soon as a developer places a <canvas> element on a given page, in supporting browsers that becomes a bitmapped drawing, immediate rendering area:
<canvas id=”coolgame” width=”500” height=”500”>
<!-- some developers leave a message here for non-supporting browsers -->
</canvas>
There are some additional attributes for the <canvas> element, but most of the drawing power for <canvas> is found within the API and the use of JavaScript.
The <canvas> API
The <canvas> Application Programming Interface provides a means of drawing in 2D. Designers and developers use this to create numerous effects including:
- Shapes
- Colors
- Rich Animations
- Rotations
- Lines and curves
Because the API can be modified with JavaScript to include user inputs via mouse, keyboard, and other devices such as joysticks, this is how HTML5, the <canvas> element and API, and JavaScript work together to create rich, interactive demos, games, and apps (Figure 2).
Figure 2.0. Magnetic by Hakim El Hattab. In this <canvas> application, particles orbit around magnetic points. Dragging around the magnets creates different effects, and you can add and remove magnets as you wish.
But wait! You might be thinking, “We have other means to do that without having to use so much (or any) scripting.” That’s true. The power and the problems with <canvas> emerge from the fact that the entire experience is script generated, including how users interact with <canvas>-based applications.
JavaScript and <canvas>
A key issue is that <canvas> relies on JavaScript to render the pixels of each given application phase. The immediate mode rendering allows for the screen display, and developers must anticipate each display frame of the application so as to render it appropriately.
Because of the immediacy of <canvas> and its very rich programmatic options via JavaScript, it’s become a choice among many game and apps developers. As time goes on, popular open-source libraries such as jQuery are adding modules that allow a wider range of designers and developers to work with <canvas>. There are also numerous folks working on <canvas> image libraries (Figure 3).
Figure 3.0. Rendering 2d geometric objects using <canvas> in Oswald Campesato’s open-source library. Download and view source, and you’ll see that except for a basic HTML5 document with the <canvas> element intact, the rest is all JavaScript.
However, despite its power, <canvas> has a few fundamental problems. To provide a deeper understanding of the various tools available, a look at the challenges with <canvas> and its unique approach is invaluable.
Challenges Using <canvas>
Because <canvas> relies on a distinctive mode within the browser to work, and also requires very clever developers to make <canvas> drawings and animations more accessible, it’s important to review the inherent challenges found in <canvas> technology.
The Document Object Model (DOM)
Most readers are familiar with the DOM, which historically was the only API available in browsers prior to HTML5. The DOM essentially is the tree structure interface created by the elements in use. JavaScript has been the primary scripting choice to manipulate elements within the DOM.
In the case of <canvas> the model changes once the browser is in the 2D context created by the way canvas rendering occurs. The end result is that the actual element <canvas> (which appears in the DOM) can be manipulated with script, but any drawn objects on the individual frames within the immediate bitmap mode are not reachable via JavaScript.
Not only can we not grab anything that’s within the immediate mode and manipulate it with script, but we have no way to easily reuse any data within a <canvas> application. Essentially, everything in that immediate mode is encapsulated. The main difference between the type of encapsulation we see in retained modes and what <canvas> provides is that it is native to the browser. As with other aspects of HTML5 in general and media in specific, the elimination of third-party plug-ins is a specific goal of HTML5 itself.
Accessibility
Another problem related to <canvas> rendering and encapsulation is that it becomes extremely difficult to make software accessible to assistive technologies such as screen readers. That is because “there’s no there ‘there’” in the sense that the content is closed, with no way to clearly mark objects and actions, even using Web Accessibility Initiative’s Accessibility for Rich Internet Applications (WAI-ARIA).
The good news is that a working group at W3C is committed to examining and providing solutions to <canvas> accessibility. Suggestions have ranged anywhere from using image maps as fallbacks (as they are open and support alternative text) to using audio description for each interaction.
While no broad best practices have emerged for making <canvas> easier for everyone to use, it’s generally a good idea to follow conventional accessibility guidelines such as ensuring keyboard access, not relying on color alone to signify an interaction, and providing clear instructions for user interactions within the prose.
Decisions, Decisions!
There are three major questions that developers and designers considering <canvas> for applications or games must ask before adoption:
- Is the API and associated rendering mode in <canvas> sufficient for the application’s needs?
- Is there any way to provide an alternative experience for users who require assistive technologies?
- Is there a better technology that lets me accomplish my end goals?
In most use cases, the likelihood that you will come back to another technology such as SVG for <canvas> is quite high. Because unlike <canvas>, SVG is XML-based and therefore is open data that can be reused, described, and categorized by Google and other engines. SVG is also vector-based, which allows for easy scaling across devices. Finally, SVG allows access to the DOM, so you have a lot of options to create shapes and even animations with a naturally more open technology.
Support for <canvas> is advancing rapidly in Web browser development. IE10 and current versions of Chrome, FireFox, and Opera all support the <canvas> element.
For IE9 or below, try this polyfil.
This should not, however, discourage you from working with and trying some <canvas> based games and apps to see how they are built, how they are rendered, and how accessible they are via keyboard and descriptors. Finally, <canvas> is fast, very responsive to user input, and excellent for complex animations, so it’s more than worth a try!
See also:
