Sounds form the background of our life. Today
the HTML5 <audio> element enables Web
developers to embed sounds in their applications. The flexibility of the
control coupled with the integration with the rest of the platform allows
several scenarios, from simple sound effects to background audio to gaming
experiences to more sophisticated audio engines.
This blog post walks through some of the best
practices for using the <audio> tag in your Web applications, and
includes useful tips from real-world sites.
Adding an Audio Element to Your Page
The very first step is to add the audio
element to your page. You can do this by declaring an <audio> tag in your
markup, by instantiating a new audio element in the JavaScript code, or by
embedding the audio stream in the page:
<audio src="audio/sample.mp3" autoplay>
</audio>
Run live demo
var audio = document.createElement("audio");
if (audio != null && audio.canPlayType && audio.canPlayType("audio/mpeg"))
{
audio.src = "audio/sample.mp3";
audio.play();
}
Run live demo
<audio src="data:audio/mpeg,ID3%02%00%00%00%00%..." autoplay>
</audio>
Run live demo
The first approach allows you to initialize
the audio components during the page load. The second approach gives you more
flexibility and better management of the network flow, as it defers the loading
of the audio clip to a specific time during the application lifecycle. The third
approach (less recommended) consists in embedding the audio files as data-uri in the page, reducing the number of requests to the server.
Note that you can play an audio element
generated by JavaScript even if it’s not been actually added to the DOM tree
(like in the code snippet above). However adding the audio element to the page
will allow you to display the default control bar.
Although not covered in this post, you can
support more than one audio file format. Also, if
you are hosting the audio files on your server, remember to register the MIME
type for mp3 files ("audio/mpeg") on the server side. Here, for instance, is
the setting on Internet Information Services (IIS).
Preloading the Audio Before Playing
Once you have your audio element, you can
choose the best preloading strategy. The HTML5 <audio> specification
describes a preload property with three possible values:
- "none": hints to the user agent that either
the author does not expect the user to need the media resource, or that
the server wants to minimize unnecessary traffic.
If your scenario is a podcast blog with an audio file for each post, this
option works particularly well, as it reduce the initial preload
bandwidth. Once the user plays the file (either through the default visual
controls or the JavaScript methods load() or play()), the browser
will start fetching the audio stream.
- "metadata": hints to the user agent that the
author does not expect the user to need the media resource, but that
fetching the resource metadata (dimensions, duration, etc.) is reasonable.
This option is recommended if you are building an audio player control and
you need basic information about the audio clip, but don’t need to play it
yet.
- "auto": hints to the user agent that the
user agent can put the user's needs first without risk to the server, up
to and including optimistically downloading the entire resource.
If you are building a game, this approach is probably the best one, as it
allows you to preload all the audio clips before actually starting the
game experience.
Note that when you set the src property
of the audio element programmatically, the browser will set the preload
property – unless otherwise specified – to "auto." For this reason, if your
scenario needs a different value, make sure to specify it in a line of code
before setting the src.
You can preview the impact over the network of
these three options by running this
page using the F12 Developer Tools (Network Tab).
For debug purposes, you can simulate new calls and disable the local cache by
checking the "Always refresh from sever" menu.
preload=none:
preload=metadata:
preload=auto:
While this property is great for the
initialization phase, you might also need to know when the browser actually
downloaded the audio clip and is ready to play it. You can get this information
by listening to the "canplaythrough" event; this event is called by the User
Agent once it estimates that if playback were to be started now, the media
resource could be rendered at the current playback rate all the way to its end
without having to stop for further buffering.
var audio = document.createElement("audio");
audio.src = "audio/sample.mp3";
audio.addEventListener("canplaythrough", function () {
alert('The file is loaded and ready to play!');
}, false);
Run live demo
Loops
Another frequent request for scenarios with
audio is the ability to loop a sound clip. With the HTML5 <audio>, you
can do this using the "loop" property; this setting will loop your clip
forever, or until the user or the application activates the pause()
audio control.
<audio src="audio/sample.mp3" autoplay loop>
</audio>
Run live demo
Another approach to loop an audio file is to
programmatically call the play() method when the audio clip ends; doing
so will allow you eventually to manage the delay between one loop and the
other.
var audio = document.createElement("audio");
audio.src = "piano/3C.mp3";
audio.addEventListener('ended', function () {
setTimeout(function () { audio.play(); }, 500);
}, false);
audio.play();
Run live demo
Note that any play() call executed on
the audio element before the sound actually ended won’t have any effect. If you
are interested to "cancel and restart" the current sound, you will need to
reset the currentTime.
var audio = null;
audio = document.createElement("audio");
audio.src = "piano/3C.mp3";
audio.addEventListener('ended', function () {
audio.play();
}, false);
function play() {
audio.play();
}
function restart() {
audio.currentTime = 0;
audio.play();
}
Run live demo
Multiple Audio Tags
If your scenario needs the same audio file to
be played several times concurrently (that is, with overlapping sounds), you
can achieve this result by creating multiple audio tags pointing to the same
file. Obviously the same approach also works if you are using different audio
files at the same time. As we explained earlier in this post, you can either
add those programmatically or by instantiating them in the markup.
The following code snippet shows how to load
and play multiple audio files using markup. The audio samples all have the same
length; at the end of the execution, they will loop starting from the
beginning. As you play them in Internet Explorer 9, you can notice that they are
automatically synchronized throughout various loops. You will notice that the
combination of these 5 sounds will play like the audio file used in the
previous demo ("sample.mp3").
<body>
<audio src="audio/Bass.mp3" autoplay loop>
</audio>
<audio src="audio/Drum.mp3" autoplay loop>
</audio>
<audio src="audio/Crunch.mp3" autoplay loop>
</audio>
<audio src="audio/Guitar.mp3" autoplay loop>
</audio>
<audio src="audio/Pizzicato.mp3" autoplay loop>
</audio>
</body>
Run live demo
While this approach is very simple and
straightforward, in most scenarios developers prefer to create the audio clips
programmatically. The following code snippet shows how to add 3 audio clips
dynamically using code. As you play them together, you will get the C Major
chord!
AddNote("3C");
AddNote("3E");
AddNote("3G");
function AddNote(name) {
var audio = document.createElement("audio");
audio.src = "piano/" + name + ".mp3";
audio.autoplay = true;
}
Run live demo
This code pattern works on any browser and
will allow you to build very compelling scenarios!
It’s important to keep in mind that as your application
or game becomes more complex you might eventually reach two limits: the number
of audio elements you can preload on the same page and the number of audio
elements you can play at the same time.
These numbers depend on the browser and the
capabilities of your PC. Based on my experience, Internet Explorer 9 can handle
dozens of concurrent audio elements simultaneously with no issues. Other
browsers don’t do as well – you might encounter evident
delays and distortions as you play multiple files
in a loop.
Synchronization Strategies
Depending on the characteristics of the
network, you should always consider the delay involved between adding the tag,
getting the content and being ready to play it. In particular, when you are
handling multiple files, each file might be ready to play earlier or later.
Here, for instance, is a capture with the 3 files used previously, loaded from
the local host.
As you can see in the Timings column,
different files might be ready at a different time.
A very common synchronization strategy is to
preload all the files first. Once they are all ready, you can quickly iterate
through a loop and start playing them.
var audios = [];
var loading = 0;
AddNote("2C");
AddNote("2E");
AddNote("2G");
AddNote("3C");
AddNote("3E");
AddNote("3G");
AddNote("4C");
function AddNote(name) {
loading++;
var audio = document.createElement("audio");
audio.loop = true;
audio.addEventListener("canplaythrough", function () {
loading--;
if (loading == 0) StartPlayingAll();
}, false);
audio.src = "piano/" + name + ".mp3";
audios.push(audio);
}
function StartPlayingAll() {
for (var i = 0; i < audios.length; i++)
audios[i].play();
}
Run live demo
Let’s bring everything together now! The
following demo simulates a piano playing Frère
Jacques (also known as Brother John, Brother
Peter…or Fra Martino). The page starts fetching all the notes, showing the
progress as they get preloaded on the client. Once they are all ready, the song
starts and keeps playing in a loop.
Run live demo
Audio in Real World Sites
Now that we’ve seen the common patterns to
handle multiple audio files, I’d like to highlight a few Web sites as examples
of best practice uses of the <audio> tag.
Pirates Love Daises: www.pirateslovedaisies.com
In another blog post I talked about Pirates
Love Daises, an awesome HTML5 game built by Grant Skinner. In addition to great
game play and compelling visual effects, Grant’s team also developed a
sophisticated audio library that plays several audio samples throughout the
game. The main logic is encapsulated within the AudioManager class. As
suggested earlier, before actually starting the game the site preloads all the
audio clips and display the cumulative progress in the initial loading screen.
The site takes also into consideration the case of network timeouts or errors
occurred while downloading an audio file.
addAudioChannel:function(b,a,f){
var h=document.createElement("audio");
if(f!=true){
this.currAsset=h;
this.timeoutId=setTimeout($.proxy(this,"handleAudioTimeout"),e.AUDIO_TIMEOUT);
h.addEventListener("canplaythrough",$.proxy(this,"handleAudioComplete"),false);
h.addEventListener("error",$.proxy(this,"handleAudioError"),false)
}
h.setAttribute("id",a);
h.setAttribute("preload","auto");
$("<source>").attr("src",b).appendTo(h);
$("<source>").attr("src",b.split(".mp3")[0]+".ogg").appendTo(h);
document.body.appendChild(h)
}
,handleAudioComplete:function(b){
if(LoadedAssets.getAsset(b.target.id)!=true){
LoadedAssets.addAsset(b.target.id,true);
clearTimeout(this.timeoutId);
this.calculatePercentLoaded(true)
}
}
,handleAudioError:function(b){
trace("Error Loading Audio:",b.target.id);
LoadedAssets.addAsset(b.target.id,true);
clearTimeout(this.timeoutId);
this.calculatePercentLoaded(true)
}
,handleAudioTimeout:function(){
trace("Audio Timed Out:",this.currAsset.id);
LoadedAssets.addAsset(this.currAsset.id,true);
this.calculatePercentLoaded(true)
}
Run live demo
Grant is currently working on a Sound Library
project that will allow developers to use their sound engine’s logic with any
other application. Looking forward to that!
Firework (by Mike Tompkins): www.beautyoftheweb.com/firework
The Firework demo is particularly interesting, as it allow you to interact with
several audio tracks at the same time, dynamically changing the volume of each
track. Moreover, as you interact with the audio channels, the interface
dynamically reacts to different inputs or settings.
This time the audio tags have been declared in
the HTML markup (there are just 6 tracks). The progress is tracked
programmatically by listening to the canplaythrough event. Once all
audio files are ready to play, a loop goes through the list and starts playing.
video.addEventListener('canplaythrough', onCanPlayAudio, false);
for (var i = 0; i < 5; i++) {
var aud = document.getElementById("aud" + i);
targetVolumes.push(0);
aud.volume = 0;
audioTags.push({
"tag": aud,
"ready": false
});
aud.addEventListener('canplaythrough', onCanPlayAudio, false);
}
document.getElementById("tompkins").src = MediaHelper.GetVideoUrl("Firework_3");
for (var i = 0; i < audioTracks.length; i++) {
document.getElementById("aud" + i).src = MediaHelper.GetAudioUrl(audioTracks[i]);
}
Run live demo
The developers in this case also made the
decision to start with the volume set to 0 and to increase it dynamically to 1
as soon as the experience is ready to play. Depending on the quality of your
audio card and drivers, this little trick reduces the likelihood of hearing an
initial "knock" noise when the audio starts.
BeatKeep: www.beatkeep.net
The last scenario is probably the most
complicated of the examples shown here. In this example, you can build your own
songs using a beat machine and playing several audio clips in a loop. In this
application, it’s critical to have perfect synchronization of the audio
channels and an agile buffering system to load
multiple clips.
The beat machine gives you full control over
the tempo and the time signature; using sophisticated timer logic and binding
model—the end result is a very smooth experience!
Conclusions
I encourage you to try all the samples and
applications in this post using Internet Explorer 9 or other browsers and let
us know what your experience was like! You can download all the sample code
used in this article here.
If you want to learn more about the audio and
video controls, I recommend that you watch 5
Things You Need To Know To Start Using <audio> and <video> Today or read these interesting articles on the MSDN.
Thanks to DoubleDominant for the audio clips used in this blog post and to Grant Skinner and Archetype for their great HTML5
experiences.
Submit an article or tip
Post your Blog
