captioning for hearing impaired

10 posts / 0 new
Last post

I don't think DAISY is likely involved, but correct me if I am wrong. I am looking for information regarding a timeline for captioning of audio/video in epubs. This will be specifically for people with hearing loss issues. I am a member of the Hearing Loss Association of America (formerly SHHH) and am also a vendor for publishers, providing epub conversion services.

I would want to know which file types I would need to ask for from my customers.

Some examples:

An iOS app containing video footage from a concert and interviews with the artists. Would a machine create the captioning from the audio, or would the epub operator or someone at the publisher's end have to listen to the audio and create their own text?

One the text is available, how to place the captioning? Can it be turned on and off depending on user needs?


Captioning is already supported in EPUB 3 via the HTML5 "track" element, although I don't know how EPUB 3 reading systems are/will support this element.

See the section "Time tracks" on Matt Garrish's (free) book "Accesible EPUB 3":

You'll also find many articles on HTML5 tracks on the web.

More elements of answers on your specific questions:

1. Good captioning requires human authoring, it shouldn't be generated by a machine.

2. being able to turn on/off captioning is the responsibility of reading systems, nothing is said in the EPUB specifications. I expect most reading systems supporting the "track" element to provide an easy way to select the track(s) to be displayed, since there can be many (different level of captions, different languages, etc).

You can embed captions directly in your video, but I'm not aware of any reading systems that natively support the track element at this time, as I mentioned in the guide. None of the major browser cores currently support the track element, so it's hard to say even as EPUB 3 reading systems get released what the timeline will be for native support. It's not something the IDPF can control, as Romain has already noted.

There are javascript "polyfill" libraries that claim support for the track element, but then you run into the problem of javascript support in reading systems, too.

The site is a good resource to find out the state of the art in browser cores. Where the cores are will often give you a good idea of where general reading system support is likely to develop (not to over-generalize).

The mozilla site also has a chart of the current implementation status at

By the note there, Opera is getting close to support, which is a good sign if not an end unto itself.

I don't know the timeline for getting accessibility support into Readium, but I'm certainly hoping to see progress soon (on all fronts).

I wouldn't leave captioning or subtitling to a machine to do, as Romain also wisely notes, although it would certainly be an effective way to speed up the process. Machines are never great at voice recognition/conversion, so you'd want a human quality checking the output regardless of the type of timed track. Giving a reader an error-filled subtitle/caption isn't going to enhance the experience for them. And for captioning, a machine couldn't capture the other ambient information that would be necessary to give the quality the reader would require.

Thank you so very much! I appreciate your time and help.

One final question: Would either of you allow me to quote you in an article for Publishers Weekly about the Hearing Loss community's access to ebooks? I am gathering "user experience" quotes from members of the HLAA and want to incorporate thoughts from the IDPF user group.

It is my opinion that School (K-12) systems will be looking at buying content from publishers/devices who can provide this. The Charlotte, NC, School system just approved $10M to buy iPads for all the students.

No problem on my end if you want to quote from here or the book. My opinion doesn't necessarily reflect that of the IDPF, of course!

Fine by me too.

thanks so much!

thanks for your submitted

Secondary menu