Auto-transcribing / Auto-captioning Live Broadcasts

Technically, what happens is your audio goes through Automatic Speech Recognition (ASR) or speech-to-text software that tries to guess at what someone is saying. It bases its guesses on voice or speech patterns.

If you've seen it in action, you know it isn't entirely accurate. That's because ASR alone does not recognize the context of the speech pattern. Was it supposed to be "2", "to", or "two"? That's where Natural Language Understanding (NLU) engines get involved. They try to look at all the words near the one that sounds like "2", "to", or "two", and determine which version it really was supposed to be in that context.

The Federal Communications Commission (FCC) regulates television broadcasts requiring a 99% accuracy in captions. A 2021 study by 3Play Media divided out the issues into two categories:

• Word Error Rate (WER): Where the translation uses the incorrect word.
• Formatting Error Rate (FER): Where the translation does not place punctuation, capitalization, sound effects, speaker changes, and other notations in the right location if at all.

In their study of the top automatic speech recognition engines:

• Word Error Rate (WER): accuracy was in the 80-90% range
• Formatting Error Rate (FER): accuracy was in the 64-79% range

Audio quality:

• Is there noise or music in the background?
• Is it over cellular service or Wifi/Internet connection?
• Is the audio equipment good?

Content:

• Is complex jargon used, like the terms in science, technology, engineering, and mathematics?
• Does it include words that are not easy to spell or have uncommon spellings, like proper names?

Duration affects prerecorded audio/video content. The longer the duration, the more inaccuracies.

If your web conferencing or online meeting platform allows your audience to turn on ASR during a live broadcast, it is a good minimum requirement, especially where no Americans with Disabilities Act (ADA) accommodation has been requested.

However, if you do have audience members with known disabilities (e.g. students registered with the Disability Services Office, audience members requesting accommodations during an event) you should consider the above items that can affect ASR accuracy and whether you need a live transcriber:

• If you tested the ASR on your web conferencing or online meeting platform, and it handled your terms well, then use ASR.
• If you are doing something performative where you need to use words that are otherwise censored, you may need to use a live transcriber instead.
• If it will be hard to identify speakers or translate jargon, you may need to use a live transcriber instead.

Instead of ASR: If you use a prerecorded video in a live broadcast, make sure it has closed captions, then play the video during the broadcast with the closed captions on for the whole audience to see. Tip: Make sure the closed captions do not cover important areas on the video frame, since they cannot be turned off in these scenarios.

Record the audio/video content using any option to auto-caption the finished recording where possible. It might be inaccurate, but it can be cleaned up later, or the recorded audio/video content can be submitted to a captioning services provider to improve or add the closed captions.

It can take some time to fix the ASR inaccuracies, so deadlines for viewing audio/video content should treat people with and without disabilities in an equitable manner.

For example, if an exam will cover the content in the audio/video, then extend the period of time between the exam and the presentation of the audio/video for all students, so those students who need the captions have time to review the content before the exam. To give everyone the same amount of time to review the content, only post it for all the students once the captions have been added.

Cisco Webex is no longer licensed at TAMU-CC. However, you can use the guest option to attend meetings from external organizations. If you need accommodations, contact them for assistance.

Some versions of Cisco Webex provide live ASR through the Webex Assistant. If the meeting host has them configured, you will be able to show/hide captions in Webex meetings.

Where needed, ASR is available for meetings that are recorded.

Google Live Transcribe is the Google app equivalent of the Microsoft Translator. They both provide open captions in multiple languages and for all audience members using the app. it is available on Google Chromebooks and Android devices.

Google Slides is the Google Docs equivalent of Microsoft PowerPoint. Google Slides allows for live open captioning. Keep in mind placement of the open captions on the screen you are sharing, since they cannot be turned on/off by the audience. They also do not listen to all members of the meeting, only the one sharing the Google Slides presentation.

Microsoft PowerPoint allows for live open captioning. Keep in mind placement of the open captions on the screen you are sharing, since they cannot be turned on/off by the audience. They also do not listen to all members of the meeting, only the one sharing the Microsoft PowerPoint presentation.

Microsoft Teams allows everyone to turn on live ASR:

• auto-generated captioning in meetings
• auto-generated captioning in live events

When an immediate accommodation is needed due to inaccurate auto-captioning, please use the chat area to communicate.

Where needed, ASR is available for meetings that are recorded.

Microsoft Translator provides open captions in multiple languages and for all audience members using the app. it is available as a web application or mobile app. Start translating by first identifying your designated language. Each member will speak in their own language, and it will translate what they say in your designated language and vice versa for them in their own language.

Some versions of Zoom provide live ASR through a beta testing program or third-party software. Zoom may launch their own live ASR in Fall 2021.

If the meeting host has live ASR configured, you will be able to show/hide captions in Zoom meetings.

Where needed, ASR is available for meetings that are recorded.