A new project from Microsoft’s in-house incubator, Microsoft Garage, introduces a different take on meeting transcriptions. While today there are a number of real-time transcription apps to use on your phone — like Otter.ai or Google’s Recorder app for Pixel devices, for example — Microsoft’s new Group Transcribe app reimagines meeting transcriptions as a more collaborative process, where everyone simultaneously records the meeting on their own device for higher accuracy. It also offers real-time translation for languages spoken in over 80 distinct locales.
To use the app, one person would first initiate the meeting in their own device. They can then invite the other meeting attendees to join the session via Bluetooth, a scannable QR code or by sharing a link. After the other participants join the session and the meeting begins, each person will see the transcript appear in real-time on their own device.
Image Credits: Microsoft
The app, which is powered by A.I. speech and language technology, is able to transcribe with higher accuracy and speaker attribution based on the volume of the speaker captured by the microphone of each phone being used in the meeting.
By comparing the level of a person’s voice volume, the cloud service attempts to determine which device is closest to the speaker and the language preferences of that speaker. This means speakers are also accurately labeled in the app, which can be a challenge for other transcription apps where only one person is recording.
In addition, if meeting participants want to speak in their own language, the app can provide the translation to others’ devices in their own language.
Image Credits: Microsoft
Microsoft says the app is designed with accessibility in mind, as it makes it easier for people who are deaf, hard of hearing, and non-native speakers to more fully participate in meetings by following along through the live transcriptions and translations.
The project itself was built by Microsoft employees who collectively speak over a dozen different languages and dialects.
“This can be a fantastic tool for communication. What I would love to see is for this to break down barriers for people speaking across multiple languages,” said Franklin Munoz, Principal Development Lead, when introducing the project.
Like most cloud-based transcription services, the app should not be used for highly confidential meetings. However, Microsoft has built granular data and privacy controls that allow users to decide if or when they want to share their conversation data.
Image Credits: Microsoft
To work, the audio and text input data collected is sent to Microsoft’s online speech recognitions and translation technologies — though with a randomly generated identifier, not your real name.
While Microsoft doesn’t save the meeting transcripts and recordings itself after the fact — they’re saved on your device — the app does encourage participants to “contribute” their meetings recordings to Microsoft so it can improve the service.
This allows Microsoft to retain the audio and speech recognition-generated text transcriptions when all meeting participants agree to opt in for that session. By reviewing the data, Microsoft aims to improve its speech recognition and speaker attribution capabilities over time, it says. The user data will then be accessed under NDA by both Microsoft employees and contractors from other companies who work for Microsoft, but won’t include any of the speakers’ account credentials.
Reviewers will also only have access to randomized snippets of audio, not full recordings. And Microsoft says it “de-identifies” meeting recordings by removing long strings of numbers that could represent things like credit card numbers or phone numbers, for example. Users can delete their previously shared recordings at any time, but otherwise they’re retained for up to 2 years on encrypted servers, the company says.
Because there’s not a way for a business, at an admin level, to configure or block the “contribution” setting for all users, people should carefully weigh the advantages and risks of such a service. It’s also a Microsoft Garage project, meaning it’s meant to be more experimental and could be shuttered at any time.
Currently, the Group Transcribe app is available on iOS only.
