In August of this year, Meta introduced SeamlessM4T, a multimodal AI translation model that can translate text into approximately 100 languages and speech into 36 languages.
New AI Suite of Meta SeamlessM4T Makes Your Voice More Seamless
The tech giant is currently developing this tool further to make conversational translations more expressive and spontaneous (the latter being a crucial component of a genuine cross-language dialogue) with an updated “v2” architecture.
“SeamlessExpressive” is the first of the two new features; as the name suggests, it transfers your expressions to your translated speech.
These consist of your speech rate, pauses, pitch, loudness, and emotional tone (excitement, sadness, or whispers).
This discovery has the potential to revolutionize both content creation and daily life, since translated speeches had previously always sounded robotic.
Supported Languages
English, Spanish, German, French, Italian, and Chinese are among the languages supported.
However, at the time this article wrote, Italian and Chinese were not available on the demo page.
“SeamlessStreaming,” the second feature, enables others to hear a translation more quickly by beginning translation of a speech while the speaker is still speaking.
You won’t have to wait for someone to finish a sentence, but there is still a slight latency of less than two seconds.
The difficulty here, according to Meta, is that different languages have different sentence structures.
Specially Designed Algorithm
For this reason, an algorithm specifically designed to analyze partial audio input had to be developed in order to determine whether there was sufficient context to begin producing a translated output or if it was better to continue listening.
The “Seamless Communication” suite that Meta has recently developed appears to be more advanced than the mobile interpreting services that companies like Google and Samsung are providing.
Although the public’s access to these new features is unknown, you can already picture Meta incorporating them into its smart glasses in the future, making them even more useful than before.
To read our blog on “Meta’s new paid subscription deletes Insta & Facebook Ads,” click here.
