{"id":129173,"date":"2024-02-14T16:00:18","date_gmt":"2024-02-14T16:00:18","guid":{"rendered":"https:\/\/blogs.nvidia.com\/?p=69734"},"modified":"2024-02-14T16:00:18","modified_gmt":"2024-02-14T16:00:18","slug":"speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2","status":"publish","type":"post","link":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/","title":{"rendered":"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge"},"content":{"rendered":"

Thanks to their work driving AI forward, Akshit Arora and Rafael Valle could someday speak to their spouses\u2019 families in their native languages.<\/p>\n

Arora and Valle \u2014 along with colleagues Sungwon Kim and Rohan Badlani \u2014 won the LIMMITS \u201924 challenge<\/a> which asks contestants to recreate in real time a speaker\u2019s voice in English or any of six languages spoken in India with the appropriate accent. Their novel AI model only required a three-second speech sample.<\/p>\n

The NVIDIA team advanced the state of the art in an emerging field of personalized voice interfaces for more than a billion native speakers of Bengali, Chhattisgarhi, Hindi, Kannada, Marathi and Telugu.<\/p>\n

Making Voice Interfaces Realistic<\/b><\/h2>\n

The technology for personalized text-to-speech translation is a work in progress. Existing services sometimes fail to accurately reflect the accents of the target language or nuances of the speaker\u2019s voice.<\/p>\n

The challenge judged entries by listening for the naturalness of models\u2019 resulting speech and its similarity to the original speaker\u2019s voice.<\/p>\n

The latest improvements promise personalized, realistic conversations and experiences that break language barriers. Broadcasters, telcos, universities, as well as e-commerce and online gaming services are eager to deploy such technology to create multilingual movies, lectures and virtual agents.<\/p>\n

\u201cWe demonstrated we can do this at a scale not previously seen,\u201d said Arora, who has two uses close to his heart.<\/p>\n

Breaking Down Linguistic Barriers<\/b><\/h2>\n

A senior data scientist who supports one of NVIDIA\u2019s biggest customers, Arora speaks Punjabi, while his wife and her family are native Tamil speakers.<\/p>\n

It\u2019s a gulf he\u2019s long wanted to bridge for himself and others. \u201cI had classmates who knew their native languages much better than the Hindi and English used in school, so they struggled to understand class material,\u201d he said.<\/p>\n

The gulf crosses continents for Valle, a native of Brazil whose wife and family speak Gujarati, a language popular in west India.<\/p>\n

\u201cIt\u2019s a problem I face every day,\u201d said Valle, an AI researcher with degrees in computer music and machine listening and improvisation. \u201cWe\u2019ve tried many products to help us have clearer conversations.\u201d<\/p>\n

Badlani, an AI researcher, said living in seven different Indian states, each with its own popular language, inspired him to work in the field.<\/p>\n

A Race to the Finish Line<\/b><\/h2>\n

The initiative started nearly two years ago when Arora and Badlani formed the four-person team to work on the very different version of the challenge that would be held in 2023.<\/p>\n

Their efforts generated a working code base for the so-called Indic languages. But getting to the win announced in January required a full-on sprint because the 2024 challenge didn\u2019t get on the team\u2019s radar until 15 days before the deadline.<\/p>\n

Luckily, Kim, a deep learning researcher in NVIDIA\u2019s Seoul office, had been working for some time on an AI model well suited to the challenge.<\/p>\n

A specialist in text-to-speech voice synthesis, Kim was designing a so-called P-Flow model prior to starting his second internship at NVIDIA in 2023. P-Flow models borrow the technique large language models employ of using short voice samples as prompts so they can respond to new inputs without retraining.<\/p>\n

\u201cI created the model for English, but we were able to generalize it for any language,\u201d he said.<\/p>\n

\u201cWe were talking and texting about this model even before he started at NVIDIA,\u201d said Valle, who mentored Kim in two internships before he joined full time in January.<\/p>\n

Giving Others a Voice<\/b><\/h2>\n

P-Flow will soon be part of NVIDIA Riva<\/a>, a framework for building multilingual speech and translation AI software, included in the NVIDIA AI Enterprise<\/a> software platform.<\/p>\n

The new capability will let users deploy the technology inside their data centers, on personal systems or in public or private cloud services. Today, voice translation services typically run on public cloud services.<\/p>\n

\u201cI hope our customers are inspired to try this technology,\u201d Arora said. \u201cI enjoy being able to showcase in challenges like this one the work we do every day.\u201d<\/p>\n

The contest is part of an initiative<\/a> to develop open-source datasets and AI models for nine languages most widely spoken in India.<\/p>\n

Hear Arora and Badlani share their experiences in a session<\/a> at GTC next month.<\/p>\n

And listen to the results of the team\u2019s model below, starting with a three-second sample<\/a> of a native Kannada speaker:<\/p>\n\nhttps:\/\/blogs.nvidia.com\/wp-content\/uploads\/2024\/02\/pr_kannada_f_indictts_prompt_3s-1.mp3<\/a><\/audio>\n

 <\/p>\n

Here\u2019s a similar-sounding synthesized voice reading the first sentence of this blog in Hindi:<\/p>\nhttps:\/\/blogs.nvidia.com\/wp-content\/uploads\/2024\/02\/pr_kannada_f_indictts_speaking_hindi_3-2.mp3<\/a><\/audio>\n

 <\/p>\n

And then in English:<\/p>\nhttps:\/\/blogs.nvidia.com\/wp-content\/uploads\/2024\/02\/pr_kannada_f_indictts_speaking_english-1.mp3<\/a><\/audio>\n

See <\/i>notice<\/i><\/a> regarding software product information.<\/i><\/p>\n<\/div>","protected":false},"excerpt":{"rendered":"

Thanks to their work driving AI forward, Akshit Arora and Rafael Valle could someday speak to their spouses\u2019 families in their native languages. Arora and Valle \u2014 along with colleagues Sungwon Kim and Rohan Badlani \u2014 won the LIMMITS \u201924 challenge which asks contestants to recreate in real time a speaker\u2019s voice in English or\t\t
\n\t\t\tRead Article\t\t\t<\/span>
\n\t\t<\/a><\/p>\n","protected":false},"author":316,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[343],"tags":[4223,13591,13623,2043,701,13534],"yoast_head":"\nSpeak Like a Native: NVIDIA Parlays Win in Voice Challenge - WebDomino.NET<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge - WebDomino.NET\" \/>\n<meta property=\"og:description\" content=\"Thanks to their work driving AI forward, Akshit Arora and Rafael Valle could someday speak to their spouses\u2019 families in their native languages. Arora and Valle \u2014 along with colleagues Sungwon Kim and Rohan Badlani \u2014 won the LIMMITS \u201924 challenge which asks contestants to recreate in real time a speaker\u2019s voice in English or Read Article\" \/>\n<meta property=\"og:url\" content=\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/\" \/>\n<meta property=\"og:site_name\" content=\"WebDomino.NET\" \/>\n<meta property=\"article:published_time\" content=\"2024-02-14T16:00:18+00:00\" \/>\n<meta name=\"author\" content=\"Rick Merritt\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rick Merritt\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/\",\"url\":\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/\",\"name\":\"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge - WebDomino.NET\",\"isPartOf\":{\"@id\":\"https:\/\/webdomino.net\/#website\"},\"datePublished\":\"2024-02-14T16:00:18+00:00\",\"dateModified\":\"2024-02-14T16:00:18+00:00\",\"author\":{\"@id\":\"https:\/\/webdomino.net\/#\/schema\/person\/f69da20fdf1afc997e397f8112ff8976\"},\"breadcrumb\":{\"@id\":\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/webdomino.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/webdomino.net\/#website\",\"url\":\"https:\/\/webdomino.net\/\",\"name\":\"WebDomino.NET\",\"description\":\"Global NEWS WordPress site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/webdomino.net\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/webdomino.net\/#\/schema\/person\/f69da20fdf1afc997e397f8112ff8976\",\"name\":\"Rick Merritt\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/webdomino.net\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/5bcaf9a1641bedb8a997b158e169032d?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/5bcaf9a1641bedb8a997b158e169032d?s=96&d=mm&r=g\",\"caption\":\"Rick Merritt\"},\"sameAs\":[\"https:\/\/blogs.nvidia.com\"],\"url\":\"http:\/\/webdomino.net\/index.php\/author\/rick-merritt\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge - WebDomino.NET","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/","og_locale":"en_US","og_type":"article","og_title":"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge - WebDomino.NET","og_description":"Thanks to their work driving AI forward, Akshit Arora and Rafael Valle could someday speak to their spouses\u2019 families in their native languages. Arora and Valle \u2014 along with colleagues Sungwon Kim and Rohan Badlani \u2014 won the LIMMITS \u201924 challenge which asks contestants to recreate in real time a speaker\u2019s voice in English or Read Article","og_url":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/","og_site_name":"WebDomino.NET","article_published_time":"2024-02-14T16:00:18+00:00","author":"Rick Merritt","twitter_misc":{"Written by":"Rick Merritt","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/","url":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/","name":"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge - WebDomino.NET","isPartOf":{"@id":"https:\/\/webdomino.net\/#website"},"datePublished":"2024-02-14T16:00:18+00:00","dateModified":"2024-02-14T16:00:18+00:00","author":{"@id":"https:\/\/webdomino.net\/#\/schema\/person\/f69da20fdf1afc997e397f8112ff8976"},"breadcrumb":{"@id":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/webdomino.net\/index.php\/nvidia\/speak-like-a-native-nvidia-parlays-win-in-voice-challenge-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/webdomino.net\/"},{"@type":"ListItem","position":2,"name":"Speak Like a Native: NVIDIA Parlays Win in Voice Challenge"}]},{"@type":"WebSite","@id":"https:\/\/webdomino.net\/#website","url":"https:\/\/webdomino.net\/","name":"WebDomino.NET","description":"Global NEWS WordPress site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/webdomino.net\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/webdomino.net\/#\/schema\/person\/f69da20fdf1afc997e397f8112ff8976","name":"Rick Merritt","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/webdomino.net\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/5bcaf9a1641bedb8a997b158e169032d?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/5bcaf9a1641bedb8a997b158e169032d?s=96&d=mm&r=g","caption":"Rick Merritt"},"sameAs":["https:\/\/blogs.nvidia.com"],"url":"http:\/\/webdomino.net\/index.php\/author\/rick-merritt\/"}]}},"_links":{"self":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts\/129173"}],"collection":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/users\/316"}],"replies":[{"embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/comments?post=129173"}],"version-history":[{"count":2,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts\/129173\/revisions"}],"predecessor-version":[{"id":129299,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/posts\/129173\/revisions\/129299"}],"wp:attachment":[{"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/media?parent=129173"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/categories?post=129173"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/webdomino.net\/index.php\/wp-json\/wp\/v2\/tags?post=129173"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}