• Login
Fintedex — Business, Fincance & Investment News
  • Contact
  • Submit a News Releases
No Result
View All Result
  • Contact
  • Submit a News Releases
No Result
View All Result
Fintedex — Business, Fincance & Investment News
No Result
View All Result
Home Technology

Meta wants the virtual landscape to sound like real life

Timothy Wilson by Timothy Wilson
26.06.2022
in Technology
0
Meta wants the virtual landscape to sound like real life
0
SHARES
2
VIEWS
Share on FacebookShare on Twitter

Meta and a group of researchers from the University of Texas at Austin (UT Austin) are working on bringing realistic audio to the metaverse.

As Kristen Garuman, Research Director at Meta AI, explains, there’s more to augmented and virtual reality (AR and VR, respectively) than just visuals. Audio plays a very important role in making a world feel alive. Garuman says “audio is shaped by the environment that [it’s] in.” There are various factors that influence how sound behaves like the geometry of a room, what’s in said room, and how far someone is from a source.

To achieve this, Meta’s plan is to use AR glasses to record both audio and video from one location, then using a set of three AI models, transform and clean the recording so it feels like it’s happening in front of you when you play it back at home. The AIs will take into account the room that you’re in so it can match the environment. 

Looking at the projects, it appears Meta is focusing on AR glasses. Meta’s plan for VR headsets includes replicating the sights and sounds of an environment, like a concert, so it feels like you’re there in person.

We asked Meta how can people listen to the enhanced audio. Will people need a pair of headphones to listen or will it come from the headset? We didn’t get a response.

We also asked Meta how can developers get a hold of these AI models. They’ve been made open source so third-party developers can work on the tech, but Meta didn’t offer any further details.

Transformed by AI

The question is how can Meta record audio on a pair of AR glasses and have it reflect a new setting.

The first solution is known as AViTAR which is a ”Visual Acoustic Matching model.” This is the AI that transforms audio to match a new environment. Meta offers the example of a mother recording her child’s dance recital at an auditorium with a pair of AR glasses. 

One of the researchers claims that the mother in question can take that recording and play it back at home where the AI will morph the audio. It’ll scan the environment, take into account any obstacles in a room, and have the recital sound like it’s happening right in front of her with the same glasses. The researcher states the audio will come from the glasses.

To help clean up audio, there is Visually-Informed Dereverberation. Basically, it removes distracting reverb from the clip. The example given is recording a violin concert at a train station, taking it home, and having the AI clean up the clip so you hear nothing but music.

The last AI model is VisualVoice, which uses a combination of visual and audio cues to separate voices from other noises. Imagine recording a video of two people arguing. This AI will isolate one voice so you can understand them while silencing everything else. Meta explains visual cues are important because AI needs to see who’s talking in order to understand certain nuances and know who’s speaking.

In relation to visuals, Meta states they plan on bringing in video and other cues to further enhance AI-driven audio. Since this technology is still early in development, it’s unknown if and when Meta will bring these AIs to a Quest headset near you.

Be sure to read our latest review on the Oculus Quest 2 if you’re thinking of buying one. Spoiler alert: we like it.


Previous Post

LG’s unique new Dolby Atmos soundbar is now on sale

Next Post

Project Syndicate: Green hydrogen is the ultimate answer to high gas prices, climate change, and saber-rattling dictators

Related Posts

Fujifilm X-T5: everything we know so far and what we want to see
Technology

Fujifilm X-T5: everything we know so far and what we want to see

by Timothy Wilson
20.07.2022
This devious attack turns a SATA cable into a data-stealing antenna
Technology

This devious attack turns a SATA cable into a data-stealing antenna

by Timothy Wilson
20.07.2022
Twitter could be banned in Indonesia if failing to comply with new law
Technology

Twitter could be banned in Indonesia if failing to comply with new law

by Timothy Wilson
20.07.2022
This creepy macOS backdoor spies on you without you noticing
Technology

This creepy macOS backdoor spies on you without you noticing

by Timothy Wilson
20.07.2022
Namecheap introduces vault-like domain security protection tool
Technology

Namecheap introduces vault-like domain security protection tool

by Timothy Wilson
20.07.2022
Next Post
Project Syndicate: Green hydrogen is the ultimate answer to high gas prices, climate change, and saber-rattling dictators

Project Syndicate: Green hydrogen is the ultimate answer to high gas prices, climate change, and saber-rattling dictators

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Premium Content

TaxWatch: When does a solo 401(k) make sense for self-employed people, and what are the contribution limits?

TaxWatch: When does a solo 401(k) make sense for self-employed people, and what are the contribution limits?

16.04.2022

: Amazon has mostly avoided antitrust scrutiny, but that may change in 2022

28.12.2021

Why India’s stockmarket is roaring

18.02.2022

Browse by Category

  • Business
  • Finance
  • Stock Market
  • Technology

Browse by Tags

Europe Oleg Volin Russia Ukraine

Fintedex delivers real-time news about the financial industry: feature stories, industry developments, opinions plus the latest on people and trends.

Categories

  • Business
  • Finance
  • Stock Market
  • Technology

Browse by Tag

Europe Oleg Volin Russia Ukraine

Recent Posts

  • Fujifilm X-T5: everything we know so far and what we want to see
  • This devious attack turns a SATA cable into a data-stealing antenna
  • How American banks are responding to rising interest rates

© 2021 Fintedex.

No Result
View All Result
  • Contact
  • Submit a News Releases

© 2021 Fintedex.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Are you sure want to unlock this post?
Unlock left : 0
Are you sure want to cancel subscription?