• Login
Fintedex — Business, Fincance & Investment News
  • Contact
  • About us
No Result
View All Result
  • Contact
  • About us
No Result
View All Result
Fintedex — Business, Fincance & Investment News
No Result
View All Result
Home Technology

Meta wants the virtual landscape to sound like real life

Timothy Wilson by Timothy Wilson
29.09.2022
in Technology
0
Meta wants the virtual landscape to sound like real life
0
SHARES
14
VIEWS
Share on FacebookShare on Twitter

Meta and a group of researchers from the University of Texas at Austin (UT Austin) are working on bringing realistic audio to the metaverse.

As Kristen Garuman, Research Director at Meta AI, explains, there’s more to augmented and virtual reality (AR and VR, respectively) than just visuals. Audio plays a very important role in making a world feel alive. Garuman says “audio is shaped by the environment that [it’s] in.” There are various factors that influence how sound behaves like the geometry of a room, what’s in said room, and how far someone is from a source.

To achieve this, Meta’s plan is to use AR glasses to record both audio and video from one location, then using a set of three AI models, transform and clean the recording so it feels like it’s happening in front of you when you play it back at home. The AIs will take into account the room that you’re in so it can match the environment. 

Looking at the projects, it appears Meta is focusing on AR glasses. Meta’s plan for VR headsets includes replicating the sights and sounds of an environment, like a concert, so it feels like you’re there in person.

We asked Meta how can people listen to the enhanced audio. Will people need a pair of headphones to listen or will it come from the headset? We didn’t get a response.

We also asked Meta how can developers get a hold of these AI models. They’ve been made open source so third-party developers can work on the tech, but Meta didn’t offer any further details.

Transformed by AI

The question is how can Meta record audio on a pair of AR glasses and have it reflect a new setting.

The first solution is known as AViTAR which is a ”Visual Acoustic Matching model.” This is the AI that transforms audio to match a new environment. Meta offers the example of a mother recording her child’s dance recital at an auditorium with a pair of AR glasses. 

One of the researchers claims that the mother in question can take that recording and play it back at home where the AI will morph the audio. It’ll scan the environment, take into account any obstacles in a room, and have the recital sound like it’s happening right in front of her with the same glasses. The researcher states the audio will come from the glasses.

To help clean up audio, there is Visually-Informed Dereverberation. Basically, it removes distracting reverb from the clip. The example given is recording a violin concert at a train station, taking it home, and having the AI clean up the clip so you hear nothing but music.

The last AI model is VisualVoice, which uses a combination of visual and audio cues to separate voices from other noises. Imagine recording a video of two people arguing. This AI will isolate one voice so you can understand them while silencing everything else. Meta explains visual cues are important because AI needs to see who’s talking in order to understand certain nuances and know who’s speaking.

In relation to visuals, Meta states they plan on bringing in video and other cues to further enhance AI-driven audio. Since this technology is still early in development, it’s unknown if and when Meta will bring these AIs to a Quest headset near you.

Be sure to read our latest review on the Oculus Quest 2 if you’re thinking of buying one. Spoiler alert: we like it.


Previous Post

LG’s unique new Dolby Atmos soundbar is now on sale

Next Post

Project Syndicate: Green hydrogen is the ultimate answer to high gas prices, climate change, and saber-rattling dictators

Related Posts

Bondstream
Technology

Bondstream™ receives Prestigious Nomination for the 2023 Go Global Awards

by Lillie Hull
07.06.2023
This James Webb telescope image may be hiding more than just the stars
Technology

This James Webb telescope image may be hiding more than just the stars

by Timothy Wilson
29.09.2022
New God of War Ragnarok abilities revealed, including incinerating blades
Technology

New God of War Ragnarok abilities revealed, including incinerating blades

by Timothy Wilson
29.09.2022
DualSense Edge: everything you need to know about the PS5 pro controller
Technology

DualSense Edge: everything you need to know about the PS5 pro controller

by Timothy Wilson
29.09.2022
Bluehost vs GoDaddy: Two top web hosting providers compared
Technology

Bluehost vs GoDaddy: Two top web hosting providers compared

by Timothy Wilson
29.09.2022
Next Post
Project Syndicate: Green hydrogen is the ultimate answer to high gas prices, climate change, and saber-rattling dictators

Project Syndicate: Green hydrogen is the ultimate answer to high gas prices, climate change, and saber-rattling dictators

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Browse by Category

  • Business
  • Opinion
  • Stock Market
  • Technology
  • Без рубрики

Fintedex delivers real-time news about the financial industry: feature stories, industry developments, opinions plus the latest on people and trends.

Categories

  • Business
  • Opinion
  • Stock Market
  • Technology
  • Без рубрики

Recent Posts

  • How to Develop a Winning Content Marketing Plan
  • Where to Discover New Trends in Your Industry for Business Growth
  • Navigating the Business Landscape in Europe: How PR Firms from Washington DC Make a Global Impact

© 2021 Fintedex. Submit news release

No Result
View All Result
  • Contact
  • About us

© 2021 Fintedex. Submit news release

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
Are you sure want to unlock this post?
Unlock left : 0
Are you sure want to cancel subscription?