Meta Platform’s AI division broke through the noise of XR headsets this Monday by announcing a new generative AI tool: VoiceBox.
The firm’s new AI solution creates high-quality audio clips and matches pre-existing audio samples with additional support for six languages.
Voicebox provides a rich advertising avenue for marketing teams. Moreover, Meta supports marketing opportunities via a toolset that the firm is currently testing with select partners before a July debut.
Voicebox also allows developers to leverage generative AI for immersive applications like voice avatars.
The news comes after Meta announced the Quest 3. As the firm ramps up to introduce more information regarding the MR headset, such as its AI and Metaverse ambitions, Meta is working on a deep technology stack to empower next-generation MR experiences, which includes AI.
Where does Meta AI Fit into the XR Picture?
Meta AI and its Reality Labs divisions for XR R&D have always existed side-by-side.
Since its rebranding and diversion towards XR technology, Meta has been championing its AI research as a tool to enhance immersive experiences.
Moreover, Meta operates an AI Research SuperCluster (RSC) for accelerating Metaverse, natural language processing (NLP), computer vision processing, and deep learning research.
Meta has been advertising prototypes of smart glasses, such as Project Nazare, which leverage voice-activated assistants that scan a user’s surroundings to help users with their daily operations.
Additionally, Meta is employing an AI platform to debut Universal Speech Translator (UST) project, an open-source, real-time speech-to-speech translation system for primarily oral languages.
Meta is releasing its integrated ML Hokkien translation data and research papers as open resources, enabling AI developers to create UST projects which cover more languages.
More recently, Meta announced the segment anything model (SAM) and the development of AI content creation tools for Messenger and WhatsApp earlier this year. Moreover, recent reports show that both social media platforms will debut on the Meta Quest portfolio.
Meta did not discuss AI during its reveal earlier this month. However, this year, the firm may introduce more AI integration during its Connect showcase event.
Builder Bot and Project CAIRaoke
In February 2022, Meta outlined how AI is crucial to developing Metaverse services. At the time, the Menlo Park-based firm introduced Builder Bot, a self-learning AI platform that generates virtual environments via voice input recognition.
Builder Bot can automatically create or import design features into an immersive environment, such as parks and beaches. Builder Bot can also add smaller objects, such as clouds or trees, with a user’s voice command.
The voice-activated bot can also add sound effects and music to immerse users in digital environments. Builder Bot appears to be part of Meta’s initiative to give developers tools for building custom XR content on the Horizon platform, where users can develop and monetize immersive UGC.
Moreover, Meta is working on Project CAIRaoke, an in-house conversational AI platform. The firm is developing the platform to support everyday use cases, digital assistants, Horizon Worlds, and future Builder Bot functions.
Project CAIRaoke employs an end-to-end neural model, basing itself on a foundation of integrated technology, including natural language understanding (NLU), dialogue state tracking (DST), dialogue policy (DP) management, and natural language generation (NLG), to create real-time, contextualized and personalized conversation flows.
Microsoft adds AutoPilot Integration for Hololens 2
This week, Microsoft also introduced AI to its XR headset product offering.
Late last year, the firm bought significant stakes in OpenAI, the firm behind ChatGPT. Since the close partnership, Microsoft debuted Autopilot, a generative, prompt-based AI tool to help users complete tasks in Office workflows.
Microsoft released its Hololens 2 23H1 update, which added generative AI tools for enterprise end-users to the Hololens 2 headset.
The firm’s Dynamics 365 Remote Assist and Dynamics 365 Guides applications for Hololens 2 now support generative-AI inputs. Microsoft will deploy the AI features once it completes its Azure Active Directory project. Moreover, the Hololens 2 update introduces an AI-specific reset option for select fail states alongside Autopilot optimization.
It appears that, for now, the hype cycle for AI has calmed down, and leading technology firms are working out how the leading emerging technology opportunities – XR and AI – can work together to enhance end-product offerings.
