NVIDIA announced on Tuesday at the SIGGRAPH 2023 it had expanded its CloudXR software developer kit (SDK) to deliver more scalable, customisable extended reality (XR) experiences for enterprises and developers.
At the event in Los Angeles, Jensen Huang, Chief Executive, NVIDIA, gave a keynote address to unveil the news. In it, he revealed a host of updates to the industrial-grade cloud streaming XR platform.
Regarding industrial extended reality (XR), Huang said that one of the biggest opportunities for the world, software, and AI was to revolutionise heavy industries. For him, such industries would benefit ‘enormously’ from digital transformations via digital twinning, simulation, and computer-aided design (CAD).
Explaining, he said,
“There’s just enormous amounts of waste [among] $50 trillion worth of dollars of new electric vehicle factories, battery factories, and new chip fabs that are going to be built all over the world. Not to mention, the enormous number of factories that are already in operation some 10 million factories in operations today”
He added that the world’s heavy industries, from design, styling, engineering, simulation, testing, factory planning and building, and others operated mostly mechanically to date.
Additionally, he revealed that major firms would use the platform to streamline and transform their digital workflows. This included adopters of the Omniverse such as Amazon, Deutsche Bahn, and others.
He explained,
“Amazon is using Omniverse to visualise their warehouse, [a] giant robotic system, to help their workers [and] to simulate their fleet of AMRs. [We are] creating a digital twin of the [Earth’s climate system]. Deutsche Bahn is using Omniverse to create a digital twin of their entire railway network, so they can operate it completely [digitally]. In order for that to happen Omniverse has to be real-time.
Additional early adopters of the updated CloudXR platform included Chinese EV automaker BYD and Swedish telecoms giant Ericsson.
Finally, he spoke of the company’s advancement of OpenUSD. Such technologies integrated into the NVIDIA CloudXR platform, where Omniverse would facilitate greater interoperability and adoption of the industrial metaverse and cloud streaming technologies.
Jensen said,
“Just as HTML ignited a major computing revolution of the 2D internet, OpenUSD will spark the era of collaborative 3D and industrial digitalization. NVIDIA is putting our full force behind the advancement and adoption of OpenUSD through our development of NVIDIA Omniverse and generative AI.”
(Re)introducing CloudXR
With the latest iteration of CloudXR, major businesses can integrate their immersive workflows on standalone headsets. This will upscale compute and rendering power to boost graphical fidelity for streamed content.
Additions to the CloudXR tech stack include,
- ChatUSD, which uses large language models (LLMs) to allow developers to answer USD knowledge questions and generate Pything-USD scripts.
- RunUSD, a cloud-based API tool for converting OpenUSD files to “fully path-traced rendered images.” It does so by verifying the compatibility of uploaded files with OpenUSD release versions and rendering objects in the Omniverse Cloud.
- DeepSearch, the LLM tool for semantic searches across untagged asset databases.
- USD-GDN Publisher, which enables one-click functions for enterprises and software developers to produce “high-fidelity, OpenUSD-based experiences to the Omniverse Cloud Graphics Delivery Network (GDN).” This works from Omniverse-based applications like USD Composer and offers real-time streaming to web browsers and mobile devices.
NVIDIA’s CloudXR Suite will also support standalone, all-in-one headsets and mobile XR devices. NVIDIA will empower enterprises with device-agnostic immersive environments through its cloud computing technologies.
CloudXR Suite will leverage NVIDIA’s RTS graphical processing units (GPUs) and virtual workstation (vWS) software. With it, companies streaming XR content across Android and iOS devices can adapt content to network conditions at the proper framerates.
Virtual, augmented, and mixed reality (VR/AR/MR) experiences can then become ubiquitous across company hardware. This allows workers across public and private networks to access critical XR content in real time.
Furthermore, independent software vendors (ISVs) can utilise the streaming platform for their headsets, mobile XR devices, and others. This conveniently allows access to applications with low-latency, industry-grade GPU acceleration and streaming capabilities for multimodal use.
NVIDIA’s CloudXR: The Triple Threat for XR Cloud Streaming?
For users of the updated platform, CloudXR provides the following:
- CloudXR Essentials to stream XR content across 5G low-latency, low-loss (L4S) infrastructure for fully optimised content and workflows. It also provides StreamVR plugins along with sample clients and server-side application programme interfaces (APIs) for developers.
- CloudXR Server Extensions additionally boost service-side interfaces via additional source-code upgrades to Monando OpenXR runtimes. With CloudXR Essentials’ novel CloudXR Server API and OpenXR API, enterprises can collaborate with orchestration partners to scale their immersive distribution operations.
- CloudXR Client Extensions will also offer a key plugin for Unity Editor, providing developers with bespoke CloudXR applications. These can include bespoke interfaces, lobbies, and other tools using the CloudXR streaming service.
Additionally, collaborators on the platform can use the tools to scale their immersive content to data centres, edge networks, and concurrent users. It will also support streaming from OpenVR, OpenXR, and other key applications operating at the edge of cloud networks.
NVIDIA-Ericsson Use Case
For example, Ericsson, a major global telecom, will work with NVIDIA’s CloudXR tools to lower lag across their networks.
Speaking further in a statement, Sibel Tombaz, Head of Product Line for 5G Radio Access Network, Ericsson, said,
“As a key part of Ericsson’s Time-Critical Communication capabilities, L4S will significantly improve user experience for use-cases like cloud gaming, and its great news that NVIDIA is making L4S a production element of CloudXR”
Devs and professionals can also incorporate VMware Workspace ONE XR Hub, which adds CloudXR, in their workflows. This allows industry professionals to collaborate with teams worldwide with their headsets and devices, along with stellar customisation and authentication tools.
Matt Coppinger, Director of Product Management for End-User Computing, VMware, added users could access RTX GPU capabilities from anywhere with the platform.
He added: “Workspace ONE XR Hub and CloudXR will allow our customers to stream rich XR content, and more teams can boost productivity and integrate realistic, virtual experiences into their workflows.”
Across the Omniverse
In a recent presser with global media, Rev Lebaredian, Vice President, Omniverse and Simulation Technology, NVIDIA, explained the demand for enterprise metaverse solutions.
He began, stating that industrial enterprises were “racing to reinvent themselves” and that opportunities for industrial digitalisation were “enormous.”
Citing figures, he said that manufacturing, transport, and telcos were $46 trillion, $10 trillion, and $2 trillion, respectively. These and similar verticals sought to optimise their physical processes.
He said that NVIDIA’s Omniverse solution was “built from the ground up” on the Universal Scene Description (OpenUSD) as a USD-native platform. This connected the world’s largest “design, computer-aided design (CAD), and simulation tool ecosystems such as Autodesk, Siemens, Adobe, PTC, and Epic Games.”
Pixar developed OpenUSD in 2012 and open-sourced the file format in 2015 to the global community. The file format has since become a major global standard for world-building, visual effects (VFX), digital twins, architecture, engineering, and construction (AEC), and robotics.
Currently, NVIDIA aims to boost USD for high-performing, real-time industrial applications. This includes interchanging CAD to USD, enabling geospatial coordinates and metrics assemblies, and other tools, he added.
This would provide a simulation-ready (SIM-ready) specification for adding physics properties to USD assets. Incorporating this into workflows would allow “AI agents such as robots and autonomous vehicles” to “learn about our real world from completely within simulated worlds.”
NVIDIA’s ISAAC SIM and DRIVE SIM are the APIs used for robot and automotive simulations, respectively.
NVIDIA Industry Collaborations
Regarding collaboration with the industry, Lebaredian stated that NVIDIA had numerous announcements at SIGGRAPH 2023.
For example, Shenzhen, China-based luxury electric vehicle (EV) automaker Denza would “build and deploy its next generation of our configurations on Omniverse Cloud” in a joint venture with BYD and Mercedes Benz.
The project aims to build on additional work with the world’s largest advertising firm, WPP. For the advertising giant, it would build a portfolio of AI-enabled marketing services using Omniverse Cloud.
This also allowed the company to integrate CAD, 3D, and other digital content creation tools to build a “super digital twin of the Denza car model.”
He continued, stating,
“To build a configurator today, you need to individually render hundreds of thousands of 2D images with every possible [variation] of the car. If the car model changes, you need to start the entire process all over again. This is extremely costly and time-consuming. With Omniverse, WPP and BYD can create a single super digital twin of a car model, based on all the original engineering and design data of the car. No manual 2D rendering required”
The companies could also add features to the super digital twin and deploy it “across all the marketing channels instantaneously” without manual reworks.
NVIDIA Alliance for OpenUSD, Additional Tools
Additionally, NVIDIA launched its Alliance for OpenUSD with major industry players such as Pixar, Apple, Adobe, and Autodesk.
Lebaredian stated the move was a “major inflexion point for the development and adoption of OpenUSD.”
He explained further,
“Today, there are many applications of USD, but there is no core specification. [This] will develop a standard specification for OpenUSD, which will then super accelerate USD adoption across the tool ecosystems and interoperability between software”
The executive also announced merging Generative AI and USD with several Cloud APIs to “simplify developer implementation and engagement with USD.”
Features for this included generating real-time, fully path trace interactive rendering of USD files. Developers can validate and check USD for compatibility and compliance of their USD implementation against core specifications.
He concluded that its USD DeepSearch would open access to LLM-based, semantic 3D search services. Developers and users can use the service to browse massive “untagged asset databases with text or image inputs.”
NVIDIA’s Omniverse Cloud API would offer the service, “enabling developers to access and build their own tools, services, and applications,” he continued.
Finally, NVIDIA’s Picasso Cloud API tool would provide developers and content creators with GenAI for visual design. This allows users to train NVIDIA’s foundation models, fine-tune pretrained models “on their own responsibly-sourced data.”
Additional Omniverse tools would include “OpenXR-compliant devices such as Magic Leap, Varjo, HTC, Meta Quest, or USD-compliant platforms such as Reality Kit and iOS,” Lebaredian explained.
Comments on Alliance for OpenUSD
Prior to SIGGRAPH, several key executives from across the tech industry explained the need for building standards in spatial computing.
In a recent comment, Mike Rockwell, Vice President, Vision Products Group, Apple, said that the alliance would help “accelerate the next generation of AR experiences [and] produce an ever-widening array of spatial computing applications.
Rockwell added,
“Apple has been an active contributor to the development of USD, and it is an essential technology for the groundbreaking visionOS platform, as well as the new Reality Composer Pro developer tool. We look forward to fostering its growth into a broadly adopted standard”
According to Gordon Bradley, Fellow, Media and Entertainment, Autodesk, content creators building computer graphics (CG) worlds, digital twins, or the 3D web would need a “cohesive” platform for collaborating and sharing data “across tools, services, and platforms.”
Continuing, Bradley said: “Autodesk is excited to support the Alliance for OpenUSD as it drives 3D interoperability for visual effects, animation, and beyond, and supports our vision to help customers design and make a better world.”
Finally, Guy Martin, Director of Open Source and Standards, NVIDIA explained how OpenUSD provided developers with “the complete foundation to tackle” large-scale workloads.
This included building content for “industrial, digital content creation, and simulation workloads” using “broad multi-app interoperability.
He concluded: “This alliance is a unique opportunity to accelerate OpenUSD collaboration globally by building formal standards across industries and initiatives to realize 3D worlds and industrial digitalization.”
SIGGRAPH 2023 is taking place in Los Angeles, California from 6 to 10 August. The massive event includes several of the world’s largest XR firms and organisations like The Metaverse Standards Forum, Meta Platforms, NVIDIA, Adobe, Unity Technologies, and Vicon.
The event is celebrating its 50th anniversary with a massive lineup of panel talks, fireside chats, product demos, and use cases. Thousands are attending the event to showcase industry verticals from computer vision, AI, XR, the industrial metaverse, and graphics design.
