Niantic’s 8th Wall is set to enhance its WebAR platform by integrating generative AI (GenAI) modules, aiming to transform augmented reality experiences.
This technological fusion offers developers, brands, and agencies enhanced creative possibilities while enriching users’ interactions with augmented reality. In recent months, developers at 8th Wall have started leveraging GenAI technologies to augment the WebAR platform.
Current Integrations
Among the current integrations are:
- DALL·E and Stable Diffusion for generating images from text inputs.
- Blockade Labs for incorporating 360-degree images.
- Inworld AI for creating realistic interactive characters.
- Hugging Face and ControlNet for enhancing facial expressions with depth.
- Move.ai and Wonder Dynamics for simulating realistic movement.
To ease the integration of GenAI modules into WebAR projects, Niantic has released sample projects for Inworld and OpenAI. These samples demonstrate how developers can incorporate GenAI technologies into their projects, simplifying the process and eliminating common barriers.
Interactive Experiences
This synergy between GenAI and WebAR enables the creation of dynamic, interactive experiences that respond to user input. Users shift from being passive viewers to active participants, engaging in immersive interactions that evolve based on their actions.
Inworld AI: Enhancing Interactions
Inworld AI brings a groundbreaking Character Engine that endows AI-driven characters with lifelike attributes, including emotional intelligence, memory, and autonomy. Together with 8th Wall, Inworld AI has introduced the Inworld AI Integration Module along with the Rainbow Crunchies sample project. This module facilitates the seamless incorporation of lifelike characters into WebAR experiences, allowing for natural conversations and expressions.
By streamlining server provisioning and authentication, the Inworld Integration Module enables developers to integrate these characters without technical complications. The Rainbow Crunchies sample project illustrates how developers can craft vibrant, interactive experiences featuring characters like Rainbow Ray.
OpenAI: Powering Creativity
8th Wall is also rolling out a new suite of developer tools that utilize OpenAI’s DALL·E and ChatGPT for image generation and interactive text communication. DALL·E allows for the creation of detailed images based on text prompts, giving users creative control over their WebAR experiences. ChatGPT enables personalized, real-time interactions, allowing developers to create user-driven content.
Similar to the Inworld Integration Module, the 8th Wall Integration Module for OpenAI simplifies server setup and token management, letting developers concentrate on crafting compelling WebAR experiences. The GenAI Sample Project powered by OpenAI showcases the integration of these technologies, highlighting DALL·E’s capacity to generate 3D textures in real-time and ChatGPT’s role in user engagement.
Developer Insights
Ian Curtis, XR Developer at 8th Wall, expressed enthusiasm about the integration of generative AI into augmented reality, stating, “What excites me most about integrating generative AI into augmented reality is that it empowers every user to become an architect of their digital world. They’re no longer passively consuming content but actively creating it, shaping their experiences with the power of their imagination.”
Elevating WebAR with GenAI Modules
The potential for GenAI to revolutionize AR is significant. The new developer tools seamlessly integrate with 8th Wall’s key features, including Image Targets, Face Effects, World Effects, Sky Effects, Lightship VPS, and Shared AR tools.
To begin exploring these possibilities, developers can check out the Inworld AI Rainbow Crunchies sample project and the GenAI sample project powered by OpenAI. They can also import the Inworld AI Integration Module or 8th Wall’s Integration Module for OpenAI into their projects.