Sensorium has teamed up with Mubert to develop AI-driven DJs that can create music at virtual concerts in virtual reality. It’s the latest in an effort to create artificial people, or virtual beings.
Los Angeles-based Sensorium is known for providing simulated virtual experiences of real-world concert venues. The company creates photorealistic avatars, or virtual characters, that look like real people. The virtual DJs in this case will autonomously create a constant flow of ever-changing music that adapts to all kinds of moods and environments. It’s another attempt to stave off human loneliness by filling our lives with — artificial people.
Sensorium has raised more than $100 million in pursuit of this mission, and it will produce a diverse series of photorealistic virtual artists featuring unique identities and musical preferences. Genetic algorithms behind each of these all-digital characters will drive their behavior and allow them to absorb data from the environment to constantly nurture their knowledge and evolve their music styles. And yes, this is one more attempt to create the metaverse, the universe of virtual worlds that are all interconnected, like in novels such as Snow Crash and Ready Player One. (We’re putting on our own GamesBeat metaverse conference on January 27.)
Mubert’s technology will empower these virtual performers with the ability to create artificially generated music in real time based on an extensive database of samples. From hip-hop and EDM to K-pop and indie rock, these DJs will handle over 60 genres of music.
Sensorium Galaxy is the stage where the virtual DJs perform, and product head Ivan Nikitin said in an interview with VentureBeat that AI combined with VR can push the boundaries of computing. He sees Sensorium as a combination of a content platform and a social network.
Since 2017, Mubert has generated and delivered millions of multi-genre tunes to music lovers. Last year, the company’s mobile application became the “App of the Day” on the App Store in 170 countries and was awarded “App of The Year” by Google Play. The AI technology developed by Mubert is used by services like Crello, PicsArt, Sleep Cycle, and Insight Timer. Today, companies leverage Mubert’s copyright-free music for a range of purposes.
Virtual DJs will host shows in Sensorium Galaxy, along with famous artists like David Guetta, Armin van Buuren, Carl Cox, and Dimitri Vegas & Like Mike. The company says AI-powered artists will complement the talent of real-life artists in the alternate universe of Sensorium Galaxy, with a balanced mix of human and AI-generated art.
Above: Sensorium’s AI virtual DJs can host concerts in VR.
Image Credit: Sensorium
Sensorium Galaxy is a multi-user social VR platform aimed at remaking the way people interact with each other and experience the arts. The Galaxy is being built in partnership with artists, producers, and entertainment companies. Among them are Yann Pissenem, creator of nightlife hubs Ushuaïa Ibiza and Hï Ibiza, and music and entertainment streaming service Tidal, which is owned by artists such as JAY-Z, Lil Wayne, Rihanna, Daft Punk, and others.
Nikitin said the Sensorium Galaxy is the evolution of social networking and takes users beyond today’s one-dimensional digital experiences. It has a rival in Wave, which also stages virtual concerts. The difference is that Sensorium is a persistent virtual world, while Wave is event-based, and Sensorium is striving to make its avatars look like real people. Both companies have expanded beyond the world of VR into other devices. You can view Sensorium Galaxy with a VR headset, a PC or Mac, or a mobile device.
Sensorium was founded in 2018 by multi-billionaire Mikhail Prokhorov and an owner of the Brooklyn Nets. Mubert has more than 200 musicians and 2,000 pro users for its real-time generative music.
Sensorium makes money through avatar customization. Users can invest their own money in clothes or other features for their avatars, and people can buy digital tickets to go see events. You can pay more for VR or watch a video livestream for free.
“We see it as a virtual space where people can join together, meet new people, join with friends, and visit different activities spread out across different worlds,” Nikitin said. “The first world that we’re releasing is called Prism. It’s created by the people who know how to create the goosebumps across audiences through electronic dance music. They gave us great ideas how to approach it in virtual reality. The audience and the artificial DJs can become active participants of the show.”
Sensorium is creating other worlds, like Motion, with different settings and ways to show off photorealistic avatars, built with photogrammetry techniques, Nikitin said. Ultimately, he wants to create worlds where you can have free-flowing conversations with the AI characters, dubbed NPCs, or non-player characters.
The company has 100 people, including a team of more than 60 in Montreal working in production. Starting in March or so, the company plans to launch a closed beta test of Prism, as well as its lobby, where users can gather as if they were in their own personal cabins aboard a starship.
“We don’t approach this like game developers,” Nikitin said. “We find the people who already achieved a lot in their respective fields, and we unleash their creativity in the virtual experience. We use the tools of VR and try to give the audience experiences that they can’t get in a real event.”
