The founders at Survios are true believers in virtual reality, and they’ve poured a lot of effort into the hit VR games Raw Data and Sprint Vector. Now they’re switching from games to something more like an immersive music experience with Electronauts.
The founders at Survios are true believers in virtual reality, and they’ve poured a lot of effort into the hit VR games Raw Data and Sprint Vector. Now they’re switching from games to something more like an immersive music experience with Electronauts.
The VR app will enable music fans to remix, compose, and perform their own music, riffing on works by artists such as The Chainsmokers, Odesza, Steve Aoki & Boehm, and many other bands that I am intimately familiar with (not).
At the core of Electronauts is the Music Reality Engine, which lets anyone perform and produce their own versions of the hits. It doesn’t skip a beat thanks to a technology called quantization. I spoke with Nathan Burba, CEO and cofounder of Survios in Los Angeles, about the new technology and the creativity that it brings to VR music.
“You can take a song by the Chainsmokers, ‘Roses,’ and determine when the different elements in the song will play,” Burba said. “It’s like you are playing inside a song.”
He said you get a sensation of playing the song at the right tempo, thanks to the quantization and a little mind trick that helps you deal with latencies in music headsets.
Electronauts debuts on August 7. It will be available on Steam and Oculus Home for HTC Vive and Oculus Rift at $20, and PlayStation Store for PS VR at $18. It’s also launching in VR arcades across 38 countries worldwide.
Here’s an edited transcript of our conversation.
GamesBeat: You were excited about the technology behind Electronauts. What makes it work? Can you explain that?
Nathan Burba: The project started with us creating quantized instruments. What that means is that you can perform an action in VR, and with a certain amount of latency that’s added, we then play a sound. The reason we do that is because we can make the sound happen at the correct tempo. That way you don’t have to worry about the tempo yourself. You’re not pounding your hands in perfect time like a drummer. You just perform your actions in the game and it sounds like you’re in time.
The best way to describe it is, it’s similar to the trick of how the HMDs work themselves. Some of the people working on Electronauts are former hardware engineers that also worked on our hardware back in the day. The latency trick we’re doing fools the brain into thinking it’s played a note at the right tempo, even though it hasn’t, because we delay it slightly. But that delay isn’t long enough for your brain to pick up on it. Certain tempos allow for that delay to be under 30 milliseconds, and with that timing, your brain says, “Did that happen in time? Sure, why not?” It plays along.
That creates the sensation of playing music in the right tempo, even you aren’t necessarily. It’s an amazing experience, to think you’re making this music. That’s at the core of the quantized instruments, in addition to the fact that they’re always in the right key. They always sound good, no matter what’s going on with the rest of the audio sync. Then we started layering other pieces on top of it, playing a bunch of samples in real time.
The way the game works, we request the stems for a song from a musician. These stems are the entire songs they’ve constructed, and then what we do is we put those into the game and allow you to play around with them through various interfaces, as much as you want. You can completely change the mix of the song at any given time. You can take out the vocals and play a solo, or bring everything down to a simpler version of the track. You can build up to a drop. All those kinds of arranging that a composer normally does — you can rearrange a song at will. You can take a song that never even had a drop and add one that sounds correct and relevant to the song, even though it was never there to begin with. You can play with all these pieces like they’re Legos.
It’s a revolutionary technology that has applications in a lot of places outside VR, but it synced up really well with VR, and we’re a VR company. It ended up working out well in Electronauts.
GamesBeat: If you’re mixing your own sounds that you’re creating into sounds that you’re hearing — are you making a song together with the game?
Burba: There are sounds that are taken from the song itself. Let’s say we take a popular song, like “Roses” by the Chainsmokers. That song has various elements. It has vocals. It has guitars playing in different keys. It has a drum kit. We’re letting the user determine when those different elements play, or whether they play or don’t play.
Some of those are turned on and off in real time and some of them aren’t. It depends on the kind of instrument. With the vocals, you queue them up and they play on time, and you can kind of remix them. With a guitar, you can play the actual notes yourself and sequence those notes to create melodies. It’s a big sonic playground. You play inside of the song.
It’s different from remixing a song using, say, a typical DJ turntable, because that’s designed to mix two different songs, alternating between one or the other, and all you have is the final mix. It’s like a cake. We’re not just giving you cake and frosting. We’re giving you the eggs, milk, and flour, everything, before it’s all put together. You get to decide how much or how little of those elements it has, and when it has those elements.
It’s the first time musicians have ever worked with a company with the raw stems, as they’re called, to allow this to occur. They did this a bit in Rock Band and Guitar Hero, but it wasn’t quite to the depth that we’re seeing with this project.
GamesBeat: You can be inside VR and then touch something with your hands, then, to turn on or turn off something like the vocals?
Burba: It’s a combination of interfaces that you touch — or bang on, so to speak — and interfaces where you turn buttons on and off. There’s what we call the orb kit. You always have two drumsticks, one in each hand, and you bang on the orbs. The orbs themselves are designed to be incredibly juicy, like how it feels to play a real drum. There’s a few instruments like that. There’s a laser harp. There’s a pinball, a kind of electric ball you can play around with.
Then there’s various buttons you can turn on and off. There’s a backing track, which has six different tracks. Usually they’re part of how the song itself is arranged. There’s an intro, a build, a drop, all the way to an outro. You play those in order. Then there are stems that are pieces of those tracks, and you can turn those on and off any time you want. You can make the entire track silent if you want.
A good example of why this is useful — have you played a game with the sub packs, the vibrating backpacks you can buy to go with VR games?
GamesBeat: Not really, no.
Burba: Imagine just having a giant rumble pack on your back. It adds a nice element to a game. But with Electronauts, because you’re in full control of the audio, you can control exactly how that thing massages your back. You can turn the bass drum on and start doing these giant pulses in certain parts of your back.
