STEAM GROUP: bashandslash

CoD:BO First Look, Part VIII
Written by Jock   
Sunday, 30 May 2010

CoD:BO First-Look, Part VIII "Full Performance Capture"

This weekend Treyarch's top staffers were in England to preview CoD:BO to a few mainstream gaming outlets, prior to the full onslaught debut at E3.

A really good article on what went on can be found here.  In the article, "Full Performance-Capture" is mentioned. That term might be new to some of you out there. Many of you might be aware of "Motion-Capture"  (also known as mocap); that is, the process of recording the motion of real-life items which can then be mapped onto virtual elements in the game. Using mocap, the movement of AI players, for example, becomes very realistic. 

Well, a revolution in technology has occurred in literally the last two years and a subset of mocap, known as performance-capture has really blossomed and is now the state of the art in the industry. Anyone seeing James Cameron's Avatar understands what that tech can do.

Call of Duty: Black Ops will be the first CoD game to have full performance capture (face+sound+motion) in their game. During the First-Look event, we got a sneak peak at that tech and talked to the folks who will be implementing it. Here's a taste of what went on during that presentation:


With the main presentations over Mark Lamia and Josh Olin gave us a walkabout through Treyarch's studios. Given how bright California days are, the building's interiors are very, very dark. No surprise really, given that the windows are heavily tinted and the overhead lights are kept off.

There are two main sections to the building those sections seem to intersect at two locations:  reception and near the atrium in the middle of the building. A well-stocked kitchen is attached to the atrium. From my own experience, well-stocked kitchens always suggest that employees work long hours...and that I believe is not uncommon at Treyarch whose staff seems unusually dedicated.

Josh ushers us through the kitchen and the community invitees stock up on sugary and salty treats. Looking around the studio, you can tell that the employees work really hard...but they love what they do, so can it really be considered work?
As an example of how great it must be to turn work into a hobby is Paulo88, one of the best mappers and modders this community ever developed. He started working at Treyarch a few months ago. I remember him telling me that he'd spend all his waking hours working on his maps. As an employee working on level design for Black Ops, not much has changed in his routine...only he's actually getting paid for it now!

We head towards the audio studios now. Josh Peckler and myself take it upon ourselves to see if we can find Paulo88 as we file one by one through the maze of darkened cubicles. Josh starts to chant 'Paulo....Paulo...', as though he was playing the Treyarch version of the Marco-Polo game. Some of us join in. All we get are weird stares from the denizens behind the cubicle walls.

The smell of hot audio equipment is unmistakable now, we follow it and eventually thread ourselves into the building's sound studio.

As we enter, we see a large wall-to-wall mixer board and several overhead LCD monitors...a Les Paul guitar stands at attention in a side room. Various speakers are strewn about the room's walls.

Mark Lamia and Josh introduce us to our new presenters: Adam Rosas, Lead Cinematics Animator for Call of Duty: Black Ops and Brian Tuey the Audio Director.

Note that there is a Cinematics Animation team and a Core-game Animation team. Adam will be talking about cinematics.

Adam Rosas: "One of the reasons why we decided to go to cinematics is so that we could focus on the performance of the characters. So, in essence the worlds are becoming more realistic, the story lines are becoming more intriguing, but the animations (core-game) weren't really keeping up with that realistic detail. So we wanted to inject more into what you were seeing from when AI is running around engaging you we have better intros better outros, so they feel more natural, they feel more connected to the world. And also, to tell our story, we have characters that can 'emote' to you. Show their emotions, so you know where they're coming from. If they're really angry, you can tell, even when their head's about this big, <puts fingers close together>, you'll get certain 'character lines'.

We're focusing on their performance, on how they read to you, their body language...everything. From that we took it up a notch and started doing full performance capture.
Essentially what that is, is that we capture everything: the body, the sound, and the face.

In essence, think of Avatar. With 'House of Moves' (ed: the company contracted to do the performance capture for Treyarch), they built a brand new 'volume' for us. Two hundred cameras. Really pushing technology. We've blown circuit breakers there because we have so many people on the stage that it just overloads the system. But these guys are great, they're solid.

The actors that we have are great performers, they know how to emote with their bodies and connect to the audience's back row (without being over-the-top). We've also been working with sound very closely, because it goes hand-in-hand with the voice performance."
Brian Tuey: "Typically when you're doing something in a booth you're working with someone's interpretation of someone else's body motions (ed: that is, a voice actor is recorded saying his lines, but there is no physicality in his performance...he is just standing still and must 'pretend' he's running/jumping/leaning...etc.). As opposed, to just getting an actor to actually capturing his performance. It's really about the characters at this point. We're getting a lot of subtleties that we're getting that we wouldn't get in the booth. For example, before our character speaks, he breathes. We see his chest move. That subtlety really comes through. Makes it feel really natural."

Adam: "Say in our conversation here, both of us start talking at the same time and our voices overlap because we're so excited about what we're talking about. Through full performance capture we get all that. We get all those emotions coming through at once. Whereas before, we would do it in three passes. We'd capture the body, then go record the voice and then go back and record the face. And when you put all that together, you come up with a pretty flat performance."

At this point Adam rolled some video of a performance capture session and superimposed on the vid was what the AI character would look like. This way you could see how the human performance was directly mapped onto the AI. The vid was what Adam and his team used to sell the technology to Mark Lamia, who in turn sold it to his bosses. Given that this was an early, early performance, the tech was remarkable.

Like many hardcore gamers, I don't spend much time playing SP. However, the technological advancements I saw in the vid might change my mind. The AI started to look like a real character to me and I actually felt bad when I saw it undergo pain. I've never had the reaction to an inanimate character before (...ok, maybe once when that boulder fell on Wile E. Coyote...someone should seriously arrest that damned Roadrunner, but I digress).
Adam: "Capturing that performance is key as opposed to us trying to animate it. It would have taken us forever."

Mark Lamia: "That performance blurs the lines. So when I tell you when we have a focus and character and story you can see that we're serious about having you engross in the story and these characters."

Adam: "What makes it tricky is that we can't go over-the-top. We have this new shiny toy that we want to use all over the place. But we have to be disciplined about how much we use it".

Mark: "...we could use it (all over the place) but there'd be a different guy talking to you over there." <pointing a threatening finger at Adam>

<Everyone laughs>
Mark then shows us a full motion capture of an ensemble piece, with two actors. The interaction is eerily realistic.
Brian: "What is your biggest scene?"

"We have a scene where we had four, full performance guys and then three others in the scene, so seven total. This is a big, intense, dramatic scene. But it had so many people in there that needed to be full 'markered', that's when we blew the circuit breakers."

Josh:  "How many cameras go into that?"

"Two hundred. And it's the biggest in the world. House of Moves really went out of their way to work with us and set up and push the technology. There's nothing else bigger in the world or more advanced right now. New software was written for it and we're still developing new techniques with them, new processes, new ways of packaging data."

"There's a big production behind it. About thirty people involved including the makeup artists, actors,'s pretty epic."

"Do the production costs blow what you've spent in the past?"



Josh Peckler:
"Is it worth it?"

"Yes. That's what I keep telling Activision."


"We believe, we'll do stuff that we couldn't have done two years ago. We're making the best entertainment package of the year."
Mark also went into a really interesting discussion of how in the past, actors were chosen for their voice-over abilities. That may change as this technology leaps forward. Studios will look for  actors with the 'total package' from now on. Sure, the voice will still be  important but they'll now need to act well and they need to 'look' like the character they're playing. The physicality needed to make video games now really does blur the line between making a movie and making a game.
Watch for this tech in CoD:BO, it was breathtaking and will go hand-in-hand with the fact that your player will have its own voice. The combination of the two should really suck you into the storyline. This will undoubtedly make me want to play SP for once.  For those of you that are into eye candy and you thought Avatar was cool...well prepare to have your minds blown.

< Prev   Next >

BASHandSlash Network


BASH on YouTube


The BASHandSlash Forums

original solarflare design by rhuk
lunarized by joomlashack