00:00:00
DELETE ME
00:00:01
It was like the army.
00:00:02
You followed orders,
00:00:04
you did the right thing,
00:00:07
you got rewarded.
00:00:11
Some of the greatest films
00:00:12
follow characters whose journeys
00:00:14
span many years.
00:00:15
Where are we gonna go
00:00:16
but up?
00:00:17
[cheering]
00:00:19
But as filmmaking has evolved,
00:00:21
so, too, have the tools filmmakers use
00:00:23
to show characters aging and de-aging.
00:00:26
You're right, Mr. Morgan!
00:00:29
As filmmaking techniques
00:00:30
have grown to include
00:00:31
not just makeup and costumes
00:00:33
but also digital technology,
00:00:35
so has the need for innovations
00:00:38
that ensure the integrity of actors' performances
00:00:41
at any age shown on screen.
00:00:44
Together, Martin Scorsese and his production crew,
00:00:47
Netflix, and ILM
00:00:49
have teamed up on The Irishman
00:00:52
to push the boundaries
00:00:53
of this visual effects process.
00:00:56
The film takes place from 1949 to 2000,
00:00:59
and it goes back and forth in time continuously.
00:01:04
Problem is by the time I was ready to make the film,
00:01:07
Bob De Niro and Al Pacino and Pesci
00:01:09
could no longer play these characters younger
00:01:11
in makeup
00:01:12
and when I looked at the script, it turns out that,
00:01:14
you know, it means I make half the film with Bob.
00:01:16
Why do it?
00:01:17
I'm shooting Silence in Taiwan
00:01:19
and Pablo of ILM came up to me and said
00:01:23
I think I could make them look younger,
00:01:25
and I said I don't know,
00:01:26
I can't have the actors talking to each other
00:01:29
with golf balls on their faces,
00:01:30
it maybe gets in the way with the actors
00:01:32
and with the kind of film this is,
00:01:34
they need to play off each other.
00:01:36
I said, if you could find a way
00:01:38
to lessen the technical aspects of it,
00:01:40
that could work.
00:01:42
So I just kind of took a breath
00:01:43
and I said, you know, we'll develop the technology.
00:01:47
I got on the phone with ILM and I said,
00:01:49
I got a project!
00:01:50
First thing that Danny said to me
00:01:53
is this is too risky,
00:01:54
don't do it.
00:01:55
Right.
00:01:55
I said well, is this the way
00:01:57
you felt when you did Jurassic Park?
00:02:00
And he just went like okay, fine,
00:02:02
you got me there.
00:02:03
I'm Rob Bredow,
00:02:04
I'm the executive creative director
00:02:06
and the head of Industrial Light and Magic.
00:02:08
To get to partner with Netflix and take a risk
00:02:11
on creating that technology while we're
00:02:13
making the film
00:02:13
is the kind of thing kind of ILM lives for.
00:02:15
There have been many movies that have de-aged
00:02:17
over the years,
00:02:18
but this one really, where it's front and center
00:02:21
and is part of the storytelling.
00:02:23
We are used to difficult projects here at ILM,
00:02:26
but on this one we have to take to the next level
00:02:30
the performance capture, the lighting acquisition,
00:02:33
the set scanning, because we need
00:02:36
way more accuracy than ever before.
00:02:38
The challenge of creating compelling
00:02:40
and believable digital humans
00:02:42
is really the holy grail of visual effects.
00:02:45
My name's Paul Giacoppo
00:02:46
and I was the digital character model supervisor
00:02:49
for Irishman.
00:02:50
It was important to us that this process not interfere
00:02:53
with the acting or directing in any way.
00:02:56
Knowing that, we decided
00:02:58
what if we can't come up with a system
00:03:00
that actually captures the performance
00:03:02
and doesn’t touch anything else?
00:03:04
The only way to convince anybody
00:03:06
that this could be done is to do a test.
00:03:09
I said why don't we shoot a scene that we all know
00:03:12
that will ground us into the ages
00:03:15
that we want for the movie?
00:03:16
In New York on that test shoot,
00:03:18
Bob came out and got in character,
00:03:21
got in front of the camera, started reenacting
00:03:23
the scene from Goodfellas…
00:03:25
You're gonna get us all pinched,
00:03:26
you fucking bozo.
00:03:27
What's the matter with you?
00:03:28
You're gonna get us all pinched,
00:03:29
you fucking bozo.
00:03:31
What's the matter with you?
00:03:32
Everyone was completely silent, just in awe
00:03:34
watching him reproduce the scene.
00:03:36
Then, all of the sudden we heard booming laughter
00:03:39
and everyone got super nervous, oh my god,
00:03:42
someone's in big trouble, and all of the sudden,
00:03:45
Marty popped up from behind the monitors
00:03:47
and was exclaiming it's just like it was 25 years ago!
00:03:52
When you came in to Netflix and you presented
00:03:54
that test, that first opportunity,
00:03:55
I remember before you walked in the meeting,
00:03:57
I was very sort of doubtful, obviously
00:04:00
the fact that it could be done.
00:04:01
We found with the test that we did,
00:04:03
it really was possible.
00:04:07
So we immediately divided into two groups,
00:04:11
one that was developing the camera system
00:04:13
and the capture itself,
00:04:15
and the other one that kept working on the software.
00:04:17
One thing that was very important was that
00:04:19
Marty would not feel restricted in any way
00:04:22
in the way he shot this film
00:04:23
because of the technology.
00:04:25
To do that, we developed this new rig,
00:04:28
which is the camera surrounded by
00:04:30
two what we call witness cameras,
00:04:32
which are infrared cameras.
00:04:34
And so if you if you were to take a look
00:04:35
at those infrared images,
00:04:36
what you would see is that there's no shadows.
00:04:39
Every one of the cameras is giving you
00:04:41
a different point of view
00:04:43
of that specific performance,
00:04:45
and the more point of views you have,
00:04:47
the better your 3D translation.
00:04:50
Great.
00:04:52
We shot a lot of the movie with two
00:04:54
and sometimes three cameras simultaneously at least,
00:04:57
with the complication of this technology.
00:04:59
We had to test different rigs, different materials,
00:05:01
cables, to figure out how to make a rig
00:05:04
with three cameras that you could operate.
00:05:08
Super balanced.
00:05:09
We designed it to be rigid but
00:05:11
at the same time allow us the flexibility
00:05:13
to pan and tilt each of the cameras independently.
00:05:16
It was called the Three Headed Monster.
00:05:18
It didn't look like a monster to me though,
00:05:19
it was kind of nice. Three sets of different videos
00:05:21
and I'm just trying to deal with the actors.
00:05:26
The biggest challenge I think was to try
00:05:28
to distill what their younger selves
00:05:30
looked like in this film.
00:05:32
See if I can give you a hand.
00:05:35
We spent about two years researching
00:05:37
footage for the three actors.
00:05:40
ILM downloaded a ton of their films so that we could
00:05:43
study and research them and take screen grabs
00:05:46
and just collect this huge library of their work.
00:05:50
We went through an entire library of scenes
00:05:53
extracted from all of their past films
00:05:56
across a spectrum of ages.
00:05:58
We realized that there was no
00:06:00
one stereotypical Robert De Niro,
00:06:02
there was no one idealized Joe Pesci.
00:06:04
They bring something to the performance
00:06:07
that changes the way they look.
00:06:10
So we needed to actually come up
00:06:12
with what was their persona
00:06:14
for this movie, for instance Robert De Niro's
00:06:17
around like mid 30s when he was in Taxi Driver
00:06:19
and Deer Hunter, two very iconic roles for him,
00:06:22
and even if you look at him in those movies,
00:06:25
he looks slightly different,
00:06:27
he took on a persona for each of those movies.
00:06:29
Keep in mind that we were not looking to re-create
00:06:32
younger versions of the actors,
00:06:33
but rather new creations that are
00:06:35
younger versions of the characters.
00:06:38
What the fuck is this?
00:06:40
It's empty.
00:06:41
I know it is. I know it is.
00:06:46
A big point of discovery for us was to learn about
00:06:49
how each actor's face performed and how
00:06:52
they used the anatomy of expression to create
00:06:56
the characters that they were playing.
00:06:58
From working with Marty, I knew
00:06:59
it was going to be a very extensive and long movie,
00:07:03
and so we knew we were going to have 1700 shots.
00:07:07
Everything was about performance,
00:07:08
so we had to be incredibly meticulous about
00:07:12
preserving every detail of each of the actors.
00:07:16
We didn't use our traditional animation pipeline,
00:07:18
so we didn't have animators basically re-creating
00:07:21
the performance of the actors in the film.
00:07:25
My name is Douglas Moore,
00:07:26
I'm a layout supervisor here at ILM.
00:07:28
One of the very first things as part of this process
00:07:30
was using our technology called Medusa.
00:07:33
We did a Medusa capture of Robert De Niro,
00:07:36
Pacino, and Pesci.
00:07:39
And what that it is having them sit in a chair,
00:07:41
and they'd go through a series of expressions
00:07:43
that we'd call the facts expressions,
00:07:45
and from that we would
00:07:46
generate a series of models that can move
00:07:49
then from one expression to another.
00:07:53
What the software then would do would solve
00:07:56
the way that animation, those shapes would
00:07:58
blend together on a particular frame.
00:08:01
The software's called Flux, the F stands for facial
00:08:04
and the lux is for the lighting component of it.
00:08:08
With Flux what we were trying to do is
00:08:09
we were trying to capture
00:08:10
the facial performance of the actors.
00:08:13
We would take the images that were shot on set
00:08:14
and we would use our tools to analyze the performance
00:08:17
in order to create a 3D model of what was shot on set.
00:08:21
Flux is able to see how the face is shaded
00:08:24
and how the lighting is hitting it,
00:08:25
and capture the subtle nuances, a little twitch here,
00:08:29
a little wrinkle up in the nose, actually
00:08:31
generate a lot of detail directly from the plate.
00:08:35
Whenever our faces are moving or emoting or such,
00:08:38
there's a lot going on that, as humans,
00:08:41
our eyes and our brains know about but we ourselves
00:08:45
don't actually think about.
00:08:47
As your muscles move in your face and contract and relax,
00:08:51
you're actually changing how blood flows
00:08:53
through your face, and it's something
00:08:54
that we had to work into our assets
00:08:56
so that when they were actually talking and emoting,
00:08:59
there was some life to their face
00:09:00
that we wouldn't have had otherwise.
00:09:03
We also studied many details like the pores of the skin,
00:09:07
the way light scatters through skin,
00:09:09
details like De Niro's classic mole,
00:09:11
that characteristic feature, and each one of them
00:09:15
we tried to create their most realistic younger self
00:09:19
according to every piece of information
00:09:21
and photographic and filmic evidence we could find.
00:09:24
Not being required, for an actor, to have to wear
00:09:27
all this bulky gear and, you know, all these things
00:09:29
that distract from the performance,
00:09:31
I think that technology is a game changer
00:09:34
for these kinds of movies.
00:09:35
Everybody's going to appreciate it.
00:09:37
It's a good thing.
00:09:39
It becomes a matter of taking all of these different
00:09:42
tools in our pipeline and putting them together
00:09:46
to solve these facial performances and then
00:09:48
also take the solved facial performances
00:09:50
and put them on their younger selves.
00:09:55
The re-targeting part. Ah, damn, so difficult.
00:09:59
The camera is making sense out of the pixels
00:10:01
that it sees and the lighting and the texture
00:10:04
and the infrared light. and you know,
00:10:06
it's all kinds of, you know, a bunch of calculations.
00:10:10
Then you take that and you make it look like
00:10:12
a younger person, there's all kinds of
00:10:15
different things going on
00:10:16
there where you actually design the character,
00:10:19
you pick up what's important, what is not,
00:10:21
you know, what you want
00:10:23
that behavior and likeness to be.
00:10:25
And that is, like I said, not a math problem,
00:10:28
it's an imagination problem.
00:10:31
If we'd had to do this all work by hand,
00:10:32
there's no way we could have done it
00:10:33
to the level of detail that we did,
00:10:36
because an animator isn't going to get
00:10:38
all the details that we were able to get
00:10:40
from these tools
00:10:42
There was not an animator on this show,
00:10:44
we had the Flux team and that was really it.
00:10:47
With this new system, we were trying different
00:10:50
disciplines of people. We started thinking,
00:10:53
well what is this similar to?
00:10:55
It is very technical, so we pulled in creature TDs
00:11:00
and layout artists and effects artists that normally
00:11:02
do particle sim, and they all brought
00:11:04
something different to it.
00:11:07
The first reaction is like,
00:11:09
what so am I looking at?
00:11:12
There's a lot of details that hopefully if we've done
00:11:14
our work well, will be invisible.
00:11:18
When you actually get them in a shot
00:11:20
and you see them moving and talking
00:11:22
and acting and it works,
00:11:24
it's just an amazing feeling.
00:11:27
It's a treat to know that
00:11:29
what we're producing is something
00:11:31
that is not only just a real recreation
00:11:33
of the moment and the scene and the motion,
00:11:35
but yeah that somebody like Martin Scorsese
00:11:37
can look at that and say yeah, this is what I was looking for,
00:11:39
this is what I was going for,
00:11:41
this is the Robert De Niro young version that I remember.
00:11:44
I think it's the first time this kind of thing is done
00:11:47
for this type of narrative film,
00:11:49
this type of character film.
00:11:51
I think this is gonna be another one of those things
00:11:54
that really does change the way films
00:11:57
can be approached and the way actors can play
00:11:59
themselves at different ages in a way that just
00:12:01
wasn't possible before.
00:12:03
I think I would want these to be
00:12:05
a referendum on technology.
00:12:10
If after they see this movie a bunch of actors
00:12:12
say, you know what? I'm not wearing any markers.
00:12:16
Right.
00:12:16
Why do I have to wear markers?
00:12:20
Why do I have to be in an environment where
00:12:22
it's not conductive to my job,
00:12:24
Right.
00:12:24
my performance?
00:12:26
That is the achievement that we want to portray.
00:12:30
No one has taken audiences on a journey
00:12:32
quite like Martin Scorsese with The Irishman,
00:12:36
but as with every innovation in film,
00:12:39
the technology is only as powerful as
00:12:41
the performances that push the story forward,
00:12:44
and this story
00:12:45
can be felt in every frame.
