00:00:00
- Thank you to Curiosity Stream
for sponsoring this episode.
00:00:03
Get access to my streaming
video service Nebula
00:00:06
when you sign up for Curiosity Stream
00:00:08
using the link in the description.
00:00:11
Hello, welcome to "Up and Atom". I'm Jade.
00:00:14
Did you know that you
can write any function
00:00:17
just by summing up sine and cosine waves?
00:00:20
Thrilling, isn't it?
00:00:22
This is known as the Fourier series,
00:00:24
and it, along with an accompanying idea,
00:00:27
the Fourier transform,
can be found in everything
00:00:30
from digital music, to quantum mechanics,
00:00:33
to image recognition.
00:00:35
If you've never heard of
the Fourier series before
00:00:38
you might be skeptical.
00:00:40
Can we really write any function
00:00:42
in terms of sine and cosine functions?
00:00:45
They have a very smooth, wavy shape,
00:00:48
so how would you make something
like this square wave,
00:00:50
which has sharp corners?
00:00:54
First, let's take a step back.
00:00:58
What happens when we add
two functions together?
00:01:02
If sine X looks like this
00:01:04
and sine three X looks like this,
00:01:07
a regular sine function
with a smaller frequency,
00:01:10
then sine X plus sine
three X takes the Y values
00:01:14
of each function at each value of X
00:01:17
and adds them together.
00:01:19
For example, at three pi on four,
00:01:22
sine X is equal to one on root two
00:01:25
and sine three X is also
equal to one on root two.
00:01:28
So adding them together gives us root two.
00:01:32
At pi on two,
00:01:33
sine X is one and sine
three X is negative one.
00:01:36
So when we add them together, we get zero.
00:01:40
If we do that with all the values,
00:01:42
this is the resulting graph.
00:01:45
When the original functions
are both positive,
00:01:47
the sum gets bigger.
00:01:49
And when one is positive
and the other negative,
00:01:52
the functions can cancel each other out.
00:01:54
This may already look slightly
closer to a square wave
00:01:57
than a sine wave does.
00:01:59
Now let's try giving sine
three X a smaller amplitude.
00:02:03
Then the added functions
would look like this.
00:02:08
This is already much
closer to the square wave.
00:02:11
By adding more and more terms
00:02:13
it will get closer and
closer to the square wave.
00:02:16
Fourier series are infinite sums,
00:02:18
meaning you can add infinitely many waves
00:02:21
so you will always
reach an exact solution.
00:02:24
Another example is the Fourier
series of a sawtooth wave.
00:02:28
The first few terms are sine X
00:02:31
minus half sine two X
00:02:33
plus 1/3 sine three X.
00:02:36
When we keep adding terms
00:02:37
we can see the sawtooth wave taking form.
00:02:45
Now, you may be wondering,
what's the point?
00:02:47
If it's just a different
way to write the same thing,
00:02:50
why bother?
00:02:51
Especially if it means
dealing with an infinite sum?
00:02:54
Well, in practice, we often
don't need a perfect solution,
00:02:58
we just need a solution
00:02:59
that's good enough for our application.
00:03:01
Like if we had these two functions
00:03:03
and we wanted to write a program
00:03:05
that could tell the
difference between them.
00:03:07
The first few terms of the
square wave Fourier series
00:03:10
are sine X plus zero sine
two X plus 1/3 sine three X,
00:03:16
while the first few terms
of the sawtooth wave
00:03:18
are sine X minus 1/2 sine
two X plus 1/3 sine three X.
00:03:25
Simply by knowing the
amplitude of that second term,
00:03:28
the program would know which was which.
00:03:30
Now, while it comes to the simple example
00:03:32
of finding the difference
00:03:33
between a square wave and a sawtooth wave,
00:03:36
finding the Fourier series
00:03:37
might be more work than it's worth.
00:03:40
But the idea can be extended
00:03:42
to pattern and shape
recognition in general.
00:03:45
If you take two objects like
an apple and a slice of pizza,
00:03:48
their outlines can be
analyzed using this technique
00:03:51
to figure out which is which.
00:03:54
Fourier analysis is an important tool
00:03:56
in image analysis and shape recognition.
00:03:59
The trickiest part of this whole process
00:04:01
is trying to figure out how
to find the Fourier series,
00:04:05
or in other words, which
waves will add up correctly
00:04:08
to make the shape you want.
00:04:10
After all, how would you go
about adding waves together
00:04:12
to make a pizza shape,
00:04:14
or knowing which frequencies
get added into a square wave?
00:04:18
It's almost like asking someone
00:04:20
to figure out the instructions
00:04:21
for how to put individual atoms
together to make your body.
00:04:25
But before we talk about that,
00:04:27
let's look at one more useful
feature of the Fourier series.
00:04:31
Our original square wave function
00:04:32
tells us how the function
changes with time,
00:04:35
while the Fourier series is a
sum of different frequencies.
00:04:39
Now, imagine if you have a recording
00:04:41
with a high pitch sound
that you want to get rid of.
00:04:44
(piano music)
(tone screeching)
00:04:47
If you found the Fourier
series of that recording,
00:04:50
you could just get rid
of the terms of the sum
00:04:52
that are associated with
the high pitched sound,
00:04:55
add the sum back up,
00:04:57
and get your original recording
00:04:58
without the distracting high pitch.
00:05:01
There's a tool associated
with the Fourier series
00:05:04
called the Fourier transform.
00:05:06
It can be used to pick
out which frequencies,
00:05:09
or which sines and cosines,
go into the Fourier series.
00:05:13
More powerfully, it
can be used to pick out
00:05:16
and remove frequencies from a function
00:05:18
as we just did with the audio recording,
00:05:20
even without needing to
find the Fourier series.
00:05:25
The basic idea of the Fourier transform
00:05:27
is that you feed it your
amplitude versus time function
00:05:30
and it spits out the same function
00:05:33
as an amplitude versus frequency function.
00:05:36
This is because the Fourier transform
00:05:38
decomposes the function
into sine and cosine waves
00:05:41
but we only really care
00:05:43
about the amplitude and frequency pairing.
00:05:45
If we were to graph all the
sine and cosine waves like this,
00:05:48
as we have been doing,
that's pretty messy.
00:05:51
We're only interested in the
frequencies and amplitudes,
00:05:55
so let's rewrite the decomposition
00:05:56
in terms of just those two properties.
00:06:02
Here we've just collapsed
each wave into a bar
00:06:04
representing its amplitude on the Y axis
00:06:07
and its frequency on the X axis.
00:06:09
This is how the Fourier series
is typically represented.
00:06:13
So to sum it up, the output
of the Fourier transform
00:06:16
is nothing more than a
frequency domain view
00:06:19
of the original time domain function.
00:06:22
So if I go back to our first few terms
00:06:25
of the Fourier series of a square wave,
00:06:27
we expect sine X zero sine
two X and 1/3 sine three X.
00:06:33
This new function, which
is dependent on frequency,
00:06:36
gets large at the frequency
associated with sine X,
00:06:39
is zero at the frequency
associated with sine two X,
00:06:43
and has a spike at the frequency
00:06:44
associated with sine three X,
though of a smaller amplitude.
00:06:48
To understand how the
Fourier transform figures out
00:06:51
how much of each sine or cosine
is in the Fourier series,
00:06:54
let's look at how the
Fourier transform works.
00:06:57
The mathematical equation
for the Fourier transform
00:07:00
is surprisingly simple, given its power.
00:07:03
Here F of omega is the Fourier transform
00:07:05
from which we get the Fourier series.
00:07:07
Notice how it's the
subject of the equation,
00:07:10
the thing we're trying to find.
00:07:12
So basically, if we do all of this stuff
00:07:15
we can figure out the Fourier series.
00:07:17
So what is all of this stuff?
00:07:20
F of X is the time function
00:07:21
we're calculating the Fourier series for.
00:07:24
Then we have this exponential
term, and an integral.
00:07:27
Why would multiplying the time function
00:07:29
by an exponential term
00:07:31
and taking the integral of that result
00:07:33
give us the sine and cosine waves
00:07:35
that that function is made of?
00:07:37
Well, there's a famous
result in mathematics
00:07:39
called Euler's Formula.
00:07:41
It tells us that you can
write this exponential
00:07:43
in terms of sine and cosine waves.
00:07:46
This is a very beautiful result,
00:07:48
and there are already a lot
of great videos about it
00:07:50
which I've linked in the description.
00:07:51
So that's where the sine and cosine waves
00:07:53
come into the Fourier transform equation.
00:07:56
But still, why would multiplying
the original time function
00:08:00
by this sine and cosine
term and taking the integral
00:08:03
tell us the frequencies
00:08:04
that make up the original time function?
00:08:07
To understand, let's do an example.
00:08:10
Say we want to know whether
a wave with frequency three
00:08:13
is used to make up this square wave,
00:08:15
and if so, how much of it?
00:08:17
Like will we need its full amplitude,
00:08:19
a quarter of its amplitude?
00:08:21
The reason we have an imaginary component
00:08:23
is to handle the general case
00:08:25
where there may be
different phases necessary
00:08:27
to make up the function.
00:08:29
But this square wave starts in phase
00:08:31
with all sine waves at X equals zero,
00:08:33
so we know we won't need any of that.
00:08:36
So without loss of generality,
we can drop the cosine term
00:08:39
and treat the sine term
as the real component.
00:08:43
This term is telling us
00:08:44
to multiply each Y
value of the square wave
00:08:46
with the same Y value of the sine wave.
00:08:49
The value it returns
00:08:50
will tell us how correlated the waves are,
00:08:52
or, more scientifically, how
much they groove together.
00:08:56
Like at pi on six, the square
wave has a value of one
00:09:00
and the sine wave has a
value of one, so we get one.
00:09:03
At pi on four,
00:09:04
we get a positive value less than one.
00:09:07
This tells us that the
waves are more correlated
00:09:09
at pi on six than at pi on
four, which we can see is true.
00:09:14
When the waves are correlated,
00:09:15
the multiplication will always
return a positive value.
00:09:18
When the waves are anti-correlated
00:09:20
the multiplication will
return a negative value.
00:09:23
And when the waves
aren't correlated at all,
00:09:25
it will return a zero.
00:09:27
Now we come to the integral,
00:09:29
which is the continuous version of a sum.
00:09:32
The sum of all of these values
00:09:34
will tell us whether this sine wave
00:09:35
is necessary to build the square wave.
00:09:38
If the sum is positive, the
waves are overall correlated,
00:09:41
and this sine wave is
in the Fourier series
00:09:44
of this square wave.
00:09:45
If the sum is zero, the waves
are not correlated at all,
00:09:48
and none of this wave is used
to make up the square wave.
00:09:52
If the sum is negative,
00:09:53
the waves are overall anti-correlated,
00:09:55
which is kind of like
grooving upside down.
00:09:58
So the negative of the sine wave
00:10:00
goes into the Fourier series.
00:10:03
It turns out that the sum is positive,
00:10:05
so sine three X does indeed
00:10:07
go into the making of this square wave.
00:10:09
But how much of it?
00:10:11
After normalization,
we get a value of 1/3.
00:10:14
This tells us that only 1/3 of
the amplitude of sine three X
00:10:17
is used to make the square wave.
00:10:19
And when we compare with
our earlier example,
00:10:22
we do indeed see the
term 1/3 sine three X.
00:10:26
So to recap, the multiplication tells us
00:10:29
how correlated the waves
are at each time step,
00:10:32
and the integral tells us how correlated
00:10:34
the waves are overall.
00:10:36
Now notice how the square wave is made up
00:10:38
of only odd sine terms, and
it's pretty easy to see why.
00:10:42
For every period of the square wave,
00:10:44
there are always two
more correlation humps
00:10:47
than anti-correlation humps,
00:10:50
so the sum will always
add to a positive number.
00:10:53
Whereas with even frequencies,
00:10:55
the positive and negative
humps exactly cancel out,
00:10:58
leaving us with a sum of zero.
00:11:00
This is also why we don't see
00:11:02
any cosine waves in the series.
00:11:04
The Fourier transform
00:11:05
does this same process for any frequency,
00:11:07
thereby telling us which frequencies
00:11:09
go into any specific function.
00:11:12
I think this process is super beautiful.
00:11:14
You can also think of it
00:11:15
as changing the basis of the function
00:11:18
in an infinite dimensional space.
00:11:20
That was a jargon dump,
so let's break it down.
00:11:23
Just like we can change the
basis of a vector space,
00:11:26
choosing new coordinates to
describe the same vector,
00:11:29
we can also change the
basis in function space.
00:11:32
We started with a function
expressed as amplitudes
00:11:34
at an infinite number of time positions,
00:11:37
and we changed the basis
00:11:39
so that it was described
in terms of amplitudes
00:11:41
at an infinite number of frequency values.
00:11:45
This is only possible because
our sine and cosine waves
00:11:48
make up an orthogonal basis,
00:11:50
which is just a fancy way of
saying they can be combined
00:11:53
to make any function in function space,
00:11:56
the same way an orthogonal vector basis
00:11:58
can be combined to make any
vector in a vector space.
00:12:01
Transforming functions back and forth
00:12:03
between bases is super useful
00:12:05
in lots of areas of math and engineering,
00:12:07
but the pure algebraic
treatment only works
00:12:10
if you have a mathematical description
00:12:11
of the input function.
00:12:13
Often in real world applications
00:12:15
you only have the raw data to work with.
00:12:18
Right now your computer is
using Fourier transforms
00:12:20
to play this video,
00:12:22
but it has to handle the messiness
of the real world's data.
00:12:25
For that, we need to write algorithms.
00:12:28
Algorithms are step by step instructions
00:12:30
that break down a really
complicated process
00:12:33
into small, manageable tasks.
00:12:36
The pattern recognition
00:12:37
and audio engineering example we used
00:12:39
would both need an algorithm to work.
00:12:42
It might amaze you how
many algorithms are working
00:12:45
quietly behind the scenes
in our everyday lives.
00:12:48
There's a great documentary
on Curiosity Stream
00:12:50
called "The Secret
Rules of Modern Living",
00:12:52
which explores how these algorithms work,
00:12:55
from how TV streaming services choose
00:12:58
which shows to recommend to you,
00:13:00
to matching profiles on dating websites,
00:13:02
to saving lives with the best
kidney transplant solution.
00:13:05
I was surprised at just
how simple and clever
00:13:08
some of these algorithms are,
00:13:09
but also at how effective they are
00:13:11
in helping us make the best decisions.
00:13:13
If you're interested in learning more,
00:13:15
you can watch this documentary for free
00:13:18
by signing up to Curiosity Stream
00:13:20
with the link in the description.
00:13:22
Curiosity Stream is an
award-winning documentary service
00:13:25
with thousands of titles,
from space exploration,
00:13:28
to the history of mathematics,
to engineering of the future.
00:13:31
They're also supporting
00:13:32
a bunch of us educational YouTube creators
00:13:35
with our own streaming platform, Nebula.
00:13:37
Where Curiosity Stream
00:13:38
is all about big budget documentaries,
00:13:40
we are building Nebula
because we want a place
00:13:43
for education-y creators to
try out new content ideas
00:13:46
that might not work on YouTube.
00:13:48
I've made a documentary
00:13:49
about whether math is
invented or discovered,
00:13:51
which you can check out if you like.
00:13:53
Curiosity Stream loves
independent creators
00:13:56
and wants to help us grow our platform,
00:13:58
so they're offering "Up and Atom" viewers
00:14:00
free access to Nebula
00:14:01
when you sign up at
curiositystream.com/upandatom.
00:14:05
They're also offering a 26% discount
00:14:08
to "Up and Atom" viewers,
00:14:09
so that's two streaming services
for just $14 79 for a year.
00:14:14
By signing up to Curiosity Stream,
00:14:16
you will be helping not just me,
00:14:18
but the entire educational community.
00:14:21
Thank you for watching,
00:14:21
and thank you to all
my Patreon supporters.
00:14:24
I'll see you in the next episode. Bye.
00:14:27
(techno music)
