00:00:04
So anyone who's been paying attention
for the last few months
00:00:08
has been seeing headlines like this,
00:00:10
especially in education.
00:00:12
The thesis has been:
00:00:14
students are going to be using ChatGPT
and other forms of AI
00:00:18
to cheat, do their assignments.
00:00:20
They’re not going to learn.
00:00:21
And it’s going to completely undermine
education as we know it.
00:00:25
Now, what I'm going to argue today
00:00:27
is not only are there ways
to mitigate all of that,
00:00:29
if we put the right guardrails,
we do the right things,
00:00:32
we can mitigate it.
00:00:33
But I think we're at the cusp of using AI
00:00:35
for probably the biggest
positive transformation
00:00:40
that education has ever seen.
00:00:42
And the way we're going to do that
00:00:44
is by giving every student on the planet
00:00:47
an artificially intelligent
but amazing personal tutor.
00:00:51
And we're going to give every teacher
on the planet an amazing,
00:00:54
artificially intelligent
teaching assistant.
00:00:57
And just to appreciate
how big of a deal it would be
00:01:01
to give everyone a personal tutor,
00:01:04
I show you this clip
00:01:07
from Benjamin Bloom’s 1984
2 sigma study,
00:01:10
or he called it the “2 sigma problem.”
00:01:12
The 2 sigma comes
from two standard deviation,
00:01:14
sigma, the symbol for standard deviation.
00:01:16
And he had good data that showed
that look, a normal distribution,
00:01:20
that's the one that you see
in the traditional bell curve
00:01:23
right in the middle, that's how the world
kind of sorts itself out,
00:01:26
that if you were to give personal
1-to-1 to tutoring for students,
00:01:31
then you could actually get a distribution
that looks like that right.
00:01:34
It says tutorial 1-to-1
with the asterisks,
00:01:36
like, that right distribution,
00:01:37
a two standard-deviation improvement.
00:01:39
Just to put that in plain language,
00:01:41
that could take your average student
and turn them into an exceptional student.
00:01:45
It can take your below-average student
00:01:47
and turn them into
an above-average student.
00:01:50
Now the reason why he framed it
as a problem, was he said,
00:01:54
well, this is all good,
00:01:55
but how do you actually scale
group instruction this way?
00:01:58
How do you actually give it
to everyone in an economic way?
00:02:02
What I'm about to show you is I think
the first moves towards doing that.
00:02:06
Obviously, we've been trying
to approximate it in some way
00:02:08
at Khan Academy for over a decade now,
00:02:10
but I think we're at the cusp
of accelerating it dramatically.
00:02:14
I'm going to show you
the early stages of what our AI,
00:02:17
which we call Khanmigo,
00:02:20
what it can now do
00:02:22
and maybe a little bit
of where it is actually going.
00:02:25
So this right over here
is a traditional exercise
00:02:28
that you or many of your children
might have seen on Khan Academy.
00:02:31
But what's new is that little
bot thing at the right.
00:02:35
And we'll start by seeing one
of the very important safeguards,
00:02:39
which is the conversation is recorded
and viewable by your teacher.
00:02:42
It’s moderated actually by a second AI.
00:02:45
And also it does not tell you the answer.
00:02:47
It is not a cheating tool.
00:02:48
When the student says,
"Tell me the answer,"
00:02:50
it says, "I'm your tutor.
00:02:51
What do you think is the next step
for solving the problem?"
00:02:54
Now, if the student makes a mistake,
and this will surprise people
00:02:57
who think large language models
are not good at mathematics,
00:03:00
notice, not only does it
notice the mistake,
00:03:02
it asks the student to explain
their reasoning,
00:03:05
but it's actually doing what I would say,
00:03:07
not just even an average tutor would do,
but an excellent tutor would do.
00:03:10
It’s able to divine what is probably
the misconception in that student’s mind,
00:03:16
that they probably didn’t use
the distributive property.
00:03:18
Remember, we need to distribute
the negative two
00:03:21
to both the nine and the 2m
inside of the parentheses.
00:03:24
This to me is a very, very, very big deal.
00:03:26
And it's not just in math.
00:03:29
This is a computer programming
exercise on Khan Academy,
00:03:32
where the student needs
to make the clouds part.
00:03:36
And so we can see the student starts
defining a variable, left X minus minus.
00:03:40
It only made the left cloud part.
00:03:42
But then they can ask Khanmigo,
what’s going on?
00:03:44
Why is only the left cloud moving?
00:03:46
And it understands the code.
00:03:48
It knows all the context
of what the student is doing,
00:03:51
and it understands that those ellipses
are there to draw clouds,
00:03:54
which I think is kind of mind-blowing.
00:03:57
And it says, "To make the right
cloud move as well,
00:03:59
try adding a line of code
inside the draw function
00:04:01
that increments the right X variable
by one pixel in each frame."
00:04:05
Now, this one is maybe even more amazing
because we have a lot of math teachers.
00:04:10
We've all been trying to teach
the world to code,
00:04:12
but there aren't a lot of computing
teachers out there.
00:04:15
And what you just saw,
even when I'm tutoring my kids,
00:04:17
when they're learning to code,
00:04:19
I can't help them this well, this fast,
00:04:21
this is really going to be a super tutor.
00:04:25
And it's not just exercises.
00:04:26
It understands what you're watching.
00:04:28
It understands the context of your video.
00:04:30
It can answer the age-old question,
“Why do I need to learn this?”
00:04:33
And it asks Socratically,
"Well, what do you care about?"
00:04:36
And let's say the student says,
"I want to be a professional athlete."
00:04:40
And it says, "Well, learning
about the size of cells,
00:04:43
which is what this video is,
00:04:44
that could be really useful
for understanding nutrition
00:04:47
and how your body works, etc."
00:04:49
It can answer questions, it can quiz you,
00:04:51
it can connect it to other ideas,
00:04:53
you can now ask
as many questions of a video
00:04:55
as you could ever dream of.
00:04:57
(Applause)
00:05:01
Another big shortage out there,
00:05:03
I remember the high school I went to,
00:05:04
the student-to-guidance counselor ratio
was about 200 or 300 to one.
00:05:10
A lot of the country,
it's worse than that.
00:05:12
We can use Khanmigo to give
every student a guidance counselor,
00:05:16
academic coach, career coach, life coach,
00:05:19
which is exactly
what you see right over here.
00:05:22
And we launched this
with the GPT-4 launch.
00:05:25
We have a few thousand people on this.
00:05:27
This isn't a fake demo,
00:05:28
this is really it in action.
00:05:32
And then there is, you know,
00:05:34
things that I think it would
have been even harder,
00:05:36
it would have been a little
science fiction to do
00:05:39
with even a traditional tutor.
00:05:40
We run an online high school
with Arizona State University
00:05:43
called Khan World School,
00:05:45
and we have a student who attends
that online school, based in India.
00:05:48
Her name's Saanvi.
00:05:50
And she was doing a report
on "The Great Gatsby."
00:05:53
And when she was reading
"The Great Gatsby,"
00:05:55
Jay Gatsby keeps looking
at the green light off into the distance.
00:05:58
And she's like, "Why does he do that?"
00:06:00
She did some web searches,
and people have obviously studied this
00:06:03
and commented about the symbolism of that,
00:06:05
but none of it was really
resonating with her.
00:06:08
And then she realized
that she had Khanmigo
00:06:10
and that she could talk
to Jay Gatsby himself.
00:06:13
And so, "Ah, splendid choice, old sport.
00:06:15
I am now Jay Gatsby,
the enigmatic millionaire
00:06:18
from F. Scott Fitzgerald’s classic.”
00:06:20
And so, "Why do you keep staring
at the green light?"
00:06:23
"Ah, the green light, old sport.
00:06:24
It's a symbol of my dreams
and desires, you see.
00:06:27
It's situated at the end
of Daisy Buchanan's dock
00:06:30
across the bay from my mansion.
00:06:32
I gaze at it longingly
as it represents my yearning for the past
00:06:35
and my hope to reunite with Daisy,
the love of my life."
00:06:39
And what was cool is, Saanvi then said,
“I had this long conversation,”
00:06:43
she called him “Mr. Gatsby,”
00:06:45
and at the end she actually apologized
for taking his time,
00:06:48
which I thought was very polite of her.
00:06:51
But you can imagine this unlocks
learning literature, learning ...
00:06:56
You could talk to historical figures.
00:06:58
We're even probably
going to add an activity
00:07:00
you can talk to like,
the Mississippi River.
00:07:02
It brings things to life in ways
that really were science fiction
00:07:06
even six months or a year ago.
00:07:10
Students can get into debates with the AI.
00:07:12
And we’ve got this here
is the student debating
00:07:14
whether we should cancel student debt.
00:07:16
The student is against
canceling student debt,
00:07:18
and we've gotten very clear feedback.
00:07:20
We started running it at Khan World School
in our lab school that we have,
00:07:24
Khan Lab School.
00:07:25
The students, the high school
students especially,
00:07:27
they're saying "This is amazing to be able
to fine-tune my arguments
00:07:30
without fearing judgment.
00:07:32
It makes me that much more confident
00:07:33
to go into the classroom
and really participate."
00:07:36
And we all know that Socratic dialogue
debate is a great way to learn,
00:07:39
but frankly, it's not out there
for most students.
00:07:42
But now it can be accessible
to hopefully everyone.
00:07:48
A lot of the narrative,
we saw that in the headlines, has been,
00:07:51
"It's going to do the writing for kids.
00:07:53
Kids are not going to learn to write."
00:07:55
But we are showing that there's ways
that the AI doesn't write for you,
00:07:58
it writes with you.
00:08:00
So this is a little thing,
00:08:01
and my eight year old is addicted to this,
00:08:03
and he's not a kid
that really liked writing before,
00:08:05
but you can say,
00:08:07
“I want to write a horror story,”
00:08:08
and it says, "Ooh, a horror story,
how spine-tingling and thrilling.
00:08:12
Let's dive into the world of eerie shadows
and chilling mysteries."
00:08:15
And this is an activity where
the student will write two sentences,
00:08:19
and then the AI will write two sentences.
00:08:21
And so they collaborate
together on a story.
00:08:24
The student writes,
"Beatrice was a misunderstood ghost.
00:08:26
She wanted to make friends
but kept scaring them by accident."
00:08:29
And the AI says, "Poor Beatrice, a lonely
spirit yearning for companionship.
00:08:33
One day she stumbled
upon an old abandoned mansion," etc.
00:08:36
I encourage you all
to hopefully one day try this.
00:08:39
This is surprisingly fun.
00:08:42
Now to even more directly
hit this use case.
00:08:45
And what I'm about to show you,
00:08:47
everything I showed you so far
00:08:48
is actually already part of Khanmigo,
and what I’m about to show you,
00:08:52
we haven't shown to anyone yet,
this is a prototype.
00:08:54
We hope to be able to launch it
in the next few months,
00:08:57
but this is to directly use AI,
use generative AI,
00:09:00
to not undermine English
and language arts
00:09:02
but to actually enhance it in ways
00:09:04
that we couldn't have even
conceived of even a year ago.
00:09:08
This is reading comprehension.
00:09:09
The students reading Steve Jobs's
famous speech at Stanford.
00:09:13
And then as they get to certain points,
00:09:15
they can click on that little question.
00:09:18
And the AI will then Socratically,
almost like an oral exam,
00:09:22
ask the student about things.
00:09:24
And the AI can highlight
parts of the passage.
00:09:26
Why did the author use that word?
00:09:28
What was their intent?
00:09:30
Does it back up their argument?
00:09:31
They can start to do stuff
that once again,
00:09:33
we never had the capability
to give everyone a tutor,
00:09:36
everyone a writing coach to actually
dig in to reading at this level.
00:09:41
And you could go on the other side of it.
00:09:43
And we have whole work flows
that helps them write,
00:09:45
helps them be a writing coach,
draw an outline.
00:09:48
But once a student actually
constructs a draft,
00:09:51
and this is where
they're constructing a draft,
00:09:53
they can ask for feedback once again,
00:09:56
as you would expect
from a good writing coach.
00:09:58
In this case, the student
will say, let's say,
00:10:01
"Does my evidence support my claim?"
00:10:03
And then the AI, not only
is able to give feedback,
00:10:05
but it's able to highlight certain parts
of the passage and says,
00:10:08
"On this passage, this doesn't
quite support your claim,"
00:10:11
but once again, Socratically says,
"Can you tell us why?"
00:10:14
So it's pulling the student,
making them a better writer,
00:10:17
giving them far more feedback
00:10:18
than they've ever been able
to actually get before.
00:10:20
And we think this is going to dramatically
accelerate writing, not hurt it.
00:10:25
Now, everything I've talked
about so far is for the student.
00:10:29
But we think this could be equally
as powerful for the teacher
00:10:32
to drive more personalized
education and frankly
00:10:34
save time and energy for themselves
and for their students.
00:10:37
So this is an American history
exercise on Khan Academy.
00:10:40
It's a question
about the Spanish-American War.
00:10:44
And at first it's in student mode.
00:10:47
And if you say, “Tell me the answer,”
it’s not going to tell the answer.
00:10:51
It's going to go into tutoring mode.
00:10:52
But that little toggle
which teachers have access to,
00:10:55
they can turn student mode off
and then it goes into teacher mode.
00:10:58
And what this does is it turns into --
00:11:01
You could view it
as a teacher's guide on steroids.
00:11:03
Not only can it explain the answer,
00:11:05
it can explain how you might
want to teach it.
00:11:08
It can help prepare
the teacher for that material.
00:11:10
It can help them create lesson plans,
as you could see doing right there.
00:11:14
It'll eventually help them
create progress reports
00:11:17
and help them, eventually, grade.
00:11:18
So once again, teachers spend
about half their time
00:11:21
with this type of activity,
lesson planning.
00:11:23
All of that energy can go back to them
00:11:25
or go back to human interactions
with their actual students.
00:11:29
(Applause)
00:11:34
So, you know, one point I want to make.
00:11:37
These large language models
are so powerful,
00:11:39
there's a temptation to say like, well,
00:11:41
all these people are just going
to slap them onto their websites,
00:11:44
and it kind of turns the applications
themselves into commodities.
00:11:47
And what I've got to tell you
00:11:49
is that’s one of the reasons
why I didn’t sleep for two weeks
00:11:51
when I first had access
to GPT-4 back in August.
00:11:55
But we quickly realized
that to actually make it magical,
00:11:58
I think what you saw
with Khanmigo a little bit,
00:12:00
it didn't interact with you
the way that you see ChatGPT interacting.
00:12:03
It was a little bit more magical,
it was more Socratic,
00:12:06
it was clearly much better at math
00:12:08
than what most people
are used to thinking.
00:12:10
And the reason is,
00:12:11
there was a lot of work
behind the scenes to make that happen.
00:12:14
And I could go through the whole list
of everything we've been working on,
00:12:18
many, many people for over six,
seven months to make it feel magical.
00:12:21
But perhaps the most
intellectually interesting one
00:12:24
is we realized, and this was an idea
from an OpenAI researcher,
00:12:27
that we could dramatically improve
its ability in math
00:12:30
and its ability in tutoring
00:12:32
if we allow the AI to think
before it speaks.
00:12:35
So if you're tutoring someone
00:12:36
and you immediately just start talking
before you assess their math,
00:12:39
you might not get it right.
00:12:41
But if you construct
thoughts for yourself,
00:12:43
and what you see on the right
there is an actual AI thought,
00:12:46
something that it generates for itself
but it does not share with the student.
00:12:49
then its accuracy went up dramatically,
00:12:51
and its ability to be a world-class tutor
went up dramatically.
00:12:54
And you can see it's talking
to itself here.
00:12:56
It says, "The student got a different
answer than I did,
00:12:59
but do not tell them they made a mistake.
00:13:01
Instead, ask them to explain
how they got to that step."
00:13:05
So I'll just finish off, hopefully,
00:13:08
you know, what I’ve just shown you
is just half of what we are working on,
00:13:11
and we think this is just
the very tip of the iceberg
00:13:15
of where this can actually go.
00:13:17
And I'm pretty convinced, which I wouldn't
have been even a year ago,
00:13:21
that we together have a chance
of addressing the 2 sigma problem
00:13:25
and turning it into a 2 sigma opportunity,
00:13:28
dramatically accelerating
education as we know it.
00:13:33
Now, just to take a step back
at a meta level,
00:13:35
obviously we heard a lot today,
the debates on either side.
00:13:38
There's folks who take
a more pessimistic view of AI,
00:13:41
they say this is scary,
00:13:42
there's all these dystopian scenarios,
00:13:45
we maybe want to slow down,
we want to pause.
00:13:48
On the other side,
there are the more optimistic folks
00:13:51
that say, well, we've gone
through inflection points before,
00:13:54
we've gone through
the Industrial Revolution.
00:13:56
It was scary, but it all
kind of worked out.
00:13:59
And what I'd argue right now
00:14:01
is I don't think this is like
a flip of a coin
00:14:04
or this is something
where we'll just have to,
00:14:06
like, wait and see which way it turns out.
00:14:09
I think everyone here and beyond,
00:14:11
we are active participants
in this decision.
00:14:14
I'm pretty convinced
that the first line of reasoning
00:14:17
is actually almost
a self-fulfilling prophecy,
00:14:20
that if we act with fear and if we say,
00:14:22
"Hey, we've just got to stop
doing this stuff,"
00:14:25
what's really going to happen
is the rule followers might pause,
00:14:28
might slow down,
00:14:30
but the rule breakers,
as Alexandr [Wang] mentioned,
00:14:32
the totalitarian governments,
the criminal organizations,
00:14:35
they're only going to accelerate.
00:14:36
And that leads to what I am pretty
convinced is the dystopian state,
00:14:40
which is the good actors
have worse AIs than the bad actors.
00:14:45
But I'll also, you know,
talk to the optimists a little bit.
00:14:49
I don't think that means that,
00:14:50
oh, yeah, then we should just relax
and just hope for the best.
00:14:53
That might not happen either.
00:14:55
I think all of us together
have to fight like hell
00:14:59
to make sure that we put the guardrails,
00:15:02
we put in -- when the problems arise --
00:15:05
reasonable regulations.
00:15:07
But we fight like hell
for the positive use cases.
00:15:10
Because very close to my heart,
00:15:12
and obviously there's many
potential positive use cases,
00:15:15
but perhaps the most powerful use case
00:15:17
and perhaps the most poetic use case
is if AI, artificial intelligence,
00:15:22
can be used to enhance HI,
human intelligence,
00:15:26
human potential and human purpose.
00:15:29
Thank you.
00:15:30
(Applause)
