00:00:00
I've been experimenting with AI video
00:00:01
tools and covering them on this channel
00:00:03
for a long time and right now is the
00:00:06
most exciting and fun time in AI video
00:00:09
yet Runway and Luma Labs have been in
00:00:10
the spotlight and I'll cover some
00:00:12
immediate real world use cases for them
00:00:14
but my personal favorite tool is
00:00:16
actually not one of those making the
00:00:17
headlines I'll get to that a little
00:00:18
later plus some amazing lip syncing
00:00:20
tools and some open source models
00:00:22
there's been a lot of advancements I've
00:00:24
got to start with Runway gen 3 since
00:00:26
it's been dominating my timeline it is
00:00:28
the best textto video model available to
00:00:30
use right now something that's
00:00:31
particularly useful with Gen 3 is text
00:00:33
like these title sequences it is amazing
00:00:35
at this I'll show a few of my favorite
00:00:37
examples I've
00:00:39
[Music]
00:00:40
seen this is such a perfect title
00:00:43
sequence for a cooking
00:00:45
channel this one also showcases how good
00:00:48
it is at fluid simulation the physics in
00:00:50
a lot of these is really good adding
00:00:52
some sound design would take these to
00:00:53
the next level for sure and be
00:00:55
production ready title sequences so I'll
00:00:57
try one for future pedia I'm logged in
00:00:59
already and here's a whole Suite of
00:01:00
image and video tools that Runway offers
00:01:02
but for Gen 3 I'll just click right here
00:01:05
then type in a prompt a title screen
00:01:07
with Dynamic movement the scene starts
00:01:09
with intricate neon circuitry patterns
00:01:11
that light up and move on a dark
00:01:13
background suddenly the circuits
00:01:14
converge and form the word future pedia
00:01:16
with a glowing pulsating effect then I
00:01:18
can choose between 5 seconds and 10
00:01:20
seconds I'll lower it to five then click
00:01:22
generate that's all there is to it right
00:01:24
now this is how that came
00:01:27
back and that looks amazing since
00:01:31
futurepedia is a longer word I have
00:01:33
gotten some misspellings but it gets it
00:01:34
right A lot of the time this prompt
00:01:37
structure has worked really well so I
00:01:38
used one of the sample prompts in the
00:01:40
Gen 3 prompting guide and modified it I
00:01:42
modified it a lot really just kept the
00:01:43
basic structure so I'd recommend trying
00:01:45
to use the prompt structure they give
00:01:46
here when you're starting out camera
00:01:48
movement establishing scene then
00:01:50
additional details hopefully that will
00:01:51
help you cut down on rolls they do also
00:01:53
have a lot of solid keywords in this
00:01:55
guide to help give you some inspiration
00:02:00
another thing it's really good at is
00:02:01
transforming between scenes this is
00:02:03
another prompt you can modify for
00:02:05
yourself that comes back with great
00:02:06
results pretty
00:02:08
[Music]
00:02:12
consistently I will copy that and paste
00:02:15
it in
00:02:16
here I'll change this to a wormhole and
00:02:19
into an alien civilization
00:02:21
[Music]
00:02:30
that's not bad with some rerolls and
00:02:32
modifying The Prompt I'm sure I could
00:02:33
get a better generation here's a couple
00:02:35
other examples using this same prompt
00:02:37
[Music]
00:02:43
[Music]
00:02:48
[Music]
00:02:54
idea now I've also had a lot of Misses
00:02:57
with runaway 2 that's the biggest caveat
00:02:59
here the amazing results you see posted
00:03:01
are definitely cherry-picked that's the
00:03:03
case with all AI video models including
00:03:05
Sora but Runway can do some amazing
00:03:07
things especially if you have a lot of
00:03:08
credit here's a few more of my favorite
00:03:10
Runway examples this was really hard to
00:03:12
narrow down
00:03:13
[Music]
00:03:33
stop
00:03:36
[Music]
00:04:11
the best image to video right now is
00:04:13
dream machine from Luma Labs you can
00:04:15
also do text to video which gets some
00:04:16
good results but where Luma shines is
00:04:18
image to video and even better than that
00:04:20
is with key frames I'll show a couple
00:04:21
straight image to video examples first
00:04:23
it's easy to use upload an image then
00:04:26
add a prompt volcano erupting contained
00:04:29
within a drinking glass surrounded by
00:04:31
peaceful
00:04:33
Tranquility this was the original image
00:04:35
then here's the result and that looks
00:04:38
perfect I'll show a few more straight
00:04:40
image to video examples then we'll move
00:04:41
on to key
00:04:53
frames these are all within one or two
00:04:55
tries it does weird things on occasion
00:04:57
like this astronaut grows in extra
00:05:00
finger or here's a weirder
00:05:05
one or how about a much weirder
00:05:10
one most of the time they do look good
00:05:12
without having to do too many rolls the
00:05:15
next step is adding an ending frame and
00:05:16
you can start doing some really cool
00:05:17
stuff you upload a starting frame then
00:05:20
also upload an ending frame now add a
00:05:23
prompt for it to use to create the whole
00:05:24
clip in between those frames this works
00:05:26
most consistently between things that
00:05:28
are logical to transition between if
00:05:30
they're not then try to be extra
00:05:31
descriptive of what you want to happen
00:05:33
in between it works great for
00:05:35
transformation shots that are in the
00:05:36
same or a similar
00:05:39
scene but you can also do more out there
00:05:45
[Music]
00:05:55
transitions this one looks like it could
00:05:57
legitimately be a Pixar movie
00:06:01
you can extend that out by just taking
00:06:03
your ending frame and using that as the
00:06:05
starting frame for a new generation then
00:06:07
adding a new ending frame you can do
00:06:09
really long sequences this way there's
00:06:10
been these long mashups of memes that
00:06:12
people have been posting which are fine
00:06:15
I am definitely more interested in the
00:06:16
creative stuff this with this girl and
00:06:18
the Wolf is one of the coolest I've seen
00:06:20
using that
00:07:04
it's amazing to keep that all consistent
00:07:05
and there's a ton of possibilities for
00:07:07
this you can do this for free but the
00:07:09
generations take a long time like
00:07:11
multiple hours when I tried before
00:07:12
signing up paying will get you to the
00:07:14
front of the line and a lot more
00:07:16
Generations there's one more out of
00:07:17
China that's similar in quality to these
00:07:19
two called cling however there is a huge
00:07:21
weight list for it and the process to
00:07:23
sign up is difficult without a Chinese
00:07:25
phone number so I'm going to cover that
00:07:26
one at the end moving on to LTX Studio
00:07:29
they they have the most control out of
00:07:31
any of these platforms and the fastest
00:07:33
speed they can build out an entire short
00:07:35
film in a few minutes how it works is
00:07:37
you can start from scratch input an
00:07:39
entire script or start from a simple
00:07:40
prompt I'll go with the prompt option
00:07:42
for this example I will paste this in
00:07:45
it's about a futuristic city controlled
00:07:46
by AI entities and a hacker that can
00:07:49
communicate with them he joins a
00:07:50
resistance they battle all that good
00:07:52
stuff hit next and it starts
00:07:55
working it will show the basic story and
00:07:58
the cast then you can select a style so
00:08:00
this is Cinematic but let's say I want
00:08:02
an anime I can direct it more with words
00:08:05
as well I'll add
00:08:07
cyberpunk there's all sorts of other
00:08:10
styles here to
00:08:15
try I like comic book it will use these
00:08:18
as consistent characters throughout the
00:08:20
story which is amazing because that's
00:08:22
usually a struggle and I can change
00:08:24
anything about them that I want I think
00:08:25
this will be better without a last name
00:08:27
I could change the essence appearance
00:08:29
the clothes I can test out different
00:08:31
voices if I want as an organizer I start
00:08:34
from where the world is as it is not as
00:08:37
I would like it to be you can even face
00:08:39
swap these with a face you upload I want
00:08:42
to change the title to how about Neon
00:08:44
Flux see how many people know that
00:08:45
reference to the anime not the movie
00:08:47
they made from it this looks good here
00:08:49
so I'll click Start and just from that
00:08:51
it builds out an entire short film with
00:08:53
fully editable
00:08:55
scenes this whole thing was generated in
00:08:58
less time than it took either of the
00:08:59
others to generate a single clip and I
00:09:01
want to show the style reference feature
00:09:02
again now that this is open because it's
00:09:04
even cooler to do it in here I'll try
00:09:06
uploading my own style reference so this
00:09:07
is a mid-journey image when I upload
00:09:10
that style every shot across every scene
00:09:12
regenerates in the new style that's
00:09:14
super helpful and doing it here will
00:09:16
give you a much better idea of how it
00:09:17
will look and that looks awesome I'm
00:09:19
actually going to run with that style
00:09:21
now these are just static images
00:09:22
currently I could generate them all into
00:09:24
videos at once or open each one to
00:09:26
customize it that gives you the most
00:09:27
flexibility so I'll do that I can
00:09:29
regenerate the image or upload my own
00:09:31
even generate or remove objects from the
00:09:33
image I'll just get rid of this little
00:09:35
blob
00:09:36
[Music]
00:09:38
here I can choose from four types of
00:09:40
motion and adjust how much of it most of
00:09:42
the options are pretty straightforward
00:09:43
but this one is really amazing when you
00:09:45
switch to custom then orbit then you can
00:09:47
have total control over the start and
00:09:49
end of the shot and it will generate the
00:09:51
motion in
00:09:52
[Music]
00:09:56
between you can also adjust how long the
00:09:59
clip is I want this to be like a movie
00:10:00
trailer so 3 seconds will be
00:10:05
plenty you can also add music sound
00:10:08
effects it generates a voice over all of
00:10:10
that can be customized as well if it
00:10:11
hasn't hit home yet everything can be
00:10:13
customized I'll actually change the
00:10:14
voiceover a little because this ended up
00:10:16
telling the entire story I didn't
00:10:18
mention in the prompt that it was for a
00:10:19
trailer I don't want to spoil the ending
00:10:21
the track it used is awesome so I'll
00:10:22
keep it that's a lot of the features
00:10:24
probably enough for this showcase so I'm
00:10:25
going to go through and generate each
00:10:27
scene how I want it then come back all
00:10:28
right finished all the scenes it's
00:10:30
really nice that you can export all the
00:10:31
files so I can bring it into premiere
00:10:33
for more fine grained editing the other
00:10:34
really cool option is to export it as a
00:10:36
pitch deck I'll do that as well first
00:10:39
here's the pitch deck it has the story
00:10:41
all of the characters and their
00:10:44
backgrounds the mood board and color
00:10:47
palette then the full story board people
00:10:49
have already made some amazing things
00:10:51
this way where they storyboarded a short
00:10:52
film with LTX then actually filmed and
00:10:55
produced the entire thing they posted
00:10:57
this one called winged on X where this
00:10:59
turns into this like human head with
00:11:01
bird wings it's pretty good you should
00:11:03
go watch
00:11:04
it then with this premere option it
00:11:07
gives me the project file that I can
00:11:08
edit further I already opened that and
00:11:10
synced things up to the music a little
00:11:11
more and made just a couple other simple
00:11:13
edits here is the final result in the
00:11:16
neon lit City I discovered a secret
00:11:18
power the ability to communicate with AI
00:11:23
Spirits joining the resistance I became
00:11:26
a spirit Guardian fighting alongside
00:11:28
comrades unique Tech
00:11:32
Powers we Face a rogue AI threatening
00:11:35
our City's balance blending ancient
00:11:37
wisdom with cuttingedge technology as
00:11:39
the battle intensifies the fate of our
00:11:41
world hangs in the
00:11:45
balance I've been having a ton of fun
00:11:47
with LTX studio and side note I did
00:11:49
actually meet some people from the team
00:11:51
at the Imagine AI conference a while
00:11:52
back at one point they were just giving
00:11:53
a little demo at their booth and like
00:11:55
everyone crowded around to watch they
00:11:57
really stole the show make sure to sign
00:12:00
up for the weit list they're letting
00:12:01
lots of people in the link is in the
00:12:02
description and thank you to LTX for
00:12:04
sponsoring this video the platform I've
00:12:06
been having the most fun with out of all
00:12:08
of these is Korea and with this one you
00:12:09
can do quite a bit for free it's a lot
00:12:11
different than the other ones we've
00:12:12
covered it's more for abstract stuff not
00:12:14
so much for realism so it focuses more
00:12:17
on these like trippy morphing type
00:12:18
animations which I like a lot personally
00:12:21
I like to use AI to create things that
00:12:23
would be impossible to create any other
00:12:25
way to open up entirely new creative
00:12:27
Avenues rather than replace EX in ones
00:12:29
so I have a ton of fun in Kaa I'll go up
00:12:31
to generate then video they also have a
00:12:34
creative upscaler that's really useful
00:12:36
I'll show that one next these are the
00:12:37
three images I want to use they're these
00:12:39
translucent kind of bioluminescent
00:12:41
flower jellyfish and dragon I think they
00:12:44
look cool together so I'll click add key
00:12:46
frame select the flower then I'll add
00:12:49
another key frame for the
00:12:51
jellyfish and another for the dragon you
00:12:55
can make these longer or shorter I'll
00:12:57
lengthen it a little bit
00:12:59
then you can add a text prompt I just
00:13:01
want these to morph into each other so I
00:13:03
don't need any longer prompt I'll just
00:13:05
say what they are then drag the length
00:13:07
of the prompt and that's where the
00:13:08
transitions will start at least that's
00:13:10
where they're supposed to it's not
00:13:11
perfect there's a couple other settings
00:13:13
for the aspect ratio motion intensity I
00:13:16
like it around 60 usually then you can
00:13:18
switch the looping on or off it defaults
00:13:20
to on where the end will morph back into
00:13:22
the first frames so it can just
00:13:23
endlessly Loop then you have four styles
00:13:25
to choose from I'll start with film then
00:13:27
click generate video and I'll generate
00:13:29
one at each of those for comparison in a
00:13:30
second this will do a first pass at
00:13:32
generating then give a preview at a low
00:13:34
quality that'll give you a good idea
00:13:36
what it will end up looking like if it's
00:13:38
not going in the direction you'd like
00:13:39
you can go to where it says enhancing
00:13:41
and select click to cancel that will
00:13:43
save you some generation
00:13:44
time now I actually think it's moving
00:13:46
away from the jellyfish too fast so I
00:13:48
will cancel I'll add another key frame
00:13:50
and select the jellyfish again so it
00:13:52
hangs out on that for longer then we'll
00:13:54
generate again that cancel option is
00:13:56
especially helpful if you're on the free
00:13:57
plan since your Generations are more
00:13:59
limited I am on the basic plan which
00:14:01
gets plenty of time for me in about a
00:14:02
minute it's all done and here's what I
00:14:05
[Music]
00:14:08
got I think that looks really cool now
00:14:11
I'll generate with render animation and
00:14:13
experimental so you can see the
00:14:14
difference
00:14:18
[Music]
00:14:33
I think these are amazing really fun and
00:14:35
unique here's another quick example I
00:14:37
did with these
00:14:42
[Music]
00:14:47
images I am working on some longer stuff
00:14:50
with tracks from suo but they're not
00:14:51
finished yet so here's one more weird
00:14:54
one I did
00:15:02
I love this style I know it's not for
00:15:03
everyone but it's really great for
00:15:04
something like a music video or an
00:15:06
opening sequence but these are a couple
00:15:08
of my favorite examples I found on x
00:15:14
[Music]
00:15:26
[Music]
00:15:29
oh
00:15:34
[Music]
00:15:41
[Music]
00:16:00
now I also want to demo their video
00:16:01
upscaler this is not like a traditional
00:16:03
upscaler where it resembles the original
00:16:05
video as much as possible just at a
00:16:07
higher resolution it does a creative
00:16:09
upscale similar to how magnific works
00:16:11
but for video it stays really close but
00:16:13
kind of reimagines everything with AI so
00:16:16
I'll use a clip from LTX video I made
00:16:18
this one has a face that was just
00:16:19
completely warped we'll see if it can
00:16:21
fix
00:16:22
that I don't need it upscaled that much
00:16:25
I'll leave that at 1.5 frame rate 30 FPS
00:16:28
write out a prompt of what it thinks is
00:16:30
in the video that looks good so I'll
00:16:31
leave the strength and resemblance at
00:16:33
the defaults first and start with
00:16:35
cinematic turn Loop off and enhance this
00:16:39
took around 30 seconds and here's the
00:16:41
result you can see that there's a face
00:16:42
on here now that's pretty amazing it was
00:16:45
able to fix that I ran one at each of
00:16:47
the different presets so you can see the
00:16:48
difference cinematic and render are
00:16:50
fairly similar then animated makes it
00:16:52
more of an anime style I'll show that
00:16:54
with a few clips from that LTX video
00:17:03
[Music]
00:17:05
I hope to see them expand that into more
00:17:07
Styles because it is very useful overall
00:17:09
I think Kaa is a ton of fun and there's
00:17:11
some additional options to play around
00:17:13
with in here as well lip syncing has
00:17:15
made some big improvements recently
00:17:16
there's tons of impressive demos that
00:17:18
have come out that we don't have access
00:17:19
to use but there's two platforms I want
00:17:21
to show that we do have access to one
00:17:23
that's completely free and one that's
00:17:24
free for five uses per day it's hedra
00:17:27
and live portrait hedra has some of the
00:17:29
most expressive talking avatars I've
00:17:31
seen it's pretty easy to use so either
00:17:32
generate the audio or upload some of
00:17:34
your own I'll use the classic Fight Club
00:17:36
line now I'll upload a character you can
00:17:38
also generate one here if you need now I
00:17:40
just generate the video it works pretty
00:17:42
quickly and here's what I
00:17:45
got the first rule of Fight Club is you
00:17:48
do not talk about Fight
00:17:50
Club second rule of Fight Club is you do
00:17:54
not talk about Fight
00:17:57
Club that looks great super expressive
00:18:00
let's do another one but with a
00:18:01
non-human character this time I'll use
00:18:03
the line from
00:18:08
Taken what I do have are a very
00:18:12
particular set of skills skills I have
00:18:14
acquired over a very long career skills
00:18:17
that make me a nightmare for people like
00:18:21
you so that one looks great but it can
00:18:23
definitely struggle with non-human
00:18:25
characters and the less human they look
00:18:27
the harder it is a dog just didn't work
00:18:28
at all or even some of those gnomes
00:18:30
especially the ones with a beard it was
00:18:31
having a hard time with you do not talk
00:18:34
about Fight Club Runway also has a lips
00:18:37
syncing feature that they implemented it
00:18:38
works really well very high quality but
00:18:40
it's much less expressive the first rule
00:18:42
of Fight Club is you do not talk about
00:18:45
Fight
00:18:45
Club second rule of Fight Club is you do
00:18:49
not talk about Fight
00:18:52
Club live anime is also amazing but
00:18:55
takes a different route you upload a
00:18:56
reference video and it will map that
00:18:58
onto the Avatar this allows you to have
00:19:00
more control over the expressiveness
00:19:01
it's on hugging face so you can use it
00:19:03
for free upload a source portrait I'll
00:19:05
give my face a try then upload a driving
00:19:08
video it works best with a straight on
00:19:09
shot like the ones they have in these
00:19:11
examples I'll just use one of those
00:19:12
here's a pretty classic one although I
00:19:14
won't be able to use the song that goes
00:19:15
along with it on YouTube now click
00:19:18
animate and in under 30 seconds here's
00:19:21
the
00:19:24
result that worked pretty perfectly I'll
00:19:26
show an example that didn't work this is
00:19:28
a CLI clip from Breaking Bad it shows
00:19:30
his face the whole time but he does move
00:19:32
a bit so with that it's just not able to
00:19:34
map that onto my face at all so there's
00:19:35
limitations here but you can still do
00:19:37
some pretty great stuff with it things
00:19:38
you would never be able to get through
00:19:39
other methods currently and really quick
00:19:41
here I want to shout out the open source
00:19:43
community and the tools and workflows
00:19:45
they've been developing and pioneering
00:19:47
those have really been the foundation of
00:19:48
some of these other tools that will
00:19:49
mimic or outright take those workflows
00:19:52
and Implement them into a paid model the
00:19:53
things you can get from Korea and some
00:19:55
of these other video models are possible
00:19:57
with open source tools most notably
00:19:59
comfy UI and animate diff that method is
00:20:01
significantly more complicated although
00:20:03
it offers more customization and control
00:20:05
the paid platforms tend to offer a nice
00:20:07
UI and ease of use so far all of these
00:20:09
examples are from James G not sure if
00:20:12
that's how you say his last name he has
00:20:13
so many amazing Creations on Instagram
00:20:15
I'm only showing short clips from them
00:20:17
they're all much longer and have music
00:20:19
to them as well so I have been following
00:20:20
his work for a while and it is some of
00:20:22
my
00:20:24
favorite this marble statue dance was
00:20:26
one of his really early ones to go viral
00:20:28
his Instagram is full of amazing stuff
00:20:30
but I'll show some other people too this
00:20:32
dancing spider is
00:20:35
[Music]
00:20:38
awesome these have all been from driving
00:20:40
these animations using real footage but
00:20:42
that's not the only way to use this at
00:20:44
all this one by Fab dream going through
00:20:46
the evolution of money was
00:20:52
[Music]
00:20:53
great or these weird frog animations
00:20:56
from boltron I love these
00:21:00
[Music]
00:21:03
to learn more about how to do this I am
00:21:05
not the person to be teaching it but if
00:21:06
you want to do a deep dive to install
00:21:08
and run it locally and learn how to do
00:21:09
all of that I'll have a couple resources
00:21:11
in the
00:21:13
description cling is the platform that's
00:21:15
as good or better than Runway and dream
00:21:17
machine with text to video and image to
00:21:19
video but the weight lless is huge and
00:21:21
the process is more complicated if you
00:21:22
don't have a Chinese phone number so I'm
00:21:24
going to link to a thread by tile AI he
00:21:27
walks through the process of downloading
00:21:28
a couple apps and getting a burner
00:21:30
number to use I was able to sign up for
00:21:31
the wait list using my own number on
00:21:33
their mobile app but then the new
00:21:34
website doesn't have an option for a
00:21:36
non-chinese number to sign in so it is
00:21:38
probably best to use his method and fair
00:21:40
warning I signed up for the wait list
00:21:42
about 2 weeks ago it said there were
00:21:43
around 500,000 people on it I still
00:21:46
don't have access and now the wait list
00:21:48
says it has around a million people on
00:21:50
it so it is free to use right now but I
00:21:52
imagine that will change by the time
00:21:53
people that are just now signing up for
00:21:55
the wait list get access so it may not
00:21:56
be worth the hassle that's why I didn't
00:21:58
want to spend all time going over that
00:21:59
here follow that thread if you want to
00:22:00
try but the quality coming out of cing
00:22:03
is amazing
00:22:07
[Music]
00:22:42
we are finally to a point where you can
00:22:44
use AI to make video that's usable in
00:22:46
the real world outside of memes not
00:22:48
usable for everything of course there's
00:22:49
still plenty of limitations these tools
00:22:51
have come a very long way and are a ton
00:22:53
of fun right now if you want to keep up
00:22:54
with everything in AI make sure to visit
00:22:56
futurepedia find the best AI tools stay
00:22:59
up to date with Innovations learn how to
00:23:01
use it all with curated tutorials
00:23:03
there's tons of new features on the site
00:23:05
so check it out if you haven't for a
00:23:06
while then subscribe so more videos like
00:23:08
this show up on your homepage thank you
00:23:10
so much for watching I'll see you in the
00:23:12
next one
![Nervous System & The Neuron [AP Psychology Unit 2 Topic 3]](https://ruhnrawpgxrzdrgaohnf.supabase.co/storage/v1/object/public/images/video/cVwHaPvIFkg/thumbnail.jpg)