英伟达2025 CES炸裂发布:自动驾驶与通用机器人ChatGPT时刻来临!|新闻特写20250107
摘要
TLDR视频中讨论了物理人工智能的未来愿景,主要是通过将AI应用在物理环境中,使其能够生成实际动作而不是文本。Nvidia 宣布了 Cosmos 世界基础模型,这是一种旨在理解物理世界的 AI 模型,并已开放许可进行使用。通过将 Cosmos 与 Omniverse 联合,提供了一个物理基础的多元宇宙生成器,可用于训练和模拟机器人在真实环境中的操作。这些工具的应用范围广泛,包括工业机器人、自主车辆等领域。Nvidia 还介绍了三台关键计算机系统:用于训练 AI 的 DGX 计算机、用于模拟和生成合成数据的 Omniverse 系统,以及用于部署 AI 的 AGX 计算机。此外,Nvidia 还推介了 Thor 处理器,这是一种适用于自动驾驶和其他机器人应用的强大处理器。
心得
- 🔄 物理AI可将AI应用于实际环境执行动作,而非仅生成文本。
- 🌍 Nvidia宣布了Cosmos,一个物理世界基础AI模型。
- 🔗 Cosmos与Omniverse结合实现物理世界的模拟与AI生成。
- 🚗 自动驾驶领域将借助这些技术进行合成数据生成和AI训练。
- 💻 需要三种计算机系统:训练、模拟以及部署。
- 🚀 Thor处理器,具备高处理能力,适合复杂机器人应用。
- 🔍 AI在现实环境中的训练可通过数字孪生技术增强。
- 🛡 Nvidia致力于确保AI系统的功能安全。
- 🌐 这些技术涵盖了广泛的应用领域,如工业机器人和自动驾驶。
- 📈 Nvidia的技术进步推动了物理AI和机器人行业的发展。
时间轴
- 00:00:00 - 00:05:00
在本视频的开头,讲者介绍了一种称为“物理AI”的概念,类似于大语言模型,但应用于物理世界的模型。这种模型不是通过生成文本,而是通过生成行动来响应物理指令。Nvidia推出的新产品“Cosmos”就是一个世界基础模型,旨在理解和模拟物理世界的功能。同时,该模型与Omniverse的结合使生成的内容有真实性和准确性,为机器人技术和工业AI提供新的科技基础。每个机器人系统需具备三台电脑:训练AI的电脑,部署AI的电脑,以及数字孪生模拟电脑。
- 00:05:00 - 00:10:00
接下来,视频讨论了Nvidia在工业领域三台电脑系统的策略:用于训练AI的DGX电脑,负责实际应用的AGX电脑,以及连接两者的数字孪生模拟系统。有诸多汽车公司已经开始与Nvidia合作开发新一代自主车辆,如丰田、特斯拉、沃尔沃等。Nvidia推出新一代汽车处理器“Thor”,相较于其前代产品更加高效,并且具有AI功能安全认证。这些技术进步展示了AI和自动驾驶车辆在工业中的广阔应用前景。
- 00:10:00 - 00:17:34
视频的最后部分展示了如何使用Omniverse和Cosmos来生成用于训练AI的合成数据。这种生成方法允许从现实驾驶场景创建大规模的数据,以支持自动驾驶车辆的发展。此外,还讨论了未来“类通用机器人时代”的来临,以及通过模拟生成训练数据以培养不同类型的机器人能力的可能性。Nvidia在推动机器人和自主车辆技术发展的同时,致力于构建大型世界基础模型,以更好地支持全球工业的转型。
思维导图
视频问答
物理AI是什么?
物理AI是指通过将AI应用于物理世界的模型,以产生实际动作,而不是生成文本。
视频中提到的Cosmos与Omniverse是什么关系?
Cosmos是一个世界基础模型,Omniverse是一个物理基础的模拟系统,二者结合可提供物理模拟和多元宇宙生成器。
Nvidia在视频中宣布了什么新项目?
Nvidia宣布了开源的Cosmos世界基础模型,并展示了与Omniverse的结合。
为何视频中提到的AI需要三个不同的计算机系统?
AI的发展需要训练系统、数字孪生模拟系统和部署系统三者协同工作。
Cosmos模型为何重要?
Cosmos模型可以帮助机器人理解和处理物理动态,进而提升工业AI和机器人技术。
为什么与Omniverse结合是重要的?
Omniverse的物理基础特征提供了AI生成内容的真实感和可靠性。
视频中展示了什么应用案例?
视频展示了自动化车辆如何利用AI和数字孪生技术生成合成数据来增强训练。
Nvidia的Thor处理器有什么特点?
Thor是一种具有高处理能力的通用机器人计算机,适合自动驾驶和其他机器人应用。
如何使用Cosmos和Omniverse训练AI模型?
通过合成驾驶场景生成大量虚拟数据,借助真实世界数据校准和反馈进行模型训练。
Nvidia的新技术在自动驾驶领域的发展前景如何?
Nvidia的技术将推动自动驾驶技术的发展,产生安全性和效率提升,成为一个大型产业。
查看更多视频摘要
- 00:00:00okay let's talk about physical AI So
- 00:00:02Physical
- 00:00:03AI imagine
- 00:00:07imagine whereas your large language
- 00:00:09model you give it your context your
- 00:00:14prompt on the left and it generates
- 00:00:19tokens one at a time to produce the
- 00:00:22output that's basically how it works the
- 00:00:25amazing thing is this model in the
- 00:00:27middle is quite large has billions of
- 00:00:29parameters
- 00:00:30the context length is incredibly large
- 00:00:33because you might decide to load in a
- 00:00:35PDF in my case I might load in several
- 00:00:37PDFs before I ask it a question those
- 00:00:41PDFs are turned into tokens the
- 00:00:43attention the basic attention
- 00:00:45characteristic of a transformer has
- 00:00:47every single token find its relationship
- 00:00:49and relevance against every other token
- 00:00:53so you could have hundreds of thousands
- 00:00:55of tokens and the computational load
- 00:00:58increases quadratically
- 00:01:00and it does this all of the parameters
- 00:01:03all of the input sequence process it
- 00:01:05through every single layer of the
- 00:01:06Transformer and it produces one token
- 00:01:09that's the reason why we need a
- 00:01:10Blackwell and then the next token is
- 00:01:13produced when the current token is done
- 00:01:16it puts the current token into the input
- 00:01:18sequence and takes that whole thing and
- 00:01:21generates the next token it does it one
- 00:01:22at a time this is the Transformer model
- 00:01:26it's the reason why it is so so
- 00:01:28incredibly effective computationally
- 00:01:31demanding What If instead of PDFs it's
- 00:01:35your surrounding and what if instead of
- 00:01:37the prompt a question it's a request go
- 00:01:40over there and pick up that you know
- 00:01:42that box and bring it back and instead
- 00:01:44of what is produced in tokens that's
- 00:01:46text it produces action
- 00:01:49tokens well that I just described is a
- 00:01:54very sensible thing for the future of
- 00:01:56Robotics and the technology is right
- 00:01:58around the corner but what we need need
- 00:02:00to do is we need to create the effective
- 00:02:03effectively the world
- 00:02:05model of you know as opposed to GPT
- 00:02:09which is a language model and this world
- 00:02:11model has to understand the language of
- 00:02:13the world it has to understand physical
- 00:02:15Dynamics we know that most models today
- 00:02:19have a very hard time with and so we
- 00:02:21would like to create a world we need a
- 00:02:24world Foundation model today we're
- 00:02:25announcing a very big thing we're
- 00:02:28announcing Nvidia Cosmos a world
- 00:02:32Foundation model that is designed that
- 00:02:35was created to understand the physical
- 00:02:37world and the only way for you to really
- 00:02:39understand this is to see it today we're
- 00:02:41announcing that Cosmos is open licensed
- 00:02:45it's open available on
- 00:02:53GitHub we hope we hope that this moment
- 00:02:57and there's a there's a small medium
- 00:02:58large for uh uh very fast models um you
- 00:03:02know mainstream models and also teacher
- 00:03:04models basically not knowledge transfer
- 00:03:07models Cosmo Cosmos World Foundation
- 00:03:10model being open we really hope will do
- 00:03:13for the world of Robotics and Industrial
- 00:03:15AI what llama 3 has done for Enterprise
- 00:03:18AI the magic happens when you connect
- 00:03:23Cosmos to Omniverse and the reason
- 00:03:25fundamentally is this Omniverse is a
- 00:03:30physics grounded not physically grounded
- 00:03:33but physics grounded it's algorithmic
- 00:03:36physics principled physics simulation
- 00:03:39grounded system it's a simulator when
- 00:03:42you connect that to
- 00:03:44Cosmos it provides the grounding the
- 00:03:47ground truth that can control and to
- 00:03:50condition the Osmos generation as a
- 00:03:53result what comes out of Osmos is
- 00:03:55grounded on Truth this is exactly the
- 00:03:57same idea as connecting a large language
- 00:03:59model to a rag to a retrieval augmented
- 00:04:03generation system you want to ground the
- 00:04:05AI generation on ground truth and so the
- 00:04:09combination of the two gives you a
- 00:04:12physically simulated a physically
- 00:04:15grounded Multiverse generator and the
- 00:04:19application the use cases are really
- 00:04:21quite exciting and of course uh for
- 00:04:24robotics uh for industrial applications
- 00:04:26uh it is very very clear this Cosmos
- 00:04:31plus Omniverse plus Cosmos represents
- 00:04:34the Third computer that's necessary for
- 00:04:36building robotic systems every robotics
- 00:04:39company will ultimately have to build
- 00:04:41three computers a robotics the robotics
- 00:04:44system could be a factory the robotics
- 00:04:45system could be a car it could be a
- 00:04:47robot you need three fundamental
- 00:04:49computers one computer of course to
- 00:04:51train the AI we call it the dgx computer
- 00:04:54to train the AI another of course when
- 00:04:58you're done to deploy the AI
- 00:05:00we call that agx that's inside the car
- 00:05:02in the robot or in an AMR or you know at
- 00:05:05the uh in a in a stadium or whatever it
- 00:05:07is these computers are at the edge and
- 00:05:11they're autonomous but to connect the
- 00:05:13two you need a digital twin and this is
- 00:05:16all the simulations that you were seeing
- 00:05:17the digital twin is where the AI that
- 00:05:20has been trained goes to practice to be
- 00:05:24refined to do its synthetic data
- 00:05:26generation reinforcement learning AI
- 00:05:28feedback such and such and so it's the
- 00:05:31digital twin of the AI these three
- 00:05:33computers are going to be working
- 00:05:34interactively nvidia's strategy for uh
- 00:05:37the industrial world and we've been
- 00:05:39talking about this for some time is this
- 00:05:41three computer
- 00:05:43system you know instead of a three three
- 00:05:46body problem we have a three Computer
- 00:05:48Solution and so it's the Nvidia robotics
- 00:05:51the AV revolution has
- 00:05:53arrived after so many years with weo
- 00:05:57success and Tesla's success it has very
- 00:06:00very clear autonomous vehicles has
- 00:06:02finally arrived well our offering to
- 00:06:05this industry is the three computers the
- 00:06:07training systems to train the AIS the
- 00:06:10simulation systems and and the and the
- 00:06:12synthetic data generation systems
- 00:06:14Omniverse and now Cosmos and also the
- 00:06:16computer that's inside the car each car
- 00:06:19company might might work with us in a
- 00:06:21different way use one or two or three of
- 00:06:23the computers we're working with just
- 00:06:25about every major car company around the
- 00:06:27world weo and zuk and Tesla of course in
- 00:06:30their data center byd the largest uh EV
- 00:06:33company in the world jlr has got a
- 00:06:35really cool car coming Mercedes because
- 00:06:37a fleet of cars coming with Nvidia
- 00:06:39starting with this starting this year
- 00:06:41going to production and I'm super super
- 00:06:43pleased to announce that today Toyota
- 00:06:47and Nvidia are going to partner together
- 00:06:48to create their next Generation AVS just
- 00:06:51so many so many cool companies lucid and
- 00:06:54rivan and xiaomi and of course Volvo
- 00:06:58just so many different companies wabby
- 00:07:00is uh building uh self-driving trucks
- 00:07:02Aurora uh we announced this week also
- 00:07:05that Aurora is going to use Nvidia to
- 00:07:06build self-driving trucks autonomous a
- 00:07:10100 million cars build each year a
- 00:07:12billion cars vehicles on a road all over
- 00:07:15the world a trillion miles that are
- 00:07:17driven around the world each year that's
- 00:07:20all going to be either highly autonomous
- 00:07:23or you know fully autonomous coming up
- 00:07:25and so this is going to be a very L very
- 00:07:27large industry I predict that this will
- 00:07:29likely be the first multi-trillion
- 00:07:31dollar robotics industry this IND this
- 00:07:35business for us um notice in just just a
- 00:07:38few uh of these cars that are starting
- 00:07:41to ramp into the world uh our business
- 00:07:43is already $4 billion and this year
- 00:07:45probably on a run rate of about $5
- 00:07:47billion so really significant business
- 00:07:49already this is going to be very large
- 00:07:51well today we're announcing that our
- 00:07:53next generation processor for the car
- 00:07:56our next generation computer for the car
- 00:07:57is called Thor I have right here hang on
- 00:08:00a
- 00:08:02second okay this is
- 00:08:05Thor this is
- 00:08:07Thor this is this is a robotics
- 00:08:12computer this is a robotics computer
- 00:08:14takes sensors and just a Madness amount
- 00:08:18of sensor information process it you
- 00:08:22know een cameras high resolution Radars
- 00:08:27Liars they're all coming into this chip
- 00:08:29and this chip has to process all that
- 00:08:31sensor turn them into tokens put them
- 00:08:34into a Transformer and predict the next
- 00:08:37PATH and this AV computer is now in full
- 00:08:41production Thor is 20 times the
- 00:08:45processing capability of our last
- 00:08:47generation Orin which is really the
- 00:08:49standard of autonomous vehicles today
- 00:08:51and so this is just really quite quite
- 00:08:53incredible Thor is in full production
- 00:08:55this robotics processor by the way also
- 00:08:57goes into a full robot and so it could
- 00:09:00be an AMR it could be a a human or robot
- 00:09:03it could be the brain it could be the
- 00:09:05manipulator uh this Ro this processor
- 00:09:07basically is a universal robotics
- 00:09:11computer the second part of our drive
- 00:09:14system that I'm incredibly proud of is
- 00:09:17the dedication to safety Drive OS I'm
- 00:09:21pleased to announce is now the first
- 00:09:23softwar defined programmable AI computer
- 00:09:28that has been certified if IED up to
- 00:09:30asold D which is the highest standard of
- 00:09:34functional safety for automobiles the
- 00:09:37only and the highest and so I'm really
- 00:09:40really proud of this asold ISO
- 00:09:4326262 it is um the work of some 15,000
- 00:09:47engineering years this is just
- 00:09:49extraordinary work and as a result of
- 00:09:52that Cuda is now a functional safe
- 00:09:55computer and so if you're building a
- 00:09:57robot Nvidia CA
- 00:10:04yep okay so so now I wanted to I told
- 00:10:06you I was going to show you what would
- 00:10:08we use Omniverse and Cosmos to do in the
- 00:10:13context of self-driving cars and you
- 00:10:16know today instead of showing you a
- 00:10:18whole bunch of uh uh videos of of cars
- 00:10:21driving on the road I'll show you some
- 00:10:23of that too um but I want to show you
- 00:10:25how we use the car to reconstruct
- 00:10:28digital twins automatically using Ai and
- 00:10:32use that capability to train future AI
- 00:10:36models okay let's play
- 00:10:40it the autonomous vehicle Revolution is
- 00:10:44here building autonomous vehicles like
- 00:10:47all robots requires three computers
- 00:10:51Nvidia dgx to train AI models Omniverse
- 00:10:54to test drive and generate synthetic
- 00:10:56data and drive agx a supercomputer in
- 00:11:00the car building safe autonomous
- 00:11:03vehicles means addressing Edge scenarios
- 00:11:07but real world data is limited so
- 00:11:09synthetic data is essential for
- 00:11:13training the autonomous vehicle data
- 00:11:15Factory powered by Nvidia Omniverse AI
- 00:11:19models and Cosmos generates synthetic
- 00:11:22driving scenarios that enhance training
- 00:11:24data by orders of
- 00:11:27magnitude first omnimap fuses map and
- 00:11:31geospatial data to construct drivable 3D
- 00:11:38environments driving scenario variations
- 00:11:41can be generated from replay Drive logs
- 00:11:43or AI traffic
- 00:11:46generators next a neural reconstruction
- 00:11:49engine uses autonomous vehicle sensor
- 00:11:51logs to create High Fidelity 4D
- 00:11:55simulation
- 00:11:56environments it replays previous drives
- 00:11:59in 3D and generates scenario variations
- 00:12:02to amplify training
- 00:12:04data finally edify 3DS automatically
- 00:12:08searches through existing asset
- 00:12:11libraries or generates new assets to
- 00:12:14create Sim ready
- 00:12:18scenes the Omniverse scenarios are used
- 00:12:21to condition Cosmos to generate massive
- 00:12:24amounts of photorealistic data reducing
- 00:12:27the Sim to real Gap
- 00:12:30and with text prompts generate near
- 00:12:33infinite variations of the driving
- 00:12:37scenario with Cosmos neotron video
- 00:12:40search the massively scaled synthetic
- 00:12:42data set combined with recorded drives
- 00:12:46can be curated to train
- 00:12:49models nvidia's AI data Factory scales
- 00:12:53hundreds of drives into billions of
- 00:12:56effective miles setting the standard for
- 00:12:59safe and advanced autonomous
- 00:13:01[Music]
- 00:13:05driving is that incredible
- 00:13:09we take take thousands of drives and
- 00:13:14turn them into billions of miles we are
- 00:13:17going to have mountains of training data
- 00:13:20for autonomous vehicles of course we
- 00:13:22still need actual cars on the road of
- 00:13:25course we will continuously collect data
- 00:13:27for as long as we shall live however
- 00:13:30synthetic data generation using this
- 00:13:33Multiverse physically based physically
- 00:13:36grounded capability so that we generate
- 00:13:39data for training AIS that are
- 00:13:41physically grounded and accurate and or
- 00:13:43plausible so that we could have an
- 00:13:45enormous amount of data to train with
- 00:13:47the AV industry is here uh this is an
- 00:13:49incredibly exciting time super super
- 00:13:52super uh uh excited about the next
- 00:13:54several years I think you're going to
- 00:13:55see just as computer Graphics was
- 00:13:58revolutionized such incredible pace
- 00:14:00you're going to see the pace of Av
- 00:14:02development increasing tremendously over
- 00:14:04the next several
- 00:14:05years I I think the next part is is
- 00:14:10robotics so um the chat GPT moment for
- 00:14:14General robotics is just around the
- 00:14:16corner and in fact all of the enabling
- 00:14:19technologies that I've been talking
- 00:14:20about is going to make it possible for
- 00:14:24us in the next several years to see very
- 00:14:27rapid breakthroughs surprising
- 00:14:28breakthroughs in in general robotics now
- 00:14:30the reason why General robotics is so
- 00:14:32important is whereas robots with tracks
- 00:14:35and wheels require special environments
- 00:14:38to accommodate them there are three
- 00:14:41robots three robots in the world that we
- 00:14:44can make that require no green
- 00:14:47fields Brown field adaptation is perfect
- 00:14:51if we if we could possibly build these
- 00:14:53amazing robots we could deploy them in
- 00:14:56exactly the world that we've built for
- 00:14:58ourselves these three robots are one
- 00:15:01agentic robots agentic AI because you
- 00:15:05know they're information workers so long
- 00:15:07as they could accommodate uh the
- 00:15:09computers that we have in our offices is
- 00:15:10going to be great number two
- 00:15:13self-driving cars and the reason for
- 00:15:15that is we spent 100 plus years building
- 00:15:17roads and cities and then number three
- 00:15:20human or robots if we have the
- 00:15:22technology to solve these three this
- 00:15:25will be the largest technology industry
- 00:15:27the world's ever seen
- 00:15:29and so we think that robotics era is
- 00:15:33just around the corner the critical
- 00:15:36capability is how to train these robots
- 00:15:39in the case of human or
- 00:15:41robots the imitation information is
- 00:15:44rather hard to collect and the reason
- 00:15:47for that is uh in the case of car you
- 00:15:49just drive it we're driving cars all the
- 00:15:50time in the case of these human robots
- 00:15:53the imitation information the the human
- 00:15:56demonstration is rather laborious to do
- 00:15:58and so we need to come up with a clever
- 00:16:00way to take hundreds of demonstrations
- 00:16:03thousands of human demonstrations and
- 00:16:06somehow use artificial intelligence and
- 00:16:10Omniverse to synthetically
- 00:16:13generate
- 00:16:15millions
- 00:16:17of
- 00:16:19synthetically generated motions and from
- 00:16:22those motions the AI can learn uh how to
- 00:16:25perform a task let me show you how
- 00:16:27that's done
- 00:16:35General robotics is arriving powered by
- 00:16:38Nvidia Isaac
- 00:16:43Groot okay well let me let me let me
- 00:16:47tell you what I told you I told you that
- 00:16:49we are in production with three new
- 00:16:54Blackwells not only is the grace
- 00:16:56Blackwell supercomputers MV link7 2's in
- 00:16:59production all over the world we now
- 00:17:01have three new Blackwell systems in
- 00:17:04production one amazing AI foundational M
- 00:17:09World Foundation model the world's first
- 00:17:11physical AI Foundation model it's open
- 00:17:14available to activate the world's
- 00:17:16industries of Robotics and such and
- 00:17:20three and three robotics three robots
- 00:17:23we're working on uh agentic AI uh human
- 00:17:27or robots and self-driving cars
- 00:17:30uh it's been an incredible year I want
- 00:17:32to thank all of you for your partnership
- 物理人工智能
- Nvidia Cosmos
- Omniverse
- 机器人技术
- 自动驾驶
- AI训练
- Thor处理器
- 数字孪生
- 合成数据
- 功能安全