Hands-On: Meta Orion Augmented Reality Glasses!

00:53:18
https://www.youtube.com/watch?v=ynLm-QvsW0Q

Zusammenfassung

TLDRMeta's ambitious Orion augmented reality (AR) glasses are hailed as the most advanced personal technology device yet. With a staggering production cost of $10,000 per unit, Orion features groundbreaking technology that combines an optical display capable of projecting holograms across a wide field and an intuitive input system that responds to neural signals. These AR glasses are still at the prototype stage, with only a thousand units made for internal development. Meta's Chief Technology Officer, BOS, expresses excitement over the potential of Orion, indicating that it represents technology that could one day replace the smartphone by merging AR functionality with AI capabilities. The glasses utilize innovative technologies such as micro LEDs, silicon carbide waveguides, and neural interfaces to create an immersive and highly functional AR experience. However, due to current production challenges, Orion remains a developmental concept, with further iterations needed to hit the consumer market.

Mitbringsel

  • 👓 Orion is Meta's latest AR glasses prototype, claimed to be highly advanced technology.
  • 💲 Each Orion prototype costs approximately $10,000 to make.
  • 🧠 The device features a groundbreaking neural interface for mind-like control.
  • 📊 It uses innovative waveguide technology for its display, enhancing AR experiences.
  • 🔬 Silicon carbide is used for its high refractive index, aiding in display mechanics.
  • 🤖 AI plays a crucial role in enhancing user interaction with the device.
  • 📱 Meta sees potential in Orion to eventually replace smartphones.
  • 🔍 The glasses are loaded with sensors for precise world perception and interaction.
  • 🚫 Orion is not yet available for consumers, aimed at internal development.
  • 🔮 The technology in Orion draws parallels with science fiction possibilities.

Zeitleiste

  • 00:00:00 - 00:05:00

    Orion Meta's augmented reality glasses are dubbed as the most advanced personal technology by Meta, taking years and millions in R&D. Each prototype is costly at about $10,000, made for internal development only. It features an optical display that shows holograms everywhere and an input system reading user intentions, described as a 'time machine'.

  • 00:05:00 - 00:10:00

    BOS, Meta's CTO, discusses Orion, stating it was intended as a product with a 10% chance of success. Pioneering over ten new technologies, it's now fully functional but not cost-effective for consumer release. The glasses are recognized for having an operating system, being internet-connected, and aiding in software intuition development.

  • 00:10:00 - 00:15:00

    The glasses have micro-LED projectors and are described as having super high clarity using silicon carbide with a high refractive index for the wave guides, reducing visual artifacts. The custom technology allows for lightweight, rigid structure necessary for optical element precision and future design improvements.

  • 00:15:00 - 00:20:00

    The Orion system includes a compute puck and relies on custom silicon to minimize power while enhancing battery life. Features include unique radio protocols for power efficiency in thermal management, and world-sensing via sensors and cameras. A fundamental leap towards phone replacement ambition shows in the software experiences enhancing personal technology utility.

  • 00:20:00 - 00:25:00

    AI integration has accelerated, allowing Orion to offer substantial value as a personal assistant device with contextual awareness beyond typical phone use. Orion was initially conceptualized to prioritize hologram technology, however, AI's advancement now presents new interactive possibilities and contextual utilities.

  • 00:25:00 - 00:30:00

    With AI and AR capabilities, Orion aims to expand its utility dramatically by enhancing user interaction with the environment. Future prospects include world-locked content sharing, advanced interaction capabilities, and intelligent data retrieval, e.g., remembering where items were left, hinting at its potential role as a memory aid.

  • 00:30:00 - 00:35:00

    Orion's optical innovations include silicon carbide wave guides for a wider field of view and miniaturization, despite high manufacturing costs. There's a trade-off between current limitations and achieving everyday usability. Future iterations are expected to decrease thickness and increase resolution and immersion.

  • 00:35:00 - 00:40:00

    The system's EMG wristbands offer innovative input through subtle gestures, complementing the glasses' hand tracking, allowing interaction even when hands are not visible. Orion supports unique experiences with multimodal input, promising an efficient and intuitive user interface without substantial physical effort.

  • 00:40:00 - 00:45:00

    The demo highlighted Orion's capabilities with seamless interactions, real-time world overlay, potent AI interaction experience, and potential for significant user engagement via social and individual applications. The future improvements are targeted towards a higher resolution, better physical integration, and more affordable consumer configuration.

  • 00:45:00 - 00:53:18

    Despite the high production cost of silicon carbide elements, Orion aims for ultimate utility beyond current devices. Meta envisions Orion not as another gadget but as a device potentially replacing phones, suggesting a high consumer price point but promising a transformative personal computing experience.

Mehr anzeigen

Mind Map

Mind Map

Häufig gestellte Fragen

  • What is Orion's main feature?

    Orion AR glasses feature a holographic display and mind-reading input system.

  • How much does each Orion prototype cost?

    Each Orion prototype costs around $10,000 to produce.

  • Who developed the Orion AR glasses?

    Meta developed the Orion AR glasses.

  • How many Orion prototypes exist?

    There are only around a thousand Orion prototypes.

  • What makes Orion unique?

    Orion's advanced holographic technology and neural input system set it apart.

  • Can the Orion replace a smartphone?

    Meta envisions the Orion as a potential successor to the smartphone.

  • What technology is used in Orion's display?

    The display uses waveguide technology with a high refractive index.

  • What is the primary use case of Orion?

    Orion is primarily focused on augmented reality applications.

  • Is Orion available for consumer purchase?

    No, Orion is not currently available for consumer purchase.

  • What role does AI play in Orion?

    AI aids in world perception and user interaction with the Orion glasses.

Weitere Video-Zusammenfassungen anzeigen

Erhalten Sie sofortigen Zugang zu kostenlosen YouTube-Videozusammenfassungen, die von AI unterstützt werden!
Untertitel
en
Automatisches Blättern:
  • 00:00:01
    this is Orion meta's augmented reality
  • 00:00:04
    glasses prototype that they consider the
  • 00:00:06
    most advanced piece of personal
  • 00:00:09
    technology ever created it's taken them
  • 00:00:12
    years and countless millions in R&D to
  • 00:00:15
    get to this point a 98 G pair of glasses
  • 00:00:18
    with an optical display system that can
  • 00:00:20
    put Holograms almost everywhere your
  • 00:00:23
    eyes can look and an input system that
  • 00:00:25
    seemingly reads your mind the catch each
  • 00:00:29
    Orion prototype costs about $10,000 to
  • 00:00:32
    make and meta is only making about a
  • 00:00:34
    thousand of them for internal
  • 00:00:35
    development it's a kind of device that
  • 00:00:37
    they call a time machine and we're one
  • 00:00:40
    of the few people at this year's connect
  • 00:00:42
    conference to go Hands-On with one to
  • 00:00:45
    dive deep into how Orion Works what it's
  • 00:00:47
    like to use them and what this time
  • 00:00:50
    machine reveals about the future of
  • 00:00:51
    augmented reality let's start with a
  • 00:00:54
    conversation I had with BOS meta's Chief
  • 00:00:57
    technology officer okay BOS we're here
  • 00:00:59
    to talk about um Orion or is it project
  • 00:01:02
    Orion Orion Orion we're calling it Orion
  • 00:01:05
    Orion um in conversations with you know
  • 00:01:08
    some of the teams here like displays
  • 00:01:09
    research team I know you guys sometimes
  • 00:01:11
    create products that you think of as
  • 00:01:13
    time machines right where it's like put
  • 00:01:15
    the the best and regardless of cost um
  • 00:01:18
    just to see what it would be like if
  • 00:01:21
    your your team and researchers and
  • 00:01:22
    developers is this representative of
  • 00:01:24
    what you would call like a time machine
  • 00:01:25
    yeah this this is an incredible piece of
  • 00:01:27
    work when we started this Pro program we
  • 00:01:29
    had hoped it would would be a product um
  • 00:01:32
    that we could ship um but we gave
  • 00:01:34
    ourselves only this is even 5 years ago
  • 00:01:37
    uh we gave ourselves only a 10% chance
  • 00:01:39
    of being able to build one we just we
  • 00:01:40
    really had so many uh new technologies
  • 00:01:43
    that we were pioneering at least seven
  • 00:01:45
    probably really I would say 10
  • 00:01:46
    technologies that had never been done
  • 00:01:48
    before yeah so we really weren't sure if
  • 00:01:50
    it could even be built at all and so the
  • 00:01:52
    product was this idea of it being a
  • 00:01:53
    product was our goal we weren't sure if
  • 00:01:55
    we'd get there well we ended up building
  • 00:01:56
    we ended up building not just one we
  • 00:01:58
    ended up building uh you know quite a
  • 00:01:59
    few them uh they work they're
  • 00:02:01
    spectacular we are like Overjoyed with
  • 00:02:04
    with the devices uh ended up not being
  • 00:02:06
    cost effective as a product and so yeah
  • 00:02:08
    it does stand as a time machine but a
  • 00:02:10
    lot of times when when we talk about
  • 00:02:11
    time machines we're talking about um you
  • 00:02:13
    know the version of this it's a time
  • 00:02:14
    machine would be on a bench you know the
  • 00:02:16
    size of of this room plug into all kinds
  • 00:02:19
    of of Power with with lasers is so so
  • 00:02:22
    this is something different this is
  • 00:02:23
    we're going to call this a prototype and
  • 00:02:25
    what's important for us is this is a
  • 00:02:26
    fully working it's got It's got an
  • 00:02:27
    operating system it's it's live uh it's
  • 00:02:30
    Internet connected and we now have a
  • 00:02:33
    device that's allows us to develop
  • 00:02:35
    software um and develop an intuition
  • 00:02:38
    around what software experiences will be
  • 00:02:39
    great for what we believe is the type of
  • 00:02:42
    device that will replace the phone
  • 00:02:44
    someday um and yeah it's it's hard to
  • 00:02:47
    describe the emotion I went through the
  • 00:02:49
    first time I tried it on when the
  • 00:02:50
    software was running it was a few months
  • 00:02:52
    ago and uh the team came in and the team
  • 00:02:55
    is very nervous to show this to me
  • 00:02:56
    because it's been obviously five years
  • 00:02:58
    and who knows how much money in the
  • 00:02:59
    making and we are uh so they're they're
  • 00:03:02
    just apologizing for everything they're
  • 00:03:04
    just oh you know I know that that that
  • 00:03:06
    pixel's a little rough and we'll get the
  • 00:03:07
    color uniformity right meanwhile I'm
  • 00:03:09
    having one of the greatest experiences
  • 00:03:11
    of my professional career what was about
  • 00:03:13
    it that that experience that so
  • 00:03:14
    different than previous you've got a
  • 00:03:17
    it's a full color wide field of view
  • 00:03:19
    display that's responding to neural
  • 00:03:22
    interfaces uh you know that you're
  • 00:03:24
    sending through your wrist yeah um and
  • 00:03:26
    it's it was very responsive and you're
  • 00:03:28
    looking at videos and you're doing video
  • 00:03:29
    calls and it's spectacular it's just
  • 00:03:31
    spectacular just you feel like it's a
  • 00:03:34
    sense of of real Wonder um and so you
  • 00:03:37
    know but of course the team has been in
  • 00:03:38
    it for all this time they just they see
  • 00:03:39
    all the problems it was such an early
  • 00:03:41
    demo that they were they had this the
  • 00:03:43
    compute was they kept replacing a cold
  • 00:03:46
    soda they kept putting new cold sodas on
  • 00:03:47
    the compute because they had they hadn't
  • 00:03:49
    even done the work yet on the thermal so
  • 00:03:51
    they were just trying to cool it down
  • 00:03:52
    with cold sodas but it was just it was a
  • 00:03:54
    glimpse and and even just in the last
  • 00:03:56
    couple of months since I did that demo
  • 00:03:58
    it's now gotten to this point where it's
  • 00:04:00
    it's a fully functioning system you
  • 00:04:01
    could use it you can walk around you can
  • 00:04:02
    spend you know it's got two-hour battery
  • 00:04:04
    life you're walking you can spend time
  • 00:04:05
    in meetings you're able to have a really
  • 00:04:07
    rich um computer available to you
  • 00:04:10
    without taking anything out of your of
  • 00:04:11
    your pocket without losing the context
  • 00:04:13
    of where you are in the world without
  • 00:04:15
    breaking eye contact with someone that
  • 00:04:16
    you're talking to um it's it's
  • 00:04:18
    spectacular and bring new context oh
  • 00:04:20
    absolutely absolutely talk about some of
  • 00:04:21
    those Technologies you said you know
  • 00:04:23
    seven have time to you all of them but
  • 00:04:25
    obviously the display systems a big part
  • 00:04:27
    of it the displays a really really
  • 00:04:28
    important when I have a very fun clear
  • 00:04:30
    demo here and so you've got microed
  • 00:04:32
    projectors which are putting out
  • 00:04:34
    tremendous brightness hugely efficient
  • 00:04:37
    micro LEDs um you know all three colors
  • 00:04:39
    RGB um they're putting out you know
  • 00:04:42
    hundreds of thousands of knits um but of
  • 00:04:44
    course to have a really big eye box so
  • 00:04:46
    you don't have to have this thing where
  • 00:04:47
    you can move the glasses on your face
  • 00:04:48
    and you can still see the image you're
  • 00:04:49
    doing people replication so you're
  • 00:04:50
    shooting those uh photons in lots of
  • 00:04:52
    different places towards the eye um and
  • 00:04:55
    so you're it's a big discount and then
  • 00:04:57
    of course there's inefficiency and
  • 00:04:58
    there's just loss as photons escape the
  • 00:05:00
    substance um and so you've got these
  • 00:05:02
    tiny projector which we completely
  • 00:05:04
    invented ourselves including
  • 00:05:06
    manufacturing the micro LEDs ourselves
  • 00:05:08
    you're shooting it back towards the the
  • 00:05:11
    wave guide the wave guide itself is is
  • 00:05:13
    nano etched into silicon carbide which
  • 00:05:15
    is a a novel material with a super high
  • 00:05:17
    index of refraction which allows us to
  • 00:05:19
    minimize the Lost photons and minimize
  • 00:05:21
    the number of photons we capture from
  • 00:05:22
    the world so it minimizes things like
  • 00:05:24
    ghosting and Haze and rainbow all these
  • 00:05:26
    artifacts while giving you that field of
  • 00:05:28
    view that you want well it's not
  • 00:05:29
    artifact free it's very close to
  • 00:05:31
    artifact free like you you'll get your
  • 00:05:32
    own experience and you you actually it's
  • 00:05:33
    like it's amazing the the optical
  • 00:05:36
    Clarity the see-through Clarity of these
  • 00:05:38
    is really really high um and the image
  • 00:05:41
    is super wide field of view full color
  • 00:05:43
    um and it's it's tremendous so you've
  • 00:05:45
    got these two systems alone working
  • 00:05:47
    together were an immense amount of of
  • 00:05:50
    investment and cost each of them
  • 00:05:51
    requiring two or three different new
  • 00:05:54
    technologies to come to Bear keep in
  • 00:05:55
    mind like red blue and green just
  • 00:05:56
    getting uh red micro EDS is hard because
  • 00:05:59
    it's a very long wavelength to generate
  • 00:06:00
    such a small space right um and then
  • 00:06:03
    silicon carbide manufacturing it and
  • 00:06:05
    growing the crystals large enough and
  • 00:06:06
    and machining it is tremendously
  • 00:06:08
    difficult um and then you have to mount
  • 00:06:10
    this whole thing on on magnesium rigid
  • 00:06:12
    magnesium frame it's lightweight good
  • 00:06:14
    thermal properties because you don't
  • 00:06:15
    have much thermal dissipation space here
  • 00:06:16
    such a small device it looks P um and
  • 00:06:19
    and it's also rigid because right now we
  • 00:06:21
    don't have disparity so you don't want
  • 00:06:22
    to you want to minimize any deflection
  • 00:06:23
    between the optical elements to keep the
  • 00:06:25
    binocular image looking correct in your
  • 00:06:27
    field of view um and these are all
  • 00:06:29
    things that we already have plans to
  • 00:06:31
    improve for the next version so you get
  • 00:06:32
    to this version one and you figure out
  • 00:06:34
    oh we can actually figure out how to do
  • 00:06:36
    disparity correction oh we can actually
  • 00:06:37
    figure out how to put the projector from
  • 00:06:38
    behind we can actually make this thing
  • 00:06:40
    thinner um and so this thing has uh you
  • 00:06:43
    know we have between the system between
  • 00:06:45
    this and the stage that powers it this
  • 00:06:47
    this Puck um you know more than 10
  • 00:06:49
    pieces of custom silicon which have
  • 00:06:51
    miniaturized uh our routines taking them
  • 00:06:53
    from on for example the quest system
  • 00:06:56
    where they might be you know the quest
  • 00:06:58
    has relatively generous envelope
  • 00:07:00
    compared to to these glasses um taking
  • 00:07:02
    them from thousands of mwatts down to
  • 00:07:04
    hundreds of mwatts or in some cases
  • 00:07:05
    dozens of Mills um to have this long
  • 00:07:08
    battery life the the radio protocol
  • 00:07:10
    completely custom radio protocol so that
  • 00:07:12
    you're sending information over in
  • 00:07:14
    bursts to minimize sustained usage of
  • 00:07:16
    the radio again for thermal reasons not
  • 00:07:18
    for power reasons as much as thermal
  • 00:07:20
    reasons which is important for um yeah
  • 00:07:22
    Ai and and continuously perceiving the
  • 00:07:24
    world to give you absolutely cont uh the
  • 00:07:27
    sensors you've got all these tiny
  • 00:07:29
    cameras built in it's actually almost
  • 00:07:30
    easier to see on this one where the
  • 00:07:32
    cameras are you know there seven
  • 00:07:33
    different cameras embedded in the frame
  • 00:07:35
    that are doing the world sensing the the
  • 00:07:37
    the camera sensors are custom sensors um
  • 00:07:39
    you know you need to have super high
  • 00:07:41
    dynamic range you need to have certain
  • 00:07:42
    properties in terms of your ability to
  • 00:07:43
    to pick from the scene what you care
  • 00:07:45
    about and focus on that um so and then
  • 00:07:49
    of course there's quite a few things
  • 00:07:50
    that are shared between this system and
  • 00:07:52
    our Quest systems so the hand tracking
  • 00:07:53
    the eye tracking um spatial audio um and
  • 00:07:57
    those pieces kind of jointly effort but
  • 00:07:59
    of course doing the hand trkking in this
  • 00:08:00
    kind of a form factor is tough and then
  • 00:08:02
    of course at the end of all that you
  • 00:08:03
    still got the novel interface right you
  • 00:08:05
    made an investment in control ABS you're
  • 00:08:07
    not going to walk around in the world
  • 00:08:08
    with controllers um and you can you're
  • 00:08:11
    want to walk around with with your hands
  • 00:08:12
    but you're not going to walk around with
  • 00:08:13
    your hands out here doing this um and so
  • 00:08:16
    and you might even want to be able to
  • 00:08:17
    control the system with your hands in
  • 00:08:18
    your pocket or obscured U and so we've
  • 00:08:21
    got these neural interfaces which are
  • 00:08:22
    just getting started but it's a you know
  • 00:08:24
    this when we started you've seen you've
  • 00:08:25
    seen the control Labs demos this didn't
  • 00:08:28
    used to be this small
  • 00:08:30
    let alone that's attractive um and it's
  • 00:08:31
    so comfortable we've been doing these
  • 00:08:32
    demos a couple weeks now and uh people
  • 00:08:34
    keep walking out of the room not realiz
  • 00:08:36
    they don't even realize they're wearing
  • 00:08:37
    it anymore um and so and then to do so
  • 00:08:40
    you're getting all these these new
  • 00:08:41
    technologies and then you're trying to
  • 00:08:42
    pack it into an under 100 grams you know
  • 00:08:45
    a 98 gram uh device that's comfortable
  • 00:08:48
    enough that you would be willing to wear
  • 00:08:49
    it for its 2-hour battery life um and
  • 00:08:51
    hopefully attractive enough that you
  • 00:08:53
    feel like yeah you could be taken
  • 00:08:54
    seriously wearing these glasses that's
  • 00:08:56
    important um and so you know and so we
  • 00:08:59
    just felt you when you look at that you
  • 00:09:01
    know 5 years AG you can imagine you're
  • 00:09:02
    just thinking man if any one of these
  • 00:09:04
    things is a swing and a miss this whole
  • 00:09:06
    system doesn't come together um and so
  • 00:09:08
    yeah that was the emot you know I have
  • 00:09:10
    all that weight coming into my first
  • 00:09:11
    demo with it and I'm feeling that
  • 00:09:13
    emotion of all that investment and all
  • 00:09:14
    that risk um and the execution that it
  • 00:09:16
    took to get to this point and at this
  • 00:09:18
    point you know and then developers can
  • 00:09:19
    start figuring out what are the best use
  • 00:09:20
    cases but you got this idea that
  • 00:09:22
    eventually you want this to replace the
  • 00:09:24
    phones so the things that people are
  • 00:09:25
    doing on the phone or at least you know
  • 00:09:26
    the media and the type of interactions
  • 00:09:29
    uh but it's also AR can do more than
  • 00:09:32
    what a phone can do so you know what are
  • 00:09:34
    the kind of dream scenarios ideally when
  • 00:09:36
    a consumer version of this hits the
  • 00:09:38
    marketplace why is what's the utility
  • 00:09:40
    that this brings to an average person
  • 00:09:42
    yeah the thing that we didn't anticipate
  • 00:09:44
    five years ago which has changed
  • 00:09:46
    obviously in the last two years is is AI
  • 00:09:49
    so five years ago we certainly imagined
  • 00:09:50
    yep all your 2D phone screen type use
  • 00:09:52
    cases could be useful and then you can
  • 00:09:53
    also have these Holograms in the world
  • 00:09:56
    and they're they're World locked and
  • 00:09:57
    they they're stable and they stay where
  • 00:09:59
    they are
  • 00:10:00
    and if we both had our glasses on we
  • 00:10:01
    could mutually enjoy them explore them
  • 00:10:03
    if it's a chess game we can Summit it
  • 00:10:05
    into existence and we can play it um and
  • 00:10:08
    that's all very cool and it can do those
  • 00:10:09
    things uh in the meantime AI has come up
  • 00:10:12
    and we always thought it would be
  • 00:10:13
    Holograms first and AI would be later um
  • 00:10:15
    and we we have the Holograms but AI is
  • 00:10:17
    coming up fast and so I think a lot of
  • 00:10:19
    the use cases that we'll use with this
  • 00:10:20
    actually will be querying the world
  • 00:10:22
    around you um you know querying uh your
  • 00:10:25
    own day your own past where did I leave
  • 00:10:27
    my keys without having to separately do
  • 00:10:29
    anything to remind it to to remember
  • 00:10:32
    where your keys were it just has you
  • 00:10:34
    know the ability to keep track of these
  • 00:10:35
    things by virtue of of processing what's
  • 00:10:38
    going on as you go about the day um and
  • 00:10:40
    so that probably is for me the phone use
  • 00:10:43
    cases combined with an AI that unlike
  • 00:10:46
    your phone has full context on what
  • 00:10:49
    you've been seeing and hearing yeah
  • 00:10:51
    right and and how much more helpful is
  • 00:10:53
    that going to be as an assistant as you
  • 00:10:55
    go about your life well also doing the
  • 00:10:57
    the basic things that people expect the
  • 00:10:58
    phone to do of course video calls seeing
  • 00:11:00
    messaging video calls are delightful in
  • 00:11:02
    this can I just tell you that it's such
  • 00:11:03
    a funny thing and that's where you see
  • 00:11:04
    some of our other Investments like in
  • 00:11:05
    our investments in codak avatars super
  • 00:11:07
    realistic avatars obviously when you're
  • 00:11:09
    wearing these on your face you're not
  • 00:11:10
    going to have a camera facing you and
  • 00:11:13
    one of the real Delights is not taking a
  • 00:11:14
    phone and holding it way out here this
  • 00:11:16
    the muscles arms get tired you know I'm
  • 00:11:18
    not that strong um and so we animate it
  • 00:11:20
    by using um a codic avatar that you
  • 00:11:23
    already created and then the person
  • 00:11:25
    you're talking to you know it's it's it
  • 00:11:27
    can fool them in the good cases like
  • 00:11:28
    they they they will not necessarily
  • 00:11:30
    immediately realize that they're not
  • 00:11:31
    talking to a camera of you you you talk
  • 00:11:34
    about fatigue and the idea that you know
  • 00:11:35
    you want to make these frictionless is
  • 00:11:37
    possible Right seamless as far as and
  • 00:11:39
    these the the the wristbands here are
  • 00:11:41
    part of that the fact that you can
  • 00:11:43
    control them what are the kind of like
  • 00:11:44
    what's the end state of of that because
  • 00:11:47
    ml has allowed even basic gyro to to in
  • 00:11:50
    infer tap gestures right that's right
  • 00:11:52
    yeah IM usage on on the Apple watch and
  • 00:11:55
    other watches is increasing popularity
  • 00:11:57
    for some use cases I mean we're really
  • 00:11:59
    just getting started with this thing so
  • 00:12:00
    today you're making relatively small
  • 00:12:02
    gestures and those gestures can be
  • 00:12:03
    obscured from the the hand tracking of
  • 00:12:06
    the glasses and you can they still
  • 00:12:08
    registered that's very nice it's just
  • 00:12:10
    the beginning um you know with this
  • 00:12:12
    device for example we have working
  • 00:12:13
    internally um like the ability to do
  • 00:12:15
    handwriting um so you're like you're
  • 00:12:16
    already wearing this on your dominant
  • 00:12:18
    wrist you can do handwriting um as a as
  • 00:12:20
    a as a silent text input and a very like
  • 00:12:23
    um you know a Dem text input if you will
  • 00:12:26
    it's a very so with this device we're
  • 00:12:28
    only just tapping the beginning of it
  • 00:12:29
    you know the ability today to use these
  • 00:12:32
    gestures that are you know very small
  • 00:12:35
    and maybe even out of view of the
  • 00:12:37
    glasses is nice um but that's just the
  • 00:12:39
    beginning you want the ceiling to be
  • 00:12:40
    raised totally in the future you this is
  • 00:12:42
    a device that can do handwrite detection
  • 00:12:43
    so you can just handw write and you can
  • 00:12:44
    do that handwriting very subtly uh and
  • 00:12:47
    have it be text entry for the glasses
  • 00:12:49
    that isn't this big dramatic you know
  • 00:12:50
    you're talking out loud or you're using
  • 00:12:52
    a keyboard in front of your face um and
  • 00:12:54
    so we see a huge ceiling in uh the
  • 00:12:57
    neural interfaces and as you work to
  • 00:12:59
    make this you know smaller andin and
  • 00:13:02
    it's amazing that you said you already
  • 00:13:03
    have plans or ideas to get this to even
  • 00:13:05
    more compact thinner more socially
  • 00:13:08
    acceptable uh which of these
  • 00:13:10
    Technologies make most sense to to
  • 00:13:12
    distribute among the other products you
  • 00:13:14
    know things like metay bands absolutely
  • 00:13:16
    like what what would be the easiest or
  • 00:13:18
    things that make the most sense yeah and
  • 00:13:20
    this is where the the AI bridge I think
  • 00:13:22
    is one of the really critical ones so
  • 00:13:24
    you know when you're when you're
  • 00:13:25
    building a device in a in a very phone
  • 00:13:27
    Centric world the Rayband metag glasses
  • 00:13:29
    are a wonderful device you know of
  • 00:13:30
    course it's it's cameras it's audio it's
  • 00:13:33
    it's really you know good-looking it's
  • 00:13:35
    it's out of the way you don't have to
  • 00:13:36
    take your phone of your pocket to get
  • 00:13:37
    the image to get the video to take the
  • 00:13:40
    call um as soon as you add AI now
  • 00:13:42
    they've transformed into a completely
  • 00:13:44
    different category now you can query the
  • 00:13:45
    world around you with the Rayband metag
  • 00:13:47
    glasses and you can do live translations
  • 00:13:49
    with the Rayband metag glasses
  • 00:13:50
    incredible um and that's just the
  • 00:13:52
    beginning what if you now had a a simple
  • 00:13:55
    small display well when you have a phone
  • 00:13:57
    already a simple small display is an
  • 00:13:59
    obviously my favorite thing to add
  • 00:14:00
    because it's like yeah like you can kind
  • 00:14:01
    of look if you need to but once you're
  • 00:14:03
    asking AI questions and you get so
  • 00:14:06
    annoyed with it reading you the answer
  • 00:14:07
    back even if it's reading the answer
  • 00:14:08
    back to you exactly the question you
  • 00:14:10
    asked it's like just takes time yeah
  • 00:14:12
    just read this read it to me in text
  • 00:14:15
    binary binary yes or no so much faster
  • 00:14:17
    in text um likewise I'm taking an image
  • 00:14:19
    I want to share it is it good like I
  • 00:14:21
    want to see it before I share it so AI
  • 00:14:23
    feels like this bridge use case that
  • 00:14:25
    makes um even smaller displays feel more
  • 00:14:28
    valuable and then those can get richer
  • 00:14:30
    and richer over time adding more and
  • 00:14:31
    more value um and so while we are
  • 00:14:34
    absolutely all you know full steam ahead
  • 00:14:36
    on the next version of these um which
  • 00:14:38
    you know we plan to have be a consumer
  • 00:14:41
    product um we also are taking technology
  • 00:14:44
    from this line and saying hey what what
  • 00:14:46
    is a miniaturized version of this what's
  • 00:14:48
    a version of this that we can take and
  • 00:14:49
    and integrate into um our you know more
  • 00:14:53
    affordable already existing very popular
  • 00:14:55
    line of glasses doesn't somebody doesn't
  • 00:14:57
    need to do everything that's right all
  • 00:14:59
    these the AI glasses have changed our
  • 00:15:01
    outlook on the entire category we were
  • 00:15:03
    worried before that we would have to
  • 00:15:04
    like it would be a big gap between
  • 00:15:06
    things like the rayan metas and the
  • 00:15:08
    Orion and now actually we see an entire
  • 00:15:11
    spectrum of products in the space and
  • 00:15:13
    along the quest ecosystem those are
  • 00:15:15
    parallel tracks that you guys see not
  • 00:15:17
    necessarily converging to a single
  • 00:15:18
    product that's right and and they have a
  • 00:15:20
    lot of relationships so certainly when
  • 00:15:21
    you're doing slam Vio when you're doing
  • 00:15:23
    some of the work on co-processors or
  • 00:15:24
    custom silicon on wireless that work is
  • 00:15:27
    shared between the two teams that's
  • 00:15:28
    wonderful and there's a there's a big
  • 00:15:29
    base of shared work um but there's also
  • 00:15:32
    a lot of work that is very different and
  • 00:15:34
    the thing that I think we always we're
  • 00:15:36
    always very motivated by like who is the
  • 00:15:37
    person whose life is better because this
  • 00:15:39
    technology exists what are they trying
  • 00:15:40
    to do that is now better um and I think
  • 00:15:44
    the use cases for glasses and for um Mr
  • 00:15:49
    headsets is very different even though
  • 00:15:50
    they do things that look very similar if
  • 00:15:52
    you write them down on paper like you
  • 00:15:53
    can use similar words to describe them
  • 00:15:54
    in practice they're very different I
  • 00:15:56
    kind of liken it to the difference
  • 00:15:57
    between like my phone and my laptop top
  • 00:15:59
    M um like I love my phone I have it all
  • 00:16:01
    the time sometimes like you're on your
  • 00:16:03
    phone and you're like I'm writing an
  • 00:16:04
    email you're like no this is a laptop
  • 00:16:06
    email you're just like no like your
  • 00:16:07
    workload like I need to like open
  • 00:16:09
    multiple tabs in a browser I need to
  • 00:16:10
    like be comparing things I need to like
  • 00:16:12
    be doing research like I just need more
  • 00:16:14
    real estate that's like a different job
  • 00:16:15
    and I use a different device for that
  • 00:16:17
    job I do sometimes make that analogy
  • 00:16:19
    between AR and Mr um like they do some
  • 00:16:22
    similar things and that's great we
  • 00:16:24
    should make that as easily kind of
  • 00:16:26
    transferable for the consumer as we can
  • 00:16:28
    in terms of how it feels to do those
  • 00:16:30
    things um but they also do some very
  • 00:16:32
    different things the ability to be
  • 00:16:33
    completely immersed in uh in a VR
  • 00:16:35
    headset is a tremendously valuable asset
  • 00:16:37
    it can be done no other way and you
  • 00:16:39
    should build a device that's really
  • 00:16:40
    really good at that um rather than
  • 00:16:42
    trying to force everything into a
  • 00:16:44
    convergence which I think is a false
  • 00:16:45
    convergence yeah and last question you
  • 00:16:46
    know the VR headsets you guys do a great
  • 00:16:47
    job Connecting People seems to be a big
  • 00:16:49
    mission statement for the company you
  • 00:16:50
    know remotely as they are physically far
  • 00:16:52
    apart imagine if these become more like
  • 00:16:56
    distributed and more mainstream how are
  • 00:16:58
    these connecting people if people are
  • 00:17:00
    sitting across from each other or in a
  • 00:17:01
    collocated space yeah it's for me it's
  • 00:17:03
    two different parts of the story I think
  • 00:17:05
    one of which is Imagine we're distant um
  • 00:17:07
    and certainly these can do video calls
  • 00:17:08
    and that can be very cool um and I can
  • 00:17:10
    share with you what I'm seeing which can
  • 00:17:12
    be kind of a cool thing we can do that
  • 00:17:13
    today with the Rayband metas and you can
  • 00:17:15
    get on a video call and share what
  • 00:17:16
    you're seeing with the person you're
  • 00:17:17
    talking
  • 00:17:18
    to but a lot of this is also when I'm
  • 00:17:21
    with you um am I on my phone because if
  • 00:17:24
    I'm on my phone I am not with you like
  • 00:17:26
    physically I may be here but like
  • 00:17:27
    mentally I am some place I've gone deep
  • 00:17:29
    into the device um whereas if I'm able
  • 00:17:32
    to feel like no no I'm on top of my
  • 00:17:34
    things um it's all here if I need it I
  • 00:17:37
    hopefully am more physically present and
  • 00:17:38
    connected with you um because I have
  • 00:17:41
    this technology versus the the the
  • 00:17:42
    technologies that we use today which are
  • 00:17:45
    kind of famous for being not that like
  • 00:17:47
    they're famous for being things that we
  • 00:17:48
    kind of escape the present moment to get
  • 00:17:49
    into um I'm sure this will be capable of
  • 00:17:52
    both if it's the case that I would
  • 00:17:53
    prefer I'm on a long car ride sure let
  • 00:17:56
    me dive into my videos and get my
  • 00:17:58
    content it's all fun fine um but one of
  • 00:18:00
    the things we do hope is like these are
  • 00:18:01
    technologies that don't just help people
  • 00:18:03
    connect at a distance but also help them
  • 00:18:05
    feel connected when they're together and
  • 00:18:07
    even if they're wearing together there
  • 00:18:08
    are features that will allow them to
  • 00:18:10
    share the same content and see the same
  • 00:18:12
    content I've gotten quite good we have
  • 00:18:14
    we have quite a few fun social games
  • 00:18:16
    that you play which when two players uh
  • 00:18:18
    both have the Orion on you can play
  • 00:18:20
    these kind of games together and uh two
  • 00:18:22
    things are true one of which is um
  • 00:18:25
    people on the outside you look kind of
  • 00:18:26
    ridiculous because you are doing a thing
  • 00:18:28
    that like they can't see the artifacts
  • 00:18:31
    and the second thing which is more
  • 00:18:32
    important is even given the first you
  • 00:18:33
    don't care because you are fully engaged
  • 00:18:35
    in a very cool shared experience um that
  • 00:18:38
    you are experiencing as very much as
  • 00:18:40
    real as anything um so I think there's
  • 00:18:42
    um I think the world gets really fun
  • 00:18:44
    when everyone has access to the
  • 00:18:46
    Holograms can't wait yeah too so much
  • 00:18:48
    cheers thank you so much yeah cheers all
  • 00:18:50
    right let's dive into the nitty-gritty
  • 00:18:52
    details of Ryan and what it was like to
  • 00:18:56
    actually use them and go through that
  • 00:18:58
    whole demo process uh I want to break
  • 00:19:00
    this down into the three discret
  • 00:19:03
    Hardware components that we saw and
  • 00:19:05
    you've seen so far and those are the
  • 00:19:07
    glasses the wireless compute Puck
  • 00:19:10
    Wireless being the operative word there
  • 00:19:12
    and then as well as these neural
  • 00:19:14
    wristbands these EMG electromyography
  • 00:19:18
    wristbands that allow for one of the
  • 00:19:20
    ways you can provide input to Orion
  • 00:19:22
    along side hand tracking eye tracking
  • 00:19:25
    and voice of course uh starting off with
  • 00:19:28
    the glasses it was very clear from
  • 00:19:30
    everyone I talked to that the goal here
  • 00:19:32
    the stated North Star goal was to have
  • 00:19:36
    wide field of view in a socially
  • 00:19:38
    acceptable form factor and previously
  • 00:19:41
    you could have a 70° diagonal field of
  • 00:19:44
    view magic leap to had that but you
  • 00:19:46
    couldn't have that in a small form
  • 00:19:48
    factor or you could have something that
  • 00:19:51
    was more in the glasses form factor like
  • 00:19:53
    these X re but a fundamentally different
  • 00:19:56
    type of augmented reality display system
  • 00:19:59
    this one using bird bath Optics with the
  • 00:20:01
    pros and cons of that so uh Orion uses
  • 00:20:05
    wave guides and if you've been following
  • 00:20:07
    uh augment reality Hardware wave guides
  • 00:20:10
    as opposed to using uh a bird bath
  • 00:20:12
    combiner it's not having like displays
  • 00:20:15
    that bounce through a series of mirrors
  • 00:20:18
    uh to combine the image the rendered
  • 00:20:20
    image with the world image it's using a
  • 00:20:22
    display projector that shoots off AIS an
  • 00:20:26
    angle across a either printed or etched
  • 00:20:30
    guide these pathways on some type of
  • 00:20:33
    Clear Optical material traditionally
  • 00:20:36
    glass and the material that's chosen
  • 00:20:38
    allows has a certain property glass has
  • 00:20:41
    a refractive index I believe of at most
  • 00:20:44
    two um which means that and I'm not an
  • 00:20:47
    optical specialist here but the way I
  • 00:20:49
    understand it is that limits one the
  • 00:20:52
    angle that you have to project the light
  • 00:20:55
    into the wave guide and also the amount
  • 00:20:58
    the degree and the field of view the
  • 00:21:00
    angle which light can exit the wave
  • 00:21:02
    guide into your retina uh and with the
  • 00:21:06
    higher the refractive index index of
  • 00:21:09
    refraction the steeper the angle that
  • 00:21:11
    you're able to get that light in and
  • 00:21:13
    then wider then it bounces around and
  • 00:21:16
    bounces around to a point where you can
  • 00:21:17
    have a wider field of view exiting here
  • 00:21:20
    so combining it combines the benefits of
  • 00:21:22
    a wider field of view as well as a
  • 00:21:24
    smaller form factor the higher that
  • 00:21:26
    refractive index the wave guide material
  • 00:21:28
    they're using here uh that is it's
  • 00:21:30
    silicon carbide you may have seen that
  • 00:21:32
    if you've looked around or watched um
  • 00:21:35
    videos or read stories about Orion
  • 00:21:37
    they're talking about what a novel
  • 00:21:38
    material this is uh and so previously
  • 00:21:41
    silicon carbide as far as they know not
  • 00:21:43
    use uh in Optics it's used in EVS for
  • 00:21:48
    example um and there are a lot of these
  • 00:21:50
    properties they talk about that make it
  • 00:21:52
    ideal for AR so high refractive index
  • 00:21:55
    it's very lightweight lighter than glass
  • 00:21:57
    durable stronger as well and so you
  • 00:22:00
    don't need to have things like flexers
  • 00:22:02
    that around the actual frames to prevent
  • 00:22:05
    it from cracking it's all these benefits
  • 00:22:08
    it's just very expensive to produce so
  • 00:22:11
    when they talk about you know somewhere
  • 00:22:13
    around that $10,000 price point um to
  • 00:22:16
    manufacture one of these 9/10 of that uh
  • 00:22:19
    is actually in manufacturing growing
  • 00:22:22
    layer by layer atom by atom uh these
  • 00:22:25
    silicon car carbide wave guides which
  • 00:22:27
    have very low low yields because you're
  • 00:22:30
    generating them kind of off of a wafer
  • 00:22:32
    it's not like you can bend them and have
  • 00:22:35
    um have some that are not as good that
  • 00:22:37
    you can use in other products they have
  • 00:22:39
    to be perfect and then they also go
  • 00:22:41
    through a physically etched process to
  • 00:22:44
    etch those wave guides at this Nano
  • 00:22:46
    scale versus an imprinting of the wave
  • 00:22:49
    guides in previous AR Technologies uh
  • 00:22:52
    long story short a very complicated and
  • 00:22:55
    expensive way to get this property this
  • 00:22:58
    lens property that's uh allows it to be
  • 00:23:01
    that smallish form factor and I have to
  • 00:23:04
    be up front my Impressions my initial
  • 00:23:06
    Impressions and current Impressions
  • 00:23:07
    still of Orion is I don't think it's
  • 00:23:10
    achieved uh the everyday wear form
  • 00:23:13
    factor yet it yes much Slimmer than it's
  • 00:23:16
    not the headset like uh you know a
  • 00:23:18
    Vision Pro it's not a headset like
  • 00:23:20
    Hollow lens or magic La it does look
  • 00:23:23
    it's closer on that Spectrum to the
  • 00:23:26
    Rayman metas but there's still a little
  • 00:23:29
    bit the rims are still a little bit
  • 00:23:30
    thick and if you look at the the profile
  • 00:23:32
    the side of the Orion the thick there's
  • 00:23:34
    actually some thickness as well to the
  • 00:23:36
    actual frames themselves and that's
  • 00:23:37
    because if you look up close it's not
  • 00:23:40
    just like one piece of glass or one
  • 00:23:43
    piece of optical material there's
  • 00:23:45
    actually an optical stack that's
  • 00:23:47
    sandwiched in those frames um I don't
  • 00:23:51
    know the exact details of what every
  • 00:23:53
    layer uh of that Optical stack does but
  • 00:23:56
    my understanding is the the frontmost
  • 00:23:57
    layer can can be like a polarized layer
  • 00:23:59
    so if you're using this Outdoors you get
  • 00:24:01
    better better visibility for the
  • 00:24:03
    projected images uh there's the display
  • 00:24:06
    projector itself that has a liant layer
  • 00:24:09
    so it's using here U micro leads not
  • 00:24:12
    micro ol Leeds that you might have seen
  • 00:24:14
    on like the big screen Beyond or even
  • 00:24:16
    the the xreal uh glasses these are micro
  • 00:24:19
    Leeds and they're not organic they are
  • 00:24:21
    high efficiency uh displays that provide
  • 00:24:24
    a massive amount of brightness we're
  • 00:24:26
    talking about and the hundreds of
  • 00:24:28
    thousands of knits uh which are required
  • 00:24:30
    because that all gets reduced down as
  • 00:24:32
    the light goes to the wave guide so
  • 00:24:34
    you're talking about a a 300 to 400 nit
  • 00:24:37
    display with the current implementation
  • 00:24:40
    of the prototypes uh that display
  • 00:24:42
    projector lives in front of the silicon
  • 00:24:46
    carbide wave guide and then behind that
  • 00:24:48
    there is a combiner as well that the uh
  • 00:24:51
    aligns and rectifies the uh display
  • 00:24:54
    image with the outside world image so
  • 00:24:57
    make sure that's all matched up and then
  • 00:24:59
    the closest layer also has things that
  • 00:25:02
    support eye tracking uh this is actually
  • 00:25:04
    something really novel the closest layer
  • 00:25:06
    if you look up close to Orion um I
  • 00:25:10
    noticed like these little squiggly lines
  • 00:25:11
    almost look like dust or hairlines what
  • 00:25:14
    they actually what's embedded on this
  • 00:25:17
    frontmost layer uh of the of the optical
  • 00:25:20
    stack closest to your eyes are tiny IR
  • 00:25:24
    light emitters used for of course ey
  • 00:25:27
    tracking cuz you have have cameras in
  • 00:25:30
    the arms of the glasses looking at the
  • 00:25:34
    uh looking at the lens and so those tiny
  • 00:25:37
    squiggly lines are actually uh the way
  • 00:25:39
    those light emitters IR light emitters
  • 00:25:41
    are powered and they're squiggly because
  • 00:25:43
    they have to be this randomized pattern
  • 00:25:45
    so that when you're wearing the glasses
  • 00:25:48
    your eyes don't see the squiggly lines
  • 00:25:50
    you actually you can't see them because
  • 00:25:52
    they are a random pattern it's really
  • 00:25:54
    interesting idea of getting uh instead
  • 00:25:56
    of having a ring of IR lights that shoot
  • 00:25:59
    into your IE beams they're actually on
  • 00:26:01
    the Optics themselves but invisible at
  • 00:26:04
    that distance to the naked eye I did ask
  • 00:26:07
    about you know corrective lensing as
  • 00:26:09
    well um if people right now are using
  • 00:26:12
    contacts um to to use Orion but uh they
  • 00:26:16
    did say their plan and a lot of their
  • 00:26:18
    team do need corrective lensing so it's
  • 00:26:20
    going to that's something they want to
  • 00:26:21
    solve it's going to be built in um it's
  • 00:26:23
    like a thing where you'll want to submit
  • 00:26:25
    a prescription and when it's in the
  • 00:26:27
    assembly state
  • 00:26:28
    the lens element that's closest to your
  • 00:26:31
    eye will support some range of um some
  • 00:26:34
    range of RX correction they don't want
  • 00:26:38
    uh something you're going to snap on
  • 00:26:40
    they don't want an additional layer that
  • 00:26:42
    was really important to them again with
  • 00:26:44
    trying to achieve this goal of having a
  • 00:26:47
    display profile a a physical profile as
  • 00:26:50
    close to glasses as possible uh they
  • 00:26:53
    want that distance between your eyeball
  • 00:26:55
    and the lens to be know 1.5 cm 15 mm
  • 00:26:59
    which they're really proud of now in
  • 00:27:02
    terms of what this display looks like uh
  • 00:27:06
    70° diagonal fovs translate to 60°
  • 00:27:10
    horizontal and 40° vertical in using the
  • 00:27:13
    demo it seemed expansive and it wasn't I
  • 00:27:17
    couldn't see the edges of uh the limits
  • 00:27:20
    of where can render if I move my head
  • 00:27:22
    side to side uh the images I could force
  • 00:27:25
    a rendered window or an avatar I was
  • 00:27:27
    looking at to be clipped off at the
  • 00:27:29
    edges but I don't need 90° field of view
  • 00:27:33
    for augmented reality as I might want
  • 00:27:35
    you know more than that for virtual
  • 00:27:37
    reality I'm also wearing the glasses
  • 00:27:40
    themselves the frames themselves you're
  • 00:27:41
    also seeing the outside world the
  • 00:27:43
    benefit of having the real photons from
  • 00:27:46
    the world come to your eyes uh means
  • 00:27:48
    that as long as the rendered image kind
  • 00:27:50
    of fits in that 60x 40° field of view uh
  • 00:27:55
    it's wide enough to support multiple
  • 00:27:57
    windows so play a game to not feel
  • 00:28:00
    constrain not looking like you're
  • 00:28:01
    looking at a postage stamp version of a
  • 00:28:04
    a rendered image as we've seen before in
  • 00:28:06
    earlier Optical AR headsets it was also
  • 00:28:10
    one of the best AR systems I've used in
  • 00:28:12
    terms of not having a ton of visual
  • 00:28:15
    artifacts now there it wasn't zero
  • 00:28:18
    visual artifacts I did notice some
  • 00:28:20
    ghosting on the inner periphery if I
  • 00:28:23
    looked my eyes are there like a ghosting
  • 00:28:25
    of the image uh but there weren't stray
  • 00:28:28
    lights casting in it was like rainbow
  • 00:28:30
    effects so even though in some of the
  • 00:28:32
    images that you may have seen looking
  • 00:28:34
    into the glasses of like this kind of um
  • 00:28:37
    chromatic prism um it's not actually
  • 00:28:40
    what I see as I was looking through the
  • 00:28:43
    glasses looking out into the real world
  • 00:28:45
    uh the actual Holograms themselves were
  • 00:28:49
    bright um and not fully opaque but it
  • 00:28:53
    had like a glowy look to them uh one of
  • 00:28:56
    the things I noticed was the the
  • 00:28:58
    resolution wasn't that high either
  • 00:29:00
    they're saying with the current microed
  • 00:29:02
    displays they're getting about a 13
  • 00:29:04
    pixels per uh pixels per degree density
  • 00:29:07
    uh so you could read some text you could
  • 00:29:10
    browse a web page but it's not sharp uh
  • 00:29:13
    it's comparable to like early VR
  • 00:29:15
    headsets like the valve index is right
  • 00:29:18
    around 13 14 pixels per degree uh they
  • 00:29:22
    are working on different types of
  • 00:29:25
    display projectors in the future so I
  • 00:29:26
    did get to try a 26 pixel per degree
  • 00:29:30
    version of the Orion prototype and that
  • 00:29:31
    one was much sharper although less
  • 00:29:33
    bright and images also less opaque so
  • 00:29:37
    they're finding where those trade-offs
  • 00:29:39
    live uh when they want to ship this as a
  • 00:29:41
    product they're saying that they know
  • 00:29:43
    the road map and there is a path to
  • 00:29:45
    getting above 30 pixels per degree they
  • 00:29:48
    wouldn't want to ship something without
  • 00:29:49
    that they know that's a really important
  • 00:29:51
    part uh but the objects they they look
  • 00:29:54
    there they look present but they're
  • 00:29:56
    they're bright and glowy it's not
  • 00:29:58
    looking like you wouldn't confuse one of
  • 00:30:00
    those Holograms with a real object in
  • 00:30:02
    the real world it's very clearly that
  • 00:30:04
    holographic look of s that's very
  • 00:30:06
    colorful and bright the display system
  • 00:30:09
    is also fix Focus so it's not that holdy
  • 00:30:12
    Grail of having some type of light field
  • 00:30:15
    display where you can have version
  • 00:30:18
    accommodation across any focal distance
  • 00:30:21
    uh the fix Focus here is 1.5 M so
  • 00:30:23
    similar to VR systems and they know
  • 00:30:26
    there's a buffer range in which they can
  • 00:30:29
    display content combined with you know
  • 00:30:31
    stereo Imaging that gives you uh
  • 00:30:34
    comfortable 3D looking images without
  • 00:30:35
    straining our eyes without straining the
  • 00:30:37
    accommodation it is a I think a
  • 00:30:40
    limitation or parameter of the type of
  • 00:30:43
    wave guide Optics they're going towards
  • 00:30:45
    so these are not you light field Optics
  • 00:30:47
    I don't think we're going to get those
  • 00:30:49
    in a glasses form factor uh but it
  • 00:30:52
    really was also emphasized how important
  • 00:30:54
    it was that the display images not only
  • 00:30:57
    between left and right eye but also with
  • 00:31:00
    the camera systems the outward World
  • 00:31:02
    facing camera systems are perfectly
  • 00:31:05
    aligned there's kept on stressing it is
  • 00:31:07
    aligned the Precision of less than a
  • 00:31:09
    tenth of the thickness of a human hair
  • 00:31:12
    uh because any disparity like vertical
  • 00:31:14
    disparity between left and right eye or
  • 00:31:17
    disparity between the images and the
  • 00:31:19
    world calibration which can be affected
  • 00:31:22
    by things like thermals expansion of the
  • 00:31:25
    hardware on a hot day that would be be
  • 00:31:28
    uncomfortable because you also have the
  • 00:31:30
    real world as a reference and so the
  • 00:31:32
    rendered images have to look perfect
  • 00:31:34
    relative to each other as well as the
  • 00:31:37
    real world that's really important for
  • 00:31:38
    AR they could not stress heavily enough
  • 00:31:41
    how important it was for that uh for
  • 00:31:44
    that alignment to me in place that's
  • 00:31:45
    reinforced by the Magnesium that they're
  • 00:31:47
    using for the frames as well as even
  • 00:31:50
    some real time adjustment and a digital
  • 00:31:54
    um calibration um based on the sensing
  • 00:31:56
    of how the glasses are being being worn
  • 00:31:59
    and this glasses form factor which I
  • 00:32:01
    think is probably the biggest this type
  • 00:32:03
    of glass will ever be hopefully as they
  • 00:32:06
    turn this into a real product they did
  • 00:32:08
    say that they had ways to make it
  • 00:32:10
    thinner you know BOS talked about like
  • 00:32:12
    not only they had ways to condense the
  • 00:32:14
    thickness of the frames by maybe moving
  • 00:32:18
    the projector instead of in front of the
  • 00:32:21
    wave gu to behind the wave guide um but
  • 00:32:24
    you need more uh calibration or real
  • 00:32:27
    time adjustment but that that actually
  • 00:32:29
    gets you half the thickness uh they also
  • 00:32:32
    think the rims themselves can be less
  • 00:32:34
    thick as well as well as a more
  • 00:32:36
    ergonomically designed uh arms so much
  • 00:32:40
    of the the physical ID is also
  • 00:32:42
    constrained by how many how much
  • 00:32:44
    Electronics they need to pack in the ear
  • 00:32:46
    and the the power envelope of this and
  • 00:32:48
    so they've developed a bunch of custom
  • 00:32:51
    silicon Hardware to run things at very
  • 00:32:53
    low power so display itself that micr
  • 00:32:57
    Leed that that's consuming most of the
  • 00:32:59
    power but you also have onboard
  • 00:33:01
    processing for slam World tracking for
  • 00:33:04
    hand tracking for ey tracking that's all
  • 00:33:06
    running locally uh spread across weight
  • 00:33:10
    distributed on these glasses and then
  • 00:33:12
    you also need battery as well so things
  • 00:33:15
    that traditionally would be consuming a
  • 00:33:17
    lot of power on a phone they're trying
  • 00:33:20
    to get to you know a third A fifth or a
  • 00:33:22
    tenth of power required so the dozens of
  • 00:33:26
    milliamps aggregated to you know a
  • 00:33:28
    couple hundred milliamps versus a couple
  • 00:33:31
    hundred milliamps aggregated to you know
  • 00:33:33
    like a l of power being used because
  • 00:33:35
    then you wouldn't be able to use this
  • 00:33:36
    for the know 2 and a half to three hours
  • 00:33:38
    of use time that they say those glasses
  • 00:33:40
    currently have so battery chemistry
  • 00:33:43
    there's just real physics limitations
  • 00:33:45
    you know the shape of the batteries they
  • 00:33:46
    can kind of mix around but it's the
  • 00:33:48
    classic you know every cubic millimeter
  • 00:33:51
    space need to be occupied and Balan in
  • 00:33:54
    some way and every Dimension that you
  • 00:33:57
    change affects some other Factor so
  • 00:34:01
    that's the that's the I think the real
  • 00:34:02
    breakthrough and innovation in the
  • 00:34:05
    glasses part of a Rion it's not that it
  • 00:34:07
    has you know the widest perfect field of
  • 00:34:10
    view or you know it has the longest
  • 00:34:13
    battery life for the most powerful and
  • 00:34:15
    smooth compute it's getting all of those
  • 00:34:18
    separate ingredients together in the
  • 00:34:20
    physical limitation of that form factor
  • 00:34:23
    that they've been able to achieve and
  • 00:34:24
    then they can say okay now that they got
  • 00:34:26
    70° diagonal field of view in this 98 G
  • 00:34:29
    package where the place is they can
  • 00:34:32
    tweak to make it more affordable as a
  • 00:34:35
    real consumer product um and a lot of
  • 00:34:38
    that has to do with the second device
  • 00:34:40
    the compute Puck as well so while there
  • 00:34:42
    is local processing on the glasses for
  • 00:34:45
    things like slam uh app logic so what
  • 00:34:47
    actually runs applications uh things
  • 00:34:50
    that don't need immediate low latency
  • 00:34:52
    that would make it uncomfortable to use
  • 00:34:54
    in AR glasses that can run on a wireless
  • 00:34:57
    computer Ute device and it being
  • 00:34:59
    Wireless such a big deal uh they talked
  • 00:35:01
    about how they've created unique
  • 00:35:04
    protocol that works on top of Wi-Fi 6
  • 00:35:06
    where they're pulsing the data not
  • 00:35:08
    streaming data non-stop which gives them
  • 00:35:11
    a lot of power savings and then of
  • 00:35:13
    course tuning the applications to work
  • 00:35:15
    with that um the compute Puck is
  • 00:35:18
    something because it's wireless it
  • 00:35:20
    doesn't even need to be in your pocket
  • 00:35:22
    it can be in a bag they talked about it
  • 00:35:24
    being in a backpack or a purse with a
  • 00:35:26
    range of about 10 to 12 ft so uh that
  • 00:35:30
    can last all day while the glasses can
  • 00:35:32
    last a couple hours and something that
  • 00:35:34
    as long as it's close to you you're
  • 00:35:35
    going to get that full functionality of
  • 00:35:38
    the glasses uh an interesting side note
  • 00:35:41
    with the compute Puck you'll see in some
  • 00:35:43
    of the video that we shot of it that it
  • 00:35:45
    looks like there are cameras on the
  • 00:35:47
    front of it as well as on the side on
  • 00:35:50
    this bevel and those aren't actually
  • 00:35:52
    cameras in use those are actually
  • 00:35:53
    artifacts uh left over from earlier
  • 00:35:56
    implementations of explorations of how
  • 00:35:58
    they could use this compute Puck and if
  • 00:36:00
    you're thinking it as a controller they
  • 00:36:03
    also thought of it as a controller uh
  • 00:36:05
    the cameras on the front and maybe the
  • 00:36:08
    the little uh divot on the top of the
  • 00:36:10
    compute Puck uh inv verions uh previous
  • 00:36:13
    versions allowed to work as a six degree
  • 00:36:15
    Freedom tracking Precision controller
  • 00:36:17
    kind of like a you know touch Pro
  • 00:36:19
    Controller that you got on the the quest
  • 00:36:20
    Pro Inside Out tracking on the device
  • 00:36:23
    itself they decided they didn't want to
  • 00:36:25
    focus on that that was maybe too much
  • 00:36:27
    spreading too thin so they have hand
  • 00:36:29
    tracking instead uh the cameras on the
  • 00:36:32
    bevel in front of it were to maybe place
  • 00:36:35
    it on a table so it can look back at you
  • 00:36:37
    and do some kind of skeletal modeling so
  • 00:36:40
    that when you are on a video call your
  • 00:36:43
    avatar could be fully tracked uh but
  • 00:36:45
    they also didn't want you to have to
  • 00:36:47
    require you know they wanted this to be
  • 00:36:48
    a device that you can be using while
  • 00:36:50
    walking around and not have to require
  • 00:36:52
    taking the compute P out of your pocket
  • 00:36:54
    putting on a bench or a table just to do
  • 00:36:56
    a video call so making those very uh
  • 00:36:59
    deliberate decisions that will then rely
  • 00:37:01
    on the camera systems B on the headset
  • 00:37:04
    itself uh inferences that they can then
  • 00:37:06
    use extrapolate over machine learning
  • 00:37:08
    for things like body tracking face
  • 00:37:10
    tracking for their codic avatars but
  • 00:37:13
    really cool to see some of those uh
  • 00:37:16
    design decisions that they explored
  • 00:37:18
    still on the Prototype Hardware itself
  • 00:37:21
    uh so the compute P doesn't do the input
  • 00:37:24
    and while there is it tracking input
  • 00:37:25
    that we talked about there is hand
  • 00:37:26
    tracking input put uh you can combine
  • 00:37:29
    those very much in like a apple Vision
  • 00:37:30
    Pro Way gaze and tap and pinch or moving
  • 00:37:33
    things around uh there's also this
  • 00:37:36
    wristband so it's also the first time
  • 00:37:38
    we're seeing in a product the investment
  • 00:37:40
    they made when they acquired control
  • 00:37:43
    labs and they've published papers since
  • 00:37:45
    then about EMG wristbands so this is a a
  • 00:37:50
    wristband that uses a non-invasive
  • 00:37:52
    sensing of the electrical signals that
  • 00:37:55
    go to your muscles in your wrists so the
  • 00:37:57
    complex sense of array of muscles that
  • 00:38:00
    are moved um as you move your fingers or
  • 00:38:03
    the orientation of your hand that can
  • 00:38:05
    get translated and processed through
  • 00:38:07
    their ml processor on board The
  • 00:38:09
    Wristband itself and then turn into
  • 00:38:12
    gestures and poses that the system can
  • 00:38:15
    recognize um I was really curious you
  • 00:38:17
    know because they invested in this uh
  • 00:38:20
    EMG system many many years ago I'm sure
  • 00:38:23
    theyve spent a lot of money developing
  • 00:38:25
    it but machine learning is also gotten a
  • 00:38:28
    lot more advanced in the past couple
  • 00:38:29
    years and something as basic as an IMU a
  • 00:38:32
    gyroscope an IMU inside you know like a
  • 00:38:35
    watchband uh we've seen uh with demos at
  • 00:38:38
    atic World Expo even in accessibility
  • 00:38:40
    options on the Apple watch you can get
  • 00:38:43
    tap uh you know pretty reliable tap
  • 00:38:46
    gestures without needing to tap into EMG
  • 00:38:50
    the IMU can actually be trained uh that
  • 00:38:52
    data can be trained to get you a pretty
  • 00:38:55
    good success rate what boss and the
  • 00:38:58
    Orion team told me is that the potential
  • 00:39:02
    of what the wristbands can deliver over
  • 00:39:04
    time that ceiling is so has so much
  • 00:39:07
    prise so much potential the very basic
  • 00:39:10
    gestures that I was used with no system
  • 00:39:13
    training no calibration training uh it
  • 00:39:15
    was just tapping so I could use a thumb
  • 00:39:18
    to index finger to tap and this doesn't
  • 00:39:21
    need to be in front of me it can
  • 00:39:22
    literally be in my hand in my pocket
  • 00:39:24
    imagine I'm walking down the street
  • 00:39:26
    hands in my jacket pocket and I can be
  • 00:39:28
    controlling the interface with Taps I
  • 00:39:31
    can be turning my wrists and tapping my
  • 00:39:33
    thumb to my middle finger and bringing
  • 00:39:35
    up a menu uh I can be scrolling so one
  • 00:39:38
    of the most uh impressive gestures was
  • 00:39:41
    just putting my thumb on my inex finger
  • 00:39:44
    and swiping up and down and scrolling
  • 00:39:47
    through like an Instagram video feed I
  • 00:39:49
    really tried to see like how much
  • 00:39:51
    pressure how much intentionality did I
  • 00:39:53
    need to put into that gesture and it
  • 00:39:55
    turned out not much at all just that
  • 00:39:57
    gentle small micro gesture as they would
  • 00:40:00
    call it was enough to get scrolling and
  • 00:40:03
    that's where it seems to be very a very
  • 00:40:05
    powerful um very powerful form of input
  • 00:40:09
    using your hand versus the uh computer
  • 00:40:12
    vision based input you know CV based
  • 00:40:14
    input of the hand tracking on Apple
  • 00:40:16
    Vision Pro I've loveed the vision proos
  • 00:40:19
    combination of gaze and pinch and
  • 00:40:21
    gesture tracking it's been really
  • 00:40:23
    intuitive and really powerful to use but
  • 00:40:25
    over this past year of using that I've
  • 00:40:26
    also realize that uh a lot of big
  • 00:40:29
    movements in my hands are are still
  • 00:40:31
    pretty straining so you know grabbing
  • 00:40:34
    something pinching something and moving
  • 00:40:35
    it with my elbow or moving with my wrist
  • 00:40:38
    which I do more than I probably should
  • 00:40:41
    that's not comfortable over time it's
  • 00:40:44
    really that our hands are built for the
  • 00:40:46
    dexterity of our finger joints and our
  • 00:40:48
    finger movements so that's why you know
  • 00:40:50
    holding a stylus holding a pencil it's
  • 00:40:52
    about moving you know your finger
  • 00:40:54
    Knuckles not about moving your wrist
  • 00:40:56
    when you're typing on the keyboard
  • 00:40:57
    you're not you don't want carpal tunnel
  • 00:40:59
    you don't want to be moving your wrist
  • 00:41:00
    so the micro gestures the smaller the
  • 00:41:02
    gesture that you can make that the
  • 00:41:04
    system can still interpret as these
  • 00:41:07
    separate discrete input types uh the
  • 00:41:10
    more powerful and the more comfortable
  • 00:41:12
    and more long-term use that you can get
  • 00:41:15
    out of this which you'll need for
  • 00:41:16
    augmented reality and that's why they're
  • 00:41:19
    committing to this EMG wristband I did
  • 00:41:22
    ask about you know multiple wristbands
  • 00:41:24
    right now there's one and it's a very
  • 00:41:25
    you know spelt form factor
  • 00:41:28
    uh but they've showed demos of people
  • 00:41:29
    using two resp spans they talk about in
  • 00:41:31
    the future being able to recognize you
  • 00:41:33
    know Handwriting Without actually
  • 00:41:35
    physically writing on on on a piece of
  • 00:41:38
    paper uh and they even uh have trained
  • 00:41:41
    versions of this where someone can be
  • 00:41:42
    typing on a keyboard that isn't there
  • 00:41:45
    didn't get to try any of those yet but
  • 00:41:47
    for the basic gestures of you know
  • 00:41:49
    tapping of opening up a menu of swiping
  • 00:41:53
    of double tapping for tapping into meta
  • 00:41:55
    AI all of those work with
  • 00:41:58
    one or two false positives you know even
  • 00:42:00
    like kind of the small ticks I had my
  • 00:42:01
    arms folded and you know just moving my
  • 00:42:03
    hands rubbing my hands together I was
  • 00:42:05
    activating their meta AI accidentally so
  • 00:42:07
    something I'll probably have to train
  • 00:42:09
    myself to use for the system as well as
  • 00:42:11
    the system learning my own proclivities
  • 00:42:13
    for how I move my hands lots of
  • 00:42:16
    interesting uh tech there and then the
  • 00:42:19
    demo experience itself there was an
  • 00:42:21
    onboarding process so a whole fitment
  • 00:42:24
    process where they got the glasses to
  • 00:42:26
    work with with my face the arms are
  • 00:42:29
    bendable so they do Bend outward and
  • 00:42:31
    they Bend not at the traditional point
  • 00:42:34
    of uh where a hinge would be it's lower
  • 00:42:36
    and that's probably for the added
  • 00:42:38
    rigidity again for the the two dis uh
  • 00:42:42
    display systems be calibrated and the
  • 00:42:44
    camera recalibrated uh but they did give
  • 00:42:46
    me a nose Bridge so it would stick to my
  • 00:42:48
    nose as well as these rubber grips to
  • 00:42:50
    make sure that the class's AR arms
  • 00:42:52
    weren't moving because uh the way their
  • 00:42:54
    eye tracking works I got a sense that if
  • 00:42:56
    I was pushing the glasses against my
  • 00:42:58
    face from time to time I would lose eye
  • 00:43:01
    track and calibration and have to do
  • 00:43:02
    that process again it was a very on
  • 00:43:05
    Rails demo lots of guard rails they
  • 00:43:07
    walked me through this very evenly
  • 00:43:09
    diffused lit room so no Bright Lights
  • 00:43:12
    lots of Hard angles probably help with
  • 00:43:15
    the slam World tracking they walk me
  • 00:43:17
    through you know from a couch to uh a
  • 00:43:20
    little display area that set up uh I did
  • 00:43:22
    very basic things there was uh a video
  • 00:43:24
    call so chat it with someone uh when in
  • 00:43:27
    which they would have seen a codak
  • 00:43:29
    avatar interestingly enough the seven
  • 00:43:32
    camera systems they there is no camera
  • 00:43:34
    system pointed down your face uh there's
  • 00:43:37
    no face tracking camera like the quest
  • 00:43:39
    Pro or the Apple Vision Pro has so
  • 00:43:41
    that's probably using your voice and
  • 00:43:44
    then using machine learning to convert
  • 00:43:46
    that into mouth movement of your Kodak
  • 00:43:48
    Avatar which probably is rendered in the
  • 00:43:50
    cloud and stream down as video so it's
  • 00:43:53
    not like a spatial persona for say it's
  • 00:43:56
    still in a windy in the video but you
  • 00:43:58
    know you had um asymmetrical experiences
  • 00:44:01
    where I was doing video calls with
  • 00:44:02
    someone just holding up on the phone uh
  • 00:44:05
    a video call with someone in a codec
  • 00:44:06
    Avatar again a flat codec Avatar and
  • 00:44:09
    then things like web browsing video
  • 00:44:11
    watching all of which looked and worked
  • 00:44:14
    as kind of I would imagine so you know
  • 00:44:18
    I'm speaking about it very plainly and
  • 00:44:19
    almost like unremarkably and the the
  • 00:44:22
    amazing part was how kind of natural and
  • 00:44:24
    unremarkable how how it the experience
  • 00:44:28
    match in my brain what I hoped it would
  • 00:44:30
    be uh performance was a little bit laggy
  • 00:44:34
    I could tell not that there was the
  • 00:44:35
    latency between the glasses and the
  • 00:44:37
    wireless compute Puck but maybe they're
  • 00:44:40
    pushing the limits of how many windows
  • 00:44:42
    they could display and uh I didn't get a
  • 00:44:45
    sense of the frame rate uh of you know
  • 00:44:47
    the images it wasn't like dragging
  • 00:44:49
    window things were lagging behind but it
  • 00:44:52
    was only the three panes they were
  • 00:44:54
    locked in line to each other uh kind of
  • 00:44:57
    like your quest dashboard um and even
  • 00:45:00
    though I could put that alignment of
  • 00:45:03
    three Windows anywhere it wasn't locking
  • 00:45:05
    those windows to the room mapping for
  • 00:45:08
    example I get to use uh one instance of
  • 00:45:11
    men AI where you know like the Rayband
  • 00:45:14
    metas I asked it a prompt to look at the
  • 00:45:16
    scene in front of me in this guided demo
  • 00:45:19
    it was a scene of ingredients to make a
  • 00:45:20
    smoothie and then it did the thing where
  • 00:45:23
    it you could hear it take a picture a
  • 00:45:25
    shutter snap it would process it and
  • 00:45:28
    after a couple seconds I would get these
  • 00:45:31
    U indications on top of the actual
  • 00:45:35
    objects themselves labeling so uh the
  • 00:45:37
    cool thing here was as opposed to
  • 00:45:39
    getting audio feedback which you can do
  • 00:45:41
    right now with the meta AI on Rayband
  • 00:45:43
    metas uh it was actually World locked
  • 00:45:46
    visual feedback identifying each of
  • 00:45:49
    these products on the table with a label
  • 00:45:52
    that was attached to them and then a
  • 00:45:54
    floating list of like smoothie making
  • 00:45:57
    instructions uh that type of world
  • 00:45:59
    loocking is what I what we want from AR
  • 00:46:01
    and we want more of that scene
  • 00:46:03
    recognition scene understanding you know
  • 00:46:06
    persistence and uh my sense is that
  • 00:46:08
    persistence isn't there yet you know
  • 00:46:10
    it's for the duration of that
  • 00:46:11
    interaction with Med AI uh but it
  • 00:46:15
    doesn't have the capabilities to know
  • 00:46:17
    that I want this XYZ recognize and have
  • 00:46:20
    it you know persistent as I'm walking
  • 00:46:22
    between rooms or walking the outside
  • 00:46:24
    world uh continuously track continuous
  • 00:46:27
    updating that might be more for you know
  • 00:46:28
    as you get more compute or they figuring
  • 00:46:30
    that out later I did a demo where I was
  • 00:46:33
    sitting at a kind of a high bench coffee
  • 00:46:36
    table and and then you know scrolling
  • 00:46:39
    through social media so scrolling
  • 00:46:41
    through Instagram again here using the
  • 00:46:44
    very small micro gestures just swiping
  • 00:46:47
    up and down that was one of the
  • 00:46:49
    Breakthrough moments for me how easy
  • 00:46:52
    intuitive and not laborious unstrained
  • 00:46:56
    it my hand was was just moving doing
  • 00:46:57
    these micro gestures it was really
  • 00:47:00
    amazing um and then there were some
  • 00:47:03
    games so they had this kind of 3D game
  • 00:47:06
    you kind of place anywhere and it's an
  • 00:47:09
    ey track and and then tap based gesture
  • 00:47:12
    based uh activation where you're looking
  • 00:47:14
    at you know where you want the enemy
  • 00:47:16
    your spaceship to shoot very simple kind
  • 00:47:18
    of asteroid style shooter not super
  • 00:47:22
    impressed with that but the final demo
  • 00:47:25
    this pong demo that one blew my mind
  • 00:47:28
    that one was done in two ways one with
  • 00:47:30
    collocation so with uh the project Le
  • 00:47:34
    Orion giving me the demo looking at him
  • 00:47:37
    he's wearing Orion glasses I'm wearing
  • 00:47:39
    Orion glasses and between us are two
  • 00:47:43
    glowing grids that are locked in place
  • 00:47:47
    for both of us and with stereo There's
  • 00:47:49
    real like sense of depth there and this
  • 00:47:52
    you know glowing pong ball that's going
  • 00:47:54
    between these two uh squares glowing
  • 00:47:58
    blue and glowing orange squares and then
  • 00:48:01
    using my hand to kind of move a gesture
  • 00:48:03
    and try to spin that pong ball as it
  • 00:48:05
    moves through 3D space that seemed so
  • 00:48:10
    futuristic so science fiction it was
  • 00:48:13
    amazing and then looking around the room
  • 00:48:15
    and of course everyone looking at us and
  • 00:48:17
    they just see two people holding their
  • 00:48:19
    hands out playing you know waving in the
  • 00:48:21
    air looking like they W were waving each
  • 00:48:23
    other but we're playing a multiplayer
  • 00:48:25
    game of holog graphic pong that was so
  • 00:48:28
    cool I did another version of pong where
  • 00:48:31
    instead of playing with someone in the
  • 00:48:33
    space I was playing against a stylized
  • 00:48:35
    full body Avatar and that Avatar was
  • 00:48:38
    maybe standing you know 7 8 ft away from
  • 00:48:42
    me and here with the 40° vertical field
  • 00:48:45
    of view I could see the Avatar in its
  • 00:48:47
    entirety you know it was there and I
  • 00:48:49
    didn't need to look up and down and have
  • 00:48:52
    their head or feet cropped off uh and
  • 00:48:54
    had them you know fully animated with
  • 00:48:57
    their hand tracked as well and talking
  • 00:48:59
    to them with spatial audio um it was
  • 00:49:02
    like a holographic presence it wasn't
  • 00:49:05
    the photorealistic codic Avatar this one
  • 00:49:07
    was way more stylized and glowy um but
  • 00:49:11
    give that sense of if you again over
  • 00:49:13
    time with more compute having more of
  • 00:49:16
    those walking around in your space that
  • 00:49:18
    feels really like the Hol like science
  • 00:49:22
    fiction throughout my entire demo
  • 00:49:25
    session it was continually reinforced to
  • 00:49:27
    me that as impressive and as proud of of
  • 00:49:31
    this prototype that they spent so many
  • 00:49:33
    years on uh they know they can do better
  • 00:49:36
    they know the paths forward to make it
  • 00:49:38
    more of a consumer device and I think
  • 00:49:40
    it's a really good decision that even
  • 00:49:42
    though they intentionally originally
  • 00:49:44
    intended to make Orion something they
  • 00:49:46
    could sell that they're not selling this
  • 00:49:48
    right now uh if I was to imagine what a
  • 00:49:51
    consumer version of Orion may look like
  • 00:49:55
    uh I don't think it's going to end up
  • 00:49:56
    using that silicon carbide I think that
  • 00:49:58
    they're realizing that the benefits of
  • 00:50:00
    it that wi fov like the costs are too
  • 00:50:03
    high they're not going to be able to
  • 00:50:04
    scale that and make it affordable so you
  • 00:50:06
    might have a little bit of a lower fov
  • 00:50:08
    or the find the trade-offs that they
  • 00:50:10
    have to figure out uh they know they
  • 00:50:12
    need a higher density display they need
  • 00:50:14
    they want to ship something with you
  • 00:50:15
    know at least 30 pixels per degree so it
  • 00:50:18
    might not be as bright or opaque as the
  • 00:50:21
    Holograms I saw in this demo but they
  • 00:50:24
    also know they're going to get a smaller
  • 00:50:25
    form factor with a thinner um thinner
  • 00:50:29
    Optical stack with thinner bezels and
  • 00:50:32
    more ergonomic arms um and it seems like
  • 00:50:36
    they're Welling their way to figure out
  • 00:50:38
    that what that means as a product
  • 00:50:39
    because they really want this to replace
  • 00:50:42
    the phone and that also means they want
  • 00:50:44
    to charge a lot of money for it it's not
  • 00:50:47
    going to be an accessory that you buy
  • 00:50:49
    just to use on airplanes or in the car
  • 00:50:52
    uh they want people to buy these as you
  • 00:50:55
    know maybe their second most important
  • 00:50:56
    Computing device and they talked about
  • 00:50:59
    maybe pricing being in the range of a
  • 00:51:00
    high-end phone or laptop so in my mind
  • 00:51:03
    maybe like 1,500 probably $2,000 is
  • 00:51:06
    where I can see this ending up um and I
  • 00:51:09
    would rather that they charge more and
  • 00:51:12
    make sure they include as much of that
  • 00:51:14
    technology as possible then try to get
  • 00:51:16
    something that's more mass Market appeal
  • 00:51:18
    and not have the experience be as good
  • 00:51:21
    but I'm also super excited about where
  • 00:51:23
    the technologies that they've built into
  • 00:51:25
    Orion are going to make their way into
  • 00:51:28
    Quest products or Rayband meta products
  • 00:51:30
    you know specifically those EMG
  • 00:51:32
    wristbands I would love to have Micro
  • 00:51:35
    gestures I could use uh while using
  • 00:51:38
    glasses to peruse you know some type of
  • 00:51:41
    social media feed or scroll have another
  • 00:51:43
    way of doing input as opposed to just
  • 00:51:45
    tapping the sides of the glasses or just
  • 00:51:47
    using uh voice control for their AI
  • 00:51:50
    system so it'll be a ways away I think
  • 00:51:52
    before we'll see the consumer product
  • 00:51:55
    that Orion and up being uh and this
  • 00:51:59
    prototype they've made the Thousand or
  • 00:52:00
    so that they've made are not for
  • 00:52:02
    developers unlike you know the snap
  • 00:52:04
    spectacles that were announced a couple
  • 00:52:06
    weeks ago uh developers can't get this
  • 00:52:09
    right now it's not about building an app
  • 00:52:11
    library or an app store you know when we
  • 00:52:14
    think of the iPhone you know that killer
  • 00:52:16
    app on the iPhone yes it's the App Store
  • 00:52:18
    it's the thing that's made it so
  • 00:52:19
    powerful but when the iPhone was
  • 00:52:21
    announced Steve Jobs walked on stage and
  • 00:52:23
    he sold it as a IP OD a phone and a
  • 00:52:28
    breakthrough internet communicator and
  • 00:52:31
    it was about getting the fundamentals
  • 00:52:33
    right having real value and utility in
  • 00:52:36
    the vice out of box so people will keep
  • 00:52:38
    on using it rather than trying to put it
  • 00:52:40
    out there and figure out and let
  • 00:52:42
    developers figure out that will come
  • 00:52:43
    later so you know meta Orion team take
  • 00:52:47
    your time get those Tech into things
  • 00:52:49
    like The rayan Meta into Quest and I
  • 00:52:52
    think they're well on their way um to
  • 00:52:55
    bring us the future
  • 00:52:57
    hope you enjoy that deep dive into Orion
  • 00:53:00
    uh if you have questions about my
  • 00:53:01
    experience please post them in the
  • 00:53:03
    comments below I'll try to reach out to
  • 00:53:05
    the folks that met to answer those as
  • 00:53:07
    best as possible um and thank you so
  • 00:53:10
    much for watching as always I'm norm and
  • 00:53:13
    we'll see you next time
Tags
  • Meta
  • Orion
  • Augmented Reality
  • Holographic Display
  • Neural Interface
  • Prototype
  • AI
  • Smartphone Replacement
  • Wearable Technology
  • Innovative Materials