Devin will take your job soon? 500$/month for a coding assistant!

00:56:06
https://www.youtube.com/watch?v=_Kp_XZV2W0k

Resumo

TLDRThe video explores the role of AI and automation in transforming society, focusing primarily on "Devon," a tool positioned as an AI-powered junior engineer designed to automate coding tasks. The speaker, a full-stack developer, provides an evaluation of Devon, examining its capabilities, value, and whether it can effectively replace human coders. Through a series of tests involving real coding issues, the speaker determines that Devon, while innovative, is not yet ready to replace engineers due to its limitations and high cost ($500 per month). It can handle simple coding tasks and integrate with platforms like Slack and GitHub, but struggles with more complex tasks without human intervention. The video highlights the need for significant improvements in the underlying language models (LLMs) and AI infrastructure. The narrator suggests that while the concept of AI in coding is promising, Devon and similar products must evolve further to justify their price and be considered fully effective. The video also advises against non-technical people investing in Devon, reinforcing that technical expertise is needed to navigate and derive value from such AI tools.

Conclusões

  • 🤖 Devon is an AI tool meant to assist in coding tasks like a junior engineer.
  • 💡 The potential of AI in coding exists, but current capabilities are limited.
  • 💸 High cost of Devon ($500/month) is seen as a barrier to widespread adoption.
  • 📉 Unable to replace engineers fully; further improvements are needed.
  • 🔗 Integrates with tools like Slack and GitHub for better workflow.
  • 🤔 Requires technical knowledge to set up and use effectively.
  • 🚧 Faces challenges with complex tasks, indicating scope for LLM improvement.
  • 🔍 Provides insights into the future of AI in the software industry.
  • 📈 Could potentially reduce repetitive coding tasks if enhanced.
  • 🛠 Full-stack developers find it promising but not ready for full implementation.

Linha do tempo

  • 00:00:00 - 00:05:00

    In the beginning, the speaker discusses the impact of AI and automation on society, focusing on a demonstration by Deon Devon of autonomous coding. Concerns about the future of coding jobs are raised, despite AI's incremental capabilities like the AI Junior Engineer.

  • 00:05:00 - 00:10:00

    The speaker evaluates the AI Junior Engineer's capabilities after purchasing a $500 subscription to test if it could replace engineers. The AI struggles with certain repository issues, only partially solving one. Its appearance and integration into tools like Slack have improved, but its foundational logic remains unchanged without significant LLM improvements.

  • 00:10:00 - 00:15:00

    Local development, preferred due to its familiarity, is deemed more practical than complete AI dependence unless substantial advancements occur. Cloud-based tools, perceived as non-essential staples, validate this sentiment. The high subscription cost makes it economically irrational for long-term use compared to GitHub Copilot.

  • 00:15:00 - 00:20:00

    The speaker describes the onboarding process for the AI tool, including connecting to GitHub and paying the subscription fee. They connect it with a repository and authorize Slack integration, noting the polished feel akin to onboarding a new engineer.

  • 00:20:00 - 00:25:00

    Upon onboarding completion, they instruct the AI to verify its capabilities by creating a draft PR. The AI generates a plan and executes steps, but faces minor configuration issues. Despite successfully creating a draft PR, the AI's proposal doesn't accurately address the underlying issue of pagination.

  • 00:25:00 - 00:30:00

    The AI is tasked to fix a pagination issue, but misunderstands the problem, wrongly altering pagination elsewhere. The fix disregards best practices like prefetching, ultimately delivering a solution that addresses non-existent issues—inadvertently breaking application behavior altogether.

  • 00:30:00 - 00:35:00

    Aiming for a successful fix, the AI is given a simpler UI task to adjust for image responsiveness. The AI suggests code changes, but they prove ineffective. The speaker considers frontend issues challenging for AI due to visual assessment limits, highlighting discrepancies between human and AI comprehension.

  • 00:35:00 - 00:40:00

    The AI's struggles with frontend tasks are reiterated, its inability to resolve issues compared against a developer's more thorough solution. A backend task is later presented, deemed more suitable given AI's logic processing strengths. The AI's potential shines through coding actions and its preference for backend duties.

  • 00:40:00 - 00:45:00

    Creating backend fixes, the AI independently addresses certain logic-heavy tasks. The discussions show it could potentially excel with clear, structured requirements, albeit frontend and visually determinative challenges necessitating human intervention for comprehensive solutions.

  • 00:45:00 - 00:50:00

    In the final summary, despite efforts for improvement, the AI remains challenged on tasks requiring cognitive flexibility and human intuition. Combining AI's backend efficiency with human-driven frontend resolutions could yield substantive results, though current AI limitations necessitate human oversight.

  • 00:50:00 - 00:56:06

    The speaker concludes with reflections on AI's current utility, noting rapid market evolution and potential pricing adjustments. They express skepticism at the AI's premium cost, with existing cheaper options providing similar functionality, highlighting areas needing improvement for feasible autonomy.

Mostrar mais

Mapa mental

Vídeo de perguntas e respostas

  • What is the video about?

    The video discusses AI, automation, and a specific product called Devon, an AI-enabled junior engineer.

  • Who is Devon in the context of the video?

    Devon is an AI tool designed to work as a junior engineer, automating coding tasks.

  • What are the concerns about AI in automation expressed in the video?

    Concerns include the reliability of AI in replacing human engineers, its current capabilities, cost, and the comparison with existing AI tools.

  • Is Devon effective in replacing human engineers?

    The video suggests that Devon is not yet effective in fully replacing engineers, requiring further improvements.

  • What improvements are suggested for Devon?

    Improvements include better alignment and progress in underlying language models (LLMs) for enhanced accuracy and usefulness.

  • How much does a Devon subscription cost?

    The Devon subscription costs around $500 per month.

  • What are the tested capabilities of Devon in this video?

    Devon was tested on solving coding issues, creating pull requests, and integrating with development environments, but faced challenges in completing these tasks efficiently.

  • Does the video recommend using Devon for non-technical users?

    No, it is suggested that non-technical users should not purchase Devon as it requires coding knowledge.

  • How does the narrator compare Devon to other AI tools?

    Devon is compared to GitHub Copilot and other local AI tools, indicating it does not provide significant advantages.

  • What are the concluding thoughts on Devon's utility?

    The narrator concludes that Devon needs more time to mature and improve in both functionality and pricing before it can be deemed truly useful for full-stack development.

Ver mais resumos de vídeos

Obtenha acesso instantâneo a resumos gratuitos de vídeos do YouTube com tecnologia de IA!
Legendas
en
Rolagem automática:
  • 00:00:02
    Ai and automation have arrived and
  • 00:00:04
    they're going to transform Society
  • 00:00:07
    [Music]
  • 00:00:15
    forever it's a full stack developer Deon
  • 00:00:18
    Devon that incredible demo that we saw
  • 00:00:20
    just a couple weeks ago where you gave
  • 00:00:22
    the agents a prompt and they built out
  • 00:00:25
    entire code Bas and so people are
  • 00:00:27
    naturally worried about where is the job
  • 00:00:28
    scenario going will coding even be a
  • 00:00:30
    thing in the
  • 00:00:31
    future
  • 00:00:37
    accoun but then co-pilots started
  • 00:00:39
    writing a little bit of code GPT 4
  • 00:00:41
    started writing a lot of code they have
  • 00:00:43
    lied about their
  • 00:00:45
    [Music]
  • 00:00:52
    capabilities so Devin the AI Junior
  • 00:00:55
    engineer is open for public release now
  • 00:00:57
    I bought the $500 subscription to go
  • 00:01:00
    through the product and understand is it
  • 00:01:02
    going to replace Engineers right now
  • 00:01:04
    anytime soon I've gone through a fairly
  • 00:01:06
    unbiased process of trying to make it
  • 00:01:08
    fix a few issues in one of our
  • 00:01:10
    repositories U I hire a few Engineers
  • 00:01:13
    now so I would love if a product like
  • 00:01:15
    this finds pmf uh I don't think we're
  • 00:01:17
    there yet let me give you Al tldr U I
  • 00:01:21
    tried to solve three issues using it on
  • 00:01:22
    a fairly simple repository U it was not
  • 00:01:25
    able to solve two of them the third one
  • 00:01:27
    it sort of solved partly and then
  • 00:01:28
    eventually got stuck uh so I think the
  • 00:01:31
    product hasn't changed in the last 6
  • 00:01:32
    months I've seen initial demos it looks
  • 00:01:34
    very similar what has changed is a lot
  • 00:01:37
    of polishes and pizar around it U it
  • 00:01:40
    connects directly to your slack uh it is
  • 00:01:42
    able to create pull requests directly
  • 00:01:44
    into your code base U so I think the the
  • 00:01:47
    thing that has changed is that they've
  • 00:01:48
    made it look very close to a real
  • 00:01:50
    engineer a real human that you can tag
  • 00:01:52
    on slack that can create issues and pull
  • 00:01:55
    requests uh that said uh the underlying
  • 00:01:58
    llm is still the same right so the
  • 00:01:59
    powers that it has are very similar to
  • 00:02:01
    what you would have locally if you're
  • 00:02:03
    using something like cursor GitHub
  • 00:02:05
    copilot super Maven um to long story
  • 00:02:08
    short uh I think unless there's a 10x
  • 00:02:11
    Improvement in llms uh you would find
  • 00:02:14
    fairly similar results if you're working
  • 00:02:16
    locally using GitHub copilot and you
  • 00:02:19
    know pushing and merging compared to
  • 00:02:20
    just delegating the whole task to uh
  • 00:02:23
    something like Devon will it find
  • 00:02:25
    product Market fit eventually unsure if
  • 00:02:27
    this product would uh I'm not being a
  • 00:02:30
    Critic here I I love GitHub copilot or
  • 00:02:32
    you know AI tools generally I feel Cloud
  • 00:02:34
    tools for you know building your
  • 00:02:35
    products haven't found pmf generally uh
  • 00:02:38
    something like replate is not used for a
  • 00:02:40
    fully fledged product people don't use
  • 00:02:41
    GitHub copilot as much people want to do
  • 00:02:43
    local development which is why it makes
  • 00:02:45
    sense to have a vs code extension
  • 00:02:47
    compared to a full-blown engineer U that
  • 00:02:50
    said um if there is a 10x Improvement
  • 00:02:52
    somewhere if they're able to do
  • 00:02:54
    something at the llm level if AGI comes
  • 00:02:56
    eventually then something like this
  • 00:02:57
    would make a lot of sense um also it's
  • 00:02:59
    fairly expensive right now I would
  • 00:03:01
    assume their margins are crazy and their
  • 00:03:03
    retention would be fairly poor because I
  • 00:03:05
    would not even though I've paid for it
  • 00:03:06
    now which is almost like 40,000 rupees
  • 00:03:08
    would have been much easier for me to
  • 00:03:10
    just give this to an engineer for a
  • 00:03:11
    month uh I don't think I'll use it after
  • 00:03:13
    today uh for no reason other than you
  • 00:03:15
    know uh it'll take me more time to
  • 00:03:16
    figure out what's up and you know uh
  • 00:03:18
    I'll spend more time looking at the
  • 00:03:19
    fanciness and understanding how this
  • 00:03:20
    product is working uh than in providing
  • 00:03:22
    any value at this point U compared to
  • 00:03:24
    you know a $20 subscription to GitHub
  • 00:03:27
    copilot U so those are the initial
  • 00:03:29
    thoughts I was super excited to try it
  • 00:03:31
    out uh I was hoping something fancy
  • 00:03:33
    based on the pricing of $500 a month but
  • 00:03:36
    it looks fairly similar to where it was
  • 00:03:38
    6 months ago that said they have great
  • 00:03:39
    GitHub demos that they've shown and you
  • 00:03:41
    know issues that it has actually solved
  • 00:03:43
    so look at them be aware of of it but
  • 00:03:45
    this video should take you through
  • 00:03:46
    full-blown demo of what they've produced
  • 00:03:48
    and what can you expect if you're
  • 00:03:50
    thinking of buying the $500 subscription
  • 00:03:53
    to a junior
  • 00:03:54
    engineer PS uh don't buy the
  • 00:03:56
    subscription if you're not technical at
  • 00:03:58
    some point you'll have to dive in and
  • 00:03:59
    you look at the code base uh so if
  • 00:04:01
    you're a I don't know marketeer thinking
  • 00:04:03
    of this providing a lot of value at $500
  • 00:04:05
    let me tell you it won't if you're an
  • 00:04:07
    engineer at may or may not provide you
  • 00:04:08
    value with that let's get into the video
  • 00:04:11
    right step one is signing up for Devon
  • 00:04:14
    um as I said it's publicly available now
  • 00:04:17
    so if you sign up as long as you pay
  • 00:04:18
    $500 you can test out the product
  • 00:04:20
    yourself
  • 00:04:32
    this is what the onboarding looks like
  • 00:04:34
    let me put my name connect my GitHub and
  • 00:04:37
    connect my
  • 00:04:44
    organization and next up I need to pay
  • 00:04:45
    for it so let me pay around $500 and
  • 00:04:49
    then we'll go from
  • 00:04:52
    there all right folks let's pay for it
  • 00:04:54
    and see how it
  • 00:04:58
    goes purchase sub subscription I think
  • 00:05:01
    it's around $500 a month
  • 00:05:09
    yep pretty expensive video guys oh you
  • 00:05:12
    guys can't see but it's 42,000 rupees
  • 00:05:14
    let's see how it goes let's get right
  • 00:05:16
    into it all right guys I've gone through
  • 00:05:19
    the onboarding um let me take you
  • 00:05:21
    through it really quickly it asks you
  • 00:05:22
    for your name um gives you some sort of
  • 00:05:25
    greetings it says gitlab and bit bucket
  • 00:05:27
    are coming soon thankfully for us most
  • 00:05:29
    of the things are on GitHub um this is
  • 00:05:31
    my organization where most of our code
  • 00:05:33
    is there uh We've purchased the meeting
  • 00:05:36
    plan um or whatever you know the $500
  • 00:05:39
    plan the team's plan for now and let's
  • 00:05:42
    click on start using de and see how it
  • 00:05:44
    goes step one unexpectedly connecting
  • 00:05:47
    GitHub um they have pretty decent
  • 00:05:49
    onboarding I would say this is how you
  • 00:05:51
    would onboard a random engineer as well
  • 00:05:53
    I have a few repositories I'll probably
  • 00:05:55
    just connect it to code 100x for now um
  • 00:05:59
    then we'll go from there it I'm giving
  • 00:06:02
    it access to all repositories in code
  • 00:06:04
    100x
  • 00:06:06
    um authorized Deon for the
  • 00:06:09
    integration connecting slack that is
  • 00:06:11
    interesting I don't think I have slack
  • 00:06:13
    yet uh so let me quickly create we do
  • 00:06:16
    have one for super 30 so let me just
  • 00:06:18
    connect it
  • 00:06:20
    there connect slack
  • 00:06:24
    organization allow it access on
  • 00:06:26
    everything and I've received an OTP
  • 00:06:31
    that's why developers are dumb at these
  • 00:06:33
    things yeah isn't slack is connected
  • 00:06:36
    link the user I'd assume this will be
  • 00:06:38
    like a de
  • 00:06:40
    user okay there we go um invite
  • 00:06:43
    teammates I think I'm fine for
  • 00:06:46
    now can I skip through that uh nope yeah
  • 00:06:50
    I can my workspace set up your first
  • 00:06:52
    deposit my workspace resets saved state
  • 00:06:54
    for every new session let's set up a
  • 00:06:55
    repository to include this saved State
  • 00:06:58
    um
  • 00:07:00
    interesting setup significantly improves
  • 00:07:02
    D Performance on your codebase I would
  • 00:07:04
    assume this is them going through our
  • 00:07:06
    repositories and you know U indexing all
  • 00:07:09
    the data somewhere in some database or
  • 00:07:11
    otherwise um so let's give it access to
  • 00:07:14
    an easy one to begin with
  • 00:07:16
    um daily code would probably be it um
  • 00:07:20
    I've lost context on this repository
  • 00:07:22
    honestly so we'll see we'll see if it
  • 00:07:25
    gets more context than I
  • 00:07:27
    do then works on the machine you're
  • 00:07:29
    seeing please install dependencies as
  • 00:07:32
    you would with a new hire missing
  • 00:07:34
    dependencies to ding being slow and
  • 00:07:35
    unreliable interesting they give you
  • 00:07:37
    access to a machine I would assume uh
  • 00:07:41
    this
  • 00:07:42
    is a pod or you know whatever a
  • 00:07:45
    container somewhere that dies from time
  • 00:07:46
    to time but I could be wrong de works on
  • 00:07:48
    the machine you're saying please install
  • 00:07:49
    dependencies as you would for a new hire
  • 00:07:51
    missing dependencies can lead for it to
  • 00:07:52
    be slow and underl all right that looks
  • 00:07:55
    fairly polished honestly uh feels like
  • 00:07:58
    I'm on boarding a new engineer um which
  • 00:08:01
    is always uh a task that Engineers dread
  • 00:08:04
    all righty seems like my machine is up
  • 00:08:06
    begin the
  • 00:08:13
    setup wide screen for a long
  • 00:08:16
    time there we go I have access to my
  • 00:08:19
    repository over here I think I'm okay
  • 00:08:22
    with this specific theme um okay they
  • 00:08:26
    don't give give you access to a machine
  • 00:08:27
    this seems like um based on how quick it
  • 00:08:30
    is it's mostly running locally inside my
  • 00:08:32
    browser and uh could either be a machine
  • 00:08:35
    or be web containers now I'm curious um
  • 00:08:38
    but let's assume it's system machine
  • 00:08:40
    let's see if it has all the dependencies
  • 00:08:41
    does it have I think we're using
  • 00:08:43
    pnpm and it does does have pnpm but we
  • 00:08:48
    don't use pnpm it seems what do we use
  • 00:08:50
    here we use Yar so let's see if it has
  • 00:08:54
    yarn and it does
  • 00:08:57
    um it seems to uh work as expected so I
  • 00:09:02
    don't think I need to add any
  • 00:09:03
    dependencies on top of this so I can say
  • 00:09:05
    done um up keep that's how you clone the
  • 00:09:09
    repository that looks fine and how do
  • 00:09:12
    you update dependencies you do a Yar
  • 00:09:14
    install here that looks decent
  • 00:09:17
    um and let's try to just confirm by
  • 00:09:20
    going through the terminal
  • 00:09:23
    here why are there two CDs
  • 00:09:29
    seems like that copy button is
  • 00:09:31
    broken when I do a y
  • 00:09:35
    install seems to work so I think we're
  • 00:09:37
    fine cool so upkeep is done this is how
  • 00:09:41
    it can pull the latest code base this is
  • 00:09:43
    how it can install new dependencies um
  • 00:09:45
    done and done and lastly tell Deon how
  • 00:09:48
    to verify and submit changes to the
  • 00:09:50
    repository Deon will always see these
  • 00:09:52
    instructions when working on this
  • 00:09:54
    repository advise on how to run tests
  • 00:09:55
    formatting PR execution so on and so
  • 00:09:57
    forth seems like they generated it for
  • 00:10:00
    me uh two options to run it locally
  • 00:10:01
    Docker compose watch or manually by
  • 00:10:03
    running this um that looks decent code
  • 00:10:07
    quality before submitting a PR run yan
  • 00:10:10
    yan run lint that looks right Yan run
  • 00:10:12
    format for formatting it PR checks
  • 00:10:14
    automatically linting informating
  • 00:10:16
    merging to main deploys to staging
  • 00:10:17
    merging to production deploys to
  • 00:10:18
    production that looks decent uh this
  • 00:10:20
    specific project is deployed on staging
  • 00:10:25
    do projects. 100ex apps.com and
  • 00:10:28
    projects. 100ex st.com based on whether
  • 00:10:30
    you're using the staging environment
  • 00:10:32
    which is synced to uh the main the
  • 00:10:35
    develop branch and then this is probably
  • 00:10:37
    sync to the master or the production
  • 00:10:38
    Branch um so this looks decent better
  • 00:10:42
    than I would have hoped for U no real
  • 00:10:46
    you know hallucinations already so it
  • 00:10:47
    seems like a polished product I click on
  • 00:10:49
    Save here and finish after completing
  • 00:10:52
    the setup it may take up to 10 minutes
  • 00:10:54
    until the save is complete any de
  • 00:10:56
    sessions started will not use the setup
  • 00:10:58
    all righty so so we'll wait for 10
  • 00:10:59
    minutes uh wait for it to set it up and
  • 00:11:02
    then we'll go from
  • 00:11:05
    there all right folks I have onboarded
  • 00:11:07
    one of my repositories uh we'll now try
  • 00:11:11
    to assign it some issues and see how
  • 00:11:13
    well it performs on some real production
  • 00:11:15
    issues on the
  • 00:11:17
    repository this is what the dashboard
  • 00:11:19
    looks like right now I've gone through
  • 00:11:20
    the onboarding process um the last step
  • 00:11:22
    is done as well I'm not inviting any
  • 00:11:24
    teammates for
  • 00:11:26
    now so I'm going to skip the rest of
  • 00:11:28
    onboarding specifically inviting
  • 00:11:30
    teammates and now it says after
  • 00:11:32
    connecting slack just tag Devon in a
  • 00:11:34
    thread to get
  • 00:11:38
    started all right that looks like a good
  • 00:11:41
    start it asks let's just try this it
  • 00:11:43
    says hey Devon please verify that you
  • 00:11:45
    can access this repository run lint and
  • 00:11:49
    open a dummy draft PR with a simple
  • 00:11:50
    change let's see if it's able to do that
  • 00:11:53
    so let me go to super 30
  • 00:11:55
    slack make people a little anxious over
  • 00:11:58
    here by tagging
  • 00:12:04
    Devin and let's see how it
  • 00:12:08
    performs oh it's not
  • 00:12:13
    invited all right let's see what it
  • 00:12:15
    replies with Devon is ready you can now
  • 00:12:17
    send messages snapshot Dev environment
  • 00:12:19
    this thing was used to mute Devon sent
  • 00:12:22
    this thing you can track the process
  • 00:12:23
    progress here I would assume that's this
  • 00:12:25
    specific thread congrats on starting
  • 00:12:27
    your first session I'm excited to work
  • 00:12:28
    with you here are the some things that
  • 00:12:31
    to know about working with me that looks
  • 00:12:35
    decent give me a task uh that you know
  • 00:12:38
    how to do yourself keep me tell me how
  • 00:12:41
    to test it and check that I'm on the
  • 00:12:42
    right track keep sessions under 3 hours
  • 00:12:44
    to break down big tasks share detailed
  • 00:12:47
    requirements upfront invest in coaching
  • 00:12:49
    Me by providing feedback in chat I'll
  • 00:12:51
    approve over time I'll verify access to
  • 00:12:53
    the reers on and so forth all right
  • 00:13:04
    let's give that another
  • 00:13:25
    try okay there you go it was just taking
  • 00:13:27
    time I'll verify all three capabilities
  • 00:13:29
    right away first I'll check if I can
  • 00:13:31
    access the
  • 00:13:46
    repo I sent the same message here
  • 00:13:49
    because I was a little
  • 00:13:53
    impatient and it's responded with a
  • 00:13:55
    similar thing it's trying to do that
  • 00:13:57
    let's see how long it takes it's 1222
  • 00:13:59
    right
  • 00:14:04
    now all right It's 12:24 so it it took
  • 00:14:08
    it 2 minutes it says I'll verify access
  • 00:14:10
    to this thing I encountered an issue
  • 00:14:11
    while trying to run the link code the
  • 00:14:12
    command failed
  • 00:14:14
    uh it appears there to be a
  • 00:14:16
    configuration issue according to best
  • 00:14:19
    practice I should not attempt to fix
  • 00:14:20
    configuration issue myself would you
  • 00:14:22
    like me to continue with a creating a
  • 00:14:24
    draft PR wait for some instructions okay
  • 00:14:26
    that's pretty good actually I think lint
  • 00:14:28
    does indeed fail um so let
  • 00:14:33
    me do that continue without creating a
  • 00:14:39
    draft PR oh sorry continue with creating
  • 00:14:42
    a draft PR let's see how long it takes
  • 00:14:45
    for it to do
  • 00:14:48
    that it responded back with I'll proceed
  • 00:14:50
    with creating a draft PR I'll make a
  • 00:14:52
    simple change to the readme formatting
  • 00:14:59
    and someone reacted with a happy
  • 00:15:05
    Emoji surprisingly the other thread it
  • 00:15:08
    said something similar but it did create
  • 00:15:10
    a PR so let's see what the pr looks like
  • 00:15:14
    it is a draft PR it's created by Devon
  • 00:15:16
    AI integration if I look at the files
  • 00:15:19
    change um it looks like a decent change
  • 00:15:21
    honestly uh adding bash over here so we
  • 00:15:25
    get you know decent color coding um
  • 00:15:29
    option one option
  • 00:15:30
    two just adding indexes everywhere which
  • 00:15:34
    honestly not the worst change in the
  • 00:15:36
    world so let's merge that and go from
  • 00:15:39
    there it seems like it's
  • 00:15:41
    testing uh which
  • 00:15:45
    oh let's mark it for ready for review
  • 00:15:47
    and just merge it and go from
  • 00:15:52
    there right it took me like 5 minutes to
  • 00:15:56
    let us know that everything is working
  • 00:15:58
    it's able to create a it created a very
  • 00:16:00
    simple PR updated readme U the one thing
  • 00:16:03
    you learn from this is you at least
  • 00:16:05
    don't need to update readms anymore and
  • 00:16:07
    AI can do that for you next up let's
  • 00:16:09
    give it a real
  • 00:16:14
    issue all right I was able to clean up
  • 00:16:16
    the readme create a simple draft PR now
  • 00:16:19
    let's try to give it a real issue here
  • 00:16:22
    are the issue lists on projects. 100ex
  • 00:16:25
    steps.com again this is the platform
  • 00:16:29
    that we're trying to work upon fixed
  • 00:16:31
    issues upon let's try to find an issue
  • 00:16:33
    over here that should be easy for an AI
  • 00:16:35
    to solve if you write the right context
  • 00:16:37
    in open AI chat GPT it gives you a
  • 00:16:40
    response that you have to manually paste
  • 00:16:41
    over here let's see if Devon can do the
  • 00:16:43
    same and create a draft PR for
  • 00:16:54
    us o that looks like a great one we
  • 00:16:58
    don't have p ination here yet so all the
  • 00:17:00
    projects that you see are you know
  • 00:17:02
    directly over here let me turn on dark
  • 00:17:05
    mode um which means there's no real
  • 00:17:07
    sense of this Bottom bar honestly either
  • 00:17:10
    we should add pagination both on the
  • 00:17:12
    back end and the front end or at the
  • 00:17:14
    very least add pagination on the front
  • 00:17:17
    end on the front end we should be able
  • 00:17:18
    to go from page one to page two to page
  • 00:17:20
    three even if we're not hitting the back
  • 00:17:22
    end to get back the you know next Les
  • 00:17:24
    list of projects so let's try to see if
  • 00:17:27
    it can fix it uh
  • 00:17:30
    I think all the notes are showing all at
  • 00:17:32
    once by default and all are selected on
  • 00:17:34
    the drop- down menu and then there is no
  • 00:17:36
    use of pagination exactly I think that's
  • 00:17:38
    what's happening as well if you want to
  • 00:17:40
    show limited notes in the first page
  • 00:17:41
    then I can fix it is this necessary can
  • 00:17:43
    you assign this to me all right someone
  • 00:17:45
    did create a PR for it let's look at the
  • 00:17:48
    files changed it's around 28 lines of
  • 00:17:50
    code that have been changed so let's see
  • 00:17:52
    if Devon can fix it um at Devon can you
  • 00:17:57
    fix this speciic specific
  • 00:18:01
    PR so let's see uh if it is able to do
  • 00:18:05
    that um it's 12:37 right now so let's
  • 00:18:08
    see how long it takes for it to create a
  • 00:18:10
    draft PR for it all right um so it
  • 00:18:14
    generated a plan for me right away it
  • 00:18:16
    said I'll take a look at the issue in
  • 00:18:17
    the daily code repository and work on
  • 00:18:20
    fixing it I'll first work on a plan if
  • 00:18:23
    you want uh me to confirm the plan with
  • 00:18:25
    you click on this button I did not
  • 00:18:27
    here's my plan this is what the plan
  • 00:18:28
    looks like it seems like it first
  • 00:18:30
    creates a very elaborate plan that's the
  • 00:18:32
    first call it's making to an llm and the
  • 00:18:35
    llm here is telling you fix pagination
  • 00:18:37
    controls that are not be properly worked
  • 00:18:39
    clicking the previous and the next
  • 00:18:40
    buttons the page content is not updating
  • 00:18:43
    when navigating so I don't think it
  • 00:18:44
    understood the problem very well uh
  • 00:18:45
    because the problem isn't that next and
  • 00:18:46
    previous aren't working the problem is
  • 00:18:48
    that we're just rendering everything in
  • 00:18:50
    the first go um create a new Branch for
  • 00:18:52
    the
  • 00:18:54
    fix which will be done by this specific
  • 00:18:56
    command um I don't know why they're
  • 00:18:58
    adding the time St here add us router
  • 00:19:01
    hook in blog. TSX to handle the route
  • 00:19:03
    change oky DOI
  • 00:19:12
    um add use effect to handle the route
  • 00:19:15
    change and force a
  • 00:19:17
    render all
  • 00:19:19
    righty update custom pagination to use
  • 00:19:24
    router. push instead of link for better
  • 00:19:25
    client side navigation
  • 00:19:28
    replace link component with router. push
  • 00:19:31
    and test the changes navigate to a track
  • 00:19:34
    with multiple problems all right I think
  • 00:19:37
    it'll be very confused when trying to
  • 00:19:39
    fix this PR based on the steps that it
  • 00:19:42
    has created it's nice what it's done
  • 00:19:44
    it's tried to create a plan um but from
  • 00:19:46
    what I can get if a developer would
  • 00:19:48
    create a plan in front of me like this I
  • 00:19:49
    would ask a few questions and you know
  • 00:19:51
    ask them to change a few things um it
  • 00:19:53
    has not done that also I don't know if
  • 00:19:55
    it has any data to test imagination so
  • 00:19:57
    there's a high probability it's not able
  • 00:19:59
    to test it very well U because well
  • 00:20:01
    there's a c. TS um file that we have let
  • 00:20:04
    me show you we have a file that you can
  • 00:20:06
    run to seed some data in there U
  • 00:20:08
    hopefully
  • 00:20:10
    packages um DB Prisma seed. TS but I'm
  • 00:20:15
    unsure if it if it ever runs this and
  • 00:20:18
    you know seeds the data in the database
  • 00:20:19
    so it can test everything well but we'll
  • 00:20:22
    see one more thing I saw was U you can
  • 00:20:24
    actually track all the changes here this
  • 00:20:26
    is the one that currently being worked
  • 00:20:28
    on if you look at this you can actually
  • 00:20:29
    follow de and see what it's doing um so
  • 00:20:34
    it created the plan which we already saw
  • 00:20:36
    and I think now it's executing the plan
  • 00:20:37
    Deon is currently running the
  • 00:20:38
    development server to test navigation
  • 00:20:39
    all right here and there you go it's
  • 00:20:41
    stuck right on uh
  • 00:20:47
    authentication it opened this browser
  • 00:20:50
    opened this I don't know how it got Auto
  • 00:20:52
    redirected to O also did not need to
  • 00:20:55
    because I'm not even if I'm not logged
  • 00:20:57
    in
  • 00:20:59
    I can still see everything um so I don't
  • 00:21:02
    have to log in to fix this issue but
  • 00:21:04
    let's see editor open. env. example file
  • 00:21:07
    which is this one right
  • 00:21:10
    here I can actually follow through
  • 00:21:12
    everything it had done the UI fantastic
  • 00:21:15
    like they everything is nailed uh all
  • 00:21:17
    the t's are crossed uh
  • 00:21:21
    but it's a fancy product but under under
  • 00:21:23
    the hood is still using you know an llm
  • 00:21:25
    which has restrictions to how well it
  • 00:21:28
    can you know solve this issue oh well it
  • 00:21:30
    did open seed data. TS I might have
  • 00:21:32
    spoken too
  • 00:21:48
    soon it
  • 00:21:50
    ran well one thing I did not expected to
  • 00:21:53
    do which it did is at least it's trying
  • 00:21:55
    to migrate the database and then it
  • 00:21:56
    probably saw there is no database URL
  • 00:21:58
    and now let's see if it creates a
  • 00:22:00
    database URL or containerize the
  • 00:22:06
    application here is the problem I think
  • 00:22:09
    the problem is you know um not all code
  • 00:22:13
    bases are written in an AI friendly
  • 00:22:15
    fashion for example there is a Docker
  • 00:22:17
    file here it did not end up using the
  • 00:22:19
    docker file um well there you go but if
  • 00:22:23
    it was I think if the code was written
  • 00:22:24
    in a more AI friendly fashion it would
  • 00:22:26
    have been easier for it to figure out
  • 00:22:27
    things it still isn't doing too bad um
  • 00:22:30
    it opened
  • 00:22:32
    NV it'll probably try to update n over
  • 00:22:35
    there but the question really
  • 00:22:38
    is will it be able to start a database
  • 00:22:44
    um it ran npx press man generate which
  • 00:22:47
    did generate everything so it seems like
  • 00:22:50
    it did create the rightn
  • 00:22:54
    file would be nice if I can see it
  • 00:23:03
    and it ran npx Prisma generate to
  • 00:23:08
    generate the Prisma client well it was
  • 00:23:09
    able to generate it which it can do
  • 00:23:11
    without the database URL hold on oh it
  • 00:23:14
    did create a database URL postgress
  • 00:23:17
    database this thing will be public with
  • 00:23:19
    public schema this thing was
  • 00:23:22
    successfully
  • 00:23:24
    reset very cool so it does create an AWS
  • 00:23:27
    postgress server for me so it's smart
  • 00:23:30
    enough to do that very surprising um
  • 00:23:33
    this is the database URL this is
  • 00:23:38
    the DB username and I'm assuming there's
  • 00:23:41
    a password somewhere
  • 00:23:43
    here this is super interesting it was
  • 00:23:47
    they've covered this corner case which I
  • 00:23:49
    thought they wouldn't which is they
  • 00:23:50
    created database for me and they're
  • 00:23:52
    migrating that database now it opened
  • 00:23:54
    tracks it's seeing a 404 for some reason
  • 00:23:57
    but I'm I'm sure it figure it out now
  • 00:23:59
    I'm getting more confident in its
  • 00:24:01
    abilities that said this seems like a
  • 00:24:03
    decentish attack Vector it's creating
  • 00:24:06
    databases for me
  • 00:24:08
    um which you know what if I create a
  • 00:24:11
    very big database or you know um create
  • 00:24:15
    multiple databases because I'm sure this
  • 00:24:17
    AI has access to create databases long
  • 00:24:19
    story short unless they have limits to
  • 00:24:21
    how many databases I can create I can
  • 00:24:22
    create thousands of databases in their
  • 00:24:23
    AWS account which isn't the best thing
  • 00:24:26
    in the world but we'll see
  • 00:24:36
    very slow very dumb no not very dumb
  • 00:24:39
    like it's actually pretty
  • 00:24:40
    smart but yeah what are you
  • 00:24:47
    doing I guess
  • 00:25:01
    all right folks um seems like it's done
  • 00:25:04
    it just let me know it has created a
  • 00:25:06
    pull request here so let's try to go
  • 00:25:09
    through its changes um more changes than
  • 00:25:11
    the other pull requests we saw but let's
  • 00:25:13
    see what it says added proper client
  • 00:25:15
    side navigation for reender whenever
  • 00:25:18
    problem changes it's like anti pattern
  • 00:25:20
    that you shouldn't follow updated this
  • 00:25:22
    thing to use router. push and test it in
  • 00:25:25
    both Legacy and new UI modes interesting
  • 00:25:28
    it says it's tested so I'm excited
  • 00:25:34
    um whenever problem. ID changes on blog.
  • 00:25:39
    TSX it refreshes that
  • 00:25:42
    looks totally wrong um because this
  • 00:25:46
    blog. TSX if I'm not wrong
  • 00:25:51
    um is for this page projects. 100ex
  • 00:25:55
    sts.com SL a specific track like this
  • 00:25:58
    renders the blog component and what we
  • 00:26:01
    wanted a fix for was here um on the
  • 00:26:04
    landing page so that looks
  • 00:26:06
    wrong let's go back here oh sorry that's
  • 00:26:09
    the original PR of a
  • 00:26:14
    human custom pagination
  • 00:26:17
    TSX uh I'm assuming it understood the
  • 00:26:21
    problem incorrectly and I think it's
  • 00:26:22
    fixing pagination or trying to fix
  • 00:26:25
    pagination here in this bot bar previous
  • 00:26:29
    and next rather than fixing it the other
  • 00:26:30
    place but this isn't broken is my
  • 00:26:33
    guess um so let's look at that
  • 00:26:41
    again it replaced these links with
  • 00:26:44
    buttons which is bad because um if you
  • 00:26:48
    have these links then next year we
  • 00:26:51
    prefetch some pages for example if you
  • 00:26:53
    go here uh and let me open the network
  • 00:26:56
    Tab and show you
  • 00:26:58
    if I click on next um we get back the
  • 00:27:02
    current page as well as the next page
  • 00:27:04
    next JS will prefetch Pages um if
  • 00:27:07
    they're already in a button somewhere
  • 00:27:08
    here if you if I add a link tag like we
  • 00:27:11
    did over here then it and if I have
  • 00:27:14
    prefetch set to true then it'll prefetch
  • 00:27:17
    the next page that gives us fast
  • 00:27:19
    navigation when I'm on this specific
  • 00:27:21
    page the next page is already fetched
  • 00:27:22
    you can also see that here okay when I'm
  • 00:27:24
    on page you know this one the next page
  • 00:27:27
    also comes let's see if I can show it
  • 00:27:29
    better if I go to the next page there's
  • 00:27:31
    not just one page that came um page
  • 00:27:35
    number six page number 7 also came so
  • 00:27:37
    that when I go to I'm on page six right
  • 00:27:39
    now when I go to page number seven it's
  • 00:27:41
    already prefetched for me if we do what
  • 00:27:44
    the AI did over here if you replace it
  • 00:27:45
    with a button then that prefetching goes
  • 00:27:47
    away which is bad then you know it'll be
  • 00:27:49
    really slow to fetch the next page
  • 00:27:51
    whenever the user clicks on the next
  • 00:27:52
    button it'll become slower so it was
  • 00:27:54
    consciously added over here with
  • 00:27:56
    prefetch u so that's bad long story
  • 00:27:58
    short it fixed pagination in the wrong
  • 00:28:01
    place where it wasn't even broken in the
  • 00:28:03
    first place and number two uh it fixed
  • 00:28:06
    it in a wrong way like I don't know why
  • 00:28:08
    it chose that specific route this seems
  • 00:28:11
    like a very anti pattern to uh you know
  • 00:28:13
    refresh do a router. refresh or Force R
  • 00:28:16
    renders U whenever the problem ID
  • 00:28:18
    changes um I would discard this for now
  • 00:28:21
    um but I'll try another issue um if a
  • 00:28:24
    user would have created this issue I
  • 00:28:26
    would immediately close it and tell them
  • 00:28:29
    this just looks wrong I wouldn't spend
  • 00:28:30
    too much time trying to explain why it's
  • 00:28:31
    wrong um unless you know I'm in the mood
  • 00:28:34
    most of the times this would be a pretty
  • 00:28:35
    bad fix but that's fine let's give it
  • 00:28:38
    the benefit of the doubt and let's try
  • 00:28:40
    to see if it can solve maybe a slightly
  • 00:28:42
    simpler
  • 00:28:45
    issue uh all righty folks I took a mild
  • 00:28:48
    detour um whenever a platform like this
  • 00:28:50
    is released you know to the public it's
  • 00:28:53
    the best time to find the vulnerability
  • 00:28:54
    the early bugs in Facebook were found
  • 00:28:56
    right after a feature was released uh in
  • 00:28:58
    Facebook um I'm not saying I found
  • 00:29:00
    anything I found something I don't know
  • 00:29:02
    I wouldn't call it trival either I
  • 00:29:03
    wouldn't call it zero day vulnerability
  • 00:29:05
    either somewhere in the middle I've
  • 00:29:06
    reported it to the team we'll see how
  • 00:29:08
    that goes uh but I'm not going to share
  • 00:29:11
    too much about what that is but you know
  • 00:29:12
    feel free to explore de and see if you
  • 00:29:14
    can find any prompt injections or some
  • 00:29:16
    way to you know steal data from other
  • 00:29:17
    users things like these these are you
  • 00:29:19
    know this is the time to do it uh
  • 00:29:21
    because one AI has been given a lot of
  • 00:29:23
    control so there's a high probability AI
  • 00:29:25
    can access data that I shouldn't have
  • 00:29:27
    access to and I can you know prompt
  • 00:29:28
    inject to get it U and there are shared
  • 00:29:31
    resources so you can look around there
  • 00:29:32
    and you know try to figure out what's up
  • 00:29:34
    and if there's a way to you know find
  • 00:29:35
    the vulnerability reported to the team
  • 00:29:36
    get a bounty for it coming back to the
  • 00:29:39
    original problem what were we doing we
  • 00:29:41
    were trying to solve slash make it solve
  • 00:29:44
    a few issues um so let's go back to our
  • 00:29:47
    repository that's code 100 x/ daily code
  • 00:29:51
    let me find another issue probably a
  • 00:29:54
    simpler one and see if it can solve it
  • 00:29:57
    uh loading skeletons is shown while
  • 00:29:59
    seeing the tracks even if the tracks are
  • 00:30:01
    empty or after filter sort
  • 00:30:04
    action
  • 00:30:08
    uh I'm just trying to find one that an
  • 00:30:10
    AI can understand better or you know
  • 00:30:12
    we're sure it should be able to solve it
  • 00:30:13
    layout issues in the PDF yeah I think
  • 00:30:16
    that's too much for it um unresponsive
  • 00:30:18
    image in search bar on smaller screens
  • 00:30:21
    um let's
  • 00:30:24
    see make it smaller
  • 00:30:30
    oh there we
  • 00:30:32
    go unsure if it's very easy for an AI to
  • 00:30:35
    fix let's look at the fix that someone
  • 00:30:38
    else made it's a 13 liner so I would
  • 00:30:41
    assume it's not too difficult to fix um
  • 00:30:44
    so let's give it that let's see if
  • 00:30:47
    um Devin can fix this specific issue
  • 00:30:52
    I'll try to recreate it as well while
  • 00:30:53
    it's trying to fix it because it of
  • 00:30:55
    course takes longer than you would hope
  • 00:30:57
    I would assume it take like I know 10
  • 00:30:58
    minutes to solve it U so while that is
  • 00:31:00
    happening let me quickly try to see is
  • 00:31:04
    this reproducible let's go to projects.
  • 00:31:06
    steps. whoopsy
  • 00:31:10
    Daisy whoopsy Daisy part two um I this
  • 00:31:13
    sounds
  • 00:31:17
    fine let's try to resize it and make it
  • 00:31:20
    really
  • 00:31:26
    small uh I'll need a
  • 00:31:28
    mobile view for
  • 00:31:31
    this yeah that looks bad on a extremely
  • 00:31:34
    small device honestly um unsure if it's
  • 00:31:37
    a very big issue but an issue
  • 00:31:39
    nonetheless let's make a
  • 00:31:42
    bigger yeah there are other issues for
  • 00:31:44
    example you know 3 months ago is
  • 00:31:46
    overflowing on a fairly thin device um I
  • 00:31:49
    would assume most mobile devices are you
  • 00:31:51
    know let's select I don't know an iPhone
  • 00:31:53
    12 Pro looks decent U if anything there
  • 00:31:56
    are other issues there's too much black
  • 00:31:57
    on the right um but we'll see we'll see
  • 00:32:00
    if AI can fix this
  • 00:32:03
    U while that happening I'd like to you
  • 00:32:05
    know do another one but let's see um
  • 00:32:07
    I'll take a look at the issue and work
  • 00:32:09
    on it I'll first work out the plan
  • 00:32:11
    access the knowledge knowledge being the
  • 00:32:12
    repo notes for codex flash daily code
  • 00:32:15
    and autogenerated knowledge Auto
  • 00:32:18
    generated knowledge for this unsure what
  • 00:32:19
    this is let's
  • 00:32:25
    see interesting
  • 00:32:28
    that's very cool um it understood my
  • 00:32:31
    code and generated a lot of context
  • 00:32:33
    around it
  • 00:32:35
    um
  • 00:32:37
    cool let's wait for it to finish let's
  • 00:32:39
    see what's what it's doing right now de
  • 00:32:41
    is working it's viewing the
  • 00:32:43
    issue yeah it's fairly slow so we'll
  • 00:32:45
    wait for it to finish uh let's look at
  • 00:32:47
    the planner clone the repo if needed
  • 00:32:50
    investigate the code base plan approved
  • 00:32:52
    equal to suggest plan if not plan
  • 00:32:54
    approved O2 if it is approved then
  • 00:32:56
    execute the uh the plan all righty these
  • 00:32:59
    are the steps it's performing right now
  • 00:33:00
    and I would assume after the plan is
  • 00:33:02
    created it'll execute the steps of the
  • 00:33:04
    plan one after the other so we'll wait
  • 00:33:05
    for it to
  • 00:33:06
    finish all righty folks um it's doing a
  • 00:33:09
    bunch of follow-ups which is great for
  • 00:33:11
    example my lock file isn't up toate it's
  • 00:33:13
    trying to update it U but before that it
  • 00:33:16
    did create a poll request for me so
  • 00:33:18
    let's check it out um oh okay it
  • 00:33:20
    probably changes package lock or
  • 00:33:22
    something yeah sorry YN lock up but
  • 00:33:24
    let's look at the change um hm
  • 00:33:28
    magnifying glass class name equal to
  • 00:33:29
    size6 and all right it
  • 00:33:34
    changed the size of an icon content
  • 00:33:39
    search
  • 00:33:41
    uh content
  • 00:33:46
    search or did I give it a wrong
  • 00:33:50
    issue
  • 00:33:51
    nope yeah I don't think it was able to
  • 00:33:55
    do what it was supposed to do I'm I'm
  • 00:33:58
    doubting my code base now is it not
  • 00:34:01
    there is there not enough context in
  • 00:34:02
    there that an AI is not able to solve
  • 00:34:04
    such a simple issue
  • 00:34:06
    um I don't know if this will this looks
  • 00:34:08
    like the wrong place for it honestly uh
  • 00:34:10
    because our tracks exist over here and
  • 00:34:12
    it's probably trying to fix something
  • 00:34:13
    over here unless that's what it was
  • 00:34:15
    supposed to be fixed and I don't
  • 00:34:16
    understand the issue let's see
  • 00:34:20
    um I was there sorry my bad um it was
  • 00:34:24
    indeed here that the problem existed
  • 00:34:27
    yeah I'm the the dumb guy of course AI
  • 00:34:29
    is smart so in the search bar is where
  • 00:34:31
    the issue was which was here and it
  • 00:34:34
    increased the size of this search I
  • 00:34:35
    which is very very cool considering it
  • 00:34:38
    does look a little small I don't know
  • 00:34:39
    how it figured out whether the r needs
  • 00:34:42
    to increase in size um so that's good
  • 00:34:44
    the other thing it did was
  • 00:34:47
    uh it updated Yan lock which is great
  • 00:34:50
    and sorry this looks like the wrong pull
  • 00:34:55
    request the other thing it did
  • 00:34:59
    was it changed the height to be or it
  • 00:35:03
    retained the height to be 400 pixels but
  • 00:35:05
    it made it 300 pixels on a smaller
  • 00:35:06
    screen which probably will fix it
  • 00:35:09
    um well it seems like it did fix a
  • 00:35:11
    simple issue um can I search for
  • 00:35:13
    something type
  • 00:35:15
    script the current issue was
  • 00:35:22
    um this there we go oh that's fairly
  • 00:35:25
    obvious issue much better and what it
  • 00:35:27
    did is let me change the Styles over
  • 00:35:30
    just apply it over here and see if it
  • 00:35:32
    does fix it
  • 00:35:33
    um it
  • 00:35:39
    changed where is 400 there you go it
  • 00:35:41
    changed h-400 to
  • 00:35:44
    be MD colon h-400 and otherwise h-300
  • 00:35:51
    pixels all right sorry about that uh
  • 00:35:54
    let's try it one more time I'm going to
  • 00:35:55
    have a single row over here um and
  • 00:35:58
    currently does the issue exist let's try
  • 00:36:00
    it
  • 00:36:01
    out it indeed exists this photo sort of
  • 00:36:04
    becomes very pixelated um and the fix
  • 00:36:08
    that it added was it made the height
  • 00:36:13
    of this whever 400 pixels is right here
  • 00:36:17
    it made it 300 on a smaller screen let
  • 00:36:19
    me make it 300 generally um and see if
  • 00:36:22
    that fixes it for a smaller screen let's
  • 00:36:24
    make it smaller and it seems like it
  • 00:36:27
    does not
  • 00:36:29
    um I would assume it does fix it a
  • 00:36:31
    little bit um like if the issue
  • 00:36:33
    originally started at this point it now
  • 00:36:35
    starts at this point um but does not
  • 00:36:37
    solve the issue at hand which probably
  • 00:36:40
    makes sense because AIS aren't very
  • 00:36:43
    visual I I'm assuming it's not taking
  • 00:36:45
    screenshots and forwarding forwarding it
  • 00:36:47
    to an AI to test this so it's really
  • 00:36:48
    hard to test for an AI whether or not a
  • 00:36:50
    visual fix is Right U so that's the AI
  • 00:36:54
    fix even though it was smaller uh
  • 00:36:57
    compared to you know the other person's
  • 00:36:59
    fix this is the original fix that the
  • 00:37:00
    other person made U and let's look at
  • 00:37:03
    the AI fix that's the one the AI fix was
  • 00:37:05
    just two lines of code U and no reason
  • 00:37:07
    for this one it was pretty much this uh
  • 00:37:10
    which did not fix it um but if you look
  • 00:37:13
    at the change that gur kirat made um
  • 00:37:20
    it's much more lines of code firstly it
  • 00:37:21
    does fix it like I'm sure there's a
  • 00:37:23
    screenshot or video here so if you look
  • 00:37:25
    at it as you scroll down
  • 00:37:28
    it does yeah it pretty much fixes the
  • 00:37:30
    issue this thing is not stretching out
  • 00:37:32
    um and if you look at the changes um
  • 00:37:36
    surprisingly he's increased the height
  • 00:37:38
    for you know throughout and um over here
  • 00:37:43
    rather
  • 00:37:44
    than having just an image right next to
  • 00:37:47
    some text or whatever you know this
  • 00:37:48
    content um header and title um
  • 00:37:53
    he still wrapped it inside a div where
  • 00:37:56
    he added items Center to probably
  • 00:37:58
    vertically align it added some more Gap
  • 00:38:00
    um and I assume this is what helps you
  • 00:38:02
    know in making sure it doesn't stretch
  • 00:38:04
    because this padding on oh that's a
  • 00:38:07
    bigger screen my bad U so this this
  • 00:38:08
    won't fix it uh this probably fixes it
  • 00:38:12
    on a smaller screen the width and height
  • 00:38:14
    is 16 bigger screen is 20 it's pretty
  • 00:38:17
    much hardcoded here so you know um
  • 00:38:18
    there's no reason for it to stretch
  • 00:38:20
    we've hardcoded the width and a height
  • 00:38:21
    on a smaller and a bigger screen um that
  • 00:38:24
    way this image isn't stretched so that
  • 00:38:25
    makes sense u a more elaborate fix um
  • 00:38:28
    but indeed one that works um and as I
  • 00:38:31
    said I don't think the AI is to blame
  • 00:38:33
    here U the reason this did not work for
  • 00:38:36
    an AI probably it's because it's a front
  • 00:38:38
    end issue U so maybe it would be good
  • 00:38:40
    for us to try a backend issue as well U
  • 00:38:42
    let's do that next let's give it the
  • 00:38:44
    benefit of the doubt second time uh and
  • 00:38:46
    see if it is able to solve a backend
  • 00:38:49
    issue that may exist in this repository
  • 00:38:50
    although I'm unsure if it does if it
  • 00:38:51
    doesn't then I'll fabricate an issue and
  • 00:38:53
    see how it performs
  • 00:38:58
    all right folks I wasn't able to find an
  • 00:39:00
    issue so I fabricated one U it's a
  • 00:39:02
    mixture of front end and back end but
  • 00:39:03
    front end should be fairly simple on
  • 00:39:05
    this one um the issue is that if you go
  • 00:39:08
    to the profile section here um you see
  • 00:39:11
    your name that comes from Google and
  • 00:39:12
    your email you should be able to edit
  • 00:39:14
    this name so you should have a pencil
  • 00:39:16
    icon over here when you click on it this
  • 00:39:18
    should become an input field you should
  • 00:39:20
    be able to change your name and then you
  • 00:39:21
    should be able to click on a button here
  • 00:39:23
    that lets you update your name that
  • 00:39:24
    would involve some changes on the back
  • 00:39:26
    end some changes on the front end but
  • 00:39:27
    fairly simple you know full stack issue
  • 00:39:29
    so I would assume um it's very clean and
  • 00:39:32
    uh well structured as to what needs to
  • 00:39:34
    be done so I think Devon should be able
  • 00:39:36
    to solve it let's go back to the landing
  • 00:39:40
    page wherever you
  • 00:39:42
    reach oh boy over here and let's ask it
  • 00:39:47
    to please
  • 00:39:49
    fix this specific issue and let's see
  • 00:39:52
    how it
  • 00:39:55
    does again take a long time so I'll wait
  • 00:39:58
    for it to finish and then we'll talk
  • 00:39:59
    back it's 4:32 right now so let's see
  • 00:40:01
    how long it takes should I compete with
  • 00:40:03
    it it's 4:32 that means I'm a little
  • 00:40:05
    sleepy um let me see how hard would it
  • 00:40:09
    have been for me to figure it out U
  • 00:40:11
    while it's coding it myself itself I'll
  • 00:40:13
    try to go it as well U maybe directly
  • 00:40:15
    over here I don't want to clone it
  • 00:40:16
    locally U app I would go to profile and
  • 00:40:20
    page. TSX
  • 00:40:24
    um user detail form that's over here
  • 00:40:27
    here so components SL user detail
  • 00:40:39
    form and over here I see the
  • 00:40:44
    user name um so here I would uh this
  • 00:40:49
    will be like yeah this will be hard not
  • 00:40:50
    hard but like figing out a lot of
  • 00:40:52
    updating the input component where
  • 00:40:54
    wherever that is coming from it's
  • 00:40:55
    probably a Shaden component um to
  • 00:40:57
    modifying it uh to add like you know an
  • 00:41:00
    End Icon which can
  • 00:41:02
    be a pencil in one case and you know a
  • 00:41:05
    submit or a cross icon in the other case
  • 00:41:07
    U so yeah that wouldn't be it was just
  • 00:41:10
    involve making this input component more
  • 00:41:13
    generic which right now probably just
  • 00:41:14
    directly comes from shatan um so yeah it
  • 00:41:17
    would take me effort for sure uh I know
  • 00:41:19
    what to do but it'll just take a lot of
  • 00:41:20
    human effort to do it um that's one and
  • 00:41:23
    then on the back end I would assume it
  • 00:41:25
    should be straightforward if if I go to
  • 00:41:28
    DB / Prisma
  • 00:41:32
    um and look at the schema the user table
  • 00:41:36
    or the account table one of these
  • 00:41:38
    probably has the Nam the user
  • 00:41:40
    table yeah there's a name field over
  • 00:41:42
    here so I just need to expose U an
  • 00:41:45
    endpoint or a server action whatever we
  • 00:41:47
    use over here
  • 00:41:49
    um there is app SL API so I would just
  • 00:41:54
    you know not too much here though re
  • 00:41:57
    validate and add tracks um which makes
  • 00:42:00
    sense like we don't let the user do lot
  • 00:42:02
    of things but let's see if we have some
  • 00:42:04
    server actions
  • 00:42:06
    somewhere um or admin marketing assets
  • 00:42:09
    yeah I don't see anything profile page.
  • 00:42:12
    TSX yeah I'd probably add an API
  • 00:42:15
    endpoint somewhere here u h that says U
  • 00:42:19
    profile name get server session and then
  • 00:42:22
    for example atrax probably gets the
  • 00:42:24
    server session as well so get the server
  • 00:42:26
    session update the username um
  • 00:42:27
    straightforward uh back end should take
  • 00:42:29
    I don't know 5 minutes to do for a human
  • 00:42:32
    and front end might take me I don't know
  • 00:42:34
    20 to 30 minutes honestly U because it's
  • 00:42:37
    making sure we create it in a very
  • 00:42:39
    generic fashion U specifically this
  • 00:42:41
    input component that right now um is
  • 00:42:45
    coming directly from Shad CNS my guess
  • 00:42:48
    um Shad /ui
  • 00:42:52
    input where are you there you go this is
  • 00:42:56
    autogenerated from Shad C and I need to
  • 00:42:58
    update it to accept more props U
  • 00:43:01
    specifically like an End Icon um or end
  • 00:43:03
    icons which could be more than one um so
  • 00:43:05
    that will be the difficult bit here or
  • 00:43:06
    you know the most time consuming this
  • 00:43:07
    should probably I mean if you have to
  • 00:43:08
    put it in human hours actually might
  • 00:43:10
    take me around an hour to do um back in
  • 00:43:12
    probably 10 minutes this guy though
  • 00:43:15
    let's see where it's at um I forgot I
  • 00:43:17
    think we started at 4:32 or 422 it's
  • 00:43:19
    been 3 5 minutes 4:32 so it's only been
  • 00:43:22
    like 4 minutes right now
  • 00:43:25
    um let it run and I'll join you back
  • 00:43:28
    once it
  • 00:43:30
    finishes all right folks that's not too
  • 00:43:33
    bad actually took it 10 minutes um in 10
  • 00:43:35
    minutes it has written this let me show
  • 00:43:37
    you guys the back end code looks exactly
  • 00:43:41
    how I would write it um gets the session
  • 00:43:44
    checks if the user session exists gets
  • 00:43:46
    the name from the request uh and then
  • 00:43:50
    updates it in the database so nothing
  • 00:43:51
    out of the
  • 00:43:53
    ordinary here oh boy
  • 00:43:58
    uh I worry this is going to return the
  • 00:44:01
    user their password uh it'll be hash
  • 00:44:05
    which is not a problem but I worry this
  • 00:44:08
    is not good um we'll see though why is
  • 00:44:12
    it rning the user no idea um and then
  • 00:44:15
    here on the front end
  • 00:44:17
    um it seems like it did what it's
  • 00:44:19
    supposed to do handle submit which sends
  • 00:44:21
    a fetch request to the back end set it
  • 00:44:23
    is editing to false a state variable
  • 00:44:25
    called is editing which toles U there's
  • 00:44:29
    a div with an input and if is editing is
  • 00:44:32
    not true then you edit you render the
  • 00:44:34
    editing or you know the pencil icon um
  • 00:44:37
    and when you click on it you set the is
  • 00:44:39
    editing variable to True which would
  • 00:44:40
    render these two buttons which look
  • 00:44:42
    right there's a um button to submit and
  • 00:44:46
    then there's a button to cancel U which
  • 00:44:50
    sets is edting to false resets the name
  • 00:44:52
    as well so looks decent there's nothing
  • 00:44:53
    wrong here U the only problem of course
  • 00:44:55
    is I need to test it locally now to to
  • 00:44:57
    see how it looks and the other doubt
  • 00:45:00
    that I had uh you know this may or may
  • 00:45:02
    not be a
  • 00:45:03
    vulnerability uh how do I pull
  • 00:45:07
    there Branch locally is the next
  • 00:45:09
    question so let me go to the repository
  • 00:45:11
    locally first uh CD projects slash uh
  • 00:45:16
    daily Dash
  • 00:45:20
    code let me pull the latest it's been a
  • 00:45:22
    while since I've pulled it
  • 00:45:27
    um now I need to get their Branch
  • 00:45:30
    locally so get remote add origin de
  • 00:45:35
    would be interesting honestly um Devon
  • 00:45:39
    AI integration is an
  • 00:45:41
    app how do you clone their codebase
  • 00:45:44
    locally or sorry their pull their Branch
  • 00:45:46
    locally I
  • 00:45:48
    have no idea honestly get fetch
  • 00:45:52
    origin can you fetch it from origin does
  • 00:45:54
    it push I don't think it has access to
  • 00:45:57
    push to my repo directly in a branch but
  • 00:45:59
    hey I could be wrong and I was wrong it
  • 00:46:01
    does have it that's scary U it can push
  • 00:46:05
    to a branch yeah Deon can push directly
  • 00:46:07
    to my origin Branch that's not great
  • 00:46:09
    sorry the original remote that's not
  • 00:46:10
    great but let's go into the Devin branch
  • 00:46:14
    that looks right let's do an npm install
  • 00:46:16
    in case I'm missing some dependencies
  • 00:46:18
    locally U followed by or whatever a Yan
  • 00:46:21
    install would have been better honestly
  • 00:46:22
    Yan install followed by Yan Dev
  • 00:46:28
    and let's see how it looks functionally
  • 00:46:31
    it should work it might look bad and the
  • 00:46:33
    vulnerability that I'm worried about is
  • 00:46:35
    the only thing um but as expected and
  • 00:46:38
    issue that's very easy for a human to
  • 00:46:39
    understand it's probably very easy for
  • 00:46:40
    it to understand um but if I give it
  • 00:46:43
    something along the lines of U I don't
  • 00:46:45
    know change something in the source code
  • 00:46:46
    of FFM Peg I would assume it would
  • 00:46:48
    hallucinate really bad U but we'll see
  • 00:46:51
    we'll see time wise much better than you
  • 00:46:53
    know how long it would have taken me if
  • 00:46:56
    I was doing it flat if I was doing it
  • 00:46:57
    with an AI helper probably similar U but
  • 00:47:01
    if I was not using super Maven or you
  • 00:47:03
    know GitHub pilot it'll probably take me
  • 00:47:04
    longer to do this and you know it's not
  • 00:47:06
    too bad to just delegate it to uh this
  • 00:47:09
    guy and you know I don't know do
  • 00:47:10
    something else and then eventually make
  • 00:47:11
    fixes on top in case that are needed um
  • 00:47:15
    so let's see I'm excited finally the
  • 00:47:17
    first issue which I feel it's close to
  • 00:47:18
    solving or you know it has actually
  • 00:47:20
    solved
  • 00:47:30
    all right let's open it
  • 00:47:37
    up let's
  • 00:47:45
    login that doesn't look great uh login
  • 00:47:49
    with Google isn't working
  • 00:47:53
    locally all
  • 00:47:55
    righty CL Cent ID is required I need to
  • 00:47:57
    update the N which would mean I would
  • 00:48:00
    need to move off the screen for a bit
  • 00:48:03
    because myn has my broad database um
  • 00:48:07
    don't ask
  • 00:48:12
    why all right folks uh I was able to run
  • 00:48:15
    it locally the first issue we see right
  • 00:48:17
    here is the changes that Devon made um
  • 00:48:22
    specifically let's look at the pr um
  • 00:48:27
    one change it made was it added use
  • 00:48:29
    State over here it did not exist before
  • 00:48:32
    U and hence it should have made this a
  • 00:48:34
    client component it did not do it so let
  • 00:48:36
    me quickly do
  • 00:48:44
    that API components
  • 00:48:58
    I think it's just easier for a human to
  • 00:49:00
    do this than you know reprompt let's see
  • 00:49:03
    if that's the only thing it
  • 00:49:20
    missed let me go to the profile section
  • 00:49:23
    now I see my name I see a very nice
  • 00:49:26
    pencil I can honestly uh click on the
  • 00:49:27
    pencil icon dude it's pretty good um few
  • 00:49:31
    things are good for example you know uh
  • 00:49:33
    name did not update here but if I
  • 00:49:36
    refresh wow did not update for some
  • 00:49:38
    reason uh what did I miss edit the UI is
  • 00:49:43
    pretty fancy basically as good as I
  • 00:49:45
    would expect it to be um click on this
  • 00:49:49
    update did get back
  • 00:49:54
    uh what do you call it um a 200 and
  • 00:49:58
    surprisingly did not send me back
  • 00:50:02
    my
  • 00:50:06
    password
  • 00:50:09
    interesting why is that I'm sure I look
  • 00:50:11
    into that later the first thing to see
  • 00:50:13
    is why is this not actually updating my
  • 00:50:16
    name um what did I send in the payload I
  • 00:50:18
    sent the new
  • 00:50:19
    name all right folks sorry the recording
  • 00:50:21
    stopped um the feature is fine the
  • 00:50:24
    problem is we store the name in the
  • 00:50:26
    session
  • 00:50:27
    so if I ever go here and update the name
  • 00:50:29
    to something new um it won't immediately
  • 00:50:32
    update everywhere uh the reason being we
  • 00:50:35
    get this name over here from the session
  • 00:50:36
    variable so I need to relog in basically
  • 00:50:38
    if I log out and then I log in again
  • 00:50:44
    um then I will see the change basically
  • 00:50:47
    uh whatever change I made a while back
  • 00:50:50
    that's 1 123123 because this name is not
  • 00:50:51
    coming from the database it's coming
  • 00:50:52
    from the session there is no easy way to
  • 00:50:54
    fix that um you have to to update the
  • 00:50:58
    session cookie which won't be very easy
  • 00:51:01
    U or you can just you know get these
  • 00:51:03
    details from the database which makes
  • 00:51:05
    more sense honestly U name probably
  • 00:51:07
    anyways shouldn't be something that
  • 00:51:08
    should be stored in your uh session it
  • 00:51:10
    should be U you know fetched from the
  • 00:51:12
    database whenever needed um even the
  • 00:51:14
    email I would say just the ID should be
  • 00:51:16
    stored in the session um so that is why
  • 00:51:19
    you would probably need a human here
  • 00:51:20
    it'll be very hard for me to I can
  • 00:51:23
    ask uh this guy to fix it but pretty
  • 00:51:27
    sure uh it'll just be easier to write it
  • 00:51:30
    myself or you know you need an architect
  • 00:51:32
    here or whatever computer engineer
  • 00:51:33
    computer person here to come in and
  • 00:51:35
    figure out which of the two routes to
  • 00:51:37
    take either refresh the session whenever
  • 00:51:38
    the user changes their name um so that
  • 00:51:41
    you know the change persists across a
  • 00:51:43
    refresh right now it doesn't persist
  • 00:51:44
    over here if I refresh um I don't see my
  • 00:51:47
    updated name until I log out and relog
  • 00:51:49
    in um and the other thing is just you
  • 00:51:52
    know fetch these details from a database
  • 00:51:53
    so long story short it did well um the U
  • 00:51:56
    is clean which is I'm something I'm very
  • 00:51:58
    surprised about because I don't know if
  • 00:52:00
    AI can see very well right now but this
  • 00:52:02
    guy you know this is what I would expect
  • 00:52:04
    so I would use it for this for sure um
  • 00:52:06
    the other thing was is the backend wable
  • 00:52:07
    it isn't but that's probably because we
  • 00:52:09
    don't have uh you know if you look at
  • 00:52:11
    our Prisma schema um we don't
  • 00:52:17
    have passwords at all uh we use
  • 00:52:25
    um oh that is why the the user table is
  • 00:52:29
    fairly
  • 00:52:33
    small it has these things only which is
  • 00:52:35
    what gets returned to the front end but
  • 00:52:36
    if there was a password Here the
  • 00:52:38
    password would get returned to the front
  • 00:52:39
    end which isn't necessarily a problem
  • 00:52:40
    it's your own password it's hashed also
  • 00:52:43
    um but still something you shouldn't
  • 00:52:45
    return to the front end doesn't matter
  • 00:52:47
    either way because you don't have a
  • 00:52:48
    password over here it's not a problem
  • 00:52:50
    the long story short it did fairly well
  • 00:52:53
    um is it mergeable no um I'm going to
  • 00:52:57
    give it one loss shot and tell it what
  • 00:52:59
    it needs to fix uh the name is stored in
  • 00:53:04
    the
  • 00:53:05
    session so we need to refresh the
  • 00:53:08
    session after the name is changed or
  • 00:53:12
    fetch the name from the DB right now
  • 00:53:17
    it I only see the updated name after I
  • 00:53:23
    refresh I had to write log out
  • 00:53:27
    honestly that's probably on me um but
  • 00:53:29
    we'll see how this performs and this
  • 00:53:31
    will probably be the end of it
  • 00:53:33
    U final thoughts
  • 00:53:36
    uh it's decent I'd rather just use an
  • 00:53:39
    llm locally
  • 00:53:42
    um then you know use this like it's very
  • 00:53:46
    fancy like having an AI engineer that's
  • 00:53:48
    you know pretty expensive $500 a month
  • 00:53:53
    um there's more Pizzaz than you know
  • 00:53:56
    substance here is what I think I think
  • 00:53:57
    it would have what what's a better
  • 00:54:00
    product is you know something closer to
  • 00:54:01
    an AGI or a better llm um than a lot of
  • 00:54:05
    layers on top um even though this is
  • 00:54:07
    great like it looks very fancy from the
  • 00:54:08
    outside U unsure if it'll be super
  • 00:54:11
    sticky uh which is not to say AI it
  • 00:54:13
    won't come will probably come but it'll
  • 00:54:15
    probably come you know directly to your
  • 00:54:16
    own vs code rather than in a cloud
  • 00:54:18
    environment
  • 00:54:19
    um historically Cloud environments be it
  • 00:54:23
    you know something like replate um have
  • 00:54:27
    been tried to be used as development
  • 00:54:28
    environments all your developer code is
  • 00:54:30
    there GitHub code spaces but rarely does
  • 00:54:32
    that actually work this is slightly
  • 00:54:33
    better U it does have your code base on
  • 00:54:35
    the cloud but it also comits to GitHub
  • 00:54:37
    um but that said unsure if this is a you
  • 00:54:40
    know 10x better outcome compared to a
  • 00:54:42
    local llm that I would just you know tab
  • 00:54:44
    away and review right there and just
  • 00:54:46
    push and merge then hand over everything
  • 00:54:49
    to an AI and look at a pull request uh
  • 00:54:52
    those are the current thoughts U that
  • 00:54:55
    said if it can be a little faster U
  • 00:54:59
    and I wouldn't call it accurate I think
  • 00:55:01
    it's fairly accurate it's probably as
  • 00:55:03
    good as it can get considering it's
  • 00:55:05
    handling everything auton autonomously U
  • 00:55:08
    but if more work can be done on you know
  • 00:55:10
    the llm layer rather than you know all
  • 00:55:13
    the the polishes around uh that might
  • 00:55:16
    make this super interesting also the
  • 00:55:18
    pricing I think is a little too
  • 00:55:20
    aggressive at the moment but I'm sure
  • 00:55:21
    someone will you know create an open
  • 00:55:22
    source version or you know there'll be
  • 00:55:24
    price Wars and eventually it'll get
  • 00:55:25
    priced I think it's like I'm sure the
  • 00:55:27
    margins here are crazy at $500 U so I
  • 00:55:30
    would assume this get squeezed down to
  • 00:55:33
    $100 whenever or maybe even lesser U as
  • 00:55:36
    the market sort of matures U but right
  • 00:55:39
    now super expensive uh not super helpful
  • 00:55:41
    $20 Super m is great U are my current
  • 00:55:44
    thoughts but we'll see how it uh matures
  • 00:55:47
    over time and you know other products
  • 00:55:49
    come and and if they out compete them
  • 00:55:52
    those are my current Thoughts with that
  • 00:55:54
    I'll wait for this to finish I'll show
  • 00:55:55
    you the final results and we'll call it
  • 00:55:57
    for this one I'll see you guys in the
  • 00:55:58
    next one bye-bye
Etiquetas
  • AI
  • automation
  • Devon
  • coding
  • engineer
  • GitHub Copilot
  • software
  • technology
  • DevOps
  • programming