Devin will take your job soon? 500$/month for a coding assistant!
Resumo
TLDRThe video explores the role of AI and automation in transforming society, focusing primarily on "Devon," a tool positioned as an AI-powered junior engineer designed to automate coding tasks. The speaker, a full-stack developer, provides an evaluation of Devon, examining its capabilities, value, and whether it can effectively replace human coders. Through a series of tests involving real coding issues, the speaker determines that Devon, while innovative, is not yet ready to replace engineers due to its limitations and high cost ($500 per month). It can handle simple coding tasks and integrate with platforms like Slack and GitHub, but struggles with more complex tasks without human intervention. The video highlights the need for significant improvements in the underlying language models (LLMs) and AI infrastructure. The narrator suggests that while the concept of AI in coding is promising, Devon and similar products must evolve further to justify their price and be considered fully effective. The video also advises against non-technical people investing in Devon, reinforcing that technical expertise is needed to navigate and derive value from such AI tools.
Conclusões
- 🤖 Devon is an AI tool meant to assist in coding tasks like a junior engineer.
- 💡 The potential of AI in coding exists, but current capabilities are limited.
- 💸 High cost of Devon ($500/month) is seen as a barrier to widespread adoption.
- 📉 Unable to replace engineers fully; further improvements are needed.
- 🔗 Integrates with tools like Slack and GitHub for better workflow.
- 🤔 Requires technical knowledge to set up and use effectively.
- 🚧 Faces challenges with complex tasks, indicating scope for LLM improvement.
- 🔍 Provides insights into the future of AI in the software industry.
- 📈 Could potentially reduce repetitive coding tasks if enhanced.
- 🛠 Full-stack developers find it promising but not ready for full implementation.
Linha do tempo
- 00:00:00 - 00:05:00
In the beginning, the speaker discusses the impact of AI and automation on society, focusing on a demonstration by Deon Devon of autonomous coding. Concerns about the future of coding jobs are raised, despite AI's incremental capabilities like the AI Junior Engineer.
- 00:05:00 - 00:10:00
The speaker evaluates the AI Junior Engineer's capabilities after purchasing a $500 subscription to test if it could replace engineers. The AI struggles with certain repository issues, only partially solving one. Its appearance and integration into tools like Slack have improved, but its foundational logic remains unchanged without significant LLM improvements.
- 00:10:00 - 00:15:00
Local development, preferred due to its familiarity, is deemed more practical than complete AI dependence unless substantial advancements occur. Cloud-based tools, perceived as non-essential staples, validate this sentiment. The high subscription cost makes it economically irrational for long-term use compared to GitHub Copilot.
- 00:15:00 - 00:20:00
The speaker describes the onboarding process for the AI tool, including connecting to GitHub and paying the subscription fee. They connect it with a repository and authorize Slack integration, noting the polished feel akin to onboarding a new engineer.
- 00:20:00 - 00:25:00
Upon onboarding completion, they instruct the AI to verify its capabilities by creating a draft PR. The AI generates a plan and executes steps, but faces minor configuration issues. Despite successfully creating a draft PR, the AI's proposal doesn't accurately address the underlying issue of pagination.
- 00:25:00 - 00:30:00
The AI is tasked to fix a pagination issue, but misunderstands the problem, wrongly altering pagination elsewhere. The fix disregards best practices like prefetching, ultimately delivering a solution that addresses non-existent issues—inadvertently breaking application behavior altogether.
- 00:30:00 - 00:35:00
Aiming for a successful fix, the AI is given a simpler UI task to adjust for image responsiveness. The AI suggests code changes, but they prove ineffective. The speaker considers frontend issues challenging for AI due to visual assessment limits, highlighting discrepancies between human and AI comprehension.
- 00:35:00 - 00:40:00
The AI's struggles with frontend tasks are reiterated, its inability to resolve issues compared against a developer's more thorough solution. A backend task is later presented, deemed more suitable given AI's logic processing strengths. The AI's potential shines through coding actions and its preference for backend duties.
- 00:40:00 - 00:45:00
Creating backend fixes, the AI independently addresses certain logic-heavy tasks. The discussions show it could potentially excel with clear, structured requirements, albeit frontend and visually determinative challenges necessitating human intervention for comprehensive solutions.
- 00:45:00 - 00:50:00
In the final summary, despite efforts for improvement, the AI remains challenged on tasks requiring cognitive flexibility and human intuition. Combining AI's backend efficiency with human-driven frontend resolutions could yield substantive results, though current AI limitations necessitate human oversight.
- 00:50:00 - 00:56:06
The speaker concludes with reflections on AI's current utility, noting rapid market evolution and potential pricing adjustments. They express skepticism at the AI's premium cost, with existing cheaper options providing similar functionality, highlighting areas needing improvement for feasible autonomy.
Mapa mental
Vídeo de perguntas e respostas
What is the video about?
The video discusses AI, automation, and a specific product called Devon, an AI-enabled junior engineer.
Who is Devon in the context of the video?
Devon is an AI tool designed to work as a junior engineer, automating coding tasks.
What are the concerns about AI in automation expressed in the video?
Concerns include the reliability of AI in replacing human engineers, its current capabilities, cost, and the comparison with existing AI tools.
Is Devon effective in replacing human engineers?
The video suggests that Devon is not yet effective in fully replacing engineers, requiring further improvements.
What improvements are suggested for Devon?
Improvements include better alignment and progress in underlying language models (LLMs) for enhanced accuracy and usefulness.
How much does a Devon subscription cost?
The Devon subscription costs around $500 per month.
What are the tested capabilities of Devon in this video?
Devon was tested on solving coding issues, creating pull requests, and integrating with development environments, but faced challenges in completing these tasks efficiently.
Does the video recommend using Devon for non-technical users?
No, it is suggested that non-technical users should not purchase Devon as it requires coding knowledge.
How does the narrator compare Devon to other AI tools?
Devon is compared to GitHub Copilot and other local AI tools, indicating it does not provide significant advantages.
What are the concluding thoughts on Devon's utility?
The narrator concludes that Devon needs more time to mature and improve in both functionality and pricing before it can be deemed truly useful for full-stack development.
Ver mais resumos de vídeos
The BEST way to learn anything: my Learning System for students!
What I Learned After Coaching 3000 Top Students
Marty Lobdell - Study Less Study Smart
13 Years of No BS Study Advice in 58 Minutes
The horrifying ‘experiment’ of Tuskegee | Planet America
The Untold Story Behind the Design of Transistor - Documentary
- 00:00:02Ai and automation have arrived and
- 00:00:04they're going to transform Society
- 00:00:07[Music]
- 00:00:15forever it's a full stack developer Deon
- 00:00:18Devon that incredible demo that we saw
- 00:00:20just a couple weeks ago where you gave
- 00:00:22the agents a prompt and they built out
- 00:00:25entire code Bas and so people are
- 00:00:27naturally worried about where is the job
- 00:00:28scenario going will coding even be a
- 00:00:30thing in the
- 00:00:31future
- 00:00:37accoun but then co-pilots started
- 00:00:39writing a little bit of code GPT 4
- 00:00:41started writing a lot of code they have
- 00:00:43lied about their
- 00:00:45[Music]
- 00:00:52capabilities so Devin the AI Junior
- 00:00:55engineer is open for public release now
- 00:00:57I bought the $500 subscription to go
- 00:01:00through the product and understand is it
- 00:01:02going to replace Engineers right now
- 00:01:04anytime soon I've gone through a fairly
- 00:01:06unbiased process of trying to make it
- 00:01:08fix a few issues in one of our
- 00:01:10repositories U I hire a few Engineers
- 00:01:13now so I would love if a product like
- 00:01:15this finds pmf uh I don't think we're
- 00:01:17there yet let me give you Al tldr U I
- 00:01:21tried to solve three issues using it on
- 00:01:22a fairly simple repository U it was not
- 00:01:25able to solve two of them the third one
- 00:01:27it sort of solved partly and then
- 00:01:28eventually got stuck uh so I think the
- 00:01:31product hasn't changed in the last 6
- 00:01:32months I've seen initial demos it looks
- 00:01:34very similar what has changed is a lot
- 00:01:37of polishes and pizar around it U it
- 00:01:40connects directly to your slack uh it is
- 00:01:42able to create pull requests directly
- 00:01:44into your code base U so I think the the
- 00:01:47thing that has changed is that they've
- 00:01:48made it look very close to a real
- 00:01:50engineer a real human that you can tag
- 00:01:52on slack that can create issues and pull
- 00:01:55requests uh that said uh the underlying
- 00:01:58llm is still the same right so the
- 00:01:59powers that it has are very similar to
- 00:02:01what you would have locally if you're
- 00:02:03using something like cursor GitHub
- 00:02:05copilot super Maven um to long story
- 00:02:08short uh I think unless there's a 10x
- 00:02:11Improvement in llms uh you would find
- 00:02:14fairly similar results if you're working
- 00:02:16locally using GitHub copilot and you
- 00:02:19know pushing and merging compared to
- 00:02:20just delegating the whole task to uh
- 00:02:23something like Devon will it find
- 00:02:25product Market fit eventually unsure if
- 00:02:27this product would uh I'm not being a
- 00:02:30Critic here I I love GitHub copilot or
- 00:02:32you know AI tools generally I feel Cloud
- 00:02:34tools for you know building your
- 00:02:35products haven't found pmf generally uh
- 00:02:38something like replate is not used for a
- 00:02:40fully fledged product people don't use
- 00:02:41GitHub copilot as much people want to do
- 00:02:43local development which is why it makes
- 00:02:45sense to have a vs code extension
- 00:02:47compared to a full-blown engineer U that
- 00:02:50said um if there is a 10x Improvement
- 00:02:52somewhere if they're able to do
- 00:02:54something at the llm level if AGI comes
- 00:02:56eventually then something like this
- 00:02:57would make a lot of sense um also it's
- 00:02:59fairly expensive right now I would
- 00:03:01assume their margins are crazy and their
- 00:03:03retention would be fairly poor because I
- 00:03:05would not even though I've paid for it
- 00:03:06now which is almost like 40,000 rupees
- 00:03:08would have been much easier for me to
- 00:03:10just give this to an engineer for a
- 00:03:11month uh I don't think I'll use it after
- 00:03:13today uh for no reason other than you
- 00:03:15know uh it'll take me more time to
- 00:03:16figure out what's up and you know uh
- 00:03:18I'll spend more time looking at the
- 00:03:19fanciness and understanding how this
- 00:03:20product is working uh than in providing
- 00:03:22any value at this point U compared to
- 00:03:24you know a $20 subscription to GitHub
- 00:03:27copilot U so those are the initial
- 00:03:29thoughts I was super excited to try it
- 00:03:31out uh I was hoping something fancy
- 00:03:33based on the pricing of $500 a month but
- 00:03:36it looks fairly similar to where it was
- 00:03:386 months ago that said they have great
- 00:03:39GitHub demos that they've shown and you
- 00:03:41know issues that it has actually solved
- 00:03:43so look at them be aware of of it but
- 00:03:45this video should take you through
- 00:03:46full-blown demo of what they've produced
- 00:03:48and what can you expect if you're
- 00:03:50thinking of buying the $500 subscription
- 00:03:53to a junior
- 00:03:54engineer PS uh don't buy the
- 00:03:56subscription if you're not technical at
- 00:03:58some point you'll have to dive in and
- 00:03:59you look at the code base uh so if
- 00:04:01you're a I don't know marketeer thinking
- 00:04:03of this providing a lot of value at $500
- 00:04:05let me tell you it won't if you're an
- 00:04:07engineer at may or may not provide you
- 00:04:08value with that let's get into the video
- 00:04:11right step one is signing up for Devon
- 00:04:14um as I said it's publicly available now
- 00:04:17so if you sign up as long as you pay
- 00:04:18$500 you can test out the product
- 00:04:20yourself
- 00:04:32this is what the onboarding looks like
- 00:04:34let me put my name connect my GitHub and
- 00:04:37connect my
- 00:04:44organization and next up I need to pay
- 00:04:45for it so let me pay around $500 and
- 00:04:49then we'll go from
- 00:04:52there all right folks let's pay for it
- 00:04:54and see how it
- 00:04:58goes purchase sub subscription I think
- 00:05:01it's around $500 a month
- 00:05:09yep pretty expensive video guys oh you
- 00:05:12guys can't see but it's 42,000 rupees
- 00:05:14let's see how it goes let's get right
- 00:05:16into it all right guys I've gone through
- 00:05:19the onboarding um let me take you
- 00:05:21through it really quickly it asks you
- 00:05:22for your name um gives you some sort of
- 00:05:25greetings it says gitlab and bit bucket
- 00:05:27are coming soon thankfully for us most
- 00:05:29of the things are on GitHub um this is
- 00:05:31my organization where most of our code
- 00:05:33is there uh We've purchased the meeting
- 00:05:36plan um or whatever you know the $500
- 00:05:39plan the team's plan for now and let's
- 00:05:42click on start using de and see how it
- 00:05:44goes step one unexpectedly connecting
- 00:05:47GitHub um they have pretty decent
- 00:05:49onboarding I would say this is how you
- 00:05:51would onboard a random engineer as well
- 00:05:53I have a few repositories I'll probably
- 00:05:55just connect it to code 100x for now um
- 00:05:59then we'll go from there it I'm giving
- 00:06:02it access to all repositories in code
- 00:06:04100x
- 00:06:06um authorized Deon for the
- 00:06:09integration connecting slack that is
- 00:06:11interesting I don't think I have slack
- 00:06:13yet uh so let me quickly create we do
- 00:06:16have one for super 30 so let me just
- 00:06:18connect it
- 00:06:20there connect slack
- 00:06:24organization allow it access on
- 00:06:26everything and I've received an OTP
- 00:06:31that's why developers are dumb at these
- 00:06:33things yeah isn't slack is connected
- 00:06:36link the user I'd assume this will be
- 00:06:38like a de
- 00:06:40user okay there we go um invite
- 00:06:43teammates I think I'm fine for
- 00:06:46now can I skip through that uh nope yeah
- 00:06:50I can my workspace set up your first
- 00:06:52deposit my workspace resets saved state
- 00:06:54for every new session let's set up a
- 00:06:55repository to include this saved State
- 00:06:58um
- 00:07:00interesting setup significantly improves
- 00:07:02D Performance on your codebase I would
- 00:07:04assume this is them going through our
- 00:07:06repositories and you know U indexing all
- 00:07:09the data somewhere in some database or
- 00:07:11otherwise um so let's give it access to
- 00:07:14an easy one to begin with
- 00:07:16um daily code would probably be it um
- 00:07:20I've lost context on this repository
- 00:07:22honestly so we'll see we'll see if it
- 00:07:25gets more context than I
- 00:07:27do then works on the machine you're
- 00:07:29seeing please install dependencies as
- 00:07:32you would with a new hire missing
- 00:07:34dependencies to ding being slow and
- 00:07:35unreliable interesting they give you
- 00:07:37access to a machine I would assume uh
- 00:07:41this
- 00:07:42is a pod or you know whatever a
- 00:07:45container somewhere that dies from time
- 00:07:46to time but I could be wrong de works on
- 00:07:48the machine you're saying please install
- 00:07:49dependencies as you would for a new hire
- 00:07:51missing dependencies can lead for it to
- 00:07:52be slow and underl all right that looks
- 00:07:55fairly polished honestly uh feels like
- 00:07:58I'm on boarding a new engineer um which
- 00:08:01is always uh a task that Engineers dread
- 00:08:04all righty seems like my machine is up
- 00:08:06begin the
- 00:08:13setup wide screen for a long
- 00:08:16time there we go I have access to my
- 00:08:19repository over here I think I'm okay
- 00:08:22with this specific theme um okay they
- 00:08:26don't give give you access to a machine
- 00:08:27this seems like um based on how quick it
- 00:08:30is it's mostly running locally inside my
- 00:08:32browser and uh could either be a machine
- 00:08:35or be web containers now I'm curious um
- 00:08:38but let's assume it's system machine
- 00:08:40let's see if it has all the dependencies
- 00:08:41does it have I think we're using
- 00:08:43pnpm and it does does have pnpm but we
- 00:08:48don't use pnpm it seems what do we use
- 00:08:50here we use Yar so let's see if it has
- 00:08:54yarn and it does
- 00:08:57um it seems to uh work as expected so I
- 00:09:02don't think I need to add any
- 00:09:03dependencies on top of this so I can say
- 00:09:05done um up keep that's how you clone the
- 00:09:09repository that looks fine and how do
- 00:09:12you update dependencies you do a Yar
- 00:09:14install here that looks decent
- 00:09:17um and let's try to just confirm by
- 00:09:20going through the terminal
- 00:09:23here why are there two CDs
- 00:09:29seems like that copy button is
- 00:09:31broken when I do a y
- 00:09:35install seems to work so I think we're
- 00:09:37fine cool so upkeep is done this is how
- 00:09:41it can pull the latest code base this is
- 00:09:43how it can install new dependencies um
- 00:09:45done and done and lastly tell Deon how
- 00:09:48to verify and submit changes to the
- 00:09:50repository Deon will always see these
- 00:09:52instructions when working on this
- 00:09:54repository advise on how to run tests
- 00:09:55formatting PR execution so on and so
- 00:09:57forth seems like they generated it for
- 00:10:00me uh two options to run it locally
- 00:10:01Docker compose watch or manually by
- 00:10:03running this um that looks decent code
- 00:10:07quality before submitting a PR run yan
- 00:10:10yan run lint that looks right Yan run
- 00:10:12format for formatting it PR checks
- 00:10:14automatically linting informating
- 00:10:16merging to main deploys to staging
- 00:10:17merging to production deploys to
- 00:10:18production that looks decent uh this
- 00:10:20specific project is deployed on staging
- 00:10:25do projects. 100ex apps.com and
- 00:10:28projects. 100ex st.com based on whether
- 00:10:30you're using the staging environment
- 00:10:32which is synced to uh the main the
- 00:10:35develop branch and then this is probably
- 00:10:37sync to the master or the production
- 00:10:38Branch um so this looks decent better
- 00:10:42than I would have hoped for U no real
- 00:10:46you know hallucinations already so it
- 00:10:47seems like a polished product I click on
- 00:10:49Save here and finish after completing
- 00:10:52the setup it may take up to 10 minutes
- 00:10:54until the save is complete any de
- 00:10:56sessions started will not use the setup
- 00:10:58all righty so so we'll wait for 10
- 00:10:59minutes uh wait for it to set it up and
- 00:11:02then we'll go from
- 00:11:05there all right folks I have onboarded
- 00:11:07one of my repositories uh we'll now try
- 00:11:11to assign it some issues and see how
- 00:11:13well it performs on some real production
- 00:11:15issues on the
- 00:11:17repository this is what the dashboard
- 00:11:19looks like right now I've gone through
- 00:11:20the onboarding process um the last step
- 00:11:22is done as well I'm not inviting any
- 00:11:24teammates for
- 00:11:26now so I'm going to skip the rest of
- 00:11:28onboarding specifically inviting
- 00:11:30teammates and now it says after
- 00:11:32connecting slack just tag Devon in a
- 00:11:34thread to get
- 00:11:38started all right that looks like a good
- 00:11:41start it asks let's just try this it
- 00:11:43says hey Devon please verify that you
- 00:11:45can access this repository run lint and
- 00:11:49open a dummy draft PR with a simple
- 00:11:50change let's see if it's able to do that
- 00:11:53so let me go to super 30
- 00:11:55slack make people a little anxious over
- 00:11:58here by tagging
- 00:12:04Devin and let's see how it
- 00:12:08performs oh it's not
- 00:12:13invited all right let's see what it
- 00:12:15replies with Devon is ready you can now
- 00:12:17send messages snapshot Dev environment
- 00:12:19this thing was used to mute Devon sent
- 00:12:22this thing you can track the process
- 00:12:23progress here I would assume that's this
- 00:12:25specific thread congrats on starting
- 00:12:27your first session I'm excited to work
- 00:12:28with you here are the some things that
- 00:12:31to know about working with me that looks
- 00:12:35decent give me a task uh that you know
- 00:12:38how to do yourself keep me tell me how
- 00:12:41to test it and check that I'm on the
- 00:12:42right track keep sessions under 3 hours
- 00:12:44to break down big tasks share detailed
- 00:12:47requirements upfront invest in coaching
- 00:12:49Me by providing feedback in chat I'll
- 00:12:51approve over time I'll verify access to
- 00:12:53the reers on and so forth all right
- 00:13:04let's give that another
- 00:13:25try okay there you go it was just taking
- 00:13:27time I'll verify all three capabilities
- 00:13:29right away first I'll check if I can
- 00:13:31access the
- 00:13:46repo I sent the same message here
- 00:13:49because I was a little
- 00:13:53impatient and it's responded with a
- 00:13:55similar thing it's trying to do that
- 00:13:57let's see how long it takes it's 1222
- 00:13:59right
- 00:14:04now all right It's 12:24 so it it took
- 00:14:08it 2 minutes it says I'll verify access
- 00:14:10to this thing I encountered an issue
- 00:14:11while trying to run the link code the
- 00:14:12command failed
- 00:14:14uh it appears there to be a
- 00:14:16configuration issue according to best
- 00:14:19practice I should not attempt to fix
- 00:14:20configuration issue myself would you
- 00:14:22like me to continue with a creating a
- 00:14:24draft PR wait for some instructions okay
- 00:14:26that's pretty good actually I think lint
- 00:14:28does indeed fail um so let
- 00:14:33me do that continue without creating a
- 00:14:39draft PR oh sorry continue with creating
- 00:14:42a draft PR let's see how long it takes
- 00:14:45for it to do
- 00:14:48that it responded back with I'll proceed
- 00:14:50with creating a draft PR I'll make a
- 00:14:52simple change to the readme formatting
- 00:14:59and someone reacted with a happy
- 00:15:05Emoji surprisingly the other thread it
- 00:15:08said something similar but it did create
- 00:15:10a PR so let's see what the pr looks like
- 00:15:14it is a draft PR it's created by Devon
- 00:15:16AI integration if I look at the files
- 00:15:19change um it looks like a decent change
- 00:15:21honestly uh adding bash over here so we
- 00:15:25get you know decent color coding um
- 00:15:29option one option
- 00:15:30two just adding indexes everywhere which
- 00:15:34honestly not the worst change in the
- 00:15:36world so let's merge that and go from
- 00:15:39there it seems like it's
- 00:15:41testing uh which
- 00:15:45oh let's mark it for ready for review
- 00:15:47and just merge it and go from
- 00:15:52there right it took me like 5 minutes to
- 00:15:56let us know that everything is working
- 00:15:58it's able to create a it created a very
- 00:16:00simple PR updated readme U the one thing
- 00:16:03you learn from this is you at least
- 00:16:05don't need to update readms anymore and
- 00:16:07AI can do that for you next up let's
- 00:16:09give it a real
- 00:16:14issue all right I was able to clean up
- 00:16:16the readme create a simple draft PR now
- 00:16:19let's try to give it a real issue here
- 00:16:22are the issue lists on projects. 100ex
- 00:16:25steps.com again this is the platform
- 00:16:29that we're trying to work upon fixed
- 00:16:31issues upon let's try to find an issue
- 00:16:33over here that should be easy for an AI
- 00:16:35to solve if you write the right context
- 00:16:37in open AI chat GPT it gives you a
- 00:16:40response that you have to manually paste
- 00:16:41over here let's see if Devon can do the
- 00:16:43same and create a draft PR for
- 00:16:54us o that looks like a great one we
- 00:16:58don't have p ination here yet so all the
- 00:17:00projects that you see are you know
- 00:17:02directly over here let me turn on dark
- 00:17:05mode um which means there's no real
- 00:17:07sense of this Bottom bar honestly either
- 00:17:10we should add pagination both on the
- 00:17:12back end and the front end or at the
- 00:17:14very least add pagination on the front
- 00:17:17end on the front end we should be able
- 00:17:18to go from page one to page two to page
- 00:17:20three even if we're not hitting the back
- 00:17:22end to get back the you know next Les
- 00:17:24list of projects so let's try to see if
- 00:17:27it can fix it uh
- 00:17:30I think all the notes are showing all at
- 00:17:32once by default and all are selected on
- 00:17:34the drop- down menu and then there is no
- 00:17:36use of pagination exactly I think that's
- 00:17:38what's happening as well if you want to
- 00:17:40show limited notes in the first page
- 00:17:41then I can fix it is this necessary can
- 00:17:43you assign this to me all right someone
- 00:17:45did create a PR for it let's look at the
- 00:17:48files changed it's around 28 lines of
- 00:17:50code that have been changed so let's see
- 00:17:52if Devon can fix it um at Devon can you
- 00:17:57fix this speciic specific
- 00:18:01PR so let's see uh if it is able to do
- 00:18:05that um it's 12:37 right now so let's
- 00:18:08see how long it takes for it to create a
- 00:18:10draft PR for it all right um so it
- 00:18:14generated a plan for me right away it
- 00:18:16said I'll take a look at the issue in
- 00:18:17the daily code repository and work on
- 00:18:20fixing it I'll first work on a plan if
- 00:18:23you want uh me to confirm the plan with
- 00:18:25you click on this button I did not
- 00:18:27here's my plan this is what the plan
- 00:18:28looks like it seems like it first
- 00:18:30creates a very elaborate plan that's the
- 00:18:32first call it's making to an llm and the
- 00:18:35llm here is telling you fix pagination
- 00:18:37controls that are not be properly worked
- 00:18:39clicking the previous and the next
- 00:18:40buttons the page content is not updating
- 00:18:43when navigating so I don't think it
- 00:18:44understood the problem very well uh
- 00:18:45because the problem isn't that next and
- 00:18:46previous aren't working the problem is
- 00:18:48that we're just rendering everything in
- 00:18:50the first go um create a new Branch for
- 00:18:52the
- 00:18:54fix which will be done by this specific
- 00:18:56command um I don't know why they're
- 00:18:58adding the time St here add us router
- 00:19:01hook in blog. TSX to handle the route
- 00:19:03change oky DOI
- 00:19:12um add use effect to handle the route
- 00:19:15change and force a
- 00:19:17render all
- 00:19:19righty update custom pagination to use
- 00:19:24router. push instead of link for better
- 00:19:25client side navigation
- 00:19:28replace link component with router. push
- 00:19:31and test the changes navigate to a track
- 00:19:34with multiple problems all right I think
- 00:19:37it'll be very confused when trying to
- 00:19:39fix this PR based on the steps that it
- 00:19:42has created it's nice what it's done
- 00:19:44it's tried to create a plan um but from
- 00:19:46what I can get if a developer would
- 00:19:48create a plan in front of me like this I
- 00:19:49would ask a few questions and you know
- 00:19:51ask them to change a few things um it
- 00:19:53has not done that also I don't know if
- 00:19:55it has any data to test imagination so
- 00:19:57there's a high probability it's not able
- 00:19:59to test it very well U because well
- 00:20:01there's a c. TS um file that we have let
- 00:20:04me show you we have a file that you can
- 00:20:06run to seed some data in there U
- 00:20:08hopefully
- 00:20:10packages um DB Prisma seed. TS but I'm
- 00:20:15unsure if it if it ever runs this and
- 00:20:18you know seeds the data in the database
- 00:20:19so it can test everything well but we'll
- 00:20:22see one more thing I saw was U you can
- 00:20:24actually track all the changes here this
- 00:20:26is the one that currently being worked
- 00:20:28on if you look at this you can actually
- 00:20:29follow de and see what it's doing um so
- 00:20:34it created the plan which we already saw
- 00:20:36and I think now it's executing the plan
- 00:20:37Deon is currently running the
- 00:20:38development server to test navigation
- 00:20:39all right here and there you go it's
- 00:20:41stuck right on uh
- 00:20:47authentication it opened this browser
- 00:20:50opened this I don't know how it got Auto
- 00:20:52redirected to O also did not need to
- 00:20:55because I'm not even if I'm not logged
- 00:20:57in
- 00:20:59I can still see everything um so I don't
- 00:21:02have to log in to fix this issue but
- 00:21:04let's see editor open. env. example file
- 00:21:07which is this one right
- 00:21:10here I can actually follow through
- 00:21:12everything it had done the UI fantastic
- 00:21:15like they everything is nailed uh all
- 00:21:17the t's are crossed uh
- 00:21:21but it's a fancy product but under under
- 00:21:23the hood is still using you know an llm
- 00:21:25which has restrictions to how well it
- 00:21:28can you know solve this issue oh well it
- 00:21:30did open seed data. TS I might have
- 00:21:32spoken too
- 00:21:48soon it
- 00:21:50ran well one thing I did not expected to
- 00:21:53do which it did is at least it's trying
- 00:21:55to migrate the database and then it
- 00:21:56probably saw there is no database URL
- 00:21:58and now let's see if it creates a
- 00:22:00database URL or containerize the
- 00:22:06application here is the problem I think
- 00:22:09the problem is you know um not all code
- 00:22:13bases are written in an AI friendly
- 00:22:15fashion for example there is a Docker
- 00:22:17file here it did not end up using the
- 00:22:19docker file um well there you go but if
- 00:22:23it was I think if the code was written
- 00:22:24in a more AI friendly fashion it would
- 00:22:26have been easier for it to figure out
- 00:22:27things it still isn't doing too bad um
- 00:22:30it opened
- 00:22:32NV it'll probably try to update n over
- 00:22:35there but the question really
- 00:22:38is will it be able to start a database
- 00:22:44um it ran npx press man generate which
- 00:22:47did generate everything so it seems like
- 00:22:50it did create the rightn
- 00:22:54file would be nice if I can see it
- 00:23:03and it ran npx Prisma generate to
- 00:23:08generate the Prisma client well it was
- 00:23:09able to generate it which it can do
- 00:23:11without the database URL hold on oh it
- 00:23:14did create a database URL postgress
- 00:23:17database this thing will be public with
- 00:23:19public schema this thing was
- 00:23:22successfully
- 00:23:24reset very cool so it does create an AWS
- 00:23:27postgress server for me so it's smart
- 00:23:30enough to do that very surprising um
- 00:23:33this is the database URL this is
- 00:23:38the DB username and I'm assuming there's
- 00:23:41a password somewhere
- 00:23:43here this is super interesting it was
- 00:23:47they've covered this corner case which I
- 00:23:49thought they wouldn't which is they
- 00:23:50created database for me and they're
- 00:23:52migrating that database now it opened
- 00:23:54tracks it's seeing a 404 for some reason
- 00:23:57but I'm I'm sure it figure it out now
- 00:23:59I'm getting more confident in its
- 00:24:01abilities that said this seems like a
- 00:24:03decentish attack Vector it's creating
- 00:24:06databases for me
- 00:24:08um which you know what if I create a
- 00:24:11very big database or you know um create
- 00:24:15multiple databases because I'm sure this
- 00:24:17AI has access to create databases long
- 00:24:19story short unless they have limits to
- 00:24:21how many databases I can create I can
- 00:24:22create thousands of databases in their
- 00:24:23AWS account which isn't the best thing
- 00:24:26in the world but we'll see
- 00:24:36very slow very dumb no not very dumb
- 00:24:39like it's actually pretty
- 00:24:40smart but yeah what are you
- 00:24:47doing I guess
- 00:25:01all right folks um seems like it's done
- 00:25:04it just let me know it has created a
- 00:25:06pull request here so let's try to go
- 00:25:09through its changes um more changes than
- 00:25:11the other pull requests we saw but let's
- 00:25:13see what it says added proper client
- 00:25:15side navigation for reender whenever
- 00:25:18problem changes it's like anti pattern
- 00:25:20that you shouldn't follow updated this
- 00:25:22thing to use router. push and test it in
- 00:25:25both Legacy and new UI modes interesting
- 00:25:28it says it's tested so I'm excited
- 00:25:34um whenever problem. ID changes on blog.
- 00:25:39TSX it refreshes that
- 00:25:42looks totally wrong um because this
- 00:25:46blog. TSX if I'm not wrong
- 00:25:51um is for this page projects. 100ex
- 00:25:55sts.com SL a specific track like this
- 00:25:58renders the blog component and what we
- 00:26:01wanted a fix for was here um on the
- 00:26:04landing page so that looks
- 00:26:06wrong let's go back here oh sorry that's
- 00:26:09the original PR of a
- 00:26:14human custom pagination
- 00:26:17TSX uh I'm assuming it understood the
- 00:26:21problem incorrectly and I think it's
- 00:26:22fixing pagination or trying to fix
- 00:26:25pagination here in this bot bar previous
- 00:26:29and next rather than fixing it the other
- 00:26:30place but this isn't broken is my
- 00:26:33guess um so let's look at that
- 00:26:41again it replaced these links with
- 00:26:44buttons which is bad because um if you
- 00:26:48have these links then next year we
- 00:26:51prefetch some pages for example if you
- 00:26:53go here uh and let me open the network
- 00:26:56Tab and show you
- 00:26:58if I click on next um we get back the
- 00:27:02current page as well as the next page
- 00:27:04next JS will prefetch Pages um if
- 00:27:07they're already in a button somewhere
- 00:27:08here if you if I add a link tag like we
- 00:27:11did over here then it and if I have
- 00:27:14prefetch set to true then it'll prefetch
- 00:27:17the next page that gives us fast
- 00:27:19navigation when I'm on this specific
- 00:27:21page the next page is already fetched
- 00:27:22you can also see that here okay when I'm
- 00:27:24on page you know this one the next page
- 00:27:27also comes let's see if I can show it
- 00:27:29better if I go to the next page there's
- 00:27:31not just one page that came um page
- 00:27:35number six page number 7 also came so
- 00:27:37that when I go to I'm on page six right
- 00:27:39now when I go to page number seven it's
- 00:27:41already prefetched for me if we do what
- 00:27:44the AI did over here if you replace it
- 00:27:45with a button then that prefetching goes
- 00:27:47away which is bad then you know it'll be
- 00:27:49really slow to fetch the next page
- 00:27:51whenever the user clicks on the next
- 00:27:52button it'll become slower so it was
- 00:27:54consciously added over here with
- 00:27:56prefetch u so that's bad long story
- 00:27:58short it fixed pagination in the wrong
- 00:28:01place where it wasn't even broken in the
- 00:28:03first place and number two uh it fixed
- 00:28:06it in a wrong way like I don't know why
- 00:28:08it chose that specific route this seems
- 00:28:11like a very anti pattern to uh you know
- 00:28:13refresh do a router. refresh or Force R
- 00:28:16renders U whenever the problem ID
- 00:28:18changes um I would discard this for now
- 00:28:21um but I'll try another issue um if a
- 00:28:24user would have created this issue I
- 00:28:26would immediately close it and tell them
- 00:28:29this just looks wrong I wouldn't spend
- 00:28:30too much time trying to explain why it's
- 00:28:31wrong um unless you know I'm in the mood
- 00:28:34most of the times this would be a pretty
- 00:28:35bad fix but that's fine let's give it
- 00:28:38the benefit of the doubt and let's try
- 00:28:40to see if it can solve maybe a slightly
- 00:28:42simpler
- 00:28:45issue uh all righty folks I took a mild
- 00:28:48detour um whenever a platform like this
- 00:28:50is released you know to the public it's
- 00:28:53the best time to find the vulnerability
- 00:28:54the early bugs in Facebook were found
- 00:28:56right after a feature was released uh in
- 00:28:58Facebook um I'm not saying I found
- 00:29:00anything I found something I don't know
- 00:29:02I wouldn't call it trival either I
- 00:29:03wouldn't call it zero day vulnerability
- 00:29:05either somewhere in the middle I've
- 00:29:06reported it to the team we'll see how
- 00:29:08that goes uh but I'm not going to share
- 00:29:11too much about what that is but you know
- 00:29:12feel free to explore de and see if you
- 00:29:14can find any prompt injections or some
- 00:29:16way to you know steal data from other
- 00:29:17users things like these these are you
- 00:29:19know this is the time to do it uh
- 00:29:21because one AI has been given a lot of
- 00:29:23control so there's a high probability AI
- 00:29:25can access data that I shouldn't have
- 00:29:27access to and I can you know prompt
- 00:29:28inject to get it U and there are shared
- 00:29:31resources so you can look around there
- 00:29:32and you know try to figure out what's up
- 00:29:34and if there's a way to you know find
- 00:29:35the vulnerability reported to the team
- 00:29:36get a bounty for it coming back to the
- 00:29:39original problem what were we doing we
- 00:29:41were trying to solve slash make it solve
- 00:29:44a few issues um so let's go back to our
- 00:29:47repository that's code 100 x/ daily code
- 00:29:51let me find another issue probably a
- 00:29:54simpler one and see if it can solve it
- 00:29:57uh loading skeletons is shown while
- 00:29:59seeing the tracks even if the tracks are
- 00:30:01empty or after filter sort
- 00:30:04action
- 00:30:08uh I'm just trying to find one that an
- 00:30:10AI can understand better or you know
- 00:30:12we're sure it should be able to solve it
- 00:30:13layout issues in the PDF yeah I think
- 00:30:16that's too much for it um unresponsive
- 00:30:18image in search bar on smaller screens
- 00:30:21um let's
- 00:30:24see make it smaller
- 00:30:30oh there we
- 00:30:32go unsure if it's very easy for an AI to
- 00:30:35fix let's look at the fix that someone
- 00:30:38else made it's a 13 liner so I would
- 00:30:41assume it's not too difficult to fix um
- 00:30:44so let's give it that let's see if
- 00:30:47um Devin can fix this specific issue
- 00:30:52I'll try to recreate it as well while
- 00:30:53it's trying to fix it because it of
- 00:30:55course takes longer than you would hope
- 00:30:57I would assume it take like I know 10
- 00:30:58minutes to solve it U so while that is
- 00:31:00happening let me quickly try to see is
- 00:31:04this reproducible let's go to projects.
- 00:31:06steps. whoopsy
- 00:31:10Daisy whoopsy Daisy part two um I this
- 00:31:13sounds
- 00:31:17fine let's try to resize it and make it
- 00:31:20really
- 00:31:26small uh I'll need a
- 00:31:28mobile view for
- 00:31:31this yeah that looks bad on a extremely
- 00:31:34small device honestly um unsure if it's
- 00:31:37a very big issue but an issue
- 00:31:39nonetheless let's make a
- 00:31:42bigger yeah there are other issues for
- 00:31:44example you know 3 months ago is
- 00:31:46overflowing on a fairly thin device um I
- 00:31:49would assume most mobile devices are you
- 00:31:51know let's select I don't know an iPhone
- 00:31:5312 Pro looks decent U if anything there
- 00:31:56are other issues there's too much black
- 00:31:57on the right um but we'll see we'll see
- 00:32:00if AI can fix this
- 00:32:03U while that happening I'd like to you
- 00:32:05know do another one but let's see um
- 00:32:07I'll take a look at the issue and work
- 00:32:09on it I'll first work out the plan
- 00:32:11access the knowledge knowledge being the
- 00:32:12repo notes for codex flash daily code
- 00:32:15and autogenerated knowledge Auto
- 00:32:18generated knowledge for this unsure what
- 00:32:19this is let's
- 00:32:25see interesting
- 00:32:28that's very cool um it understood my
- 00:32:31code and generated a lot of context
- 00:32:33around it
- 00:32:35um
- 00:32:37cool let's wait for it to finish let's
- 00:32:39see what's what it's doing right now de
- 00:32:41is working it's viewing the
- 00:32:43issue yeah it's fairly slow so we'll
- 00:32:45wait for it to finish uh let's look at
- 00:32:47the planner clone the repo if needed
- 00:32:50investigate the code base plan approved
- 00:32:52equal to suggest plan if not plan
- 00:32:54approved O2 if it is approved then
- 00:32:56execute the uh the plan all righty these
- 00:32:59are the steps it's performing right now
- 00:33:00and I would assume after the plan is
- 00:33:02created it'll execute the steps of the
- 00:33:04plan one after the other so we'll wait
- 00:33:05for it to
- 00:33:06finish all righty folks um it's doing a
- 00:33:09bunch of follow-ups which is great for
- 00:33:11example my lock file isn't up toate it's
- 00:33:13trying to update it U but before that it
- 00:33:16did create a poll request for me so
- 00:33:18let's check it out um oh okay it
- 00:33:20probably changes package lock or
- 00:33:22something yeah sorry YN lock up but
- 00:33:24let's look at the change um hm
- 00:33:28magnifying glass class name equal to
- 00:33:29size6 and all right it
- 00:33:34changed the size of an icon content
- 00:33:39search
- 00:33:41uh content
- 00:33:46search or did I give it a wrong
- 00:33:50issue
- 00:33:51nope yeah I don't think it was able to
- 00:33:55do what it was supposed to do I'm I'm
- 00:33:58doubting my code base now is it not
- 00:34:01there is there not enough context in
- 00:34:02there that an AI is not able to solve
- 00:34:04such a simple issue
- 00:34:06um I don't know if this will this looks
- 00:34:08like the wrong place for it honestly uh
- 00:34:10because our tracks exist over here and
- 00:34:12it's probably trying to fix something
- 00:34:13over here unless that's what it was
- 00:34:15supposed to be fixed and I don't
- 00:34:16understand the issue let's see
- 00:34:20um I was there sorry my bad um it was
- 00:34:24indeed here that the problem existed
- 00:34:27yeah I'm the the dumb guy of course AI
- 00:34:29is smart so in the search bar is where
- 00:34:31the issue was which was here and it
- 00:34:34increased the size of this search I
- 00:34:35which is very very cool considering it
- 00:34:38does look a little small I don't know
- 00:34:39how it figured out whether the r needs
- 00:34:42to increase in size um so that's good
- 00:34:44the other thing it did was
- 00:34:47uh it updated Yan lock which is great
- 00:34:50and sorry this looks like the wrong pull
- 00:34:55request the other thing it did
- 00:34:59was it changed the height to be or it
- 00:35:03retained the height to be 400 pixels but
- 00:35:05it made it 300 pixels on a smaller
- 00:35:06screen which probably will fix it
- 00:35:09um well it seems like it did fix a
- 00:35:11simple issue um can I search for
- 00:35:13something type
- 00:35:15script the current issue was
- 00:35:22um this there we go oh that's fairly
- 00:35:25obvious issue much better and what it
- 00:35:27did is let me change the Styles over
- 00:35:30just apply it over here and see if it
- 00:35:32does fix it
- 00:35:33um it
- 00:35:39changed where is 400 there you go it
- 00:35:41changed h-400 to
- 00:35:44be MD colon h-400 and otherwise h-300
- 00:35:51pixels all right sorry about that uh
- 00:35:54let's try it one more time I'm going to
- 00:35:55have a single row over here um and
- 00:35:58currently does the issue exist let's try
- 00:36:00it
- 00:36:01out it indeed exists this photo sort of
- 00:36:04becomes very pixelated um and the fix
- 00:36:08that it added was it made the height
- 00:36:13of this whever 400 pixels is right here
- 00:36:17it made it 300 on a smaller screen let
- 00:36:19me make it 300 generally um and see if
- 00:36:22that fixes it for a smaller screen let's
- 00:36:24make it smaller and it seems like it
- 00:36:27does not
- 00:36:29um I would assume it does fix it a
- 00:36:31little bit um like if the issue
- 00:36:33originally started at this point it now
- 00:36:35starts at this point um but does not
- 00:36:37solve the issue at hand which probably
- 00:36:40makes sense because AIS aren't very
- 00:36:43visual I I'm assuming it's not taking
- 00:36:45screenshots and forwarding forwarding it
- 00:36:47to an AI to test this so it's really
- 00:36:48hard to test for an AI whether or not a
- 00:36:50visual fix is Right U so that's the AI
- 00:36:54fix even though it was smaller uh
- 00:36:57compared to you know the other person's
- 00:36:59fix this is the original fix that the
- 00:37:00other person made U and let's look at
- 00:37:03the AI fix that's the one the AI fix was
- 00:37:05just two lines of code U and no reason
- 00:37:07for this one it was pretty much this uh
- 00:37:10which did not fix it um but if you look
- 00:37:13at the change that gur kirat made um
- 00:37:20it's much more lines of code firstly it
- 00:37:21does fix it like I'm sure there's a
- 00:37:23screenshot or video here so if you look
- 00:37:25at it as you scroll down
- 00:37:28it does yeah it pretty much fixes the
- 00:37:30issue this thing is not stretching out
- 00:37:32um and if you look at the changes um
- 00:37:36surprisingly he's increased the height
- 00:37:38for you know throughout and um over here
- 00:37:43rather
- 00:37:44than having just an image right next to
- 00:37:47some text or whatever you know this
- 00:37:48content um header and title um
- 00:37:53he still wrapped it inside a div where
- 00:37:56he added items Center to probably
- 00:37:58vertically align it added some more Gap
- 00:38:00um and I assume this is what helps you
- 00:38:02know in making sure it doesn't stretch
- 00:38:04because this padding on oh that's a
- 00:38:07bigger screen my bad U so this this
- 00:38:08won't fix it uh this probably fixes it
- 00:38:12on a smaller screen the width and height
- 00:38:14is 16 bigger screen is 20 it's pretty
- 00:38:17much hardcoded here so you know um
- 00:38:18there's no reason for it to stretch
- 00:38:20we've hardcoded the width and a height
- 00:38:21on a smaller and a bigger screen um that
- 00:38:24way this image isn't stretched so that
- 00:38:25makes sense u a more elaborate fix um
- 00:38:28but indeed one that works um and as I
- 00:38:31said I don't think the AI is to blame
- 00:38:33here U the reason this did not work for
- 00:38:36an AI probably it's because it's a front
- 00:38:38end issue U so maybe it would be good
- 00:38:40for us to try a backend issue as well U
- 00:38:42let's do that next let's give it the
- 00:38:44benefit of the doubt second time uh and
- 00:38:46see if it is able to solve a backend
- 00:38:49issue that may exist in this repository
- 00:38:50although I'm unsure if it does if it
- 00:38:51doesn't then I'll fabricate an issue and
- 00:38:53see how it performs
- 00:38:58all right folks I wasn't able to find an
- 00:39:00issue so I fabricated one U it's a
- 00:39:02mixture of front end and back end but
- 00:39:03front end should be fairly simple on
- 00:39:05this one um the issue is that if you go
- 00:39:08to the profile section here um you see
- 00:39:11your name that comes from Google and
- 00:39:12your email you should be able to edit
- 00:39:14this name so you should have a pencil
- 00:39:16icon over here when you click on it this
- 00:39:18should become an input field you should
- 00:39:20be able to change your name and then you
- 00:39:21should be able to click on a button here
- 00:39:23that lets you update your name that
- 00:39:24would involve some changes on the back
- 00:39:26end some changes on the front end but
- 00:39:27fairly simple you know full stack issue
- 00:39:29so I would assume um it's very clean and
- 00:39:32uh well structured as to what needs to
- 00:39:34be done so I think Devon should be able
- 00:39:36to solve it let's go back to the landing
- 00:39:40page wherever you
- 00:39:42reach oh boy over here and let's ask it
- 00:39:47to please
- 00:39:49fix this specific issue and let's see
- 00:39:52how it
- 00:39:55does again take a long time so I'll wait
- 00:39:58for it to finish and then we'll talk
- 00:39:59back it's 4:32 right now so let's see
- 00:40:01how long it takes should I compete with
- 00:40:03it it's 4:32 that means I'm a little
- 00:40:05sleepy um let me see how hard would it
- 00:40:09have been for me to figure it out U
- 00:40:11while it's coding it myself itself I'll
- 00:40:13try to go it as well U maybe directly
- 00:40:15over here I don't want to clone it
- 00:40:16locally U app I would go to profile and
- 00:40:20page. TSX
- 00:40:24um user detail form that's over here
- 00:40:27here so components SL user detail
- 00:40:39form and over here I see the
- 00:40:44user name um so here I would uh this
- 00:40:49will be like yeah this will be hard not
- 00:40:50hard but like figing out a lot of
- 00:40:52updating the input component where
- 00:40:54wherever that is coming from it's
- 00:40:55probably a Shaden component um to
- 00:40:57modifying it uh to add like you know an
- 00:41:00End Icon which can
- 00:41:02be a pencil in one case and you know a
- 00:41:05submit or a cross icon in the other case
- 00:41:07U so yeah that wouldn't be it was just
- 00:41:10involve making this input component more
- 00:41:13generic which right now probably just
- 00:41:14directly comes from shatan um so yeah it
- 00:41:17would take me effort for sure uh I know
- 00:41:19what to do but it'll just take a lot of
- 00:41:20human effort to do it um that's one and
- 00:41:23then on the back end I would assume it
- 00:41:25should be straightforward if if I go to
- 00:41:28DB / Prisma
- 00:41:32um and look at the schema the user table
- 00:41:36or the account table one of these
- 00:41:38probably has the Nam the user
- 00:41:40table yeah there's a name field over
- 00:41:42here so I just need to expose U an
- 00:41:45endpoint or a server action whatever we
- 00:41:47use over here
- 00:41:49um there is app SL API so I would just
- 00:41:54you know not too much here though re
- 00:41:57validate and add tracks um which makes
- 00:42:00sense like we don't let the user do lot
- 00:42:02of things but let's see if we have some
- 00:42:04server actions
- 00:42:06somewhere um or admin marketing assets
- 00:42:09yeah I don't see anything profile page.
- 00:42:12TSX yeah I'd probably add an API
- 00:42:15endpoint somewhere here u h that says U
- 00:42:19profile name get server session and then
- 00:42:22for example atrax probably gets the
- 00:42:24server session as well so get the server
- 00:42:26session update the username um
- 00:42:27straightforward uh back end should take
- 00:42:29I don't know 5 minutes to do for a human
- 00:42:32and front end might take me I don't know
- 00:42:3420 to 30 minutes honestly U because it's
- 00:42:37making sure we create it in a very
- 00:42:39generic fashion U specifically this
- 00:42:41input component that right now um is
- 00:42:45coming directly from Shad CNS my guess
- 00:42:48um Shad /ui
- 00:42:52input where are you there you go this is
- 00:42:56autogenerated from Shad C and I need to
- 00:42:58update it to accept more props U
- 00:43:01specifically like an End Icon um or end
- 00:43:03icons which could be more than one um so
- 00:43:05that will be the difficult bit here or
- 00:43:06you know the most time consuming this
- 00:43:07should probably I mean if you have to
- 00:43:08put it in human hours actually might
- 00:43:10take me around an hour to do um back in
- 00:43:12probably 10 minutes this guy though
- 00:43:15let's see where it's at um I forgot I
- 00:43:17think we started at 4:32 or 422 it's
- 00:43:19been 3 5 minutes 4:32 so it's only been
- 00:43:22like 4 minutes right now
- 00:43:25um let it run and I'll join you back
- 00:43:28once it
- 00:43:30finishes all right folks that's not too
- 00:43:33bad actually took it 10 minutes um in 10
- 00:43:35minutes it has written this let me show
- 00:43:37you guys the back end code looks exactly
- 00:43:41how I would write it um gets the session
- 00:43:44checks if the user session exists gets
- 00:43:46the name from the request uh and then
- 00:43:50updates it in the database so nothing
- 00:43:51out of the
- 00:43:53ordinary here oh boy
- 00:43:58uh I worry this is going to return the
- 00:44:01user their password uh it'll be hash
- 00:44:05which is not a problem but I worry this
- 00:44:08is not good um we'll see though why is
- 00:44:12it rning the user no idea um and then
- 00:44:15here on the front end
- 00:44:17um it seems like it did what it's
- 00:44:19supposed to do handle submit which sends
- 00:44:21a fetch request to the back end set it
- 00:44:23is editing to false a state variable
- 00:44:25called is editing which toles U there's
- 00:44:29a div with an input and if is editing is
- 00:44:32not true then you edit you render the
- 00:44:34editing or you know the pencil icon um
- 00:44:37and when you click on it you set the is
- 00:44:39editing variable to True which would
- 00:44:40render these two buttons which look
- 00:44:42right there's a um button to submit and
- 00:44:46then there's a button to cancel U which
- 00:44:50sets is edting to false resets the name
- 00:44:52as well so looks decent there's nothing
- 00:44:53wrong here U the only problem of course
- 00:44:55is I need to test it locally now to to
- 00:44:57see how it looks and the other doubt
- 00:45:00that I had uh you know this may or may
- 00:45:02not be a
- 00:45:03vulnerability uh how do I pull
- 00:45:07there Branch locally is the next
- 00:45:09question so let me go to the repository
- 00:45:11locally first uh CD projects slash uh
- 00:45:16daily Dash
- 00:45:20code let me pull the latest it's been a
- 00:45:22while since I've pulled it
- 00:45:27um now I need to get their Branch
- 00:45:30locally so get remote add origin de
- 00:45:35would be interesting honestly um Devon
- 00:45:39AI integration is an
- 00:45:41app how do you clone their codebase
- 00:45:44locally or sorry their pull their Branch
- 00:45:46locally I
- 00:45:48have no idea honestly get fetch
- 00:45:52origin can you fetch it from origin does
- 00:45:54it push I don't think it has access to
- 00:45:57push to my repo directly in a branch but
- 00:45:59hey I could be wrong and I was wrong it
- 00:46:01does have it that's scary U it can push
- 00:46:05to a branch yeah Deon can push directly
- 00:46:07to my origin Branch that's not great
- 00:46:09sorry the original remote that's not
- 00:46:10great but let's go into the Devin branch
- 00:46:14that looks right let's do an npm install
- 00:46:16in case I'm missing some dependencies
- 00:46:18locally U followed by or whatever a Yan
- 00:46:21install would have been better honestly
- 00:46:22Yan install followed by Yan Dev
- 00:46:28and let's see how it looks functionally
- 00:46:31it should work it might look bad and the
- 00:46:33vulnerability that I'm worried about is
- 00:46:35the only thing um but as expected and
- 00:46:38issue that's very easy for a human to
- 00:46:39understand it's probably very easy for
- 00:46:40it to understand um but if I give it
- 00:46:43something along the lines of U I don't
- 00:46:45know change something in the source code
- 00:46:46of FFM Peg I would assume it would
- 00:46:48hallucinate really bad U but we'll see
- 00:46:51we'll see time wise much better than you
- 00:46:53know how long it would have taken me if
- 00:46:56I was doing it flat if I was doing it
- 00:46:57with an AI helper probably similar U but
- 00:47:01if I was not using super Maven or you
- 00:47:03know GitHub pilot it'll probably take me
- 00:47:04longer to do this and you know it's not
- 00:47:06too bad to just delegate it to uh this
- 00:47:09guy and you know I don't know do
- 00:47:10something else and then eventually make
- 00:47:11fixes on top in case that are needed um
- 00:47:15so let's see I'm excited finally the
- 00:47:17first issue which I feel it's close to
- 00:47:18solving or you know it has actually
- 00:47:20solved
- 00:47:30all right let's open it
- 00:47:37up let's
- 00:47:45login that doesn't look great uh login
- 00:47:49with Google isn't working
- 00:47:53locally all
- 00:47:55righty CL Cent ID is required I need to
- 00:47:57update the N which would mean I would
- 00:48:00need to move off the screen for a bit
- 00:48:03because myn has my broad database um
- 00:48:07don't ask
- 00:48:12why all right folks uh I was able to run
- 00:48:15it locally the first issue we see right
- 00:48:17here is the changes that Devon made um
- 00:48:22specifically let's look at the pr um
- 00:48:27one change it made was it added use
- 00:48:29State over here it did not exist before
- 00:48:32U and hence it should have made this a
- 00:48:34client component it did not do it so let
- 00:48:36me quickly do
- 00:48:44that API components
- 00:48:58I think it's just easier for a human to
- 00:49:00do this than you know reprompt let's see
- 00:49:03if that's the only thing it
- 00:49:20missed let me go to the profile section
- 00:49:23now I see my name I see a very nice
- 00:49:26pencil I can honestly uh click on the
- 00:49:27pencil icon dude it's pretty good um few
- 00:49:31things are good for example you know uh
- 00:49:33name did not update here but if I
- 00:49:36refresh wow did not update for some
- 00:49:38reason uh what did I miss edit the UI is
- 00:49:43pretty fancy basically as good as I
- 00:49:45would expect it to be um click on this
- 00:49:49update did get back
- 00:49:54uh what do you call it um a 200 and
- 00:49:58surprisingly did not send me back
- 00:50:02my
- 00:50:06password
- 00:50:09interesting why is that I'm sure I look
- 00:50:11into that later the first thing to see
- 00:50:13is why is this not actually updating my
- 00:50:16name um what did I send in the payload I
- 00:50:18sent the new
- 00:50:19name all right folks sorry the recording
- 00:50:21stopped um the feature is fine the
- 00:50:24problem is we store the name in the
- 00:50:26session
- 00:50:27so if I ever go here and update the name
- 00:50:29to something new um it won't immediately
- 00:50:32update everywhere uh the reason being we
- 00:50:35get this name over here from the session
- 00:50:36variable so I need to relog in basically
- 00:50:38if I log out and then I log in again
- 00:50:44um then I will see the change basically
- 00:50:47uh whatever change I made a while back
- 00:50:50that's 1 123123 because this name is not
- 00:50:51coming from the database it's coming
- 00:50:52from the session there is no easy way to
- 00:50:54fix that um you have to to update the
- 00:50:58session cookie which won't be very easy
- 00:51:01U or you can just you know get these
- 00:51:03details from the database which makes
- 00:51:05more sense honestly U name probably
- 00:51:07anyways shouldn't be something that
- 00:51:08should be stored in your uh session it
- 00:51:10should be U you know fetched from the
- 00:51:12database whenever needed um even the
- 00:51:14email I would say just the ID should be
- 00:51:16stored in the session um so that is why
- 00:51:19you would probably need a human here
- 00:51:20it'll be very hard for me to I can
- 00:51:23ask uh this guy to fix it but pretty
- 00:51:27sure uh it'll just be easier to write it
- 00:51:30myself or you know you need an architect
- 00:51:32here or whatever computer engineer
- 00:51:33computer person here to come in and
- 00:51:35figure out which of the two routes to
- 00:51:37take either refresh the session whenever
- 00:51:38the user changes their name um so that
- 00:51:41you know the change persists across a
- 00:51:43refresh right now it doesn't persist
- 00:51:44over here if I refresh um I don't see my
- 00:51:47updated name until I log out and relog
- 00:51:49in um and the other thing is just you
- 00:51:52know fetch these details from a database
- 00:51:53so long story short it did well um the U
- 00:51:56is clean which is I'm something I'm very
- 00:51:58surprised about because I don't know if
- 00:52:00AI can see very well right now but this
- 00:52:02guy you know this is what I would expect
- 00:52:04so I would use it for this for sure um
- 00:52:06the other thing was is the backend wable
- 00:52:07it isn't but that's probably because we
- 00:52:09don't have uh you know if you look at
- 00:52:11our Prisma schema um we don't
- 00:52:17have passwords at all uh we use
- 00:52:25um oh that is why the the user table is
- 00:52:29fairly
- 00:52:33small it has these things only which is
- 00:52:35what gets returned to the front end but
- 00:52:36if there was a password Here the
- 00:52:38password would get returned to the front
- 00:52:39end which isn't necessarily a problem
- 00:52:40it's your own password it's hashed also
- 00:52:43um but still something you shouldn't
- 00:52:45return to the front end doesn't matter
- 00:52:47either way because you don't have a
- 00:52:48password over here it's not a problem
- 00:52:50the long story short it did fairly well
- 00:52:53um is it mergeable no um I'm going to
- 00:52:57give it one loss shot and tell it what
- 00:52:59it needs to fix uh the name is stored in
- 00:53:04the
- 00:53:05session so we need to refresh the
- 00:53:08session after the name is changed or
- 00:53:12fetch the name from the DB right now
- 00:53:17it I only see the updated name after I
- 00:53:23refresh I had to write log out
- 00:53:27honestly that's probably on me um but
- 00:53:29we'll see how this performs and this
- 00:53:31will probably be the end of it
- 00:53:33U final thoughts
- 00:53:36uh it's decent I'd rather just use an
- 00:53:39llm locally
- 00:53:42um then you know use this like it's very
- 00:53:46fancy like having an AI engineer that's
- 00:53:48you know pretty expensive $500 a month
- 00:53:53um there's more Pizzaz than you know
- 00:53:56substance here is what I think I think
- 00:53:57it would have what what's a better
- 00:54:00product is you know something closer to
- 00:54:01an AGI or a better llm um than a lot of
- 00:54:05layers on top um even though this is
- 00:54:07great like it looks very fancy from the
- 00:54:08outside U unsure if it'll be super
- 00:54:11sticky uh which is not to say AI it
- 00:54:13won't come will probably come but it'll
- 00:54:15probably come you know directly to your
- 00:54:16own vs code rather than in a cloud
- 00:54:18environment
- 00:54:19um historically Cloud environments be it
- 00:54:23you know something like replate um have
- 00:54:27been tried to be used as development
- 00:54:28environments all your developer code is
- 00:54:30there GitHub code spaces but rarely does
- 00:54:32that actually work this is slightly
- 00:54:33better U it does have your code base on
- 00:54:35the cloud but it also comits to GitHub
- 00:54:37um but that said unsure if this is a you
- 00:54:40know 10x better outcome compared to a
- 00:54:42local llm that I would just you know tab
- 00:54:44away and review right there and just
- 00:54:46push and merge then hand over everything
- 00:54:49to an AI and look at a pull request uh
- 00:54:52those are the current thoughts U that
- 00:54:55said if it can be a little faster U
- 00:54:59and I wouldn't call it accurate I think
- 00:55:01it's fairly accurate it's probably as
- 00:55:03good as it can get considering it's
- 00:55:05handling everything auton autonomously U
- 00:55:08but if more work can be done on you know
- 00:55:10the llm layer rather than you know all
- 00:55:13the the polishes around uh that might
- 00:55:16make this super interesting also the
- 00:55:18pricing I think is a little too
- 00:55:20aggressive at the moment but I'm sure
- 00:55:21someone will you know create an open
- 00:55:22source version or you know there'll be
- 00:55:24price Wars and eventually it'll get
- 00:55:25priced I think it's like I'm sure the
- 00:55:27margins here are crazy at $500 U so I
- 00:55:30would assume this get squeezed down to
- 00:55:33$100 whenever or maybe even lesser U as
- 00:55:36the market sort of matures U but right
- 00:55:39now super expensive uh not super helpful
- 00:55:41$20 Super m is great U are my current
- 00:55:44thoughts but we'll see how it uh matures
- 00:55:47over time and you know other products
- 00:55:49come and and if they out compete them
- 00:55:52those are my current Thoughts with that
- 00:55:54I'll wait for this to finish I'll show
- 00:55:55you the final results and we'll call it
- 00:55:57for this one I'll see you guys in the
- 00:55:58next one bye-bye
- AI
- automation
- Devon
- coding
- engineer
- GitHub Copilot
- software
- technology
- DevOps
- programming