The Marketing Formula that Propelled Duolingo to 500M Users
Summary
TLDRThe episode of the Logan Bartlett Show features an extensive conversation with Luis von Ahn, the co-founder and CEO of the language learning platform Duolingo. Duolingo, valued at $9 billion, differentiates itself through a strong culture of product iteration and user engagement. The conversation delves into how this culture is built around continuous improvement, A/B testing, and creative risk-taking. Luis discusses Duolingo's early decision to focus solely on improving user retention rather than monetization, eventually leading to significant market growth, especially in the U.S., where 80% of users were not previously learning a language before using Duolingo. Luis also narrates how Duolingo's mascot, Duo, became a viral meme, contributing to the company’s marketing strategy. The company's revenue model, which relies on a freemium approach with ads and subscriptions, evolved from a firm belief in providing free education. Duolingo also avoids relying heavily on performance marketing, instead relying on organic and creative engagement strategies. Through these approaches, the company has been able to significantly grow its user base and improve educational outcomes, pushing boundaries without becoming overly reliant on traditional marketing avenues.
Takeaways
- 📈 Duolingo's success is built on a culture of continuous product iteration and user engagement.
- 🦉 The company's mascot, Duo, has become a viral meme, boosting their marketing efforts.
- 💡 A/B testing is crucial for refining product features and understanding user behavior.
- 📊 User feedback is important, but data-driven decisions are prioritized to enhance the learning experience.
- 📱 The platform capitalized on mobile growth, bringing language learning to new audiences.
- 💸 Duolingo utilizes a freemium model, offering a blend of free access and paid features to sustain revenue.
- 🚫 By avoiding heavy reliance on performance marketing, Duolingo focuses on creative and organic growth strategies.
- 🎯 The team maintains a focus on high user retention before monetization to grow the market.
- 👥 Luis von Ahn highlights the importance of hiring exceptional and kind talent for company culture and success.
- 🔄 Duolingo's expansion beyond languages, to areas like math and music, broadens its educational impact.
Timeline
- 00:00:00 - 00:05:00
The episode features a conversation with Luis von Ahn, co-founder and CEO of Duolingo, discussing the company's culture of product iteration, AB testing, and the viral success of their mascot, Duo. Luis also reflects on Duolingo's unexpected path to becoming a billion-dollar business in language learning.
- 00:05:00 - 00:10:00
Luis explains that the key to Duolingo's success was a focus on user retention for the first five years, even without monetization. The company prioritized making the product engaging to grow the language learning market, particularly in countries where such tools were scarce.
- 00:10:00 - 00:15:00
Duolingo started testing changes to their product only after realizing they couldn't objectively measure their impact otherwise. A set of tests revealed they weren't good at predicting success, which led them to implement a rule where every change had to be tested.
- 00:15:00 - 00:20:00
Luis recounts how initially they bet on which product change would work best but found that data often contradicted their instincts. This convinced them of the necessity of an AB testing infrastructure, which now runs over 500 tests per quarter, encouraging a culture where failing tests is acceptable.
- 00:20:00 - 00:25:00
The discussion highlights how effective AB testing at Duolingo involves putting a team on optimizing significant metrics, such as 'time spent learning.' If issues arise, they are addressed to avoid issues like those experienced by Groupon due to over-testing.
- 00:25:00 - 00:30:00
They discuss the balance between using data and listening to user feedback. Duolingo employs user research but relies heavily on AB testing to identify real user struggles, supported by a knowledgeable internal team that's adept at understanding what works.
- 00:30:00 - 00:35:00
Luis talks about marketing strategies, emphasizing the importance of understanding user needs rather than just feature suggestions. Duolingo engaged social media users organically by leaning into existing memes about their brand, using this to drive effective and cost-efficient marketing.
- 00:35:00 - 00:40:00
Initially having no marketing budget, Duolingo relied on PR and the quirky appeal of their mascot to gain traction. The first significant marketing changes came with the arrival of their CMO, who steered them towards leveraging viral social media content effectively.
- 00:40:00 - 00:45:00
Duolingo capitalized on memes portraying their owl mascot humorously, which went viral. They explored creative and engaging marketing angles instead of traditional methods, emphasizing humor in their campaigns to boost engagement without heavy reliance on ROI metrics from social content.
- 00:45:00 - 00:50:00
Luis discusses the challenges of setting metrics in a way that doesn’t exhaust resources or frustrate users, like what happened with Groupon. They run rigorous reviews to ensure their product changes lead to genuine improvements without negative consequences.
- 00:50:00 - 00:55:00
Luis emphasizes Duolingo’s reliance on iterative design and rapid user feedback rather than traditional market studies, which enabled them to scale effectively. The strategy involves building a culture where it’s okay to fail fast and learn from it.
- 00:55:00 - 01:00:00
Marketing at Duolingo evolved under the new CMO, shifting from PR-focused efforts to building a brand through viral content. Luis highlights how being data-driven and responsive to user-created content allowed them to optimize their strategies.
- 01:00:00 - 01:05:00
The integration of humor and timing in their marketing proved successful, particularly leaning into user-driven content about Duolingo's owl mascot. Their marketing’s evolution shows a shift from simply gaining attention to engaging deeply with potential users.
- 01:05:00 - 01:10:00
Luis illustrates Duolingo’s agility by discussing how they leverage temporary ad space and adaptable marketing strategies to remain cost-effective while maximizing reach. They use this flexibility to focus on organic growth and user content.
- 01:10:00 - 01:15:00
He talks about Duolingo's early attempts at innovating language learning without prior knowledge in the field and how initial misconceptions were overcome through user feedback and AB testing, refining their approach to teaching languages effectively.
- 01:15:00 - 01:20:00
Luis confirms the success of Duolingo largely thrived on understanding the market was different than initially anticipated. Their strategy included focusing heavily on the user experience and integrating learning improvements based on extensive tests.
- 01:20:00 - 01:25:00
The conversation turns to how Duolingo scaled its operations to become a public company, bringing in both commercial success and social impact. The approach includes balancing between free services and premium offerings to support the business model.
- 01:25:00 - 01:30:00
Discussing the company culture, Luis highlights the importance of maintaining high hiring standards and incentivizing employees. They use team reviews to hold the line on quality, ensuring the right people are in place to maintain company values.
- 01:30:00 - 01:35:00
The discussion includes insights into maintaining innovation while managing growth, emphasizing metrics without stifling creativity. Luis stresses that understanding deeper user issues comes from staying close to user needs and experiences.
- 01:35:00 - 01:40:30
Closing the episode, Luis reflects on personal work habits and learning approaches that have contributed to his success, along with insights on maintaining discipline and work-life balance. His focus remains on impactful, mission-driven work.
Mind Map
Video Q&A
Who is Luis von Ahn?
Luis von Ahn is the co-founder and CEO of Duolingo and previously developed reCAPTCHA, which he sold to Google.
What is Duolingo?
Duolingo is a language learning platform known for its engaging, iterative approaches to education and is publicly valued at $9 billion.
How does Duolingo maintain high user engagement?
Duolingo focuses on incremental product improvements, such as higher user retention through engaging features and A/B testing.
What is the significance of A/B testing at Duolingo?
A/B testing is crucial for Duolingo's product development, allowing them to determine the actual effectiveness of features in improving user engagement.
What kind of culture does Duolingo promote?
Duolingo promotes a culture of continuous improvement, creativity, and data-driven decision making, encouraging risk-taking and learning from unsuccessful A/B tests.
How did Duolingo's mascot become viral?
The mascot, Duo, became a viral meme through strategic marketing and user engagement, leaning into internet culture and humor.
What is a major revenue strategy for Duolingo?
Duolingo employs a freemium model with optional ads and subscriptions for enhanced features to generate revenue.
Why did Duolingo initially avoid performance marketing?
Duolingo avoided performance marketing initially due to its expense and the fact that the company wasn't generating revenue at the start.
What is the philosophy behind Duolingo's product iteration?
The philosophy is to improve the product continuously, making it sticky and engaging through user-focused experiments and data utilization.
How does Duolingo balance user feedback with data?
While user feedback is considered, Duolingo prioritizes data-driven decisions to determine what truly enhances user experience.
View more video summaries
Les origines d'Halloween (Histoire)
Nicolás Promanzio con Luis Petri (ministro de defensa de Argentina) | Letra Chica - 10/07
De Vendedor Ambulante A MILLONARIO
PARA SALVARTE—Confesión [Ep. 8 T1]
Simple Ways To Stop Procrastination, Laziness & Increase Motivation | Cal Newport
The best answers to interview questions QA engineer position
- 00:00:00welcome to Logan Bartlett show on this
- 00:00:01episode what you're going to hear is a
- 00:00:02conversation I have with Louise Von on
- 00:00:05now Louise is the co-founder and CEO of
- 00:00:07dualingo a $9 billion business in the
- 00:00:09public markets that builds an
- 00:00:11application for language learning L and
- 00:00:13I talked about a number of different
- 00:00:15things including how they built a
- 00:00:16culture of iterating on product and AB
- 00:00:19testing different features as well as
- 00:00:22how they got their mascot Duo to become
- 00:00:24a meme and go viral on different social
- 00:00:27channels really fun conversation with
- 00:00:29someone that is very very bright
- 00:00:31including having a PhD in computer
- 00:00:34science but applied that to building
- 00:00:36both duol lingo as well as the prior
- 00:00:38company that he sold to Google called
- 00:00:40recapture which you might be familiar
- 00:00:42with you'll hear that conversation with
- 00:00:44Louise here
- 00:00:53now I want to talk about like building a
- 00:00:55culture of iteration AB testing
- 00:00:57continuous Improvement all that stuff I
- 00:00:59kind of have a opinion that in some
- 00:01:01companies or some markets uh there's an
- 00:01:04opportunity that opens up and it's just
- 00:01:06a debate of which compan is going to run
- 00:01:08through uh and build the equity value
- 00:01:11that comes out of that I'm not sure if
- 00:01:13you agree with this but my sense is that
- 00:01:16you guys were in the right place right
- 00:01:17time with regard to mobile and you were
- 00:01:20right about your thesis around language
- 00:01:23but it was actually the culture of
- 00:01:25incremental Improvement and just
- 00:01:28continuously getting better with the
- 00:01:30product that has led to the distance how
- 00:01:34big of a business you've been able to
- 00:01:35build I don't know if you agree with
- 00:01:37that that thesis that it wasn't
- 00:01:38necessarily inevitable that there would
- 00:01:40be an8 billion public company built
- 00:01:43around language
- 00:01:45learning yeah I I don't think it's
- 00:01:48necessarily the case that a that an8
- 00:01:50billion doll company needed to exist in
- 00:01:53language learning um it's true that we
- 00:01:55were at the at the right time with
- 00:01:57mobile for sure and we capitalized for
- 00:01:59that uh capitalize on that but I really
- 00:02:02think we were able to create a product
- 00:02:05that
- 00:02:07um got people very engaged and and such
- 00:02:11a thing didn't exist for language
- 00:02:12learning and and I'll tell you the
- 00:02:14biggest stat that that supports that 80%
- 00:02:17of our users in the United States were
- 00:02:19not learning a language before dual
- 00:02:21lingual just we basically massively
- 00:02:24grown the market in certain countries
- 00:02:26not in every country but in certain
- 00:02:27countries we've massively grown the
- 00:02:29market and it's I think it's just
- 00:02:31because I I mean the main reason for it
- 00:02:33is because for the first five years of
- 00:02:36the
- 00:02:37company the only thing we did was work
- 00:02:40on making dualingo have higher user
- 00:02:43retention and that's not paid user
- 00:02:45retention that's just free we we were
- 00:02:47not making any money for the first five
- 00:02:48years of dualingo we make zero dollars
- 00:02:50like we were not making any money and
- 00:02:54because we were not making any money we
- 00:02:55also weren't spending any money on
- 00:02:56marketing so there was kind of nothing
- 00:02:59to do I mean we we weren't spending any
- 00:03:01money on marketing so there was not much
- 00:03:02to do there we weren't making any money
- 00:03:04so there was not much to do in terms of
- 00:03:06like you know working on the
- 00:03:07monetization of dueling or or anything
- 00:03:09like that the only things we could work
- 00:03:10on are teach better and keep people
- 00:03:14engaged and so that's all we did for
- 00:03:17five years we simply made the product
- 00:03:20stickier and stickier and it really made
- 00:03:23a big difference how do you build a
- 00:03:24culture of like AB testing and iteration
- 00:03:27and people willing to take chance is
- 00:03:30knowing that you might make mistakes and
- 00:03:33that you can undo them if so it's not
- 00:03:35like when we started we thought we're
- 00:03:38going to build a company around AB
- 00:03:39testing this is not you know the way it
- 00:03:41started is we started we started with
- 00:03:44the product and we we would make some
- 00:03:45changes to it and
- 00:03:49um we would make like whatever the
- 00:03:51changes were like okay we're going to
- 00:03:53add
- 00:03:53a screen at the end of a lesson that has
- 00:03:56fireworks or whatever random
- 00:03:58changes and two things were happening
- 00:04:01number
- 00:04:03one we would add the string and we
- 00:04:06couldn't tell if the thing was better or
- 00:04:07not because you know we would we would
- 00:04:10see that the metrics were like either
- 00:04:12kind of flat but if they were flat we
- 00:04:13always found an explanation like oh it's
- 00:04:15because it's because this week is
- 00:04:16Thanksgiving or whatever it is and
- 00:04:18that's why normally we would have gone
- 00:04:20down but because we uh you know added
- 00:04:24the screen we didn't go down or
- 00:04:25something like that like whatever
- 00:04:27couldn't tell if things were actually
- 00:04:28helping or not
- 00:04:31and in addition to that we we couldn't
- 00:04:35tell which idea was good and which
- 00:04:37wasn't because we're like well if we
- 00:04:38couldn't tell if that was a good thing
- 00:04:40or not we you know I don't know was that
- 00:04:42a good idea or not and so at some point
- 00:04:45um it occurred to us you know what we
- 00:04:46should actually run one ad test and
- 00:04:49figure out if if that thing was actually
- 00:04:52good um and we run I remember we ran a
- 00:04:55test I don't remember exactly what the
- 00:04:57feature was but I remember there were
- 00:04:58like four options
- 00:05:00and we each one of us bet on which the
- 00:05:02option was going to be best and we
- 00:05:04thought of ourselves as very good
- 00:05:05product people and it turned out none of
- 00:05:08us were right and then after that we
- 00:05:11thought oh crap I guess we really had no
- 00:05:13idea you know we ran the a test the
- 00:05:15thing that one was not the thing any of
- 00:05:16us thought was was going to win and then
- 00:05:19for the next few weeks we ran a number
- 00:05:21of other AB tests and we always would
- 00:05:25say try to predict what was going to win
- 00:05:28and we realized we actually quite crappy
- 00:05:30at predicting what was going to win and
- 00:05:32that's when we really convinced
- 00:05:34ourselves well look we don't really know
- 00:05:38what is better for a product I mean we
- 00:05:41have some idea but it's not you know
- 00:05:42it's better than 50% we know much better
- 00:05:45than 50% so you know at that point we
- 00:05:48kind of started this rule that every
- 00:05:50change up du a linga is a Navy test and
- 00:05:52so we built a pretty sophisticated
- 00:05:54infrastructure which was way ahead of
- 00:05:55its time by now I think you know this
- 00:05:58was more than 10 years ago was way ahead
- 00:05:59of its time in terms of how good it was
- 00:06:01also for the size of company maybe
- 00:06:03Google at the time has such a good
- 00:06:04infrastructure much probably better than
- 00:06:05ours but for a small company you know it
- 00:06:08was kind of unheard of to have this
- 00:06:10infrastructure but we built an
- 00:06:11infrastructure that really allowed us to
- 00:06:12make everything an test and I think that
- 00:06:16that really made a made a big difference
- 00:06:19the other thing that I think in terms of
- 00:06:20a culture you we make we make it clear
- 00:06:24that it's totally okay to run AB tests
- 00:06:26that don't succeed and you know we have
- 00:06:28the stats you know I don't know off the
- 00:06:30top of my head anymore but I mean we run
- 00:06:32something around 500 a test per quarter
- 00:06:36and look something like 60% of them
- 00:06:38succeed and the other 40% fail that
- 00:06:40means hundreds of ab tests that we run
- 00:06:43per quarter
- 00:06:44fail and it's okay everybody's
- 00:06:47everybody's done that and I also like to
- 00:06:49tell the story that some of the best
- 00:06:50features of dual
- 00:06:51indle uh have been things that I've been
- 00:06:54vly against at first most notably the
- 00:06:57the streak I think was one that you said
- 00:06:59you originally pushed back on is that
- 00:07:01right no the streak actually was not the
- 00:07:04streak I was you know you don't really
- 00:07:06remember anymore but I'm pretty sure
- 00:07:09came I'm pretty sure I was the one who
- 00:07:11brought it up but there are there's more
- 00:07:13than one person that will tell you that
- 00:07:15they're pretty sure they're the one who
- 00:07:16brought it up so success has many
- 00:07:18fathers right or mothers like I'm not
- 00:07:20the thing is I'm not even lying to
- 00:07:21myself like I actually have a memory
- 00:07:24remember that I remember that but there
- 00:07:26are other people who claim that too so
- 00:07:27and I believe them so we don't really
- 00:07:29know who actually so I was not against
- 00:07:31the streak but I was against our
- 00:07:32leaderboards we have this league system
- 00:07:34in our leaderboards I was against that
- 00:07:36because we have tried leaderboards many
- 00:07:38times of before and they had failed and
- 00:07:40yeah this was like the fourth attempt
- 00:07:42and somebody came and they said we're
- 00:07:43going to do leaderboards I'm like no
- 00:07:45don't do that you we've tried it we've
- 00:07:47tried it many times you're just not
- 00:07:49going to work but they were very um
- 00:07:52committed to it and turns out it one of
- 00:07:54the best things we ever done the the
- 00:07:55league system at Theo how do you think
- 00:07:58about like listening in to people's
- 00:08:00feedback I've heard you use the example
- 00:08:02before of like people thought it was too
- 00:08:04bright the green was too bright and they
- 00:08:06all complained about it and then you
- 00:08:08said you changed it and they said okay
- 00:08:10wait never mind it's not too bright even
- 00:08:11though nothing actually happened how do
- 00:08:13you think about like trusting the data
- 00:08:15versus listening to the users and the
- 00:08:17balance between that and product
- 00:08:20design a couple of things to say the the
- 00:08:23good news about dueling or what makes it
- 00:08:24a little easier about dueling was that
- 00:08:27we can be the users too there's some
- 00:08:29that you designed where you're not the
- 00:08:30user I mean for example for a while I
- 00:08:33was you know we we we have a dualingo
- 00:08:36ABC which is an app to teach you how to
- 00:08:37read I'm not the user I know how to read
- 00:08:40I just I'm just not the user I'm also
- 00:08:42not like four years old um but for
- 00:08:44dualingo I am actually the user and
- 00:08:47so for that we we do take advantage of
- 00:08:50that I mean everybody that is in the
- 00:08:52product in a dual lle we ask them and
- 00:08:55you kind of have to really be a big user
- 00:08:56of it and that makes a big difference so
- 00:08:59that's one thing you listen to yourself
- 00:09:02um when we talk to users we have to take
- 00:09:04everything they say with a grain of salt
- 00:09:07um we just we have found that doesn't
- 00:09:09mean don't listen to them but we have
- 00:09:10found definitely do not build features
- 00:09:13based on the things that you are ask I
- 00:09:16mean usually there's a deeper reason for
- 00:09:19why they're asking for a feature so what
- 00:09:21what we're interested in is not what's
- 00:09:23your idea for a feature we're not very
- 00:09:25interested in that we're interested in
- 00:09:26what you know where you're having where
- 00:09:29you're confused or where you know where
- 00:09:31you're having trouble or something like
- 00:09:33that and and if you have an idea we want
- 00:09:35you to tell us why you have it an idea
- 00:09:38what why is that you're suggesting that
- 00:09:39because that usually gets at what the
- 00:09:41actual user problem is um so you know we
- 00:09:45we do we do use of research we try to
- 00:09:47all that but ultimately I think what
- 00:09:50works best for us is we have a group of
- 00:09:52people at dualle which fortunately has
- 00:09:54grown over time but it's a group of
- 00:09:57people that started with one or two
- 00:09:58people who just because we ran so many
- 00:10:01AB tests we had a really good idea of
- 00:10:03what worked and what didn't and by now
- 00:10:06we have a group of I don't know how many
- 00:10:09call it 15 to 20 people in the company
- 00:10:12who are just
- 00:10:15excellently um ATT tuned to what works
- 00:10:18and what doesn't with an IB cast it just
- 00:10:21it's they're very good um and and I
- 00:10:23think that's what what's worked the most
- 00:10:24we just when you have under your belt
- 00:10:26when you have hundreds of reps where
- 00:10:28you're like I tried that
- 00:10:29and you know did it work no it didn't
- 00:10:31work I tried that and you know did it
- 00:10:32work didn't work and you do that
- 00:10:34hundreds of times you just get a really
- 00:10:35really good nose and I think that's the
- 00:10:38thing that has worked the most for us
- 00:10:39having having this group of people I've
- 00:10:41heard you say that when you find a
- 00:10:42metric that you want to optimize you'll
- 00:10:44actually form a team that focuses on the
- 00:10:47optimization how does that is that a
- 00:10:48cross functional team across product
- 00:10:50engineering marketing like what does a
- 00:10:52team look like maybe give an example of
- 00:10:54a metric that you you thought about like
- 00:10:56how it actually comes to be yeah I'll
- 00:10:58give some examples I mean one for
- 00:11:00example is
- 00:11:02um a metric that we call time spent
- 00:11:05learning
- 00:11:06TSL a while ago a few years ago we
- 00:11:09decided you know we should have people
- 00:11:11spend more time in the app because the
- 00:11:13more time they spend the more they learn
- 00:11:15it's very simple uh so let's form a team
- 00:11:19around it and what a team usually is is
- 00:11:21we have um usually two team leads
- 00:11:26sometimes three uh usually there's an
- 00:11:29engineering team lead and then there's a
- 00:11:31product manager team lead sometimes
- 00:11:34there's a design team lead and sometimes
- 00:11:36there's a um a learning scientist team
- 00:11:39lead um but usually it's two and then we
- 00:11:43put in that team we put designers we put
- 00:11:46Engineers we may put one or two more
- 00:11:48program product managers depending on
- 00:11:50how how big the team is um and and then
- 00:11:54they work and they can touch any place
- 00:11:56in the app after a few months
- 00:11:59they um they figure out what moves that
- 00:12:02metric in the case of time spent
- 00:12:04learning what moves the metric is the
- 00:12:06leaderboards and so the time spent
- 00:12:08learning team mainly touches the
- 00:12:10leaderboards that's what they
- 00:12:12do and as you think about the uh the
- 00:12:16unintended consequences of setting these
- 00:12:19metrics I think I've heard the example
- 00:12:21of the streak uh people were just
- 00:12:24signing into the app just to keep their
- 00:12:26streak going maybe speak a little bit
- 00:12:29about that or the Groupon example of
- 00:12:32when a channel gets too saturated and
- 00:12:36the balance of local optimization versus
- 00:12:39long-term value creation yeah this is
- 00:12:42the tyranny of metrics I mean that we
- 00:12:44have to watch out for that a lot our
- 00:12:47themes are so good at at optimizing
- 00:12:52metrics that sometimes they lose sight
- 00:12:56of what's actually important they're
- 00:12:57just it's just a machine to optimize a
- 00:12:59metric um and they're very good item by
- 00:13:02the way the people that we hire you know
- 00:13:04are usually these people that have
- 00:13:05optimized their entire life I mean they
- 00:13:07were probably the best student in their
- 00:13:10preschool I just kind of optimized their
- 00:13:13entire life so they're massively good at
- 00:13:14optimizing and they'll optimize the hell
- 00:13:16out of something and so we have to
- 00:13:17really watch out and you know the way we
- 00:13:19watch out for that is by we have this
- 00:13:21process for product review where you
- 00:13:23know every change of the app to the app
- 00:13:26gets reviewed by a a senior group of
- 00:13:28people who are kind of overseeing the
- 00:13:31whole thing so they have to keep in mind
- 00:13:34you know they're the ones who stop
- 00:13:35spamming us and stuff like that but you
- 00:13:38know when you just let teams just fully
- 00:13:40optimize the metric this lead to familyy
- 00:13:42behavior and you know I I this we have a
- 00:13:44a story that we use inside the company a
- 00:13:46lot um which is comes from group one
- 00:13:48from the founder of group one who told
- 00:13:50us this story um who said uh look
- 00:13:54originally with group one we have this
- 00:13:56rule that we could only send one email
- 00:13:58per day per user because who wants to
- 00:14:01receive more than one you know kind of
- 00:14:02marketing email per day from a
- 00:14:04company and uh at some point some
- 00:14:08product manager said well can we try
- 00:14:11sending two and you know the founder
- 00:14:13said no we only send one and the prodog
- 00:14:15manager reled this amazing argument
- 00:14:17which is are you against knowledge we
- 00:14:19just let's just let's just see let's
- 00:14:22just see can we and of course that's a
- 00:14:23very hard argument to argue against no
- 00:14:27of course I'm not against so it's fine
- 00:14:28you but there's no way in hell that
- 00:14:30we're going to launch to like we're just
- 00:14:32going to see what it does so they they
- 00:14:34tried it they saw what it does it turned
- 00:14:36out the matches were way too good not to
- 00:14:39launch it it would have been
- 00:14:40irresponsible not to launch it like my
- 00:14:42God this been preached everything so
- 00:14:44much and so you know of course they they
- 00:14:48they launched it um and uh then you know
- 00:14:52I was like well we tried two what about
- 00:14:55three they tried three free was better
- 00:14:57they launched it and and they had a
- 00:14:59tested their way to a larger number I
- 00:15:01don't remember the exact number but like
- 00:15:02seven emails per day and then
- 00:15:04immediately after that the whole Channel
- 00:15:07died because everybody started marking
- 00:15:10their emails as spam because it's too
- 00:15:12much and then they just a tested their
- 00:15:14weight to killing the channel and the
- 00:15:16reason for that is because they didn't
- 00:15:18have full information about what was
- 00:15:19going on all the metries they could see
- 00:15:21were good but they actually couldn't see
- 00:15:22how many people are Marching their
- 00:15:23emails a Spam and that's always true you
- 00:15:25never have full information about what's
- 00:15:27going on in the user you just have some
- 00:15:29metrics and from what you can see it's
- 00:15:32good but you're not seeing for example
- 00:15:34uh that maybe you're you're getting
- 00:15:36people to spend more time in the app but
- 00:15:38you're actually frustrating them you're
- 00:15:39not seeing their frustration you're just
- 00:15:41seeing the time and you're like oh okay
- 00:15:42well that's good and so you know we we
- 00:15:45combat that by basically you know it is
- 00:15:48okay at do there are so many test we
- 00:15:50just don't run even if when people say
- 00:15:53are you against knowledge I'm like yep I
- 00:15:54actually am against knowledge I don't
- 00:15:56want to know what happens when you send
- 00:15:5830 no ifications in one minute I don't
- 00:16:00want to know sorry and so because
- 00:16:03because in general we do things and
- 00:16:05that's why it's good that we're users of
- 00:16:06the app it's like would you want to
- 00:16:07receive seven emails per day uh that are
- 00:16:11all marketing emails would you want to
- 00:16:12do that no I wouldn't okay then don't do
- 00:16:14that and that I think that has really
- 00:16:16helped us um do the right thing here and
- 00:16:19not drive our product to the ground I
- 00:16:22think a lot of brothers particularly
- 00:16:23games Dy themselves to the ground when
- 00:16:27you know they're just like well we're
- 00:16:28just going to monetize more and monetize
- 00:16:30more and monetize more and it works for
- 00:16:31a while until at some point it's just
- 00:16:33like you know I by the way I wouldn't
- 00:16:35know how it is with Facebook anymore but
- 00:16:37at some point then Facebook their
- 00:16:38Facebook app kind of just had so many
- 00:16:41ads that I think they kind of ran it to
- 00:16:43the ground too
- 00:16:45um I haven't used it in a long time but
- 00:16:47this is you know if they started with
- 00:16:48like one ad per whatever 10 posts at
- 00:16:51some point it was I don't know how many
- 00:16:53ads you know do you set constraints on I
- 00:16:56mean push notifications and emails is a
- 00:16:58very obvious one but are there
- 00:17:00constraints about things that you won't
- 00:17:03change within the app and those are
- 00:17:05non-starters or is is push notifications
- 00:17:09really a uh a oneofone thing because it
- 00:17:12can actually be marked as spam no no
- 00:17:15it's not just um we have a lot of
- 00:17:18channels in the app that our product
- 00:17:21managers get very good at so for example
- 00:17:23when you finish a lesson on dualingo
- 00:17:25there is a sequence of screens that we
- 00:17:28call session end so there course should
- 00:17:31be one that says you finished and here
- 00:17:33are your points but we start adding
- 00:17:35other ones for example you increase your
- 00:17:37streak good you increase your streak
- 00:17:39that's another screen but we have other
- 00:17:40ones too that are just like hey uh by
- 00:17:43the way do you want to subscribe to uh
- 00:17:44super dualle or we add another one that
- 00:17:47oh by the way have you tried our podcast
- 00:17:49it's basically an ad we just have
- 00:17:52decided to start adding things in there
- 00:17:54and it it works every time you add
- 00:17:55another session on screen it does what
- 00:17:57we wanted h but you can't have 30
- 00:18:00session in screens after you finish one
- 00:18:02lesson right so this is just like at
- 00:18:04some point if you finish one lesson you
- 00:18:06have 30 screens that you have to click
- 00:18:07through the channel dies I mean it just
- 00:18:10it just makes the whole thing so for
- 00:18:11example we have limits you know we just
- 00:18:14cannot have more than a certain number
- 00:18:16of screens at the end of a lesson uh and
- 00:18:19and then there's a whole pretty
- 00:18:21sophisticated system that tries to it's
- 00:18:22it's kind of like an auction it's not
- 00:18:24exactly an auction but there's there's a
- 00:18:26pretty sophisticated system that tries
- 00:18:27to figure out okay can only show three
- 00:18:29which three should I show um that's the
- 00:18:32type of thing that we done and there's a
- 00:18:33lot of channels like that in the app you
- 00:18:35know I'm is red dots turns out if you
- 00:18:37put a red dot somewhere in the screen
- 00:18:39people are going to tap on it well after
- 00:18:41a while when we when that was discovered
- 00:18:43by a uh enterprising product managers
- 00:18:46the app had like 30 red dots all over
- 00:18:48the place and then the channel dies
- 00:18:50because if there are 30 red dots you
- 00:18:52don't click on any of them there's just
- 00:18:5330 red dots um so that that's the type
- 00:18:56of thing that that you know we use the
- 00:18:58group on story unlock for and then how
- 00:19:00do those things roll up to a decision
- 00:19:02like is it does It ultimately land on
- 00:19:04your desk or someone else's that like
- 00:19:06hey we might be uh Pennywise pound
- 00:19:09foolish on uh on these types of tests
- 00:19:12and and this is a little hacky or
- 00:19:14gimmicky to approach it this way every
- 00:19:17Tuesday and Thursday for two hours we do
- 00:19:20this thing called product review which
- 00:19:21is I am there there's usually three
- 00:19:24reviewers there's me there's somebody
- 00:19:26from product manager and somebody from
- 00:19:27the
- 00:19:29and every change to the app gets 15
- 00:19:32minutes locked and we decide whether
- 00:19:34it's okay or not and after product
- 00:19:36review uh we spend uh what we call
- 00:19:39debrief for that day and we just go
- 00:19:42through everything that we approv or
- 00:19:44didn't approve and we make decisions
- 00:19:45about like hey um this seems like a like
- 00:19:49a more General thing let's let's add a
- 00:19:51we We call we we internally have been
- 00:19:53calling them FAS let's had like don't do
- 00:19:56that anymore so that's the type of stuff
- 00:19:58there's a lot of feedback in in this
- 00:20:00where we we try to make ourselves a
- 00:20:02better product team that way so we spend
- 00:20:05yeah I mean every Tuesday and Thursday
- 00:20:08multiple hours kind of
- 00:20:10discussing what is okay and what is not
- 00:20:12okay in terms of um features throughout
- 00:20:15if you were uh an entrepreneur or uh an
- 00:20:17executive at a company and were trying
- 00:20:20to copy or mimic elements of just how
- 00:20:24iterative of a culture you all have
- 00:20:26built I sort of think like consumer apps
- 00:20:28maybe gaming is the furthest extreme of
- 00:20:31just what a knif edge it's on in which
- 00:20:34you need to iterate and do as much as
- 00:20:36you can to keep users and then
- 00:20:38enterprise software is probably on the
- 00:20:40the far other side once three years ago
- 00:20:44yeah yeah exactly exactly if you were
- 00:20:46trying to inject some more of this DNA
- 00:20:49uh into a company is there any tactics
- 00:20:53or anything that uh outside of what
- 00:20:55we've spoken about here or philosophies
- 00:20:57that you would as to a Founder that's
- 00:20:59thinking about
- 00:21:01this you know one thing you did say
- 00:21:04right is I don't believe that this
- 00:21:06iterative approach is what's best for
- 00:21:08every single
- 00:21:11product it happens that with
- 00:21:13Duolingo a lot of things came together
- 00:21:16that made it so that it works really
- 00:21:18well so for example we don't have to pay
- 00:21:21licensing fees for stuff we don't
- 00:21:23license music that doesn't allow us to
- 00:21:26play this song and that country or
- 00:21:28whatever we don't have to worry about
- 00:21:30that so we don't have to worry we don't
- 00:21:31have to worry about that type of stuff
- 00:21:34um we don't we don't have we also don't
- 00:21:37run something where it's like life or
- 00:21:40death where if something changes
- 00:21:41somebody's going to you know they're
- 00:21:43going to lose their job or whatever we
- 00:21:45can easily just change the app at any
- 00:21:46point in time um so there's just a lot
- 00:21:48of things that that came together so
- 00:21:51that it makes it so that we really can
- 00:21:54just continually improve the product and
- 00:21:57with the other thing is that we can
- 00:21:58deliver the product you know we can
- 00:22:00change the app often I mean we we change
- 00:22:02the app once a week There's a new
- 00:22:03version of the app once a week so we can
- 00:22:04change the app often there's a lot of
- 00:22:06this that that makes it so that we
- 00:22:08really can
- 00:22:09change uh very fast and and can iterate
- 00:22:12and because we can then we do it um and
- 00:22:16and if what's what if that's what you
- 00:22:18want to do um I would say the most
- 00:22:21important things are make it so
- 00:22:24that uh there's no shame in failing uh
- 00:22:27for the AB tests and that that's very
- 00:22:30important make it so that you
- 00:22:33can um iterate pretty quickly
- 00:22:37so I mean that's those to me are
- 00:22:41probably the biggest things now shifting
- 00:22:43gears a little bit to marketing I'm not
- 00:22:44sure I would have guessed uh that a
- 00:22:47professor at Carnegie melon starting a
- 00:22:50company would be the the one especially
- 00:22:54computer science background right it
- 00:22:55would be the one to build a world class
- 00:22:57marketing organization and brand and
- 00:23:00it's it was not intuitive uh to to me um
- 00:23:05can we talk a little bit about how you
- 00:23:06think about marketing and where that
- 00:23:08fits into do a lingo strategy including
- 00:23:13maybe brand and paid user acquisition
- 00:23:16influencer organic all that stuff yeah
- 00:23:19marketing is has had a long history of
- 00:23:21dueling I think
- 00:23:23um you know the first several years
- 00:23:29we had a mar marketing budget of zero so
- 00:23:33can't spend any money just
- 00:23:36can't uh the reason for that was because
- 00:23:38we were making no money and I thought it
- 00:23:39was silly to spend money acquiring uses
- 00:23:42and by the way you know when we were
- 00:23:43launching everybody was that's kind of
- 00:23:45when Performance Marketing was getting
- 00:23:47like you know really growing you're like
- 00:23:49oh my God you can just go to Google and
- 00:23:51pay them five bucks and they give you a
- 00:23:52user it's amazing just do that um I
- 00:23:55thought it was silly to do that you know
- 00:23:57pay Google five bucks or however many
- 00:23:59dollars for a user when we were going to
- 00:24:00make no money from a user so that just
- 00:24:02seemed like throwing money away so this
- 00:24:04is why we never got under the
- 00:24:05Performance Marketing boat uh we just
- 00:24:07didn't but what we did you know at first
- 00:24:10marketing you for us meant
- 00:24:13PR and it actually when you were small
- 00:24:17first when you were small and be 10
- 00:24:20years ago PR was actually quite a strong
- 00:24:23Channel I think it's a much less strong
- 00:24:25Channel now because you know it's kind
- 00:24:27of social I'm taking over a lot of
- 00:24:30generally news outlets don't have as
- 00:24:32much traffic as they used to but that
- 00:24:35was quite a big channel for us um and so
- 00:24:38we had a good story for that and that
- 00:24:40helped the other thing that helped us is
- 00:24:42that we had this Green Owl that was kind
- 00:24:44of weird um and I think people started
- 00:24:46taking to it and so you would see like
- 00:24:49posts on Reddit or posts that would go
- 00:24:51viral but it would be you know kind of
- 00:24:53once every few
- 00:24:54months post on Twitter or post on Reddit
- 00:24:57or one of these would go viral and it
- 00:24:59had to do something with owl or
- 00:25:00something like that
- 00:25:03um we became significantly more
- 00:25:07professional with marketing uh when we
- 00:25:10hired um our first CMO who was Cammy
- 00:25:15dway um Cammy had been the CMO for um
- 00:25:18you know very fancy company she was in
- 00:25:20the Heyday of Yahoo she was the CMO for
- 00:25:22Yahoo then she was the head of all
- 00:25:24marketing at Nintendo which is just
- 00:25:27crazy a crazy marketing budget and then
- 00:25:29she came to us and then you know I said
- 00:25:33well we have no marketing budget and
- 00:25:35then she said I'm see I can't operate
- 00:25:37that way you got to give me something
- 00:25:39and she convinced me and I don't know I
- 00:25:41don't remember anymore the first year
- 00:25:43she may have convinced me for something
- 00:25:44like $30 million of marketing which to
- 00:25:46me seemed like a my God what are you
- 00:25:49doing and for her it must have been the
- 00:25:50smallest marketing budget she ever had
- 00:25:52in her entire life um but she was really
- 00:25:55quite a trooper for it because she was
- 00:25:57like okay well I'll operate with this
- 00:25:59and she started trying a lot of stuff
- 00:26:02and she hired a really good team um and
- 00:26:05she started trying a lot of stuff the
- 00:26:06majority of it didn't work we tried
- 00:26:08stuff like oh you know what I'm going to
- 00:26:10try a radio ad for Latinos in the US who
- 00:26:13want to learn English stuff like
- 00:26:16that kind of a lot of it wasn't working
- 00:26:19but what we did start
- 00:26:21noticing was that more and more there
- 00:26:24were uh these memes kept popping up
- 00:26:28relating to our owl and we weren't doing
- 00:26:30much for that we were just you know we
- 00:26:33were trying our own marketing stuff
- 00:26:34whatever it is we were trying it's not
- 00:26:36that it was working zero but it was not
- 00:26:37home
- 00:26:38runs but more and more we started
- 00:26:40noticing that like oh my God there's
- 00:26:42another another mean of some person uh
- 00:26:47that decided to say that they're scared
- 00:26:49of the dueling gu because it's going to
- 00:26:51kidnap their
- 00:26:52family and that's when you know Cammy
- 00:26:55and a few people in the team and me and
- 00:26:57all of us decided you know what why
- 00:27:00don't we lean in on
- 00:27:01this that works like a lot of the stuff
- 00:27:05we working like we're doing yeah know
- 00:27:07it's it's half ass working it's not not
- 00:27:10working like that so we started leaning
- 00:27:13in on it and we had a group of people
- 00:27:15that she had hired that were just really
- 00:27:17creative and really had their finger on
- 00:27:18the pulse and then we started you know I
- 00:27:21remember we did one video which was an
- 00:27:23April Fool's campaign where we hired um
- 00:27:27the idea for theol campaign was that we
- 00:27:29were going to make a video of one of
- 00:27:31these really cheap looking lawyers
- 00:27:33lawyer videos where they're like you
- 00:27:35know I will get you millions of dollars
- 00:27:38uh and and we we s we thought okay let's
- 00:27:41make a video just like that like the
- 00:27:42crappy video saying uh for a lawyer that
- 00:27:46is looking for people who had been
- 00:27:47kidnapped by The dualingo
- 00:27:50Owl and you there was so many good
- 00:27:53things by the way a really interesting
- 00:27:54thing is we actually went to try to get
- 00:27:56one of the companies that make these
- 00:27:58crappy videos and we talk to them and
- 00:27:59we're like hey but we want it like like
- 00:28:01crappy like the videos you make and
- 00:28:02they're like what do you mean crappy
- 00:28:04like we make excellent work okay and
- 00:28:07they kind of didn't and in the end we
- 00:28:08didn't hire any of them we actually
- 00:28:09hired a very very fancy company and told
- 00:28:13them to make it crappy and they actually
- 00:28:14understood how to do that and it was
- 00:28:16very good as opposed to
- 00:28:17telling just do your normal thing just
- 00:28:20yeah make that that wasn't working so
- 00:28:22well we ended up hiring a very fancy
- 00:28:24company um to to to make a very crappy
- 00:28:27looking video what year is this that you
- 00:28:29guys are doing 2019 maybe that was 2019
- 00:28:32I think so there was some precedent like
- 00:28:34I sort of think of um Dollar Shave Club
- 00:28:38in their video back in whatever
- 00:28:4120134 as kind of the first satirical
- 00:28:44brandish video out there so you guys had
- 00:28:47that was amazing precedent they did a
- 00:28:49great job with it we we did but we
- 00:28:50weren't thinking about that we I do know
- 00:28:53about that video and everything we just
- 00:28:54weren we were just like look this is
- 00:28:55what this is what people are saying
- 00:28:57anyways so we put out that video and it
- 00:28:59went really quite viral and the video by
- 00:29:03the way to be clear I mean it's a very
- 00:29:06uh inside joke of okay so so the push
- 00:29:10notifications you have which I want to
- 00:29:12talk about in a second people uh made
- 00:29:15memes that they are aggressive uh in
- 00:29:18nature or passive aggressive in nature
- 00:29:21and so that the owl's going to come find
- 00:29:24you and then you made a derivative joke
- 00:29:27about that which is if you're a lawyer
- 00:29:30uh you're a lawyer looking for the
- 00:29:32people that the owl actually stole so
- 00:29:35it's like a very inside joke right
- 00:29:38multiple we made multiple derivative
- 00:29:40jokes for it I mean we made another one
- 00:29:41where we also um as a joke product we
- 00:29:45said you can you can actually pay us to
- 00:29:46send the owl to like stand in front of
- 00:29:49you until you do your lesson and stuff
- 00:29:51like that um so yeah we lean in on that
- 00:29:54and that started working really well and
- 00:29:57and and that's by the way working really
- 00:30:00well as a percentage of how did you even
- 00:30:01think about the you guys are so data
- 00:30:03driven how did you think about the ROI
- 00:30:06or the could you just see have much we
- 00:30:08didn't have much Roi at first all we
- 00:30:10could tell was that the videos or the
- 00:30:12things we were posting were being
- 00:30:13watched a lot that's the only thing we
- 00:30:16could tell eventually we ended up
- 00:30:18getting data because um one of the
- 00:30:20marketing people decided that they were
- 00:30:22going to um that they were going to do a
- 00:30:24a you know we added a screen at the
- 00:30:26beginning when sign up to do a lingo how
- 00:30:28did you hear about us and we did a lot
- 00:30:30of um number crunching there and we
- 00:30:33figured out there's really a very high
- 00:30:34correlation people actually answer
- 00:30:36pretty truthfully there it's not 100%
- 00:30:38but it's pretty truthfully and we're
- 00:30:39able we use that data and we're now able
- 00:30:41to really figure out where things come
- 00:30:42from and that really works um but at
- 00:30:45first all we knew is that a lot of
- 00:30:47people were watching our stuff um and we
- 00:30:50weren't paying for it which which was
- 00:30:52great I mean people were watching that
- 00:30:54stuff and we weren't paying for it how
- 00:30:55did you I mean this is something
- 00:30:57innately a Founder can do I guess but
- 00:31:00you're you're taking a chance by making
- 00:31:02your brand and the icon of the owl uh
- 00:31:08into a meme even more than just the
- 00:31:11internet's already doing right and so
- 00:31:14how did you think about like clearly you
- 00:31:16must have a good sense of humor to find
- 00:31:18this funny yourself but there's a risk
- 00:31:20associated with that right you're a
- 00:31:22learning app you don't want to be taken
- 00:31:24as a joke necessarily we discussed it
- 00:31:28there was a risk and we discussed it but
- 00:31:30in the end you know it there was a risk
- 00:31:33but it
- 00:31:34wasn't it didn't seem like a existential
- 00:31:37risk at first it's like what just
- 00:31:38publish that video what's the problem
- 00:31:41and you know we kept working and we kept
- 00:31:43doing it more and more I think that was
- 00:31:45we kind of boiled the Frog into it I
- 00:31:46mean yeah it just didn't seem like that
- 00:31:48big of a risk at first um and and I did
- 00:31:51learn something pretty big here um we
- 00:31:55hire um a a a person who uh was straight
- 00:32:01out of college and Cammy hired uh this
- 00:32:04person a woman named zarya who's amazing
- 00:32:06um who um you know I remember I remember
- 00:32:10her hiring meeting um you know Cammy was
- 00:32:12trying to explain that we should hire
- 00:32:13zarya and you know I saw her wrest me I
- 00:32:15the whole thing and I'm
- 00:32:17like why are we hiring this person I
- 00:32:19don't understand Cam's like trust me
- 00:32:21look you have to hire some young people
- 00:32:23because they understand young people you
- 00:32:25don't anymore Lise and um just trust me
- 00:32:29on this one I'm like okay C me and zarya
- 00:32:33came on and pretty quickly she said you
- 00:32:35know what you know where young people
- 00:32:37are on Tik Tok I'm like my God Tik Tok
- 00:32:39[ __ ] people dancing I don't
- 00:32:42understand
- 00:32:43it and uh it's like look let let me make
- 00:32:46this video they made a video they showed
- 00:32:48it to me I'm like I I don't even
- 00:32:50understand what this
- 00:32:51is I have no idea what this is in fact
- 00:32:54this not going to work because nobody's
- 00:32:56going to understand this this what I
- 00:32:58told but they said well will you let us
- 00:33:01and sure carry on I don't think it's
- 00:33:04going to work anyways they posted a
- 00:33:07couple of videos and they went crazy
- 00:33:09viral like I don't know how many any you
- 00:33:12lost track 20 million views some crazy
- 00:33:14viral
- 00:33:15video and I still didn't understand why
- 00:33:18why it was even funny I just did not get
- 00:33:20it and one of the things that I
- 00:33:21understood which I a major shift you
- 00:33:23know most of my time at dualingo and
- 00:33:26even before dualingo SP designing
- 00:33:27products when you design a product it is
- 00:33:30important that every single one of your
- 00:33:31users understand what you're doing it
- 00:33:34needs to be understandable and I was
- 00:33:35applying that lens to our marketing I
- 00:33:37thought well everything we put out has
- 00:33:39to be at least understandable by the
- 00:33:42world this thing you put out makes no
- 00:33:44sense to me how in fact most people
- 00:33:46won't understand it and what I learned
- 00:33:49that with marketing that's okay
- 00:33:50especially with things like social media
- 00:33:53that Target it to the people you know
- 00:33:55somehow Tik Tok targeted it to the
- 00:33:56people that understood it and that was
- 00:33:59fine and so you know that that worked
- 00:34:02out and we started really leaning in on
- 00:34:05Tik Tok quite a bit and then we started
- 00:34:07getting worried that we were too reliant
- 00:34:09on Tik Tok at least for our marketing um
- 00:34:13uh so we started uh figuring out how to
- 00:34:16how to get better at that um we started
- 00:34:19this we tried the same thing on YouTube
- 00:34:20shorts and it works on YouTube as well
- 00:34:23so by now our Tik Tok and our YouTube
- 00:34:26provide about the same number of of new
- 00:34:28users to doal lingo which is awesome the
- 00:34:30other thing is we started replicating
- 00:34:32the same formula in multiple countries
- 00:34:34so now we have accounts Tik Tok accounts
- 00:34:36and and YouTube accounts on you know in
- 00:34:39Germany in Japan in uh Brazil in Spanish
- 00:34:44speaking world and in in a bunch of
- 00:34:47different countries in Vietnam and
- 00:34:48they've all succeeded uh and they all
- 00:34:51use the same
- 00:34:52formula and and so we're you know it's
- 00:34:56it's worked out really well but but the
- 00:34:57the formula you
- 00:34:59know there's a number of things but I
- 00:35:01think the biggest thing about the
- 00:35:02formula is if you look at our Tik Tok
- 00:35:05stuff and our our marketing stuff it
- 00:35:07does it's not selling to you it's just
- 00:35:10funny crap and like there's no button
- 00:35:12that says download now or buy now or
- 00:35:15anything like that it's just we now have
- 00:35:17built the personality for the owl and um
- 00:35:21people love it and I think that that
- 00:35:23really gets a lot of people you know
- 00:35:25it's basically free marketing we don't
- 00:35:27we don't pay for that it's not true that
- 00:35:28we don't pay for any marketing but that
- 00:35:31that we don't pay for was there a
- 00:35:32session in which you needed to come up
- 00:35:34with what the personality of of the owl
- 00:35:38was it's just it's iterative it's it's
- 00:35:41evolved and um it really didn't mean I
- 00:35:44would say the first glimmer of a
- 00:35:46personality came this was many years ago
- 00:35:49from the passive aggressive notification
- 00:35:51and this one was one I added and it was
- 00:35:53it was as follows and but I didn't think
- 00:35:55it was going to be this this big right I
- 00:35:56think it was um it was because you know
- 00:36:00we've
- 00:36:01always stopped sending notifications we
- 00:36:04stopped using dual lingo so we you know
- 00:36:05if you don't use dualingo we stop after
- 00:36:07like it used to be about five days we're
- 00:36:09like okay after five days you know
- 00:36:11whatever you've given up we're not going
- 00:36:12to spam you forever and it occurred to
- 00:36:14me you know what if we're going to stop
- 00:36:16we may as well tell people that we're
- 00:36:18stopping why not so the last
- 00:36:20notification on the fifth day that we
- 00:36:21that we started sending was uh these
- 00:36:24reminders don't seem to be working we're
- 00:36:26going to stop sending them for now or
- 00:36:27something to that effect and it turned
- 00:36:29out this we didn't expect it turned out
- 00:36:31that we really got people to come back
- 00:36:33because they you know it's this passive
- 00:36:35aggressive thing that they you know they
- 00:36:37felt bad that Dingo was giving up on
- 00:36:39them and that is that is the first
- 00:36:41glimmer of a personality that was just
- 00:36:43like oh this owl really wants me to be
- 00:36:45there and he's GNA be passive aggressive
- 00:36:47and that's I think that's kind of what
- 00:36:48gave rise to a lot of the memes but I
- 00:36:50mean you know we didn't and and so the
- 00:36:53personality of the owl has really
- 00:36:55evolved
- 00:36:57over time and it's been kind of
- 00:37:00co-created by a lot of different
- 00:37:02dualingo employees but also by the
- 00:37:04community because they've made a lot of
- 00:37:06this stuff and
- 00:37:07so there like there hasn't been now what
- 00:37:10we have had meetings about especially
- 00:37:12for some of the videos you know with a
- 00:37:13lot of the videos we you know we are
- 00:37:16learning brand after all and so we don't
- 00:37:17want to do things that are lewd but with
- 00:37:19a lot of the videos we are um we're
- 00:37:23unhinged and so um we do have uh a
- 00:37:28system in place for
- 00:37:30not posting things that we're going to
- 00:37:32regret later the system has failed a few
- 00:37:35times um but we do have a whole approval
- 00:37:38mechanism uh for not posting really
- 00:37:40crazy crap what what do you do when you
- 00:37:45uh when it feels it's crossed the line
- 00:37:48and uh and something just offended
- 00:37:51people in a way that you didn't expect
- 00:37:53you just take it down and and move on we
- 00:37:56usually take I mean we haven't that
- 00:37:57hasn't happened too many times um we do
- 00:38:00take it down by the way I don't know
- 00:38:02good or bad usually the most offended
- 00:38:04are the buing with
- 00:38:06employees um so we the times we've taken
- 00:38:09stuff down is more often than not is
- 00:38:10because somebody in the company or a
- 00:38:13group of people in the company are like
- 00:38:14how could you do that um and so that's
- 00:38:17that's what has happened but you know
- 00:38:20but there was only one time that where
- 00:38:23there was a major major backlash kind of
- 00:38:25publicly which was when we
- 00:38:28commented on um
- 00:38:32the you know I couldn't have told you
- 00:38:35this was going to be that bad there was
- 00:38:37a our account our Tik Tok account left a
- 00:38:39comment on a video about the Johnny Depp
- 00:38:45Amber her thing and the comment said
- 00:38:48something to the effect of like you guys
- 00:38:50think Amber Heard is on Tik Tok or
- 00:38:53something like I think that was the
- 00:38:54comment and there was a huge huge
- 00:38:56backlash because somebody posted saying
- 00:38:59that dualingo does not care about
- 00:39:03um uh like spousal violence or something
- 00:39:08like that and there was a huge backlash
- 00:39:10to the point where our um social media
- 00:39:12team had to um make their their personal
- 00:39:16accounts private because they were
- 00:39:17getting death threats wow um so that's
- 00:39:20when we learned to stay away from you
- 00:39:23know there's certain things we just stay
- 00:39:25away from even even if our comment even
- 00:39:26if our comment is you know I don't think
- 00:39:29that comment was
- 00:39:30particularly incendiary but you know
- 00:39:33there are some certain things we just
- 00:39:34stay away from we had one one time we
- 00:39:36have a a will joke on Tik Tok and
- 00:39:39Instagram reals and YouTube shorts and
- 00:39:42we had one one time that was um it was
- 00:39:45when the market had corrected and uh it
- 00:39:48was VC's six months ago versus VC's
- 00:39:52today and the joke was VC's six months
- 00:39:55ago were spend as much as you want like
- 00:40:00are you are you really burning as much
- 00:40:02cash as you can we can give you more
- 00:40:04money blah blah blah then VC's today
- 00:40:08were are you sure we need a marketing
- 00:40:11team is that something and the joke was
- 00:40:14that the VCS were flippant in their
- 00:40:16advice right uh people took it as we
- 00:40:20were making a joke about layoffs of a
- 00:40:22marketing department right and it was
- 00:40:26the joke was about us and VCS but I
- 00:40:30understood where they were coming from
- 00:40:32so we got a big that was a big outrage
- 00:40:34and you know uh it was something that we
- 00:40:37ended up I I think we apologized or
- 00:40:39something but you just never if you're
- 00:40:41going to take chances at these lines
- 00:40:42inevitably you're going to touch one and
- 00:40:44you're not sure I stand by that was just
- 00:40:46a good joke and we were making fun of
- 00:40:48EC's and not bounders it's impossible to
- 00:40:51tell it's impossible to tell what
- 00:40:53somebody's going to you know heard
- 00:40:56mentality takes it you know they kind of
- 00:40:58go against you um we haven't had a that
- 00:41:01was the last this was maybe like a year
- 00:41:03and a half ago or whatever we haven't
- 00:41:04had a big problem since then um and and
- 00:41:07I think it's important by the way if
- 00:41:08you're running a marketing team like
- 00:41:09ours which I think is is a major home
- 00:41:12run machine um to be okay with it so you
- 00:41:15know what I did when that happened and
- 00:41:17they they thought I was going to fire a
- 00:41:18few of them and I actually walked to the
- 00:41:19desk of in fact the person that posted
- 00:41:21and I said it's completely okay to stand
- 00:41:24out is the risk being polarizing in some
- 00:41:27way and so
- 00:41:29inevitably there's going to be some
- 00:41:31people that don't find it funny and
- 00:41:34inevitably there'll be some percentage
- 00:41:35of people that are offended and it's
- 00:41:38hard to say where it's going to come
- 00:41:39from so no that's that's great that you
- 00:41:41went over it and said that that's yeah
- 00:41:43the the account the checks and balances
- 00:41:45are important and trying to get enough
- 00:41:46diverse eyes on the things so that you
- 00:41:49could understand where the totality of
- 00:41:51your audience maybe is going to react to
- 00:41:53it but it it's the cost of doing
- 00:41:55business in some ways that inevitably
- 00:41:57you're going to get hit with some of
- 00:41:58these things and yeah we're going to
- 00:41:59continue I mean and and by the way we've
- 00:42:01had a number of things that we've had to
- 00:42:03pull I mean the most recent one we had a
- 00:42:05hilarious video that the marketing team
- 00:42:06just made it was
- 00:42:08hilarious um that we were going to post
- 00:42:12close to Halloween we had gotten we had
- 00:42:14gotten a famous person like in there it
- 00:42:18it was it was
- 00:42:19great but it had
- 00:42:22kidnappings and uh the Israel Palestine
- 00:42:26conflict erupted right around that time
- 00:42:28and so we didn't post it we were so sad
- 00:42:31that we couldn't I mean obviously there
- 00:42:32was no way we should have posted that
- 00:42:34but um in other times this would have
- 00:42:36been an amazing video what about being
- 00:42:38Scrappy on the brand marketing side I'd
- 00:42:40heard a story about like you guys uh
- 00:42:43just hanging around waiting for
- 00:42:45billboard inventory to pop up and and
- 00:42:48being able to take advantage of it can
- 00:42:50you can you tell that story as well as
- 00:42:51like other examples of just sort of
- 00:42:53being uh Scrappy about opportunities
- 00:42:56that present themselves you know because
- 00:42:59historically our marketing budget has
- 00:43:00been pretty small I mean it's still
- 00:43:01pretty small our marketing budget now is
- 00:43:03about 50 million a year which for a
- 00:43:06company with a revenu this it's about
- 00:43:09this about a fifth of what it should
- 00:43:11be um you know we
- 00:43:13don't we are very Scrappy I mean there's
- 00:43:16this story with you know with the
- 00:43:18Billboards we and the billboard was
- 00:43:20going to be a funny billboard it was um
- 00:43:21this was a billboard not necessarily so
- 00:43:23much for marketing dual lingo but it was
- 00:43:25for getting employees um so we just put
- 00:43:29a billboard in the 101 in San Francisco
- 00:43:32that said something or other like uh own
- 00:43:36a home work in Tech move to Pittsburgh
- 00:43:39and it just said dualingo or something
- 00:43:41like that um yeah I
- 00:43:44mean we only got that because we got a
- 00:43:47really good deal we just hung around for
- 00:43:49a long time until we got a good really
- 00:43:50good deal um and but we Ed that we use
- 00:43:53that same idea in a lot of ways by the
- 00:43:55way that is exact
- 00:43:56what we do with our Performance
- 00:43:58Marketing so we do Performance Marketing
- 00:43:59we don't do very much we
- 00:44:01do you know I don't know on the order of
- 00:44:0510 million bucks a year in Performance
- 00:44:06Marketing a little more than that but
- 00:44:07not much more than that and that's what
- 00:44:08we do um and the what we do with
- 00:44:13Performance Marketing is we we noticed
- 00:44:15we were in a really good position which
- 00:44:18is we don't particularly care which
- 00:44:20country are users are coming from uh we
- 00:44:23don't particularly care at what time we
- 00:44:25acquire them we don't we don't need to
- 00:44:26acquire them you know there are some
- 00:44:28some companies that need to acquire them
- 00:44:30you know for a specific date or for
- 00:44:31something we just don't care and so one
- 00:44:33of the things we do with our Performance
- 00:44:35Marketing is we just have a system that
- 00:44:36basically looks at the whole world and
- 00:44:39so if for example right now ads are too
- 00:44:42expensive in Italy we just don't do
- 00:44:44anything in Italy we move all our budget
- 00:44:46to France um or or or whatever if it's
- 00:44:49too expensive in France we move it to
- 00:44:51Vietnam or something so it just we're
- 00:44:53always shifting budget a lot between
- 00:44:55countries because we are we're in this
- 00:44:57great position where we you know operate
- 00:44:59in every single single country in the
- 00:45:01world uh or have users in every single
- 00:45:03country in the world and we are agnostic
- 00:45:04as to where they come from um and so
- 00:45:09that that's the type of stuff we do
- 00:45:10we're we're we're very um very Scrappy
- 00:45:12with that budget I heard when you were
- 00:45:13starting to a lingo uh you had no
- 00:45:16experience in teaching languages and so
- 00:45:20you read a bunch of books and they all
- 00:45:22contradicted each other or they they had
- 00:45:24mixed messages on how to learn how to
- 00:45:26teach I I don't know if that's true but
- 00:45:29if so like how did you go about figuring
- 00:45:31out what the core nucleus is of teaching
- 00:45:35language and how to start we didn't I
- 00:45:38didn't know how to teach languages I
- 00:45:39mean I knew I knew Spanish because
- 00:45:41that's my native language but I had
- 00:45:43never taught Spanish um s my co-founder
- 00:45:46knew German because that was his native
- 00:45:48language and he never taught German and
- 00:45:50so yeah we we read a bunch of books we
- 00:45:53you know there were some commonality
- 00:45:56about what all the books said how to
- 00:45:57best teach a language and we kind of
- 00:45:58took those unfortunately the books
- 00:46:00didn't agree with each other they all
- 00:46:01had their own method and they're all you
- 00:46:03know um so we took the commonalities and
- 00:46:06then we did what we thought made a lot
- 00:46:08of sense by the way we made a bunch of
- 00:46:09dumb mistakes that only computer
- 00:46:10scientist would make so for example this
- 00:46:13was a this was done um we thought hey we
- 00:46:17this this this kind of approach to
- 00:46:19deconstruct everything language is just
- 00:46:22words so all we need to do is teach you
- 00:46:24words and how to use them and if you
- 00:46:26know all the words and how to use them
- 00:46:27you would learn the language done and so
- 00:46:30we're luy though so that's what we're
- 00:46:32going to do and so the the first we one
- 00:46:34of the first versions of dingo had just
- 00:46:37we just were teaching you a list of
- 00:46:38words ordered by frequency so you know
- 00:46:41what the first word the most frequent
- 00:46:42word is the teach you the first then the
- 00:46:46second most common word whatever the
- 00:46:47hell that is teach you that word Etc
- 00:46:50that's a very bad way of teaching um and
- 00:46:54uh you know it's also not true that
- 00:46:57because you've learned enough words to
- 00:46:59cover the first like 30 words or
- 00:47:01whatever cover something like 20% of the
- 00:47:02language if not true you know 20% of The
- 00:47:05Language by knowing the first 30 words
- 00:47:06that is just absolutely not true um but
- 00:47:09that's kind of how we thought about it
- 00:47:10so this is the type of stuff that we
- 00:47:11were doing um over time we learned that
- 00:47:16we could run AB tests to figure out how
- 00:47:18to teach better and so that we started
- 00:47:20getting better at that so we started
- 00:47:21kind of figuring some stuff out
- 00:47:23ourselves um and then but then we got
- 00:47:26significantly more Pro um when we hired
- 00:47:30our first person with a PhD in second
- 00:47:32language acquisition um that was a we
- 00:47:36should have done that earlier um yeah we
- 00:47:39got a lot more Pro and and that that has
- 00:47:42really helped and now what's interesting
- 00:47:44is when we got these people with with ph
- 00:47:46from second language AIS it's not like
- 00:47:48they know how to teach with an app they
- 00:47:49didn't because they know how to teach in
- 00:47:51a classroom setting um but we knew how
- 00:47:54to teach with an nap now we had our own
- 00:47:56wonky ways of teaching with an app and
- 00:47:58they had their own very professional
- 00:48:01ways of teaching through classroom
- 00:48:02settings um what has been really amazing
- 00:48:05is kind of you know they've
- 00:48:08learned how to take everything they know
- 00:48:10and some of it applies to to an app some
- 00:48:13of it doesn't they learn how to make
- 00:48:14that kind of closer to the app and we
- 00:48:17with the app and with AB testing and
- 00:48:18everything have learned how to go more
- 00:48:20closer to them and so at this point you
- 00:48:23know I can I can say that compared to
- 00:48:25years ago Dule teaches I know way better
- 00:48:29um but we've understood that actually
- 00:48:31there's just all these skills that you
- 00:48:33need to learn and that they don't carry
- 00:48:35to each other like for example you can
- 00:48:36be amazing at reading a language and
- 00:48:37have no idea how to speak
- 00:48:39it and and we just didn't understand
- 00:48:42that at first um so now you know we try
- 00:48:44to teach you all the different skills
- 00:48:45that that are required to to to learn a
- 00:48:47language and you know we we understand
- 00:48:49that the way to teach better is at first
- 00:48:51you really need to teach things about
- 00:48:53how to you know uh how to talk about
- 00:48:55concretely about things and greetings
- 00:48:57and stuff like that these are not the
- 00:48:58most popular word or the most kind of
- 00:49:00highfrequency words at all uh but they
- 00:49:02are very useful to get started and so
- 00:49:05that's um you we've learned that in the
- 00:49:09in the early days when it was just you
- 00:49:11and San uh you I I think we we talked
- 00:49:14about this a little bit before we
- 00:49:15started recording but neither of you
- 00:49:17actually uh retained as users of of the
- 00:49:21app necessarily and trying to learn new
- 00:49:24languages was there was there a single
- 00:49:27product unlock or an aha that you
- 00:49:30realized about how to retain users or
- 00:49:34teach people through it or was it just a
- 00:49:35continuous process of these little tests
- 00:49:37to get Inc incrementally better every
- 00:49:40day so what happened with this is before
- 00:49:42we launched
- 00:49:44Duolingo um you know I made the first
- 00:49:46version of the Spanish course and S made
- 00:49:49the first version of the German course
- 00:49:50and we were going to learn each other's
- 00:49:52language I didn't know any German and he
- 00:49:54didn't know any Spanish and we were
- 00:49:55going to learn each other's language and
- 00:49:56we ran into this problem that neither of
- 00:49:59us wanted to do it I mean we would come
- 00:50:00into the office and we would say hey did
- 00:50:02you do your Spanish lesson you like no
- 00:50:04man so so boring and I I thought the
- 00:50:08same thing it was so boring to learn um
- 00:50:11German um and that's when that's when we
- 00:50:14realized that the hardest thing about
- 00:50:15learning a language by yourself is stay
- 00:50:17motivated and so that's when we started
- 00:50:19like adding um you know adding all kinds
- 00:50:23of game mechanics like progress bar at
- 00:50:27the top at the end of a lesson we give
- 00:50:29you points the lessons should be short
- 00:50:31there all this stuff and this is before
- 00:50:33we launched so we weren't a testing
- 00:50:35anything but we added a bunch of stuff
- 00:50:37so that at least we could swallow taking
- 00:50:42a lesson I mean it's like okay well at
- 00:50:44least it doesn't feel like a complete
- 00:50:46chore feels yeah kind of fun and that's
- 00:50:49you know we launched with that um and so
- 00:50:52by the time we launched it was kind of
- 00:50:53fun it wasn't as fun as it is today but
- 00:50:55it was kind of of fun we had done no a
- 00:50:56testing it was just at least with us we
- 00:50:59had made it you know kind of engaging
- 00:51:02and but but the the learning remained
- 00:51:05which
- 00:51:06was just um make it us engaging as
- 00:51:11possible and so that that continued what
- 00:51:14did you think this could be when you
- 00:51:17were starting the company you had been
- 00:51:19independently successful selling
- 00:51:21recapture to Google and so you were able
- 00:51:24to be a little bit more Mission driven
- 00:51:26in your Pursuits uh I think which I my
- 00:51:30understanding was that was an important
- 00:51:31consideration in the early days did you
- 00:51:32think that this would be a commercial
- 00:51:35potential public company or was it hey
- 00:51:37this is just a good service and maybe
- 00:51:39it's a good business along the way I
- 00:51:41never thought this would be a public
- 00:51:42company um I thought you know I thought
- 00:51:46we could do something that would help
- 00:51:47teach people mainly English um I thought
- 00:51:51that that that was going to be there I I
- 00:51:53didn't know how big it was going to get
- 00:51:56um when we were launching um Rosetta
- 00:52:00Stone was like huge we thought at the
- 00:52:03time was
- 00:52:04huge um it's not even clear that we
- 00:52:07thought we were going to get bigger than
- 00:52:08them we just thought well we're free and
- 00:52:10we're at least going to let people you
- 00:52:13know learn that can't afford to pay and
- 00:52:16andone was super expensive was like 500
- 00:52:18bucks or something like okay well uh
- 00:52:21we're going to let people learn that and
- 00:52:24you know when they don't have to pay 500
- 00:52:28bucks before we launched I thought if we
- 00:52:30ever get to about 100,000 monthly active
- 00:52:33users um that'll be success that's what
- 00:52:37I thought and um yeah that's funny I
- 00:52:39mean we get we get way more than that
- 00:52:41new users per day now there there's a
- 00:52:44great YouTube video I think it's around
- 00:52:46the launch of uh of dualingo whatever uh
- 00:52:5014 years ago or 12 years ago or whatever
- 00:52:53it is but I believe it's on that video
- 00:52:54you make make reference to this this
- 00:52:57concept of the cap on human coordination
- 00:53:01uh that historically had been around
- 00:53:03100,000 people I think that was true of
- 00:53:05the pyramids Panama Canal getting a man
- 00:53:07on the moon that like that's been the
- 00:53:10the when when when physical objects were
- 00:53:13involved that was the most we could
- 00:53:14scale to as as Humanity u c can you talk
- 00:53:18a little bit about that and how you how
- 00:53:20you thought that as an influence of
- 00:53:22starting Duolingo so we always wanted
- 00:53:25dealing with be free and it was one
- 00:53:27potential way to monetize it we ended up
- 00:53:30not monetizing this way but there was
- 00:53:32one potential way of monetizing which
- 00:53:33which was um look maybe we can get a lot
- 00:53:37of people learning a language on
- 00:53:38dualingo and at the same time while
- 00:53:40they're learning they helping us
- 00:53:41translate stuff because at the time
- 00:53:43computers were just not as good today
- 00:53:45they're essentially perfect but at the
- 00:53:46time they weren't as good as they are
- 00:53:48today translating stuff so we thought we
- 00:53:50could rather than charging people uh to
- 00:53:53learn a language or using ads we thought
- 00:53:56we could use their work um to help like
- 00:53:59translate stuff on the web and then we
- 00:54:01could make money from that so for
- 00:54:02example the idea was going to be
- 00:54:03something like well if you write a news
- 00:54:06piece in English like if you're CNN and
- 00:54:08you write a news piece in English you
- 00:54:09can send it to us and our users while
- 00:54:12they're learning English they could help
- 00:54:13translate it into Spanish because
- 00:54:15they're native Spanish speakers and
- 00:54:16that's part of their learning and then
- 00:54:18once it gets translated by a users we
- 00:54:19send it back to CNN and then we charge
- 00:54:21them for the translation or something
- 00:54:22like that we thought that was a
- 00:54:24potentially good business model because
- 00:54:26it you know no no money was involved uh
- 00:54:30at least for the users turns out this is
- 00:54:32a very bad business model so we ended up
- 00:54:34not doing that but that was that was one
- 00:54:36of the original ideas with dualing to
- 00:54:37try to make money while keeping it free
- 00:54:39it turns out a much better way to make
- 00:54:41money while keeping it free is to use a
- 00:54:43freemium model which is you can learn as
- 00:54:45much as you want but you may have to see
- 00:54:47some ads if you don't want to see ads
- 00:54:49you can pay us turns out that works
- 00:54:50really well that was not clear back then
- 00:54:53that this would that this would work so
- 00:54:54well yeah yeah so what year was it when
- 00:54:57uh so so Leela sturdy from capg invested
- 00:55:00and and told you hey you got to figure
- 00:55:02out how to make some money here at some
- 00:55:05point right what year was that she
- 00:55:08invested in 2015 I believe we at that
- 00:55:10point dualingo she invested a valuation
- 00:55:12of about half a billion dollars with
- 00:55:15with with roughly how many users zero
- 00:55:19Revenue plus or minus zero Revenue we
- 00:55:22must have had yeah I don't remember
- 00:55:25anymore call
- 00:55:27it I'm going to make up a number call it
- 00:55:30single digit May one two million daily
- 00:55:32active users one two million daily
- 00:55:34actives and and if you looked at monthly
- 00:55:35actives is
- 00:55:37probably call it uh I don't know six
- 00:55:41seven million monthly actives something
- 00:55:43like that so so a rich valuation uh for
- 00:55:47certainly for uh those times uh may
- 00:55:51maybe yeah we were the good news is we
- 00:55:52were growing a lot so we have just you
- 00:55:54know we the previous year we had like
- 00:55:5610x so it looks like we were drawing a
- 00:55:58lot which was good and but you know she
- 00:56:02said to me look um great product great
- 00:56:07team um you got to make money and uh
- 00:56:12yeah she she took me out to drinks after
- 00:56:14after my third drink she said look at
- 00:56:17you're not going to find uh she used
- 00:56:19exactly this words you're not going to
- 00:56:20find a bigger fool than Google like that
- 00:56:23this is it like you know we invested we
- 00:56:25like you this is great we believe in you
- 00:56:27but you're not going to find a you know
- 00:56:28you're not going to go get a valuation
- 00:56:30hip from anybody else you're not going
- 00:56:32to do that unless you make some money so
- 00:56:35get it going and then um I got start
- 00:56:38some yeah well and so now at this point
- 00:56:41you have a very Mission driven company
- 00:56:43right that are very uh aligned with your
- 00:56:46democratizing access to to learning
- 00:56:49languages and all that and so you come
- 00:56:51back to them and say hey you're a little
- 00:56:54hung over after a couple drinks with Lea
- 00:56:56you start noodling it and you say hey we
- 00:56:59we've got to figure this out I assume
- 00:57:02not all of the company was uh was
- 00:57:04excited to start showing ads or charging
- 00:57:07subscriptions is that a fair fair
- 00:57:08assumption yeah no they really were not
- 00:57:11they were in fact um entirely against it
- 00:57:14um most everybody that joined dualingo
- 00:57:17joined because of the mission to really
- 00:57:18deliver education to everyone um and so
- 00:57:23you know I said to everyone we got to we
- 00:57:24got to you know somehow pay for this
- 00:57:27whole operation and what everybody you
- 00:57:28know first question everybody said is
- 00:57:30why well got to pay for your salary and
- 00:57:34you've been paying our salary this whole
- 00:57:35time what's the what's the problem you
- 00:57:38know that's when I kind of told them
- 00:57:39kind of what Lea had said you you can't
- 00:57:41raise Venture Capital
- 00:57:43forever um and
- 00:57:47uh you know people finally were like
- 00:57:49okay okay fine I this took like months
- 00:57:52for people were like fine you got to
- 00:57:54make money how are we by the way when
- 00:57:56you have that I mean a lot of our
- 00:57:57companies have dealt with different
- 00:57:58versions of this in particular now
- 00:58:00trying to get people back into office or
- 00:58:03what whatever it is right different
- 00:58:04social issues or the mission driven
- 00:58:07nature of a of a company and trying to
- 00:58:09Corral it in some way shape or form I
- 00:58:12think as we're recording this uh there's
- 00:58:14this there's this large business called
- 00:58:16open AI That's maybe going through some
- 00:58:18some elements of of this like how did
- 00:58:20you actually go about building consensus
- 00:58:24around
- 00:58:25uh getting people on board with this it
- 00:58:28sounds like you didn't do it by edict
- 00:58:30and say Hey listen this is the way no no
- 00:58:32it had to be consensus duing was
- 00:58:34generally a consensus driven company it
- 00:58:36had to I mean the way that works we were
- 00:58:38I don't know how many employees we were
- 00:58:39at that time call it 150 maybe keep or
- 00:58:42take um you know there was clearly a
- 00:58:46spectrum of how much people cared and
- 00:58:48also spectrum of how influential they
- 00:58:51were and um you know I spent time with
- 00:58:54each with each individual who either
- 00:58:57cared a lot or was very influential like
- 00:58:59one-on-one and some of them were easy to
- 00:59:02convince they were a few stragglers who
- 00:59:03were just like really resisted the idea
- 00:59:07um but you know it took it took a period
- 00:59:10I don't know how long it was couple of
- 00:59:11months
- 00:59:13of you know people being you know
- 00:59:16uh upset at at this and then when they
- 00:59:20started seeing the mock of how they
- 00:59:22became even more upset when they started
- 00:59:23the first somebody saw a build of
- 00:59:26dualing with ads in it people were like
- 00:59:29what is this that's awful and it's
- 00:59:33because yeah it's true I mean the ads
- 00:59:34the truth is the Dual lingual product
- 00:59:35experience is quite polished and then
- 00:59:37you put an ad which is look like sorry
- 00:59:39but most of these ads that you play on
- 00:59:41apps look like crap yeah um so yeah it
- 00:59:46took it took quite a bit to to figure
- 00:59:49that out to to sorry for people to get
- 00:59:50used to it um but you know the
- 00:59:54explanation that really resonated most
- 00:59:58with everybody is if we make money we
- 01:00:01can use it to really speed up our own
- 01:00:04mission and it was 100% true as soon as
- 01:00:07we started making money we started
- 01:00:08growing the employees a lot instead of
- 01:00:10having like four separate product
- 01:00:12pipelines now we have I don't know how
- 01:00:14many product pipelines we have we have
- 01:00:15like 50 separate you know kind of lanes
- 01:00:19where people are are just improving the
- 01:00:21product um and that's all because we you
- 01:00:23know we make Revenue now how did you
- 01:00:26think about the line between
- 01:00:28subscription which my understanding is
- 01:00:30that drives is a very small percentage
- 01:00:32of the users but a high percentage of
- 01:00:34Revenue and then advertising which is
- 01:00:37the opposite when you were thinking
- 01:00:39about this philosophically how did you
- 01:00:41come up with where the lines of
- 01:00:42demarcation were we didn't have too much
- 01:00:44philosophical it was a lot AB testing
- 01:00:46the only thing that we decided to do is
- 01:00:49we're going to put an ad on the freeo
- 01:00:51that we're going to do at the end of our
- 01:00:53lesson we will put one
- 01:00:55and that you know after 5 seconds is
- 01:00:58skipable we're going to do that and we
- 01:01:01decided we were going to do and then we
- 01:01:03added an edict that said we're not going
- 01:01:05to degrade the fre years experience
- 01:01:06further than that that is it we're going
- 01:01:09to degrade it by putting one ad at the
- 01:01:11end of the lesson don't degrade it
- 01:01:13more um we didn't know whether ads or
- 01:01:17subscription was going to give us you
- 01:01:20know that much more money in retrospect
- 01:01:23I think I think subscript businesses are
- 01:01:25a lot more well understood now than they
- 01:01:27were at least digital subscriptions like
- 01:01:29that for apps than they were a while ago
- 01:01:31in retrospect this would have been
- 01:01:32obvious but at the time it was not clear
- 01:01:34to us that ads or subscriptions you know
- 01:01:37which one was going to be the killer one
- 01:01:39but after running it for about a you
- 01:01:41know a few months it became clear that
- 01:01:43subscription was so much of a Better
- 01:01:45Business um and so yes today eight give
- 01:01:49or take you know 8% of our um monthly
- 01:01:52active users are paid subscribers
- 01:01:55and 92% use it for free yet for a
- 01:01:58revenue give or take 80% comes from the
- 01:02:01subscription and you guys
- 01:02:02philosophically will not charge people
- 01:02:05for content right like that's uh you
- 01:02:09don't want to become Rosetta Stone and
- 01:02:10gatekeep content too much correct we
- 01:02:14don't charge people for Content the only
- 01:02:16things that are okay for us to charge
- 01:02:18for are things that we have to pay
- 01:02:20for so for example um we now have a here
- 01:02:24and dual lingo which is dualingo Max
- 01:02:26which has some features that have um
- 01:02:29large language model usage we have to
- 01:02:31pay for that because we don't have our
- 01:02:32own large language model so we have to
- 01:02:33pay for that and because we have a
- 01:02:35variable cost on it we do put it behind
- 01:02:38um you know you have to pay for it um
- 01:02:41because otherwise that would be a major
- 01:02:43money loser for us there's a rule I've
- 01:02:45heard you guys have that if uh if five
- 01:02:48people or less can build something as
- 01:02:50well you won't charge for it is that is
- 01:02:53that still true and how philosophically
- 01:02:55do you think about that this is you know
- 01:02:58it's one of those internally she called
- 01:03:00the five Germans and a
- 01:03:02dog five Germans and a dog can build it
- 01:03:06uh why are why why why can we Char
- 01:03:08because you know the reason we decided
- 01:03:10not to do is because that you know
- 01:03:11that's how that's how honestly that's
- 01:03:14just how we beat Rosetta Stone I mean we
- 01:03:16weren in German but we just made a thing
- 01:03:19that was free and it was not made by 300
- 01:03:22people it was made by very small number
- 01:03:25of people and very quickly we took them
- 01:03:27over in terms of usage not in terms of
- 01:03:30Revenue because we're making any Revenue
- 01:03:31but in terms of usage we took them over
- 01:03:33very
- 01:03:34quickly um and
- 01:03:37so yeah we're going to get disrupted if
- 01:03:39we build something and start charging
- 01:03:41for it that can be built you know so
- 01:03:43easily some company that has a small
- 01:03:46amount of funding I mean for you know
- 01:03:48five people you need five million bucks
- 01:03:51of funding or less than that they can
- 01:03:52build it and eat your business so that
- 01:03:55that's been the rule um we're a little
- 01:03:57less worried about that these days just
- 01:04:00you know that was more of a worry a few
- 01:04:01years ago when when our dominance was
- 01:04:04not as big I mean at this point it's
- 01:04:06just we have a lot of Moes and it's it's
- 01:04:08become pretty hard to compete with us so
- 01:04:10a little less worried about that but
- 01:04:11that was a worry that was a pretty major
- 01:04:12worry I'd say you know six years ago so
- 01:04:17now you're actually branching outside of
- 01:04:19languages uh how do you think about
- 01:04:21doing that and then what's the hardest
- 01:04:23part is it is it getting the brand is it
- 01:04:26the content is it rebuilding elements of
- 01:04:28the product what's the hardest part of
- 01:04:30doing that it's creating the whole
- 01:04:32learning experience um you know we're
- 01:04:35branching out to math and music um
- 01:04:37there's some differences in how you
- 01:04:39teach math and music to languages and so
- 01:04:41we're creating The Learning Experience
- 01:04:42that's to me that's the hardest part
- 01:04:44what's the most interesting difference
- 01:04:45of how someone would learn a language
- 01:04:47versus math or music well math needs
- 01:04:50more
- 01:04:51explanation than than languages
- 01:04:54um it's also the exercises and there's a
- 01:04:56lot more graphical elements to it it's
- 01:04:58just a lot of math I mean some of it is
- 01:05:00just numbers but a lot of it it just
- 01:05:02especially you're going to teach
- 01:05:03geometry and stuff like that it's it's a
- 01:05:05lot more graphical stuff and and and you
- 01:05:07have to start really you know you have
- 01:05:09there a lot of things you have to kind
- 01:05:11of figure out that we've now just
- 01:05:13figured out with languages I mean we
- 01:05:15figure out for example with languages
- 01:05:16I'll tell you it took us many years to
- 01:05:18figure this out and in retrospect it's
- 01:05:20obvious ready look if you're going to
- 01:05:21teach a language you you should align
- 01:05:23yourself with this thing called the
- 01:05:24common European framework of reference
- 01:05:25it's called the CFR just do that don't
- 01:05:28deviate um there's no such thing for
- 01:05:31math for math there's something like
- 01:05:32it's called like the common core but
- 01:05:34it's not clear to us that that the
- 01:05:36common core is as good as uh
- 01:05:39CFR and so that's the type of stuff that
- 01:05:42we just don't know and and also for math
- 01:05:45um we're not entirely sure who our
- 01:05:48audiences there's an audience we want
- 01:05:51but there's an audience that we may end
- 01:05:52up getting that is not what we want what
- 01:05:54we want is everyone we want to do for
- 01:05:57math what we did for languages which is
- 01:06:00you know 80% of our users were not
- 01:06:01learning a language before dual lingual
- 01:06:03we want it to be the case that anybody
- 01:06:05who's 25 years old or 30 years old or
- 01:06:08however or they're eight years old they
- 01:06:10just use dual lingle to learn math and
- 01:06:12they get better at we want that to be
- 01:06:14true it may end up being that the only
- 01:06:16people we get are the people who are
- 01:06:17forced to learn math because we just
- 01:06:19can't make math fun enough uh but what
- 01:06:22we would like is is to get to everyone
- 01:06:24and with meth this is a real open
- 01:06:26question can we get it so that your
- 01:06:28average 35y old decides to spend five to
- 01:06:3310 minutes a day getting better at math
- 01:06:35can we do that I don't know that is what
- 01:06:39the product team is T with and we have a
- 01:06:41really good product team but I don't
- 01:06:42know if they'll succeed because it's
- 01:06:44interesting and and you alluded to this
- 01:06:46of getting to spend five to 10 minutes a
- 01:06:48day but uh you don't view your
- 01:06:51competition as Rosetta Stone or a
- 01:06:54private language tutor or or whoever
- 01:06:57right how how do you think about your
- 01:06:59competition and and what you're going up
- 01:07:02against for that
- 01:07:03individual competition is time um and
- 01:07:07you know if you want to be more precise
- 01:07:08it's time on the phone so a real you
- 01:07:10know when we talk to our users and we
- 01:07:12ask them you know the ones who like sto
- 01:07:14using dualingo where you
- 01:07:17stop I don't know we've done I'm sure
- 01:07:19they exist but I I don't know if we've
- 01:07:21done any user research where they told
- 01:07:24us they've gone to a competitor like a
- 01:07:26like a language learning competitor
- 01:07:27that's not what people
- 01:07:28say the most common thing that people
- 01:07:30say well the first thing they says I
- 01:07:32haven't stopped which they're just lying
- 01:07:34about but once they once you convince
- 01:07:37them that yes you know that they have
- 01:07:38stopped they'll tell you what to be
- 01:07:40honest I'm just spending more time on
- 01:07:42Tik
- 01:07:42Tok or Instagram or whatever the hell it
- 01:07:45is so our real
- 01:07:47competition is is you know the apps
- 01:07:51games uh whatever it is the the things
- 01:07:53that they take up your
- 01:07:55time on your phone is there an
- 01:07:58optimistic point about humans and their
- 01:08:01desire to learn embedded in that that
- 01:08:03they're willing to waste time on Tik Tok
- 01:08:07or Instagram which presumably I think
- 01:08:09most of that is uh or spend it learning
- 01:08:13a language if the language experience is
- 01:08:16comparable or enjoyable or easy like how
- 01:08:19do you how do you think about the I mean
- 01:08:21I believe that obviously we haven't
- 01:08:23gotten there I mean obviously Tik Tok
- 01:08:24has more uses than dualingo but I do
- 01:08:27believe that
- 01:08:28um as long as you're close to as fun as
- 01:08:33tick Tok or as engaging or however you
- 01:08:34want to call it addictive engaging
- 01:08:36however you want to call it as long as
- 01:08:37you're close I think we have a good good
- 01:08:39shot because afterwards the good news
- 01:08:41about dueling was afterwards you don't
- 01:08:43feel like you wasted your time you at
- 01:08:45the very least got somewhat better
- 01:08:48Spanish um whereas if you spend 30
- 01:08:51minutes you know scrolling on
- 01:08:54Instagram you know you may learn a
- 01:08:57couple of things but really mostly time
- 01:09:00wastage um and so I we think that as
- 01:09:03long as we're close as long as we're
- 01:09:06close to how engaging that is we're in a
- 01:09:08good spot and and again I think we've
- 01:09:11gotten somewhat close but we're not
- 01:09:14there uh you know we have I don't know
- 01:09:17the latest numbers we released something
- 01:09:19like 24 million daily active users we
- 01:09:21don't have 500 million daily active
- 01:09:22users so we got to get there yep there's
- 01:09:25room to grow uh in terms of artificial
- 01:09:28intelligence you all have been fairly
- 01:09:30early in terms of using this can can you
- 01:09:33talk about your original desire to maybe
- 01:09:35build some of this stuff internally and
- 01:09:38then uh where you are in the evolution
- 01:09:41of of using tooling and what that's
- 01:09:44going to mean from a user experience
- 01:09:46standpoint yeah for artificial
- 01:09:48intelligence um you know we've always
- 01:09:51from the beginning for duelingo the idea
- 01:09:52was we're not going to use humans to
- 01:09:54teach
- 01:09:55you there's a reason for that uh because
- 01:09:59humans are expensive um you know if we
- 01:10:02want to teach
- 01:10:03literally whatever hundreds of millions
- 01:10:05of people you got to find a lot of
- 01:10:07teachers and they're expensive and you
- 01:10:09got to pay for them and we're just not
- 01:10:11going to use them basically we're going
- 01:10:13to teach you with with a
- 01:10:14computer um so that has always been the
- 01:10:17idea and if you're going to do it with a
- 01:10:18computer you know you got to make it so
- 01:10:20the computer can teach us well as a
- 01:10:21one-on-one human tutor and so we've been
- 01:10:23working really hard to use AI to teach
- 01:10:27as well as a one human Twitter um there
- 01:10:31was a major shift of course in the last
- 01:10:33year call it because of large language
- 01:10:36models I and I don't think almost
- 01:10:39anybody else could have told you five
- 01:10:40years ago that large language models
- 01:10:42were going to be this
- 01:10:43good um turns out they're extremely good
- 01:10:47so you know immediately when they came
- 01:10:49out um we started you know applying some
- 01:10:53of our AI efforts into that that's not
- 01:10:55the only place where we've used AI we
- 01:10:57use AI for a lot of different things um
- 01:11:00but at least for conversational practice
- 01:11:02this is the thing we were never able to
- 01:11:04do uh without large language models we
- 01:11:06just could not build a very good
- 01:11:08conversational partner but today uh we
- 01:11:12can build in a really good
- 01:11:13conversational partner with a large
- 01:11:14language model how has uh going public
- 01:11:18made you all operate differently is it
- 01:11:21been a good experience what would you
- 01:11:23recommend for for people maybe thinking
- 01:11:25about it for for us I think it's been an
- 01:11:28excellent experience um for me has been
- 01:11:30a really good experience I think there's
- 01:11:31a couple of things I think we you know I
- 01:11:33heard this from um Rich Barton uh who
- 01:11:37started Zillow and a number of other you
- 01:11:39know amazing companies he said look
- 01:11:41going public is like joining uh the
- 01:11:43major leagues like everything just
- 01:11:45operates at a higher level and I believe
- 01:11:47that I mean just everything started
- 01:11:48operating on a higher level you just are
- 01:11:50forced to do all these things that you
- 01:11:53didn't have have to do before so you
- 01:11:54don't do them it's like you know when
- 01:11:55you get no visitors in your house you're
- 01:11:57not really forced to clean but if you're
- 01:11:59going to have visitors every day you
- 01:12:00keep a pretty clean house um it's the
- 01:12:03same thing like everything just we
- 01:12:05operate a lot better I think for me in
- 01:12:08particular I've been very lucky that we
- 01:12:11have a really good CFO and finance team
- 01:12:14so I haven't had my job hasn't changed
- 01:12:17all that much because a lot of the
- 01:12:18public company stuff gets done by our
- 01:12:21finance team and I'm and other than just
- 01:12:24regular doing regular earnings calls um
- 01:12:27um it hasn't changed all that much for
- 01:12:29me in particular but I think I've had a
- 01:12:32fortunate experience for a number of
- 01:12:34reasons but I think the probably the
- 01:12:35biggest reasons reason why we've had a
- 01:12:37fortunate experience is that
- 01:12:40um
- 01:12:42we got very good at
- 01:12:45forecasting and we don't overpromise and
- 01:12:49I think we've historically haven't we we
- 01:12:52haven't just been one of these companies
- 01:12:53that completely overpromises and so I
- 01:12:55think that that has helped why do you
- 01:12:58think there there haven't been more
- 01:13:00successful consumer companies as we look
- 01:13:03today you all are at your all-time high
- 01:13:07despite having gone public you know
- 01:13:10everyone else that sort of has gone
- 01:13:11public in the 2021 vintage or whatever
- 01:13:14uh are definitely below their all-time
- 01:13:16high um I would say there there aren't
- 01:13:18that many businesses that have been
- 01:13:20created between five and
- 01:13:2320 billion do$ 25 billion do in equity
- 01:13:27value in within consumer do you think
- 01:13:30there's there's some innate cap that you
- 01:13:33either get super big like a Facebook or
- 01:13:35a a Google or a Snapchat or it's it
- 01:13:38feels like there's there's just not a
- 01:13:40lot that exists at that level versus
- 01:13:42enterprise software there's a ton do you
- 01:13:44have a perspective on that yeah I mean
- 01:13:47we asked ourselves this question um that
- 01:13:49there does seem to be something to once
- 01:13:52you get kind of big enough you just
- 01:13:54immediately turned into like a
- 01:13:56Facebook
- 01:13:58um there do seem to be a little bit of
- 01:14:00that uh I think another thing that has
- 01:14:03happened certainly for a lot of kind of
- 01:14:04the
- 01:14:05smaller consumer things I I think I
- 01:14:09think a lot of them were fooled by
- 01:14:10Performance
- 01:14:12Marketing I I think um we were very
- 01:14:16fortunate not because we were so smart
- 01:14:18it's not that we were so smart it's just
- 01:14:19we just because we were not monetized we
- 01:14:22didn't do performance Market
- 01:14:24but Performance Marketing is such a drug
- 01:14:26it's such an addiction and and once
- 01:14:28you're in it you can't stop it and the
- 01:14:30problem with it is that really cuts into
- 01:14:32your margins to the point where you just
- 01:14:35it's very hard to it's very hard to ever
- 01:14:36become you know very profitable when
- 01:14:38you're when you're relying on a lot of
- 01:14:40these consumer apps that are not the
- 01:14:44Instagrams of the world if you look at
- 01:14:46where their users are coming from it's
- 01:14:48overwhelmingly Performance Marketing
- 01:14:50when you look at like 80% of your users
- 01:14:52are acquired something like that you're
- 01:14:54going to have a shitty margin that
- 01:14:56that's that and so then then yeah
- 01:14:58investors are going to think you're a
- 01:14:59shitty business too because they have a
- 01:15:00shitty margin fortunately for us we
- 01:15:03never did rely on Performance Marketing
- 01:15:05and so our margins are really good
- 01:15:07because we don't do that um and I think
- 01:15:10that's it I think I think in retrospect
- 01:15:13I can understand that again it's not
- 01:15:14because we were so smart back then but
- 01:15:15in retrospect I can understand look the
- 01:15:17truth of the matter with Performance
- 01:15:18Marketing is that companies like
- 01:15:20Facebook like meta or or Google um are
- 01:15:24excellent at extracting all the value
- 01:15:26for that like you know if there is more
- 01:15:29value to extract if there's more margin
- 01:15:31to extract they'll take it uh because
- 01:15:34that's what they're good at and
- 01:15:37so I I think that's another big reason I
- 01:15:40think interesting so dualingo today has
- 01:15:43approximately how many people uh
- 01:15:45LinkedIn says 1700 does that sound about
- 01:15:48right no that's a lot of um um I don't
- 01:15:50know why that is there's some uh
- 01:15:52contractors is there and who knows who
- 01:15:54knows what else Randomness there is uh
- 01:15:57we have about 750 employees 750 okay uh
- 01:16:01I heard you or severon your co-founder
- 01:16:04have been or were in every hiring
- 01:16:07meeting is that still the case both of
- 01:16:10us both of you all 750 people you've met
- 01:16:15with well we've hired more than that
- 01:16:16because we've also lost some people um
- 01:16:18yeah I guess guess and by the way not
- 01:16:20only that we've also turned down a lot
- 01:16:22of people so yes we've been in hiring
- 01:16:24meetings of thousands of people how do
- 01:16:27how do you make that work TimeWise uh
- 01:16:29and how is that a principle every week
- 01:16:32we just do it every week There's a
- 01:16:33meeting every week that lasts about half
- 01:16:34an hour where they present us um you
- 01:16:37know the hiring manager comes and
- 01:16:38presents the case for the person that
- 01:16:39they're uh going to look at for some
- 01:16:41roles you know this go super fast
- 01:16:44particularly for more kind of Junior
- 01:16:47roles sometimes these meetings take a
- 01:16:50long time and and you know we may go
- 01:16:53through a lot of um back and forth but
- 01:16:56it you know it works it's not like I
- 01:16:58mean again it's not like the majority of
- 01:17:01the people that apply for dual lingal
- 01:17:02get rejected much earlier than that the
- 01:17:04only ones that get to us are the ones
- 01:17:06that where everything looks pretty good
- 01:17:08and so they get to us so it's not that
- 01:17:10many per week it's um got it so it's not
- 01:17:12that you don't meet every employee
- 01:17:15before they join but you sign off us not
- 01:17:17we used to meet every employee that was
- 01:17:19that that that was a while ago I mean
- 01:17:21that probably stopped five years ago um
- 01:17:25at this point it's it's really just the
- 01:17:27case gets made to us yeah by the way
- 01:17:29this is very important um and it's
- 01:17:31because of this everybody else but us
- 01:17:34and it's not because we're special
- 01:17:35people I'll just tell you what it is
- 01:17:37everybody else but us has a true
- 01:17:38incentive to fill the role the the
- 01:17:40person from the talent acquisition team
- 01:17:42just wants to fill the role because
- 01:17:43they're being paid to fill the role the
- 01:17:45hiring manager has a hole in their
- 01:17:46organization so they need to F the role
- 01:17:48so everybody just has an incentive so
- 01:17:50what happens is they're just like ah
- 01:17:52this this price is good enough and so
- 01:17:55they come in with a incentive to hire
- 01:17:57whereas we have an incentive to hold the
- 01:17:59line and by the way yeah again it's not
- 01:18:01because we're such great individuals
- 01:18:03with such great it's just a matter of
- 01:18:05incend us because we do the same thing
- 01:18:06because I've seen when s wants to hire
- 01:18:08somebody he also has this problem and
- 01:18:11when I want to hire somebody for my team
- 01:18:12I also lower the bar and so we try to
- 01:18:15you know it's it's simply it's not
- 01:18:18because we are such great judges of
- 01:18:21talent or whatever it's simply it's it's
- 01:18:23a matter of incentives and that's why we
- 01:18:25we we keep interesting so it's kind of
- 01:18:26human nature if it's yours then you see
- 01:18:29the pain of having to restart it and so
- 01:18:31that's and that's exactly right I mean
- 01:18:32when I'm hiring for my executive team
- 01:18:34I'm just like my god I've been looking
- 01:18:35for six months look this person is
- 01:18:38pretty good they're pretty good they're
- 01:18:39pretty good and then if I show it to
- 01:18:41somebody who's way more impartial
- 01:18:43they're like
- 01:18:45really come on and so that I think
- 01:18:48that's what it is it's it's simply that
- 01:18:50Beyond just being a good person what's a
- 01:18:52what's a non-negotiable for someone
- 01:18:55joining dualingo and maybe maybe to the
- 01:18:57good person example if you could also
- 01:18:59tell the the car ride uh of what what
- 01:19:02you all do if people hear this maybe
- 01:19:04they'll know now to be nice to we do a
- 01:19:06bunch of things and you know um we do
- 01:19:09that you know for us it's important that
- 01:19:12people are
- 01:19:13kind um
- 01:19:16and one of the places where that shows
- 01:19:18up a lot is how you treat people who are
- 01:19:21uh lower in the organizational structure
- 01:19:24than you are yeah everybody's pretty
- 01:19:27nice to me when they meet me uh and they
- 01:19:28want to work for dualingo but um you
- 01:19:31know we definitely see how people uh
- 01:19:35treat um some and and this is usually
- 01:19:39for for um more senior roles um where we
- 01:19:43look at you know how they treat much
- 01:19:45more Junior people and you know I mean
- 01:19:47this is one you know we I've told this
- 01:19:49story before where um you know we were
- 01:19:50looking for a CFO for a long time
- 01:19:53and
- 01:19:54um one person had everything looked good
- 01:19:59everything looked good but um they did
- 01:20:02not treat their driver well and the
- 01:20:04driver is a person that has worked with
- 01:20:05us you know not he not a dualing
- 01:20:07employee but it's the same you know we
- 01:20:09have a few drivers that we work with to
- 01:20:11send people to the airport you know to
- 01:20:13to pick them up and you know we asked
- 01:20:15them was this person good I like
- 01:20:17actually no they were pretty pretty
- 01:20:19nasty to me and we didn't hire them and
- 01:20:22that was it was painful for me not to
- 01:20:24hire them because I wanted to hire this
- 01:20:25person because I have been looking this
- 01:20:27is an example I had been looking for a
- 01:20:28long time at that time I proba had been
- 01:20:30looking for like nine months and this
- 01:20:33was a very smart person uh very
- 01:20:35competent but I guess they were not nice
- 01:20:37to our try and and my sense is if that
- 01:20:41happens you know they're probably not
- 01:20:43going to be very nice to a level one you
- 01:20:45know Finance associate or whatever
- 01:20:48beyond that Beyond being kind to people
- 01:20:50are there certain traits that you look
- 01:20:52for
- 01:20:53uh that are non-negotiables or is every
- 01:20:55role kind of specific beyond that it
- 01:20:57depends a lot on the role no it's not
- 01:20:59the role but more the function so you
- 01:21:02know in the case of
- 01:21:04um design you know we really look at
- 01:21:09people's portfolio and we really care
- 01:21:10about polish you know some design
- 01:21:12organizations are okay that some of the
- 01:21:15designers don't have as much polish as
- 01:21:16others for us every designer that we
- 01:21:18hire we want to have polish so we look
- 01:21:19at their portfolio a lot it just kind of
- 01:21:21depends on the function
- 01:21:23uh but I would say companywide you know
- 01:21:26we want people to be kind and obviously
- 01:21:28really good at their Craft um that
- 01:21:30that's just you know kind of obviously
- 01:21:33so there's one other thing that we do
- 01:21:34look for it's it's it's which which we
- 01:21:37try
- 01:21:38um we really want people who are
- 01:21:42um in one way or another exceptional and
- 01:21:46and we we look for that quite a bit and
- 01:21:48exceptional can mean different things I
- 01:21:49mean in some cases you know exceptional
- 01:21:51is the usual kind of tech company
- 01:21:52definition of exceptional which is like
- 01:21:54went to Stanford got up
- 01:21:564.0 um etc etc that's pretty exceptional
- 01:21:59honestly very few people were able to
- 01:22:01get a 4.0 from Stanford so that's pretty
- 01:22:03exceptional but we look for a broader
- 01:22:05definition of exceptional I mean first
- 01:22:08first person their family to go to
- 01:22:09college it's pretty
- 01:22:11exceptional um comes from a
- 01:22:14country from a very poor country
- 01:22:17actually grew up in a very poor country
- 01:22:19and somehow managed to you know go to
- 01:22:22the university
- 01:22:23France that's pretty
- 01:22:25exceptional uh so stuff like that um you
- 01:22:28know we we we we try to make it so that
- 01:22:30everybody we hire has something
- 01:22:32exceptional what have you learned about
- 01:22:34I guess both leading and managing over
- 01:22:38the course of running dual
- 01:22:40lingo managing definitely I've learned
- 01:22:42how to not become how to not be a
- 01:22:46micromanager I actually believe that if
- 01:22:48you're a small company you know under 20
- 01:22:51employees it really is actually a good
- 01:22:53thing for you to be a micro manager like
- 01:22:55why not um after a certain number you
- 01:22:59just can't beat and be and then after
- 01:23:01you start hiring very senior people you
- 01:23:03piss them off um so I've learned not to
- 01:23:05be a micromanager um you know there a
- 01:23:08number of other things I've learned that
- 01:23:10ultimately when there's a
- 01:23:12[ __ ] not to point fingers to anybody
- 01:23:15but
- 01:23:16yourself and we actually have made that
- 01:23:18pretty good part of the culture doing it
- 01:23:20especially in some functions where you
- 01:23:22know there's fuckups happening every
- 01:23:24day um and we we've gotten pretty good
- 01:23:27at you know it's just like okay well
- 01:23:30keep going and as a leader I think it's
- 01:23:33really important to be very
- 01:23:36very ready to blame yourself um I think
- 01:23:41a lot of leaders I've seen it with you
- 01:23:43know even with people who have in over
- 01:23:45the years report to me were they really
- 01:23:46trying not to get blamed for something
- 01:23:49like no no somebody else did it yeah
- 01:23:53I think people respect it a lot more
- 01:23:55when you say look
- 01:23:56ultimately I'm responsible for it and
- 01:24:00you know by that I'm responsible for all
- 01:24:02the dualingo fuckups and we've had
- 01:24:04hundreds maybe
- 01:24:07thousands it's probably safe to describe
- 01:24:09yourself as a uh accidental entrepreneur
- 01:24:13is that is that fair I did not grow up
- 01:24:15thinking I would be an entrepreneur and
- 01:24:18and and then you capture then then
- 01:24:20recapture then do lingo I guess um when
- 01:24:24recapture sold how many people did you
- 01:24:26have oh not a lot um a dozen got it what
- 01:24:31what what part of the entrepreneurial
- 01:24:33Journey uh do you innately enjoy the
- 01:24:38most I don't know if it's the
- 01:24:40entrepreneurial Journey that I enjoy
- 01:24:42it's more um I enjoy doing things that
- 01:24:45have
- 01:24:46impact that part I enjoy so I enjoy I'm
- 01:24:48enjoying a lot what I'm doing right now
- 01:24:49with do lingo but it's because I'm still
- 01:24:51allowed to touch the product
- 01:24:53um the day I'm no longer allowed to
- 01:24:54touch the product I my job satisfaction
- 01:24:57will goow down quite
- 01:24:59significantly um so I like that I mean I
- 01:25:01like tinkering I like being able to have
- 01:25:03a lot of fun say in
- 01:25:06what in in in what goes into the end
- 01:25:10product of a of a consumer product I've
- 01:25:12heard you say that working out has been
- 01:25:14a life-changing thing for you when did
- 01:25:17that start uh and and how how has it
- 01:25:21actually changed your
- 01:25:23life I mean it started out it started
- 01:25:26about right around when I started do a
- 01:25:28lingo um I just it was a streak I
- 01:25:31decided to have a workout streak and um
- 01:25:35I haven't I haven't lost it actually
- 01:25:36it's been you
- 01:25:37know however many years 13 give or tape
- 01:25:40years every every day or every every day
- 01:25:43every day every day I have not lost it I
- 01:25:45did cheat a couple of times as follows
- 01:25:47these trans these flights to to
- 01:25:50Japan there a problem with the date
- 01:25:53there Che by working out twice the day
- 01:25:56before and uh you know something like
- 01:25:59that but but yeah I haven't lost my
- 01:26:02streak of working out every single day
- 01:26:04um what's the most bare bone thing
- 01:26:06you've done to uh to make it work uh H
- 01:26:10yeah the the New York Times seven minute
- 01:26:12workout yeah got it um look because some
- 01:26:16because you can always say you can go
- 01:26:17running but sometimes it's like horrible
- 01:26:19weather you're in some random hotel that
- 01:26:22has no gym or whatever and all you got
- 01:26:25is you and yeah you do that 7-minute
- 01:26:29workout a few times and that's all you
- 01:26:30do how has it changed you is is it is it
- 01:26:34the energy from it is it the the the
- 01:26:37energy you know I used to get sick quite
- 01:26:38often which is funny which I now I am
- 01:26:40sick but I I get sick once every like
- 01:26:42two years um I used to get sick very
- 01:26:44often before that that's one thing I
- 01:26:46think the other thing is it really
- 01:26:48allows me to regulate my own emotions
- 01:26:52quite a bit like I think I you know I'm
- 01:26:55generally a pretty emotional guy but I
- 01:26:59think that um my wife says I'm I'm Latin
- 01:27:02American Latin American
- 01:27:05men but I think uh um you know it allows
- 01:27:09me to regulate my emotions I think
- 01:27:11someone that uh maybe struggles with
- 01:27:13that or or wants to get going is there
- 01:27:15is there any advice you would have to
- 01:27:17them on like how to just start making
- 01:27:19progress is it just get a streak going
- 01:27:22in a similar way to yeah yeah yeah yeah
- 01:27:24yeah and by the way what that's what
- 01:27:26matters the most just get a street going
- 01:27:28with anything I mean this is this by the
- 01:27:30way it's the same with dualingo look a
- 01:27:32lot of times people are like no no no no
- 01:27:34there's a there's a better method for
- 01:27:35learning a language and it's like look
- 01:27:37ultimately that's like saying the
- 01:27:39treadmill is better than the elliptical
- 01:27:40it probably is but you know what matters
- 01:27:4295% is that you do
- 01:27:44something that's you know did you do it
- 01:27:47with a treadmill with an elliptical or
- 01:27:49one of those weird climbing machines or
- 01:27:50whatever sure probably there there are
- 01:27:52some differences but between not having
- 01:27:54done anything and having done even the
- 01:27:56dumbest one of these it's better to do
- 01:27:58the dumbest one of these I mean and
- 01:27:59that's the thing with doing we're
- 01:28:00optimized for keeping you going is there
- 01:28:03is there another habit or anything
- 01:28:04you've picked up on maximizing
- 01:28:06productivity and balancing your your you
- 01:28:09know work life balance and all of that
- 01:28:11stuff I mean I'm pretty efficient at
- 01:28:14most everything I do and I think that's
- 01:28:15you know a lot of people comment on that
- 01:28:17I'm I do most everything very fast I eat
- 01:28:20very fast so I I don't know if it's a
- 01:28:22habit that I think for some reason I'm
- 01:28:24just very fast at most everything I do
- 01:28:26I'm the same way eating by the way I
- 01:28:28it's always been a problem people think
- 01:28:30that I've been to jail yeah people think
- 01:28:32that I've been to jail because I like
- 01:28:33eat so fast I eat so fast but the truth
- 01:28:37is you know I love food but I I don't
- 01:28:39savor it in my mouth for five hours yeah
- 01:28:42that's funny um one of the uh one of the
- 01:28:46things I'd be curious about is um
- 01:28:49recapture was sold in what was it
- 01:28:522008 nine 10 nine nine okay and at that
- 01:28:57point in time I I don't think it's ever
- 01:28:59been disclosed but you didn't raise any
- 01:29:01outside money and sold for I I've youve
- 01:29:04said tens of millions of dollars uh and
- 01:29:07so um and now dualingo is an enormous
- 01:29:10success I'm sure we can look up in the
- 01:29:11S1 how uh you know your ownership at
- 01:29:14time of IPO but for safe to say uh you
- 01:29:17you uh you've made more money than you
- 01:29:20can probably spend but between both of
- 01:29:23of these things unless you want to go
- 01:29:24buy a sports team or something uh but I
- 01:29:27assume I don't know if that's in the
- 01:29:28cards for you but what depending on the
- 01:29:30sports team depending on the sports team
- 01:29:32yeah well in that case then then there's
- 01:29:35a lot of stuff there's a lot more that
- 01:29:37you can keep going uh to BU the sports
- 01:29:39team what would be the sports team of
- 01:29:40choice if you
- 01:29:42could I mean I don't know I like
- 01:29:45soccer I'm not that crazy I mean I like
- 01:29:48soccer but I'm not it's not GNA be the
- 01:29:50Steelers or the yeah one of the
- 01:29:52Pittsburgh teams okay well the Pirates
- 01:29:55would be cheap but yeah that's right
- 01:29:57that's right how
- 01:29:59um how has that most changed your life
- 01:30:03or or how you view the the the world uh
- 01:30:07having that level of of success it seems
- 01:30:09like you you still live in Pittsburgh
- 01:30:11presumably a similarish life but is
- 01:30:13there any way that it's changed to the
- 01:30:15positive versus the negative on stuff oh
- 01:30:19yeah I mean you don't worry about some
- 01:30:20of the stuff that you used to I mean I
- 01:30:22remember when I was a graduate student
- 01:30:23you worry about like are you going to
- 01:30:25have enough money to pay the rent and
- 01:30:26stuff like that um you know those those
- 01:30:30worries entirely go away and that I
- 01:30:31think you know that's that's quite a
- 01:30:33nice thing I mean I just don't and and
- 01:30:35not only that the other thing is just
- 01:30:36not having to spend any mental energy on
- 01:30:39life you know should that does this
- 01:30:41toothpaste cost more less than that
- 01:30:43other toothpaste Etc just just I you
- 01:30:45know I basically learn just don't think
- 01:30:48about money unless again unless you're
- 01:30:49going to go doing some crazy thing like
- 01:30:51buy a sports team team kind of just
- 01:30:52doesn't matter uh and you know at least
- 01:30:56I don't spend the um mental energy on
- 01:30:58that now for me I'm you know it's not
- 01:31:00like I go around spending crazy amounts
- 01:31:03of money um so it hasn't changed that
- 01:31:07much how I live my life I mean I have I
- 01:31:10have like three pairs of jeans that's it
- 01:31:12it's kind of what I
- 01:31:15have um yeah hasn't changed that much I
- 01:31:18mean I did you know right now I'm in New
- 01:31:21York in New York and you know I did buy
- 01:31:23a pretty expensive house here but other
- 01:31:25than that I haven't done all that much I
- 01:31:27I've heard you say uh something about
- 01:31:29like to truly understand something is to
- 01:31:32be able to explain it simply or or some
- 01:31:35philosophy like that can can you exp
- 01:31:37spouse or expound on that a little bit
- 01:31:40uh for for why you I learned that's from
- 01:31:42my PhD adviser Manuel Blum I mean it is
- 01:31:45an amazing thing and it's hard to
- 01:31:48explain unless you've actually felt it
- 01:31:51um
- 01:31:52um once you actually understand
- 01:31:55something you can give an amazing clear
- 01:31:58simple explanation of it um and I used
- 01:32:00to think that I just didn't know that um
- 01:32:05and you know I learned that um from my
- 01:32:09advisor and it was as follows it was
- 01:32:12early on in my PhD you know I had to
- 01:32:14explain a scientific paper to him and he
- 01:32:17said go read this paper explain it to me
- 01:32:19because I haven't read it I would come
- 01:32:22you know I came in for a one hour
- 01:32:23meeting and I said like the first
- 01:32:26statement and he said I don't understand
- 01:32:29and I'm like how can you understand it's
- 01:32:30just a simple the simplest statement
- 01:32:32that I he's I I just don't get it why
- 01:32:34did you say that why did you start with
- 01:32:36that why I don't get it and I spent the
- 01:32:38whole hour and I couldn't even go after
- 01:32:40F past the first line of the
- 01:32:42paper um I came back the next week and
- 01:32:45it was the same thing it took me six
- 01:32:47months to finally get to a point where
- 01:32:49he said okay I understand meeting him
- 01:32:52weekly it turned out he was he was
- 01:32:54pretending the whole time actually that
- 01:32:57paper he was a real expert in um but it
- 01:33:01took me really six months and and is
- 01:33:03true I mean the explanation that I had
- 01:33:04at the end of the six months was just
- 01:33:06because I really understood it it was
- 01:33:08just crystal clear and I could explain
- 01:33:10something that at first I was planning
- 01:33:11on taking like 40 minutes to explain it
- 01:33:13to him I was able to explain the idea in
- 01:33:15like a minute like no this is the main
- 01:33:17idea of the paper blah and um yeah but
- 01:33:21that happens once you really understand
- 01:33:23something and that's I mean that's the
- 01:33:24genius of my phc adviser he's he he
- 01:33:27really understands things well I think
- 01:33:30today there's more people learning
- 01:33:31languages in the US through dualingo
- 01:33:34than the education system combined do I
- 01:33:37have that fact right correct there are
- 01:33:40more people learning languages on Dual
- 01:33:41lingual in the US than there are people
- 01:33:43learning languages across all us high
- 01:33:44schools combined but um I I heard you
- 01:33:48say recently that you're not anti the us
- 01:33:52education system uh and in fact may
- 01:33:56maybe Pro of it if Having learned so
- 01:33:59much about how people learn and built a
- 01:34:03business around it at least for uh
- 01:34:05languages and now increasingly music and
- 01:34:07math if you could wage wave a magic wand
- 01:34:10is there anything from a teaching
- 01:34:13standpoint or or how we go about the the
- 01:34:16educational system in the US that you
- 01:34:19would change if there was one thing that
- 01:34:22stuck out it's very different from what
- 01:34:25we do at dual lingo but I mean what
- 01:34:26matters the most in the educational
- 01:34:28system are the
- 01:34:29teachers um this is by far what matters
- 01:34:32the most and um there are some excellent
- 01:34:35teachers but there's also quite a high
- 01:34:37barans and you know what I would wave
- 01:34:40the most is or what I would wave want to
- 01:34:42is make sure that every teacher that is
- 01:34:46out there is
- 01:34:48amazing um that's pretty hard to do uh
- 01:34:51given a system that doesn't pay teachers
- 01:34:53very well given a system where um you
- 01:34:56know there's essentially equivalent of
- 01:34:57tenure in some of these places where you
- 01:34:59know you basically can work and not do
- 01:35:01all that much because of teacher unions
- 01:35:02so there's a lot of stuff that is
- 01:35:05working against that but the thing you
- 01:35:07know you want better education better
- 01:35:09teachers that's you know again very
- 01:35:11different from dualingo uh dualingo we
- 01:35:13don't use humans but the truth of the
- 01:35:15matter is that for the foreseeable
- 01:35:17future humans are going to be needed for
- 01:35:20you know the educational system can you
- 01:35:21tell the story of Bill Gates so Bill
- 01:35:23Gates called you once to try to
- 01:35:24personally recruit you to uh to join
- 01:35:27Microsoft what can you give that
- 01:35:29backstory I picked it up in some I was
- 01:35:31researching and I was like well I have
- 01:35:33to get that story uh or a little bit
- 01:35:35more detail into it clearly it didn't
- 01:35:37work you never went to go work at
- 01:35:38Microsoft right yeah it's when I
- 01:35:39graduated from my PhD I
- 01:35:43um there's this one of thing that
- 01:35:45happens when you and kind of the The
- 01:35:47Winner Takes call kind of thing um I I
- 01:35:50was the kind of most sought after
- 01:35:52canidate after I graduated from my PhD
- 01:35:54every University wanted me to become a
- 01:35:56professor
- 01:35:57there and um usually when you graduate
- 01:36:01from a PhD in computer science you go
- 01:36:02become a professor in computer science
- 01:36:03at our University but there's another
- 01:36:05path which is you could also join
- 01:36:06Microsoft research and I was the most
- 01:36:09sought after candidate you know every
- 01:36:11University every computer science
- 01:36:12department wanted to hire me Microsoft
- 01:36:14research wanted to hire me too and
- 01:36:16Microsoft research I guess at the time I
- 01:36:18don't know they had a thing where the
- 01:36:20person they wanted to hire the most that
- 01:36:22year um they would get Bill Gates to
- 01:36:24call you and personally pitch you for as
- 01:36:28long as you wanted to so I you know I
- 01:36:30ended up talking to them for like an
- 01:36:31hour um mainly not because I had you
- 01:36:34know I I love a lot of the stuff that
- 01:36:36they've done at Microsoft and certainly
- 01:36:38Microsoft research but I I I wanted to
- 01:36:40become a professor so I had no no desire
- 01:36:42but I wanted to talk to Bill Gates for
- 01:36:44an hour so I was happy to do that yeah
- 01:36:46well it's good I if he listen if Bill
- 01:36:48Gates ever calls me I I will gladly take
- 01:36:50his call and I'll spend an hour a smart
- 01:36:53guy he's such a smart guy it's crazy
- 01:36:55when you enter a room and you're like
- 01:36:57he's clearly smarter than me yeah you
- 01:36:59think that with him like every time
- 01:37:01because I've been in a room multiple
- 01:37:02times with him for different things
- 01:37:03different every time you're like yeah
- 01:37:05that guyy smarter than me do do you
- 01:37:07remember what you talked to him about or
- 01:37:08what questions you asked him at that
- 01:37:10time no I don't too much um but I've
- 01:37:12talked to him about a number of things I
- 01:37:13mean he's you know he's a we' talked
- 01:37:15about gling and stuff number of times
- 01:37:17again he's such a smart guy yeah now uh
- 01:37:20last one I'll let hop what's the what's
- 01:37:22the story of you setting up a Honeypot
- 01:37:25website to catch people that were
- 01:37:27cheating on your uh on your tests when
- 01:37:30you were a professor I was a professor
- 01:37:32yes I was a professor of computer
- 01:37:33science um and um uh it wasn't tests I
- 01:37:37you know we were I was assigning um you
- 01:37:40know math problems it was basically
- 01:37:42discret math but it was um math problems
- 01:37:45and we had this rule which was a dumb
- 01:37:47rule but it was a rule that I inherited
- 01:37:49from the guy who T the class before me
- 01:37:51that was you can't use you can't Google
- 01:37:52for
- 01:37:54Solutions um and so we tried every time
- 01:37:57but we never because there's only so
- 01:37:59many really awesome math problems and
- 01:38:01you know we tried to disguise them a
- 01:38:03little bit but you know we thought well
- 01:38:05probably people are Googling for
- 01:38:06Solutions and then it occurred to me
- 01:38:09well I'll teach them not to do that and
- 01:38:12so I made up a problem I specifically
- 01:38:14made up a problem and then I made a
- 01:38:16website for that problem that there a
- 01:38:19fully unique problem made a website that
- 01:38:21had the problem Etc and I owned the
- 01:38:24website and what happened is I assigned
- 01:38:26the problem and I said of course you
- 01:38:28know as usual you can't Google or
- 01:38:29anything but if you Googled that problem
- 01:38:34you got to my website and I could track
- 01:38:36who you were uh on the website and so um
- 01:38:40Turned out 30% of the class this was a
- 01:38:41class with 300 kids so about a 100 of
- 01:38:43them fell on the little
- 01:38:46website and um I found them all and then
- 01:38:50um the next lecture you know I didn't
- 01:38:53say anything I didn't contact them the
- 01:38:54next lecture I just went in I stood in
- 01:38:57front of the class I projected my screen
- 01:38:59I typed you know the query to get to
- 01:39:02that problem I got to the website and
- 01:39:04everybody's just like okay and then I
- 01:39:06said I did a you know to to know who
- 01:39:09owns a website you just do who is and
- 01:39:12the name of the website and I typed in
- 01:39:13in the prompt there who is the name of
- 01:39:15that website and then it said that is
- 01:39:17owned by me and then you see everybody's
- 01:39:20job just drop and then I'm then I Then I
- 01:39:23then I you know I'm using this this
- 01:39:25using Linux I'm I'm showing them you
- 01:39:27know the the server logs and just
- 01:39:30everybody's IP address and I'm like w
- 01:39:33and everybody was just they were um they
- 01:39:36were peeing the pets um and so what did
- 01:39:40you do with it did you did they confess
- 01:39:42or or what happened I mean I knew who
- 01:39:45they were um so I just gave them a zero
- 01:39:47all of them in that assignment I said
- 01:39:48look I'm not going to ruin your life I'm
- 01:39:50just going to give you a zero in that
- 01:39:50assignment Ian you can still get an name
- 01:39:51the class
- 01:39:53but I was not interested in ruining
- 01:39:55anybody anybody's life I just wanted to
- 01:39:56uh have them not do that anymore yeah
- 01:39:59it's a good lesson well thank you for
- 01:40:01doing this this is fun I hope hopefully
- 01:40:03you feel better powering through here uh
- 01:40:06yeah so thanks for doing this it's a
- 01:40:08pleasure and uh yeah this is great thank
- 01:40:11you same pleasure is mine thank
- 01:40:13[Music]
- 01:40:19you
- 01:40:21[Applause]
- 01:40:28he
- Luis von Ahn
- Duolingo
- language learning
- A/B testing
- product iteration
- user engagement
- viral marketing
- freemium model
- continuous improvement
- startup culture