“Duolingo” has conducted the AB Test 1,800 times so far, resulting in a daily retention (*) increase from 13% to 55%.
We received feedback from users that they wanted to do more of their native language → language of study, so we did an AB test, which resulted in a 1.5% drop in daily active users. The lesson here is: “User behavior does not always match what you say. This was a common trend in other AB tests. It’s important to recognize the problem instead of just putting in the solution as is.” - User requests do not match behavior.
[The methodology behind the foreign language learning app Duolingo, which makes learning difficult things easy GDC 2019 - Famitsu.com]
This page is auto-translated from /nishio/Duolingo using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.