Significant Difference and Break-even Point

• 2 has significant difference but not beyond [the break-even point
• Whether or not the field finds value in this situation depends on the field.
• Any system is costly to maintain.
• If the customer doesn’t pay that cost, it won’t be maintained.
• Let’s quickly and inexpensively verify “are there paying customers?”
  • This is the Lean startup concept
• Paying customers” = “people who value the product enough to want to continue using it even if they have to pay for it.”
• Even if there is a significant difference, it’s as if there isn’t enough to pay for it.
• On the other hand, scientific thinking tends to ignore 3, “a small percentage of subjects are worthwhile”
  • It can be ignored simply because the sample size is too small to make a significant difference.
  • When “one in a hundred has a significant effect.”
    • If there were 100 subjects, it would be “only one was effective”.

sequel - losses and variance

---

This page is auto-translated from /nishio/有意差と損益分岐点 using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.