Incorrect KPI setting

• Consider the state of having “objectives” and “quantitative measures” that are useful in measuring progress toward those objectives.

• This “quantitative measure” is often referred to as a KPI

• What is incorrect KPI setting?

  • Choosing a KPI that is not appropriate for measuring the objective, or
  • KPIs that were initially appropriate, but as objectives are achieved or change, they are no longer appropriate KPIs, yet they are still measured by themselves because they are “easy to understand”.

• Example

  • Researchers’ performance is measured by the number of papers they write. - No numerical criteria should be used to evaluate researchers.

  • Measure the quality of a company by the amount of profit it makes.

  • Imposing quotas on salespeople to acquire new projects even though the business is a subscription model.

    • https://cybozushiki.cybozu.co.jp/articles/m005521.html

  • @tweeting_drtaka: there has been a lot of activity on this issue Dr. Nagayasu Toyoda’s blog. While the number of papers is increasing in many countries, only in Japan, it stops growing at all after around 2005. He says that the performance of research is deteriorating rapidly, and Japan is being overtaken by other countries.

  • https://t.co/4H3euAvWES

  • 

  • @tweeting_drtaka: Since 2004, national universities have been operating according to a mid-term plan Since 2004, national universities have been operating in accordance with their mid-term plans and have been subjected to third-party evaluations. So what about the results of those evaluations? The National Institution for Academic Degrees and University Reform has published the data. The National Institution for Academic Degrees and University Reform has issued a document showing that both education and research are performing very “well” in accordance with the plan.

  • https://t.co/fMiK0NHWM3

  • 

  • @tweeting_drtaka: This point was made by Dr. Egashira of Otaru University of Commerce. It is a received opinion. But in fact, the reforms implemented by the will of the government are considered “on track” in the domestic evaluation system, while they are close to “collapsing” in the international evaluation. In short, evaluation in Japan’s governing structure is not functioning as evaluation.

  • @tweeting_drtaka: in short, the evaluation axis was completely wrong …So… If they wanted to achieve an increase in the number of papers, they should have set KPIs such as research hours of university faculty members as a science and technology policy, but they reduced the hours by tightening management, so the number of papers decreased.

  • Mistakes in setting up these evaluation axes can greatly exacerbate the situation.

  • @_anohito: Industrial competitiveness, which topped the list in the early 1990s, fell to 30th place in 2002 and has not shown any signs of recovery since then. On the other hand, as far as the number of patent applications is concerned, Japan remains in second place in the world as of 2011. This gap indicates a failure in the use and meaning making of technology.

relevance

• bias to measure only what is easy to measure.

---

This page is auto-translated from /nishio/誤ったKPI設定 using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.