nishio When LLM LLM evolves more and more, the decision to ākeep using the old version for fear of compatibility and other changes without much thoughtā, you will be forced to make a decision between āmake the latest version and some features will breakā and ākeep using the old version for fear of breaking itā, but if you learn from history, the latter is just a postponement of the cost, and eventually you will be forced to raise the Eventually, you will have to raise the price.
I tried to write a follow-up to this, but I canāt verbalize it well.
Test cases before refactoring mechanically verifiable KPI. clear needs.
Some tasks are easy or difficult to set mechanically verifiable KPIs, or difficult tasks need to be divided
When we go to realize an understandable need, we encounter the difficulty of setting this KPI. Cases where humans as changing parts are woven into the system otherwise the case In the former, it expects human learning ability, so it cannot be used for people who do not learn. Boundary Surface Issues
Hmmm, no.
Look at the discussion below, guess what theyāre trying to say, and give me a clue to verbalize what they canāt verbalize.
The following key points can be gleaned from this discussion
- In designing an AI system, it is important to set appropriate boundary surfaces for the division of tasks between humans and AI. Clarifying which tasks are to be performed by AI and which are to be performed by humans will increase the efficiency, effectiveness, and user acceptance of the system.
- another important boundary plane is between the userās need space and the functionality provided by the system. If this boundary plane is unclear, a gap between the value of the system and user expectations may arise.
- the boundary aspects of the needs space are particularly difficult to verbalize and are often ignored, but are essential for improving UX. Efforts should be made to better understand user needs through market research and feedback.
- it is effective to divide tasks for which KPI setting is difficult into smaller, more manageable parts. In processes with significant human involvement, it is often difficult to set specific numerical targets.
- incorporating human learning ability and adaptability into the system design and providing continuous training and feedback mechanisms will help improve the efficiency of human-mediated processes.
Behind their words may be an awareness of the need to optimize systems that utilize rapidly evolving AI, not only from a technical design perspective, but also from a multifaceted perspective, including how humans and AI should collaborate, matching user needs, and developing human resources.
No, itās notā¦
Systems as stable and unchanging, and systems that change. Software peer production/on-demand production/Diversity Production
nishio I feel like thereās something important there that still needs to be verbalized but hasnātā¦
nishio Prompt is indeed a new programming language, but it is a machine language for a ānew calculator called LLMā and is model dependent. If a user is satisfied with the level of writing in a natural language and it works well on any LLM, thatās fine. If the user is satisfied with that level of performance, then it is fine, but if the user is not satisfied with that level of performance, then it is not a problem. From there, it becomes a cliff due to model dependence.
nishio At this point, writing in a more abstract language and creating optimized prompts for the target LLM becomes a thing to do. This is the same thing that gave rise to C, which can be compiled against a wide variety of calculators.
nishio I see. In other words, āareas where the market has been too small for anyone to enterā where there is no competition will be supplied by natural language programming, and areas that have been covered by software will be shifted to more abstract and high-level languages by becoming red ocean in the degree of prompting. The result is a shift to higher-level languages with a higher level of abstraction by becoming a [red ocean - Diversity Production
nishio Human description of prompts in natural language is equivalent to machine language. You may think it is impossible for ordinary people to use machine language, but at present, ordinary people are in the state of realizing ācontrolling the behavior of machines by pushing physical buttonsā in the virtual reality of GUI, which is even before machine language.
nishio So only a few people get to write prompts, and many others push a button that executes a prompt that is already written. There will only be more new programmers of the kind that write prompts, just like when LL was created and more programmers said I donāt know C but I can do LL.
nishio This is not āwhat I was trying to verbalizeā, but itās a related topic, so I just spit out what came to mind
This page is auto-translated from /nishio/LLMć®ę“ę°ćØćć¼ćŗć®å¢ē® using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. Iām very happy to spread my thought to non-Japanese readers.