Discomfort-driven intellectual production is the act of freeing oneself from consistency in a narrow context width and seeking consistency in a wide context width What kind of research is needed to further deepen discomfort-driven intellectual production?
BELOW_IS_LESS_INTERESTING
digest
.
The source of discomfort in discomfort-driven intellectual production is the low probability of token generation in probabilistic language models. Predicting the probability of token occurrence in a small context width produces what appears to be “meaning” to humans. However, given the depth of intellectual production, it is important to be consistent over a wide context width. To take advantage of this, it is useful to fragment sentences by removing particles and other words from them, and to seek a consistent spatial arrangement among them.
Considerations and questions
Discomfort-driven intellectual production can be described as the act of freeing oneself from consistency in a narrow context width and seeking consistency in a wide context width. However, the question is how to capture the sense of discomfort in the process and how to resolve it. Moreover, to what extent can seeking the source of discomfort in a probabilistic language model replicate the process of human intellectual production? What kind of research is needed to further deepen discomfort-driven intellectual production?
extra info
json size: 41055043 pickle size: 778509348 previous notes size: 594 previous notes: 🤖2023-08-19 07:08 TITLES: Refactoring, Splatoon becomes a squid, Hatena2008-12-18, Explicitly telling users when keyphrase extraction speed is slow, Hatena2009-08-10, 2020-04-09 note-taking experiment, Technology supporting coding English sentence check
This page is auto-translated from [/nishio/🤖2023-08-20 07:07](https://scrapbox.io/nishio/🤖2023-08-20 07:07) using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.