Short-term memory has limited capacity, and when we read a book carefully, the author’s point of view occupies our short-term memory, leaving no room for other points of view or entry.
- Reading unsystematically allows us to store different perspectives, thoughts, and information from multiple books simultaneously in our short-term memory. This increases the likelihood of serendipity, where unexpected perspectives combine to generate new insights and knowledge. This is how Nishio interprets Shigehiko Toyama’s “The Serendipity of Random Reading.
Like humans, AI (LLM, Large Language Model) also has a limited input context width. This is the limit of the amount of information that AI can process at one time, which also limits its thinking.
However, facilitating the combination of new knowledge and perspectives through binge reading is also a useful strategy for AI: as AI freely “browse” through a variety of texts and data, its understanding becomes broader and deeper, and thus more likely to produce more creative and useful output.
Thus, the serendipity of binge reading can be a beneficial strategy for AI as well as humans. This is because it provides a path beyond the limitations of our cognitive abilities to integrate knowledge and discover new insights. Binge reading can be a compass for us to explore the sea of unknown knowledge and perspectives.
2023-07-27 Co-authored, log
This page is auto-translated from /nishio/乱読のセレンディピティとLLMのコンテキスト幅 using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.