from Diary 2023-10-23 Is it beneficial to RAG only against BOOK if one does not have written material?

I have about 60 books worth of my own writing, I have experienced RAG for this, I have cut and scanned books, I have experienced RAG for that as well. Q: Is it beneficial for someone who has no written work to RAG only against books?

A: It’s beneficial, but I don’t think its form of usefulness is what you’re expecting.

  • You don’t get the same knowledge without reading a book as you do when you read a book.
    • It’s more of a painful reading than a normal reading.
  • usefulness - rhetorical afterimage in The Serendipity of Random Reading. - syntopic reading - Read three books in the same field and you’ll become an expert.
    • These effects are beneficial
    • concrete example - Insights from inner experience are key to approaching more responsive governance
      • Issuing the appropriate query yielded five relevant document fragments.
      • This is a state of binge reading of 5 books in parallel, and luckily, serendipity drew “encountering related fragments”.
      • If you build on this over time, you can create a network of knowledge that crosses multiple books.
  • Reading a book from the beginning is usually an easy read because it is designed to flow smoothly
    • In contrast, this reading ignores context and jumps to the most relevant part.
    • Read back and forth from there.
      • As I was writing this, I was thinking that it might be assumed that books can be read in reverse order in the first place.
        • Well, but if you chop it up a little and read it, even someone who can only read from the front can probably do it…
    • I thought that Reading in an era with LLM] would be different from what it had been, but one of those “different forms” is now a reality at my fingertips!

This page is auto-translated from /nishio/書き溜めたものがない人が書籍に対してだけRAGして有益か? using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.