I want to do a vector search of Scrapboxes of people who donā€™t publish a vector index.

  • Not just Scrapbox, but blogs and the like, right?
  • In the first place, not only digital books, but also paper books, right?
  • We need a box that allows us to plug in anything we are interested in and make it vector searchable.

from /villagepump/2023/03/23 from Mapping Scrapbox rows to Kozane in Kozaneba

  • Iā€™d love to vector search Scrapbox for people who havenā€™t created a vector index.
    • shokai or rashitamemo.
    • I think it could be made on the same principle as well backups are made on Githubā€¦
    • I mean, I wish my project was updated once a day.
    • And while weā€™re at it, hereā€™s a summary of the updated sectionā€¦
    • ā†’ Vector search across multiple projects
    • Porting to Python with GPT-4
    • Itā€™s done.
    • I guess we can now take data from anyoneā€™s projectā€¦
    • So what did I want to do with it (far away)?
    • Iā€™m tired today, so letā€™s call it a day.
  • You can use this script to both automatically update your own project daily and take someone elseā€™s project, but theyā€™re two different things to begin with.
    • I donā€™t need every day except for my own projects.
  • Itā€™s convenient to be able to search vectors and ask questions, but itā€™s inconvenient to have to sit in front of a computer.
    • The problem is that the index file is too big when trying to deploy it to use from a phone.
    • Hmmm, I guess that would be Pinecoreā€¦
  • LLMs can submit search queries.
    • Experiments have actually been done to receive it and search Wikipedia.
    • Then, when you read Scrapbox and see the link notation, you can make the query ā€œRead that linkā€.
    • A system that utilizes the link structure of Scrapbox

This page is auto-translated from /nishio/他äŗŗ恮Scrapbox悂惙ć‚Æćƒˆćƒ«ę¤œē“¢ć—ćŸć„ using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. Iā€™m very happy to spread my thought to non-Japanese readers.