Nodes of Thought2019-10-31

• Facebook
  • The “link things that are close in distance” system we created yesterday: Link Creation Support.
  • It is the same as Curiosity Module written in Subjectivity and Emotion up to the point where the distance between past memories is calculated.
    • The “Curiosity Module” is not a separate page at this time, but should become one eventually.
  • Now it’s a simple Euclidean distance, but when using cosine similarity, it takes the same form as inner product attention.
    • Then, as I wrote in dimensionality reduction caution, you can make a distant association by putting Dropout
  • It seems relevant to this that I excluded the ones where the vectors are equal. - Independent Desire Device
    • Nodes of Thought 20191021 Looking Back
  • Picking up the closest past memory from the new input and linking to it is “associative” where the pickup is “associative” and stocked with - Stock of associations
  • If we can describe what would be desirable, we may be able to acquire measures - Knowledge Weaving Program
  • What we haven’t done yet
    • I am able to find the closest memory Y for input X, but when I observe myself I am doing “reading both and making new outputs with that stimulus”, how can this process be achieved?
      • (11/4)
        • Associative indication is a primitive form of output
        • 
        • If the contents are combined in a different arrangement than originally given, the corresponding vectors will naturally be different.
          • The act of selecting and pointing to the closest content from this new vector is “the act of outputting a new combination of content as input.
  • There are 2^768 ways to do a Dropout in 768 dimensions, so a full search is practically impossible; if X and Y are associated under a certain Dropout and the connection is beneficial, it suggests that “that Dropout way is a beneficial abstraction”.
    • So the connection of two points, “the connection of X and Y”, is another data point one layer above, so that should be stocked too!

---

This page is auto-translated from /nishio/思考の結節点2019-10-31 using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.