-
log-bilinear language model
- The probability of occurrence of the following words under a given context c is
- The language model is that is an embedding function for the appropriate word
- In short, take inner product and softmax.
- In A. Mnih and G. Hinton (2007), a little more advanced than mere inner product
-
Similarly, softmaxing after taking the inner product is internal volume caution (2015).
-
Logarithmic bilinear model
-
Log Bilinear Language Model
-
A. Mnih and G. Hinton (2007)
-
“Three new graphical models for statistical language modelling”,
-
ICML, pp. 641–648, http://www.cs.utoronto.ca/~hinton/absps/threenew.pdf
-
A. Mnih, Y. Zhang, and G. Hinton (2009)
-
“Improving a statistical language model through non-linear prediction”,
-
Neurocomputing, vol. 72, no. 7-9, pp. 1414 – 1418
-
http://www.sciencedirect.com/science/article/pii/S0925231209000083
This page is auto-translated from /nishio/対数双線形モデル using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I’m very happy to spread my thought to non-Japanese readers.