變分推斷

背景

${\displaystyle P(\mathbf {z} \mid \mathbf {x} )={\frac {P(\mathbf {x} \mid \mathbf {z} )\cdot P(\mathbf {z} )}{P(\mathbf {x} )}}}$

數學推導

最大化估計

${\displaystyle f(\theta )\geq g(\theta )}$

獲取啲下界函數

${\displaystyle \log p_{\mathbf {\theta } }(\mathbf {x} )=\int q(\mathbf {z} )\log p_{\mathbf {\theta } }(\mathbf {x} )d\mathbf {z} }$

${\displaystyle \log p_{\mathbf {\theta } }(\mathbf {x} )=\int q(\mathbf {z} )\log {\dfrac {p_{\theta }(\mathbf {x} ,\mathbf {z} )}{p_{\theta }(\mathbf {z} \mid \mathbf {x} )}}d\mathbf {z} }$

{\displaystyle {\begin{aligned}\log p_{\mathbf {\theta } }(\mathbf {x} )&=\int q(\mathbf {z} )\log \left({\dfrac {p_{\theta }(\mathbf {x} ,\mathbf {z} )}{q(\mathbf {z} )}}\cdot {\dfrac {q(\mathbf {z} )}{p_{\theta }(\mathbf {z} \mid \mathbf {x} )}}\right)d\mathbf {z} \\&=\int q(\mathbf {z} )\log {\dfrac {p_{\theta }(\mathbf {x} ,\mathbf {z} )}{q(\mathbf {z} )}}d\mathbf {z} +\int q(\mathbf {z} )\log {\dfrac {q(\mathbf {z} )}{p_{\theta }(\mathbf {z} \mid \mathbf {x} )}}d\mathbf {z} \end{aligned}}}

${\displaystyle \log p_{\mathbf {\theta } }(\mathbf {x} )={\mathbb {E} _{\mathbf {z} \sim q(\mathbf {z} )}[\log {\dfrac {p_{\theta }(\mathbf {x} ,\mathbf {z} )}{q(\mathbf {z} )}}]}+\mathbb {KL} [q(\mathbf {z} )\|p_{\theta }(\mathbf {z} \mid \mathbf {x} )]}$

${\displaystyle \log p_{\mathbf {\theta } }(\mathbf {x} )\geq {\mathbb {E} _{\mathbf {z} \sim q(\mathbf {z} )}[\log {\dfrac {p_{\theta }(\mathbf {x} ,\mathbf {z} )}{q(\mathbf {z} )}}]}}$