习题9.1
EM算法分为E步与M步
对于E步，计算期望。 (mu_j^{(i+1)} = frac{pi^{(i)}(p^{(i)})^{y_j}(1-p^{(i)})^{1-y_j}}{pi^{(i)}(p^{(i)})^{y_j}(1-p^{(i)})^{1-y_j} + (1 - pi^{(i)})(q^{(i)})^{y_j}(1-q^{(i)})^{1-y_j}})
对于M步，极大似然进行估计。 (pi^{(i+1)} = frac{1}{n} sum mu_j^{(i+1)}) ， (p^{(i+1)} = frac{sum mu_j^{(i+1)}y_j}{sum mu_j^{(i+1)}}) ，(q^{(i+1)} = frac{sum (1-mu_j^{(i+1)})y_j}{sum (1 - mu_j^{(i+1)})})
第一轮迭代
E步：观测值为1的 (mu^{(1)} = 0.4115) ，观测值为0的(mu^{(1)} = 0.5374)
M步：(pi^{(1)} = 0.4619) ，(p^{(1)} = 0.5346) ， (q^{(1)} = 0.6561)
第二轮迭代
E步：观测值为1的 (mu^{(2)} = 0.4116) ，观测值为0的(mu^{(2)} = 0.5374)
M步：(pi^{(2)} = 0.4619) ，(p^{(2)} = 0.5346) ，(q^{(2)} = 0.6561)
EM算法两轮收敛，得到参数的极大似然估计值。
(hat pi = 0.4619, hat p = 0.5346,hat q =0.6561)
习题9.2
引理9.2：若 (tilde P_{theta}(Z) = P(Z|Y,theta)) ， 则 (F(tilde P, theta) = log P(Y|theta))
(F(tilde P, theta) = E_{tilde P}[logP(Y,Z|theta)] + H(tilde P) \ = E_{tilde P}[logP(Y,Z|theta)] - E_{tilde P} log tilde P(Z) \ =sum_Z tilde P_{theta}(Z) logP(Y,Z|theta) -sum_Z tilde P(Z) log tilde P(Z) \ =sum_Z P(Z|Y,theta) logP(Y,Z|theta) -sum_Z P(Z|Y,theta) log P(Z|Y,theta) \ =sum_Z P(Z|Y,theta) log frac{P(Y,Z|theta)}{P(Z|Y,theta)} \ = sum_Z P(Z|Y,theta) logP(Y|theta) = logP(Y|theta))
习题9.3
调用 sklearn.mixture.GaussianMixture 这个API进行训练，可得
(alpha_1 = 0.8668, mu_1 = 32.9849, sigma_1^2 = 429.4576)
(alpha_2 = 1- alpha_1 = 0.1332, mu_2 = -57.5111, sigma_2^2 = 90.2499)
习题9.4
Mixture of Naive Bayes Model（NBMM，混合朴素贝叶斯模型）
NBMM的EM算法
E步：(w_j^{(i)} = P(z^{(i)} =1|x^{(i)};phi_z,phi_{j|z^{(i)}=1},phi_{j|z^{(i)}=0}))
M步：(phi_{j|z^{(i)}=1} = frac{sum w^{(i)}I(x_j^{(i)}=1)}{sum w^{(i)}}) ，(phi_{j|z^{(i)}=0} = frac{sum (1-w^{(i)})I(x_j^{(i)}=1)}{sum (1-w^{(i)})})， (phi_{z^{(i)}} = frac{sum w^{(i)}}{m})
文章来源: 博客园
- 还没有人评论，欢迎说说您的想法！
客服
