1. A(δ∣w)A(\delta|w)A(δ∣w)和B(δ∣w)B(\delta|w)B(δ∣w)推导

A(δw)A(\delta|w)L(w+δ)L(w)L(w+\delta) - L(w)的下界:

令:
fif_ifi(x,y)f_i(x,y)

1.1. A(δ∣w)A(\delta|w)A(δ∣w)的推导

证明:
L(w+δ)L(w)=x,yP~(x,y)i=1nδifixP~(x)logZw+δ(x)Zw(x) L(w+\delta) - L(w) = \sum_{x,y}\tilde P(x,y)\sum_{i=1}^n\delta_if_i - \sum_x\tilde P(x) \log \frac{Z_{w+\delta}(x)}{Z_w(x)} 当a>0时,loga1a-\log a \ge 1-a,得: (1)x,yP~(x,y)i=1nδifi+xP~(x)(1Zw+δ(x)Zw(x))=x,yP~(x,y)i=1nδifi+xP~(x)xP~(x)Zw+δ(x)Zw(x)) (1) \ge \sum_{x,y}\tilde P(x,y)\sum_{i=1}^n\delta_if_i + \sum_x\tilde P(x)(1-\frac{Z_{w+\delta}(x)}{Z_w(x)}) = \sum_{x,y}\tilde P(x,y)\sum_{i=1}^n\delta_if_i + \sum_x\tilde P(x) - \sum_x\tilde P(x)\frac{Z_{w+\delta}(x)}{Z_w(x)}) 根据P~(x)\tilde P(x)可知xP~(x)=1\sum_x\tilde P(x)=1,得:
(2)=x,yP~(x,y)i=1nδifi+1xP~(x)Zw+δ(x)Zw(x)) (2) = \sum_{x,y}\tilde P(x,y)\sum_{i=1}^n\delta_if_i + 1 - \sum_x\tilde P(x)\frac{Z_{w+\delta}(x)}{Z_w(x)}) 根据【?】,得:
(3)=x,yP~(x,y)i=1nδifi(x,y)+1xP~(x)yPw(yx)expi=1nδifi(x,y) (3) = \sum_{x,y}\tilde P(x,y)\sum_{i=1}^n\delta_if_i(x,y) + 1 - \sum_x\tilde P(x)\sum_yP_w(y|x)\exp \sum_{i=1}^n\delta_if_i(x,y) A(δw)A(\delta|w)

1.2. B(δ∣w)B(\delta|w)B(δ∣w)的推导

一次只优化其中一个变量δi\delta_i,而固定其它变量δj,ij\delta_j,i \neq j,得:
【?】是一只优化一个wiw_i还是一个δi\delta_i? 【?】如果是只优化一个wiw_i,为什么不能直接假设其它δj=0\delta_j=0
【?】如果是只优化一个δi\delta_i,为什么算法6.1步骤2-(b)只更新一个wiw_i

后面的推导不难,只是这一块没想通,就不往下记了。

results matching ""

    No results matching ""