3.3 Constructions

3.3.1 Construction via a Stochastic Process

Kolmogorov-consistency theorem

3.3.2 Construction through a Distribution Function

3.3.3 Construction through a Gamma Process

CRM

3.3.4 Construction through Polya Urn Scheme

de Finetti measure of polya urn scheme is DP?

3.3.5 Stick-Breaking Representation(Sethuraman Representation)

Stick-Breaking Representation, 혹은 Sethuraman Representation은 Beta 분포에 기반한 stick-breaking 과정을 통해 Dirichlet process를 construct하는 방법을 제시한다.

If $\theta_1, \theta_2, \cdots \stackrel{\textit{iid}}{\sim} G_0$ and $V_1, V_2, \cdots \stackrel{\textit{iid}}{\sim} \text{Beta}(1,M)$ are independent random variables and $W_j = V_j \prod_{l=1}^{j-1} (1-V_l) $, then $\sum_{j=1}^{\infty} W_J \delta_{\theta_j} \sim \text{DP}(M G_0)$.

증명은 아래와 같다.

\[\begin{align*} 1 - \sum_{j=1}^{\infty} W_j &= 1- V_1 + V_2(1-V_1) + V_3(1-V_2)(1-V_1) + \cdots \\ &= (1-V_1)\left[ V_2 + V_3(1-V_2) + \cdots \right] \\ &= \prod_{l=1}^{\infty} (1-V_l) \\ \text{E}\left[ \prod_{l=1}^{j} (1-V_l) \right] &\stackrel{ind}{=} \prod_{l=1}^{j} \text{E}\left[ (1-V_l) \right] \\ &= \left( \frac{M}{M+1} \right)^j \\ &\longrightarrow 0, \enspace \text{ as } j \rightarrow \infty \\ \\ \therefore \enspace \sum_{j=1}^{\infty} W_j &= 1- \prod_{l=1}^{\infty} (1-V_l) = 1,\enspace \text{ almost surely} \end{align*}\]

즉 stick-breaking weights $W_j$는 probability vector이다, with probability $1$. 따라서, 다음과 같이 정의된 random measure $P$는 probability measure이다, with probability $1$.

\[P= \sum_{j=1}^{\infty} W_j \delta_{\theta_j}\]

$j \geq 1$에 대해, $W_j^\prime = V_{j+1} \prod_{l=2}^{j}(1-V_l)$와 $\theta_j^\prime = \theta_{j+1}$을 정의하자. $j \geq 1$에 대해, 다음이 만족한다.

\[W_{j+1}=V_{j+1} \prod_{l=1}^{j}(1-V_l)=(1-V_1)\cdot V_{j+1} \prod_{l=1}^{j}(1-V_l)=(1-V_1)W_j '\] \[P= W_1 \delta_{\theta_1} + \sum_{j=2}^{\infty} W_j \delta_{\theta_j}=V_1 \delta_{\theta_1} + (1-V_1) \sum_{j=1}^{\infty} W_j ' \delta_{\theta_j '}\]

Random measure $P^\prime$를 $P^\prime= \sum_{j=1}^{\infty} W_j^\prime \delta_{\theta_j^\prime}$로 정의한다면, $P^\prime$는 $P$와 정확히 같은 structure를 가지고 있기 때문에 $P$와 분포가 같다. 또한 $P^\prime$는 $V_1$와 $\theta_1$를 포함하지 않기 때문에 $(V_1 ,\theta_1)$과 독립이다. 따라서 stick-breaking으로 construct된 random measure $P$는 다음과 같은 distributional equation을 만족한다.

\[P \stackrel{d}{=} V \delta_{\theta} + (1-V) P\]

다음 lemma를 이용하여 증명을 끝낼 수 있다.

For given independent $\theta \sim G_0$ and $V \sim \text{Beta}(1, M)$, the Dirichlet process $\text{DP}(M G_0)$ is the unique solution of the above distributional equation.

proof of the lemma

Let $P \sim \text{DP}(M G_0)$. Let $A_1, \cdots , A_k$ be any finite partition of $\mathfrak X$. Then,

\[(P(A_1), \cdots , P(A_k)) \sim \text{Dirichlet}(MG_0(A_1), \cdots , MG_0(A_k))\]

For $\theta \sim G_0$,

\[(\delta_{\theta}(A_1), \cdots , \delta_{\theta}(A_k)) \sim \text{Multinomial}[1;(G_0(A_1), \cdots , G_0(A_k))]\]

Then, for $Y_0 , Y_1, \cdots , Y_k \stackrel{ind}{\sim} \text{Gamma}(\alpha_i,1)$, and $\alpha_0 = 1, \alpha_i = MG_0(A_i)$, Then,

\[\tilde P := \left( \frac{Y_1}{\sum_{i=1}^{k} Y_i} , \cdots, \frac{Y_k}{\sum_{i=1}^{k} Y_i} \right) \sim \text{Dirichlet}(MG_0(A_1), \cdots , MG_0(A_k))\] \[\tilde P \perp\!\!\!\perp \left( Y_0, \sum_{i=1}^{k} Y_i \right)\] \[\tilde V := \frac{Y_0}{Y_0 +\sum_{i=1}^{k} Y_i } \sim \text{Beta}(1, M)\] \[(\tilde V , (1-\tilde V )\tilde P )=\frac{1}{Y_0 +\sum_{i=1}^{k} Y_i} \left( Y_0, Y_1, \cdots , Y_k \right) \sim \text{Dirichlet}(1,MG_0(A_1), \cdots , MG_0(A_k))\]

Then,for $i=1,\cdots, k$,

\[\tilde V e_i + (1-\tilde V )\tilde P \sim \text{Dirichlet}(MG_0(A_1), \cdots , MG_0(A_i) + 1, \cdots , MG_0(A_k))\]

Using the multinomial variable $N$ defined above,

\[H_i := \tilde V N+(1-\tilde V)\tilde P \Big\vert N=e_i \sim \text{Dirichlet}(MG_0(A_1), \cdots , MG_0(A_i)+1, \cdots, MG_0(A_k)).\]

We can denote the distribution of $\tilde V N+(1-\tilde V_i)\tilde P$ as a mixture of $H_i$’s.

\[\begin{align*}\tilde V N+(1-\tilde V_i)\tilde P \Big\vert N &\stackrel{d}{=} \sum_{i=1}^{k} H_i \cdot \mathbb P(N=e_i)\end{align*}\]

By the conjugacy of Dirichlet distribution and multinomial distribution,

\[\tilde V N+(1-\tilde V_i)\tilde P \sim \text{Dirichlet}(MG_0(A_1), \cdots, MG_0(A_k)).\]

Then, by the fact that $V \stackrel{d}{=} \tilde V$, $P \stackrel{d}{=} \tilde P$,

\[V N + (1-V)P \sim \text{Dirichlet}(MG_0(A_1), \cdots , MG_0(A_k))\] \[\therefore \enspace P \stackrel{d}{=} V N + (1-V)P.\]

We have chosen a finite partition of $\mathfrak X$, $A_1, \cdots , A_k$ arbitrarily. Then, $P \sim \text{DP}(M G_0)$ is a solution for the distributional equation above.

((uniqueness 보충 필요))

따라서, random measure $P$는 $\text{DP}(M G_0)$을 따른다. $\square$