Tree-based iterated local search for Markov random fields with applications in image analysis

Abstract

The maximum a posteriori assignment for general structure Markov random fields is computationally intractable. In this paper, we exploit tree-based methods to efficiently address this problem. Our novel method, named Tree-based Iterated Local Search (T-ILS), takes advantage of the tractability of tree-structures embedded within MRFs to derive strong local search in an ILS framework. The method efficiently explores exponentially large neighborhoods using a limited memory without any requirement on the cost functions. We evaluate the T-ILS on a simulated Ising model and two real-world vision problems: stereo matching and image denoising. Experimental results demonstrate that our methods are competitive against state-of-the-art rivals with significant computational gain.

Appendix: Distribution over conditional trees

We provide the derivation of Eq. (7) from a probabilistic argument. Recall that $\mathcal {N}(\tau )$ is the Markov blanket of the tree $\tau$, that is, the set of sites connecting to $\tau$. Due to the Markov property

$$\begin{aligned} P\left( \mathbf {x}_{\tau }\mid \mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right)&= P\left( \mathbf {x}_{\tau }\mid \mathbf {x}_{\lnot \tau },\mathbf {y}\right) \\&\propto \exp \left\{ -E_{\tau }\left( \mathbf {x}_{\tau },\mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right) \right\} \end{aligned}$$

where

$$\begin{aligned} E_{\tau }\left( \mathbf {x}_{\tau },\mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right)&= \sum _{i\in \tau }E_{i}(x_{i},\mathbf {y})+\sum _{(i,j)\in \mathcal {E}\mid i,j\in \tau }E_{ij}(x_{i},x_{j})\nonumber \\&+\sum _{(i,j)\in \mathcal {E}\mid i\in \tau ,j\in \mathcal {N}(\tau )}E_{ij}(x_{i},x_{j}) \end{aligned}$$

(11)

Equation (7) can be derived from the energy above by letting:

$$\begin{aligned} E_{i}^{*}(x_{i},\mathbf {y})=E_{i}(x_{i},\mathbf {y})+\sum _{(i,j) \in \mathcal {E},j\in \mathcal {N}(\tau )}E_{ij}(x_{i},x_{j}) \end{aligned}$$

(12)

Thus finding the most probable labeling of the tree $\tau$ conditioned on its neighborhood is equivalent to minimizing the conditional energy in Eq. (9):

$$\begin{aligned} \hat{\mathbf {x}}_{\tau }=\arg \max _{\mathbf {x}_{\tau }}P\left( \mathbf {x}_{\tau }\mid \mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right) \end{aligned}$$

(13)

The equivalence can also be seen intuitively by considering the tree $\tau$ as a mega-site, so the update in Eq. (9) is analogous to that in Eq. (4).

1.1 Proof of Proposition 1

Recall that the energy can be decomposed into singleton and pairwise local energies (see Eq. 1)

$$\begin{aligned} E(\mathbf {x}_{\tau },\mathbf {x}_{\lnot \tau },\mathbf {y})&= \sum _{i\in \tau }E_{i}(x_{i},\mathbf {y})+\sum _{i\notin \tau }E_{i}(x_{i},\mathbf {y})+\sum _{(i,j)\in \mathcal {E}|i,j\in \tau }E_{ij}(x_{i},x_{j})+\\&\quad +\sum _{j\in \mathcal {N}(\tau )|(i,j)\in \mathcal {E}}E_{ij}(x_{i},x_{j})+\sum _{(i,j)\in \mathcal {E}|i,j\notin \tau }E_{ij}(x_{i},x_{j}) \end{aligned}$$

where:

• $\sum _{i\in \tau }E_{i}(x_{i},\mathbf {y})$ is the data energy belonging to the tree $\tau$,

• $\sum _{i\notin \tau }E_{i}(x_{i},\mathbf {y})$ is the data energy outside $\tau$,

• $\sum _{(i,j)\in \mathcal {E}|i,j\in \tau }E_{ij}(x_{i},x_{j})$ is the interaction energy within the tree,

• $\sum _{j\in \mathcal {N}(\tau )|(i,j)\in \mathcal {E}}E_{ij}(x_{i},x_{j})$ is the interaction energy between the tree and its boundary, and

• $\sum _{(i,j)\in \mathcal {E}|i,j\notin \tau }E_{ij}(x_{i},x_{j})$ is the interaction energy outside the tree.

By grouping energies related to the tree and the rest, we have

$$\begin{aligned} E(\mathbf {x}_{\tau },\mathbf {x}_{\lnot \tau },\mathbf {y})&= E_{\tau }\left( \mathbf {x}_{\tau },\mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right) +\sum _{(i,j)\in \mathcal {E}|i,j\notin \tau }E_{ij}(x_{i},x_{j}) \end{aligned}$$

where $E_{\tau }\left( \mathbf {x}_{\tau },\mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right)$ is given in Eq. (11) for all $i\in \tau$. This leads to:

$$\begin{aligned} E(\hat{\mathbf {x}}_{\tau },\mathbf {x}_{\lnot \tau },\mathbf {y})&= E_{\tau }\left( \hat{\mathbf {x}}_{\tau },\mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right) +\sum _{(i,j)\in \mathcal {E}|i,j\notin \tau }E_{ij}(x_{i},x_{j})\\&\le E_{\tau }\left( \mathbf {x}_{\tau },\mathbf {x}_{\mathcal {N}(\tau )},\mathbf {y}\right) +\sum _{(i,j)\in \mathcal {E}|i,j\notin \tau }E_{ij}(x_{i},x_{j})\\&= E(\mathbf {x}_{\tau },\mathbf {x}_{\lnot \tau },\mathbf {y}) \end{aligned}$$

This completes the proof $\square$