CoCRF deformable model: A geometric model driven by collaborative conditional random fields

We present a hybrid framework for integrating deformable models with learning-based classification, for image segmentation with region ambiguities. We show how a region-based geometric model is coupled with conditional random fields (CRF) in a simple graphical model, such that the model evolution is driven by a dynamically updated probability field. We define the model shape with the signed distance function, while we formulate the internal energy with a C¹ continuity constraint, a shape prior, and a term that forces the zero level of the shape function towards a connected form. The latter can be seen as a term that forces different closed curves on the image plane to merge, and, therefore, our model inherently carries the property of merging regions. We calculate the image likelihood that drives the evolution using a collaborative formulation of conditional random fields (CoCRF), which is updated during the evolution in an online learning manner. The CoCRF infers class posteriors to regions with feature ambiguities by assessing the joint appearance of neighboring sites, and using the classification confidence to regulate the inference. The novelties of our approach are (i) the tight coupling of deformable models with classification, combining the estimation of smooth region boundaries with the robustness of the probabilistic region classification, (ii) the handling of feature variations, by updating the region statistics in an online learning manner, and (iii) the improvement of the region classification using our CoCRF. We demonstrate the performance of our method in a variety of images with clutter, region inhomogeneities, boundary ambiguities, and complex textures, from the zebra and cheetah examples to medical images.