Evolution analysis of Binary Partition Tree for hierarchical video simplified segmentation

This paper proposes volumetric hierarchical video segmentation and evolution analysis. The contribution of this paper is twofold: Firstly, a single Binary Partition Tree (BPT) is proposed to represent the entire video segmentation. Every node represents not only a single region but also a series of correlated regions in subsequent frames. Secondly, we propose a method to identify a stop merging criteria by exploiting the discontinuity of volume evolution. The pre-segmentation of the video is produced by 26 neighbourhood watershed. A volume adjacency graph (VAG) is constructed and merging cost between all neighbouring volume as edges. Iterative merging among neighbouring volumes is performed sequentially from the lowest volume distance. Volume model combines colour and temporal direction. Every iteration produces new larger volume; a new node is issued the VAG is updated. The edge in the VAG with the lowest merging cost a selected in every iteration. The history of merging is recorded in the BPT. In order to identify salient nodes, volume evolution analysis is proposed; progressing from the initial partition at the lowest leaf of the BPT towards the root. Discontinuity in the evolution of colour mean and temporal direction indicates a reluctance to merge between a pair of volumes. Therefore, they may belong to different objects. The evolution analysis identifies all salient nodes in the temporal domain of the BPT. In order to simplify the BPT, nodes below a salient node are pruned. The processing gives salient nodes with temporal registration over a video sequence for higher-level cognition.