Recent development of the information communication technology raises the need for an content-based video scene retrieval. Nevertheless, in order to retrieve the scenes which the user wants to see from the video under existing circumstances, we can only search them by time-consuming viewing. This is because the video data is huge, and it is difficult for computers to understand semantic contents of videos. One of the techniques for solving the retrieval costs is to give the video indexes based on audio and visual feature vectors per video packet. And, to solve the semantic gap, we cluster the feature vectors and assign the "sign" which is the cluster ID as the video index. After all, the video stream replace the "sign sequence". Using this method, the similar scenes to the query videos can be quickly retrieved by sign sequence matching.


   Video Scene Retrieval with Sign Sequence Matching Based on Audio and Visual Features

list

  Video summarization is defined as creating a shorter video clip or a video poster which includes all but only the important scenes in original video streams. In this research, we propose two methods of generating a summary of arbitrary length for large sports video archives. One is to create a concise video clip by temporally compressing the amount of the video data. The other is to provide a video poster by spatially presenting the image keyframes which together represent the whole video content. Our methods deal with the metadata to have semantic description of video contents. Summaries are created according to the significance of each video segment which is normalized in order to handle large sports video archives.

＜Video Crip＞

＜Video Poster＞

list

  Structuralization of the video stream is necessary for effective handling of video media. Especially, collecting similar scenes with sufficient accuracy contributes to the structuralization. We propose a method of clustering with relevance feedback. First,fixedlength segments are clustered according to the feature of each segment. For the results of clustering, the user gives the feedback information whether each element is relevant to the cluster it belongs to. The accuracy can be improved by re-clustering based on the feedback information. Applying this method to a variety of video streams, we demonstrated the effectiveness of the relevance feedback.


         Figure 1．Flow of clustering


         Figure 2．Clustering result

list

  We propose a video completion method to automatically remove unnecessary objects from a video without leaving artifacts. After a user selects an object from a frame on an interface, the method extracts the corresponding objects from the entire video and realizes plausible video completion with spatial-temporal features by using the temporal correlation between adjacent frames followed by the spatial correlation between adjacent pixels in a frame.

＜Example1＞

＜Example2＞

list

  Professionally edited short video clips, such as movie trailers, are created by adding music and sound effects on top of cut-and-paste of the original videos. We are trying to learn how the videos and music/sounds are physically and semantically synchronized to make such videos more attractive/intriguing in order to develop an (semi-)automatic video editing system(Fig..1). We are currently focusing on the synchronization between the accents in audio streams and shot/scene changes in video streams by focusing on their hierarchical structures(Fig..2).


      Figure1．System


Figure2．Synchronization between audio accents and shot/scene changes

list

INTRODUCTION