Abstract
We present ConvCut, an interactive tool for efficiently navigating and editing 360 video of social conversations into shareable video highlights of the memorable moments. ConvCut starts by obtaining a high-quality transcript of the conversation and uses it to segment the video into one 360 video clip per line of speech. It then applies audio, video and text analysis to label the clips with information including the spatial location of faces, the current speaker, the topics of conversation, instances of laughter and extreme changes in volume, facial expression, or gestural motion. The resulting structure lets users navigate the video using keyword search over the transcript and labels to quickly find memorable moments. Users can mark the lines corresponding to these moments and ConvCut edits together the corresponding video clips, automatically choosing a regular field of view (RFOV) framing that emphasizes the speaker of each line. If desired, users can modify the automatic edit to include alternative framings or reactions from others in the group. We demonstrate that with ConvCut , first-time users can easily edit long social conversations (25-60 min) into short highlight videos (0.27-2 min) and share them with others. The resulting highlights include jokes, reactions to pranks, funny stories and interactions with children.
