VideoCraft: A Mixed Reality-Empowered Video Generation Workflow with Spatial Layer Editing for Concept Video Creation

Boyu Li1, Linping Yuan2, Zeyu Wang1,2

The Hong Kong University of Science and Technology (Guangzhou)1

The Hong Kong University of Science and Technology2

UIST 2025

Space Restructing

Layer-edited MR Scene
V2V
Comparison Extra

Motivation: MR + V2V

V2V 示意图
MR+V2V 示意图

Motivated by the advantages of MR and V2V models, we envision a workflow that first uses MR to edit and capture mixed-reality footage, which then serves as input for V2V models to automatically refine and enhance the content.

Localized Editing with Spatial Layer

Spatial Layer

The spatial layer, created within the MR environment, serves as a guide for localized editing in the generated concept video.

Video