Video Object Replacement版本v2.0 (ID: 942535)

Introduction

This workflow uses Segment Anything v2 to segment out the part of the original video to be replaced. Then the Set Latent Noise Mask node is used before passing the VAE encoded video into the KSampler.

You will provide a reference image to be used with IPAdapter and also a prompt to guide the generation of the new object. The depth controlnet is used to give some guidance on what the shape of the replaced object should be.

Models

1. SD1.5 Checkpoint: I use a LCM checkpoint so I do not need to use the AnimateLCM LoRA. You can use your favourite SD1.5 checkpoint. https://civitai.com/models/4384?modelVersionId=252914

2. AnimateLCM_sd15_t2v.ckpt: Download from https://huggingface.co/wangfuyun/AnimateLCM/tree/main and put in models/animatediff_models

3. control_v11f1p_sd15_depth_fp16.safetensors and depth_anything_v2_vitl.pth. Download using Manager.

4. IPAdapter PLUS: download ip-adapter-plus_sd15.safetensors using ComfyUI manager. Also download ViT-H and ViT-G clip_vision models using the ComfyUI manager.

5. SAM2 models should automatically download

Custom Nodes

Install with Manager:

1. ComfyUI-VideoHelperSuite

2. Crystools

3. KJNodes for ComfyUI

4. ComfyUI's ControlNet Auxiliary Preprocessors

5. ComfyUI-Advanced-ControlNet

6. AnimateDiff Evolved

7. ComfyUI_IPAdapter_plus

描述:

Updated the workflow to use SAM2.

训练词语: