LTX Image to video版本v5.1 (ID: 1297666)

V6

Removed notes

Info here.

https://civitai.com/articles/10440

New LLM Prompting

https://github.com/SeargeDP/ComfyUI_Searge_LLM

Model

https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

Fine tuned LTX model.

https://www.seaart.ai/models/detail/ctkvacde878c73f55dc0

There is an issue when trying to extend anything. The size of the upscale is changed if not matching the proper sizes. Eg. 720 is changed to 704. This causes extends to have issues. Working on a mehood to just input your upscale size you want and then it will set base /upscale sizes.

Anyone interested in searge LLM prompting? I am experimenting with it and it seems very good.

V5.1

- Fixed the damn restart bug. Finally. 3 things...

- Added blur noise option (bug as its in the group with noise, its meant to be enabled with a noise option to dull the noise)

- added superprompt rewording for prompts.

- Added custom masked noise. I dont think it works well.

- Finished Sd1.5 frame fix, delete if you dont want it.

- made the perturbed attention optional both for base and upscale.

- added feta enhance node. I think it helps, but not really sure yet.

---

V5 - Fully redid interface now that i know main settings

- All options are the same, just now its much easier to follow and change.

- Pretty sure defaults now just work with noise/prompt/seed being main options for change.

---

Extends Beta

- Features 3 clip extends from base for total of 4 clips.

- Tail to head processing.

- Modular interface for expanding.

May be bugs, not many notes atm.

---

The Following node can help lower vram needed for the flow to work well, i will not be adding it to the flow as its a one off node that is not in the manager. Install only if you have issues with vram.

https://gist.github.com/city96/30743dfdfe129b331b5676a79c3a8a39

This node allows the offloading of the clip to cpu/system ram instead of fighting with the ltx model for room in your vram. I went from using 91% of my vram for 512x768 to using 53 and 61%. The text encode of the prompt is slower but the render works just as fast, and moreso if you were maxing out your ram before.

Sorry for the frequent changes, i wanted to get this out before i started on an extend workflow.

4.3 - small update

- changed vae decodes/encodes to tiled i dont notice any diff between this and normal besides it does not lock my pc when decoding large videos. You can change this if you prefer normal ones. (note - temporal size is max. It causes a skip in the video if not)

- Researched Perturbed attention - new base settings and explanation

- added color match node at end of base and upscale to attempt color match the video properly back to the original image.

- added main ref tagging option for extends.

- Fixed vae swap i did not know did not work. Added VAE decoder noise node.

---

4.2 - tried not to change or add much

- Added second add noise option (mtb nodes)

- removed apply stg node. either i was using it wrong or it was overridden by other nodes as it changed nothing in the renders for me.

- Changed upscaler Now upscales in 3 - 5 steps and is better (uncoupled CFG for upscale)

- fixed any scaling issues that could happen when the the input image is not divisible by 8.

- moved fps to settings area.

Note - I have tried the tiled new tiled vae, It works, you need to set your temporal size to max, or at least your frame number.

Note 2 - Try setting rescale to 0 (off) i have read details on how perturbed attention works and tested 0 and got better results. Will add more to notes next update.

---

4.1 bugfixes

- Fixed florence bypass to work with it disabled

- Fixed default upscale setting to work with defaults

- Fixed updscale(added note to explain what its doing and what can be changed)

---

If you have not updated your python lately and would like to have latest torch and such.

Inside comfy ui main folder is an update folder. This has update comfy and python dependencies. This updates everything.

V4 - useability Update

- Cleaned up the interface to make it simplified

- removed upscale options that should have been copied from base.

- Change how extend works

- Replaced all notes

- changed many small things with the latents for upscale

---

tips

• Noise (blur) is the main factor in getting movement

• Change seed if you dont like the movement you are getting

• Inserts(addnewframes) /Stretch change how it is rendered, if one does not work try another.

• Scale up gives more movement /Scale down holds to the image ref more.

• Rescale up holds to the image ref more /down allows more movement

• CFG is how much it tries to do your movement, raise this if you get movement but not what you asked for and down if the video is being burnt.

• Number of frames changes settings needed

• 10 steps is enough... 20-30 are ok but can change the video content.

• If upscaling SGM uniform 0.52 denoise works mostly when not complex movements, you can go higher here but it can change things.

How placed at option works ... i think

Latents placed are 8 latents long, meaning

0 is frame 1-8 are replaced with the image latents.

-1 is last frame meaning last 8 frames are replaced by the image latents.

Now... if you have 49 frames, you can use 49/8=6 (rounded down)

This means you can insert the frames into the middle of the video at 8 frame intervals at 6 points.

3 is mid way, 1-2 are frames 9-17 and 18-25 and same for 4-5-6.

V3

-changed interface

-added many examples for all settings

Have fun :)

https://github.com/Lightricks/LTX-Video

Search LTX in manager

Note to users. Previewing the video as its made is possible with an option in the comfy ui settings under VHS settings, Latent preview can be enabled. This shows animated previews as its rendering. Not sure if it slows things down,

Also note this needs to be disabled to use any of my other workflows as this setting makes them crash.

---

Confirmed working resolutions

512x360 wide and tall

360x360

V2

-added tools group with cropper/removebg groups

-changed prompt to allow pre/app text

-extended the replace functionality

-added upscale group

-added remove/replace after upscale

-added clip extend functionality from upscale or base video gens.

-added many test videos to show settings

-changed default attentions for many things

---

v1.1

-added video frame extract(request)

-fixed group enablers to properly turn things off

-added apply STG node

-florence prompt now optional can be turned off

-added on/off switch for sampler group to allow image prep.

-added more notes on settings

-included a few test videos with settings to show what they do better.

---

This is my first delve into LTX

This flow uses Florence and LTX to do single image to video and 2 image frame interpolation.

Few bugs i will fix tomorrow

• All option boxes are set to max one, should be default, right click on the nodes and properties to change this

Tips -

Try turning off stretch in the sampler group, It seems to allow much longer videos.

with this disabled i was able to do 500 frames, 600 caused the video to turn brown...

Features

2 image interpolation

1 image to video

THis works, probably better than sparse controls for animate diff for 2 image interp. But its not perfect, the images need to be close and the same person/scene for the effect to work properly.

Many notes in the flow

Will require the 2 extra ltx nodes from the manager to use this.

描述:

V5.1

- Fixed the damn restart bug. Finally.

- Added blur noise option

- added superprompt rewording for prompts.

- Added custom masked noise. I dont think it works well.

- Finished Sd1.5 frame fix, delete if you dont want it.

- made the perturbed attention optional both for base and upscale.

- added feta enhance node. I think it helps, but not really sure yet.

训练词语: