PonyXL LoRA Training版本kohya_ss config (ID: 951740)

Training Guide for PonyXL LoRA

Documenting my own process to create captions and train LoRAs, so a post that I can link to exists when needed~

Dataset Captions

"Explain my rather... unconventional, or even controversial, ethos on how to caption the dataset"

The majority of anime models on CivitAI were simply trained on datasets that only got auto-captioned, without any manual pruning at all, which results in the Trigger Words being a giant wall of text. So now, to recreate the look of a character alone, you already have to waste the entire default token count...

Instead, in my opinion, the caption should only consist of "variables" that do not define the subject. In essence, the model should learn to associate the trigger words with the subject. Take a character with blonde hair for example, the blonde hair tag should be removed from the caption, as that "trait" should be linked to the character's name instead. Therefore, I prefer to manually prune the captions before training.

Some people would say this method reduces the flexibility of the model, as you can no longer control each individual detail precisely. However, this goes both ways. I have tried models where if you do not prompt for every single detail, the result ends up not resembling the character at all. So all in all, personally I'd rather use few tags to recreate a character.

Though recently, I found out that when training character pack LoRAs that contain multiple subjects, the model sometimes gets confused between the details, such as the hair colors and eye colors. This issue can most likely be fixed by using a higher Network Dimension, so that it can learn more information. But this also leads to a larger model size. Therefore, in the pursuit of keeping the storage size bearable, I decided that I will also include captions for eye color/hair color/hair style in a character pack LoRA.

The first model that adopts this method is the v3.0 of Assault Lily LoRA

Training Parameters

"Explain some of the parameters that worked the best for me after hours of testing"

• mixed_precision: Use bf16 if your GPU supports it for better results

• dynamo_backend: Improves training speed by ~5% if your GPU supports it

• optimizer: Use Prodigy for automatically managed learning rate

  • learning_rate: Set to 1.0

• network_alpha: Set to 0.5x ~ 0.25x of network_dim

  • d_coef: Set to 1.2 due to the need of higher learning rate caused by network_alpha

• lr_scheduler: Use linear to combat Prodigy's tendency of keeping learning rate high

• clip_skip: Use 2 for Pony

• debiased_estimation_loss: enable to improve the color accuracy

描述:

The file is a .json config for Kohya_SS

• Simply load the config and adjust the paths to your model and dataset

• Also adjust the repeat/epoch count, so that each unique concept gets trained for ~1000 steps in total

训练词语: