Trigger word: juno
Optional Triggers: space helmet, overboots, blaster
The optional trigger will only show up when prompted for.
Does not currently work with the onsite generator
(civitai does not yet support LoKr)
----------
HOW WAS THIS MADE?
----------
This is a LoKr trained on 115 images of juno. (about 40% ingame screenshots, 40% renders I made myself from a t-posing juno, and 20% official concept art)
This was trained on SimpleTuner for 5400 steps, at batch 6, on an A6000
Training strategy:
So the point here was to train a genuinely flexible lora (LoKr) using nothing but repetitive "meh" quality data.
I limited myself to an ingame screenshots, self-made blender renders & official concept art for my dataset.
Dataset examples:
In addition to that, I've noticed that while multi-resolution training is important and needed, it isn't needed throughout the entire training run
So my theory was that I can just purposely train 50% of my total steps at a resolution of 512px. then, once it's adapted well enough, stop training, add in the 768px and 1024px datasets, and resume training for the final 3k steps. (albeit much slower, due to the increased size of the images)
The result?
a huge success!
There is no decrease in quality nor flexibility from my other LoKr tests, and more than that, it trained significantly faster!
Captioning Strategy:
-
concept art, juno -
3D rendering, space helmet, juno, t-pose, overboots -
3D rendering, juno, overboots -
ingame screenshot, blaster -
ingame screenshot of juno, blaster, space helmet, overboots
Basically, mention the style and what is visible. But dont go into too much detail. Flux can extrapolate A LOT of info. So only mention things that will be misunderstood, or things that you are specifically training and want to be able to prompt for explicitly.
lycoris settings:
{
"algo": "lokr",
"multiplier": 1.0,
"linear_dim": 10000,
"linear_alpha": 1,
"factor": 12,
"apply_preset": {
"target_module": [
"Attention",
"FeedForward"
],
"module_algo_map": {
"Attention": {
"factor": 12
},
"FeedForward": {
"factor": 6
}
}
}
}config.env:
RESUME_CHECKPOINT='latest'
DATALOADER_CONFIG='config/juno.json'
ASPECT_BUCKET_ROUNDING='2'
TRAINING_SEED='42'
USE_EMA='false'
USE_XFORMERS='false'
MINIMUM_RESOLUTION='0'
OUTPUT_DIR='output/juno'
USE_DORA='false'
USE_BITFIT='false'
LORA_TYPE='lycoris'
LYCORIS_CONFIG='config/lycoris_config.json'
PUSH_TO_HUB='false'
PUSH_CHECKPOINTS='false'
MAX_NUM_STEPS='10000'
NUM_EPOCHS='0'
CHECKPOINTING_STEPS='100'
CHECKPOINTING_LIMIT='40'
HUB_MODEL_NAME='simpletuner-lora'
DEBUG_EXTRA_ARGS=''
MODEL_TYPE='lora'
MODEL_NAME='black-forest-labs/FLUX.1-dev'
FLUX='true'
KOLORS='false'
STABLE_DIFFUSION_3='false'
STABLE_DIFFUSION_LEGACY='false'
TRAIN_BATCH_SIZE='4'
USE_GRADIENT_CHECKPOINTING='true'
GRADIENT_ACCUMULATION_STEPS='1'
CAPTION_DROPOUT_PROBABILITY='0'
RESOLUTION_TYPE='pixel_area'
RESOLUTION='1024'
VALIDATION_SEED='42'
VALIDATION_STEPS='100'
VALIDATION_RESOLUTION='1024x1024'
VALIDATION_GUIDANCE='3.5'
VALIDATION_GUIDANCE_RESCALE='0.0'
VALIDATION_NUM_INFERENCE_STEPS='20'
VALIDATION_PROMPT='concept art of juno, space helmet, white background'
ALLOW_TF32='true'
MIXED_PRECISION='bf16'
OPTIMIZER=optimi-lion
LEARNING_RATE='5e-5'
LR_SCHEDULE='polynomial'
LR_WARMUP_STEPS='100'
BASE_MODEL_PRECISION='int8-quanto'
ACCELERATE_EXTRA_ARGS=''
TRAINING_NUM_PROCESSES='1'
TRAINING_NUM_MACHINES='1'
VALIDATION_TORCH_COMPILE='false'
TRAINER_DYNAMO_BACKEND='no'
TRAINER_EXTRA_ARGS='--lr_end=1e-8 --compress_disk_cache --optimizer_config=weight_decay=1e-3'Dataset config for first 3k steps:
[
{
"id": "juno-512",
"type": "local",
"instance_data_dir": "/mnt/md/0/DATASET/juno",
"crop": false,
"resolution": 512,
"resolution_type": "pixel_area",
"minimum_image_size": 512,
"prepend_instance_prompt": false,
"instance_prompt": "",
"only_instance_prompt": false,
"caption_strategy": "textfile",
"cache_dir_vae": "/mnt/md/0/AI/SimpleTuner/SimpleTuner-Cache/vae-cache/juno-512",
"probability": 1.0,
"repeats": 2,
"preserve_data_backend_cache": true,
"cache_file_suffix": "-juno-512"
},
{
"id": "juno-text-embeds",
"type": "local",
"dataset_type": "text_embeds",
"default": true,
"cache_dir": "/mnt/md/0/AI/SimpleTuner/SimpleTuner-Cache/embed-cache/juno",
"disabled": false,
"write_batch_size": 128
}
]Dataset config for steps 3000~5400:
[
{
"id": "juno-1024",
"type": "local",
"instance_data_dir": "/mnt/md/0/DATASET/juno",
"crop": false,
"resolution": 1024,
"resolution_type": "pixel_area",
"minimum_image_size": 1024,
"prepend_instance_prompt": false,
"instance_prompt": "",
"only_instance_prompt": false,
"caption_strategy": "textfile",
"cache_dir_vae": "/mnt/md/0/AI/vae-cache/juno-1024",
"probability": 1.0,
"repeats": 1,
"preserve_data_backend_cache": true,
"cache_file_suffix": "-juno-1024"
},{
"id": "juno-768",
"type": "local",
"instance_data_dir": "/mnt/md/0/DATASET/juno",
"crop": false,
"resolution": 768,
"resolution_type": "pixel_area",
"minimum_image_size": 768,
"prepend_instance_prompt": false,
"instance_prompt": "",
"only_instance_prompt": false,
"caption_strategy": "textfile",
"cache_dir_vae": "/mnt/md/0/AI/vae-cache/juno-768",
"probability": 1.0,
"repeats": 2,
"preserve_data_backend_cache": true,
"cache_file_suffix": "-juno-768"
},{
"id": "juno-512",
"type": "local",
"instance_data_dir": "/mnt/md/0/DATASET/juno",
"crop": false,
"resolution": 512,
"resolution_type": "pixel_area",
"minimum_image_size": 512,
"prepend_instance_prompt": false,
"instance_prompt": "",
"only_instance_prompt": false,
"caption_strategy": "textfile",
"cache_dir_vae": "/mnt/md/0/AI/vae-cache/juno-512",
"probability": 1.0,
"repeats": 2,
"preserve_data_backend_cache": true,
"cache_file_suffix": "-juno-512"
},
{
"id": "juno-text-embeds",
"type": "local",
"dataset_type": "text_embeds",
"default": true,
"cache_dir": "/mnt/md/0/AI/embed-cache/juno",
"disabled": false,
"write_batch_size": 128
}
]描述:
训练词语: juno,space helmet,overboots
名称: juno_for_flux_by_caith.safetensors
大小 (KB): 189898
类型: Model
Pickle 扫描结果: Success
Pickle 扫描信息: No Pickle imports
病毒扫描结果: Success
