LoRA Extraction tests版本fantexi (ID: 137811)

https://github.com/TkskKurumi/DiffusersFastAPI/blob/main/model_as_lora.py

I've attempted to develop an algorithm that can calculate LoRA from subtracting two trained full model. The core is to compress the full Δw (weight difference) matrix into LoRA.

I think my matrix compression algorithm is not good enough. I'm not good at linear algebra (or I don't even know what fields of knowledge is needed for the task).

Compress matrix W of shape(m, n) into matul of A and B of shapes (m, rank) and (rank, n). The compress ratio according to parameter amount is rank*(m+n)/(m*n). I've found that ratio = 25% may nearly represent Fantexi subtract my anime model. And ratio=5% with lora alpha doubled has adequate result.

I've also tried the algorithm to take image as input and use 2 matrixs of shapes (width, rank) (rank, height). Following image has original 1080 pixels width and height and compressed into low rank 32.

描述:

The LoRA ≈ A - B. Compress ratio = 25%

A: https://civitai.com/models/101518

B: https://civitai.com/models/18427

训练词语: