We all enjoy LoRAs, some are trained by our self but many are from well known sources. And usually people are just happy about them with little...
This is an automated archive made by the Lemmit Bot.

The original was posted on /r/stablediffusion by /u/StableLlama on 2024-09-04 16:45:15+00:00.

We all enjoy LoRAs, some are trained by our self but many are from well known sources. And usually people are just happy about them with little diverse feedback that gives a real measurement of the quality of a LoRA. But this quality is important for the user - and also for the creator to be able to see where improvement is necessary. So I think we need to make the quality measurable.

For that I created this little list that could create a 1-5 star rating.

It should do what it is advertised to do:

  • Does the output look like it should?
    • fail: +0
    • little resemblance: +1
    • identifiable: +2
    • good match: +3
    • perfect match: +4
  • How often does the output look like it should:
    • seldom (less than every 4th image): +0
    • sometimes (every 3rd or 4th image): +1
    • half of the time (every 2nd image): +2
    • most of the time (only every 3rd or 4th image is a fail): +3
    • nearly every time (at most every 4th image is a fail): +4

It should not do what it is not advertised to do (freedom from side effects):

Test setup: make up a prompt that will work with the LoRA, fix the seed to stay the same and create image A with just the base model (i.e. without the LoRA) and without the trigger word as a base, then do exactly the same with the LoRA loaded (still without the trigger word!) as image B and finally with the trigger word as image C

  • strong side effect: image B looks like image C and not like image A: +0
  • side effect: image B looks like a mixture of image C and image A: +1
  • little side effect: image B looks mostly like image A with little deviations, image C looks very different: +3
  • no side effect: image A and image B are (nearly) identical, image C looks completely different: +4

Note: This setup works for character and object LoRAs. A style LoRA is expected to be a side effect in the classical sense, so often it doesn’t even come with a trigger word. Therefore the definition and test of freedom from side effects is for this type slightly different: create an image of a person or object (either already in the base model of added by a good LoRA) as image D first and then this side effect test should be done by additionally loading the style LoRA to create image E.

When the character/object is still looking like it should (but in the new style, of course) and anything that shouldn’t be is not affected by the style in image E, there’s no side effect.

When the character/object or anything else that shouldn’t be is mutated much more than just changing the style you have a side effect.

And it should not destroy what we have already:

  • minor anatomy issues (hands, finger, feet): -1
  • major anatomy issues (bad arms and legs): -3

It should be easy to use:

  • does it have description about how to use it? +1
  • does it have sample images with sample prompts that show its effect and do they contain the prompt used to create them? +1

Adding all together we could come to a star rating:

13 - 14: Very good, 5 stars

11 - 12: Good, 4 stars

8 - 10: acceptable, 3 stars

5 - 7: poor, 2 stars

4 or less: bad, 1 star

I’m happy to hear your feedback on this attempt to bring quality to the LoRA. So I might update the scoring according to feedback, but I will be transparent about that so that there are not bad surprises.

And I’d also be very happy to see people using this scoring to score LoRAs on the typical places like civitai. And, of course, I’d be also very happy when this helps LoRA trainers to create a good LoRA.